Java Reading data from URL

URL class provides following methods to read data from URL.

public final InputStream openStream() throws IOException

public URLConnection openConnection() throws IOException

public URLConnection openConnection(Proxy proxy) throws IOException

public final Object getContent() throws IOException

public final Object getContent(Class[] classes) throws IOException

1. public final InputStream openStream() throws IOException
Opens a connection to this URL and returns an InputStream for reading from that connection.

import java.io.IOException;
import java.io.InputStream;
import java.net.URL;

public class Main {
  public static void main(String args[]) throws IOException {

    String location = "https://self-learning-java-tutorial.blogspot.com/2014/02/creating-object.html";
    URL url = new URL(location);

    InputStream is = url.openStream();

    int c;
    char data;

    while ((c = is.read()) != -1) {
      data = (char) c;
      System.out.print(data);
    }

    is.close();
  }
}

Above program, reads data from the URL, https://self-learning-java-tutorial.blogspot.com/2014/02/creating-object.html and print on the console.

You can write the same application using BufferedReader also.

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;

public class Main {
  public static void main(String args[]) throws IOException {

    String location = "https://self-learning-java-tutorial.blogspot.com/2014/02/creating-object.html";
    URL url = new URL(location);

    InputStream is = url.openStream();
    
    BufferedReader br = new BufferedReader(new InputStreamReader(is));
    
    String line;
    
    while((line=br.readLine()) != null){
      System.out.println(line);
    }
  }
}

2. public URLConnection openConnection() throws IOException
Returns a URLConnection instance that represents a connection to the remote object referred to by the URL.

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;

public class Main {
  public static void main(String args[]) throws IOException {

    String location = "https://self-learning-java-tutorial.blogspot.com/2014/02/creating-object.html";
    URL url = new URL(location);

    URLConnection conn = url.openConnection();
    InputStream is = conn.getInputStream();
    
    BufferedReader br = new BufferedReader(new InputStreamReader(is));
    
    String line;
    
    while((line=br.readLine()) != null){
      System.out.println(line);
    }
  }
}

One advantage of URLConnection is, you can access connection metadata with the URL. You can access headers etc.,

import java.io.IOException;
import java.net.URL;
import java.net.URLConnection;
import java.util.List;
import java.util.Map;
import java.util.Set;

public class Main {
  public static void main(String args[]) throws IOException {

    String location = "https://self-learning-java-tutorial.blogspot.com/2014/02/creating-object.html";
    URL url = new URL(location);

    URLConnection conn = url.openConnection();
    Map<String, List<String>> headers = conn.getHeaderFields();

    Set<String> keySet = headers.keySet();

    for (String str : keySet) {
      System.out.println(str);

      List<String> values = headers.get(str);

      for (String tmp : values) {
        System.out.println(tmp);
      }
      System.out.println("-------------------");
    }
  }
}

Sample Output

HTTP/1.1 200 OK
-------------------
X-XSS-Protection
1; mode=block
-------------------
Expires
Wed, 06 May 2015 05:49:38 GMT
-------------------
Alternate-Protocol
80:quic,p=1
-------------------
Last-Modified
Sat, 02 May 2015 17:22:29 GMT
-------------------
Server
GSE
-------------------
X-Content-Type-Options
nosniff
-------------------
Cache-Control
private, max-age=0
-------------------
Transfer-Encoding
chunked
-------------------
Vary
Accept-Encoding
-------------------
Date
Wed, 06 May 2015 05:49:38 GMT
-------------------
Accept-Ranges
none
-------------------
Content-Type
text/html; charset=UTF-8
-------------------

3. public final Object getContent() throws IOException

getContent() returns the contents of the URL. Depends on the type of data returned by the getContent() method, it refers to specific kind of object.

import java.io.IOException;
import java.net.URL;

public class Main {
  public static void main(String args[]) throws IOException {

    String location = "https://self-learning-java-tutorial.blogspot.com/2014/02/creating-object.html";
    URL url = new URL(location);

    Object content = url.getContent();
    System.out.println(content.getClass().getName());
  }
}

Output

sun.net.www.protocol.http.HttpURLConnection$HttpInputStream

Change the location to an image, and re run the application

String location = "https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibO4MpRsExMN9vOqTNRe6xRvgW_H2sJzm0pbf1lINWKqn4F9HQmxTVP8Gbto3Vw1_J8PBv92ImlJs_0Qkpgqkgk05RmbwKTabn3-nLLSwMEbhJH7z4uoBknwJI1VidxtS9mzTYMqicU8xU/s1600/struts2+hello+world+eclipse+1.png";

For the above image, you will get following output.

sun.awt.image.URLImageSource

public final Object getContent(Class[] classes) throws IOException

It is overloaded version of getContent method. By using this method, you can choose which class you’d like the content to be returned as. This method returns the URL content in the first available format, that user specified.

import java.io.IOException;
import java.io.InputStream;
import java.io.Reader;
import java.net.URL;

public class Main {
  public static void main(String args[]) throws IOException {

    String location = "https://self-learning-java-tutorial.blogspot.com/2014/02/creating-object.html";
    URL url = new URL(location);

    Class<?>[] types = new Class[3];
    types[0] = String.class;
    types[1] = Reader.class;
    types[2] = InputStream.class;
    Object o = url.getContent(types);

    System.out.println(o.getClass().getName());
  }
}

Output

sun.net.www.protocol.http.HttpURLConnection$HttpInputStream

Prevoius                                                 Next                                                 Home