Debug a HttpURLConnection problem

Take this example.

import java.io.InputStreamReader;
import java.io.Reader;
import java.net.URL;
import java.net.URLConnection;

public class HttpConnect {
    public static void main(String[] args) throws Exception {
      URL url = new URL("http://www.rgagnon.com/howto.html");
      URLConnection con = url.openConnection();
      Reader reader = new InputStreamReader(con.getInputStream());
      while (true) {
        int ch = reader.read();
        if (ch==-1) {
          break;
        }
        System.out.print((char)ch);
      }
    }
}

The above snippet reads the HTML page and dumps it to the console.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD>
<!-- developpe par  Real Gagnon, Quebec Canada -->
<LINK REL="SHORTCUT ICON" HREF="http://www.maixuanviet.com/favicon.ico">
<META NAME="description"
      Content="Real's JAVA JAVASCRIPT WSH and PowerBuilder How-to pages with useful code snippets">
<META NAME="keywords"
      Content="java,javascript,wsh,vbscript,how-to,powerbuilder">
<LINK title="mystyle" href="howto.css" type="text/css" rel="stylesheet">
...

To trace what is going on at the HTTP protocol level, we can switch the HttpURLConnection/UrlConnection in debug mode.

You must enable the java-logging mechanism by setting a special property when starting the JVM :

java.exe -Djava.util.logging.config.file=logging.properties HttpConnect

and put in logging.properties file (by default in JRE_HOME\lib) the following property

sun.net.www.protocol.http.HttpURLConnection.level = ALL

Note :
If running from Eclipse, you put the -Djava.util.logging… via the Run configuration dialog -> Arguments Tab -> VM arguments textarea.

The result is :

2010-08-07 00:00:31 sun.net.www.protocol.http.HttpURLConnection writeRequests
FIN: sun.net.www.MessageHeader@16caf435 pairs: {GET /howto.html HTTP/1.1: null}{User-Agent: Java/1.6.0_20}{Host: www.maixuanviet.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
2010-08-07 00:00:31 sun.net.www.protocol.http.HttpURLConnection getInputStream
FIN: sun.net.www.MessageHeader@5ac0728 pairs: {null: HTTP/1.1 200 OK}{Date: Sat, 07 Aug 2010 04:00:33 GMT}{Server: Apache}{Accept-Ranges: bytes}{Content-Length: 17912}{Keep-Alive: timeout=5, max=64}{Connection: Keep-Alive}{Content-Type: text/html}
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD>
<!-- developpe par  Real Gagnon, Quebec Canada -->
<LINK REL="SHORTCUT ICON" HREF="http://www.maixuanviet.com/favicon.ico">
<META NAME="description"
      Content="Real's JAVA JAVASCRIPT WSH and PowerBuilder How-to pages with useful code snippets">
...

Done! Happy Coding!

Java 9 Stream API Improvements

Guide to java.util.concurrent.Locks

Handle EML file with JavaMail

Java Program to Implement Max-Flow Min-Cut Theorem

Composition, Aggregation, and Association in Java

Java Program to Check Whether an Undirected Graph Contains a Eulerian Path

Creating a Generic Array in Java

Model, ModelMap, and ModelAndView in Spring MVC

Java Program to Implement RenderingHints API

Mix plain text and HTML content in a mail

Java Program to Check if a Given Binary Tree is an AVL Tree or Not