[FIXED] Turning on debug output for python 3 urllib

Issue

In python 2, it was possible to get debug output from urllib by doing

import httplib
import urllib
httplib.HTTPConnection.debuglevel = 1
response = urllib.urlopen('http://example.com').read()

However, in python 3 it looks like this has been moved to

http.client.HTTPConnection.set_debuglevel(level)

However, I'm using urllib not http.client directly. How can I set it up so that my http request display debugging information in this way?

Here's what I"m using so far. What's the best way to proceed if I want to be able to get debug information?

#Request Login page
cookiejar = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cookiejar))
request = urllib.request.Request(options.uri)
add_std_headers(request)
response = opener.open(request)
response_string = response.read().decode("utf8")
# ...

Solution

You were right the first time. You can simply add the line http.client.HTTPConnection.debuglevel = 1 at the start of your file to turn on HTTP debugging application-wide. urllib.request still uses http.client.

It seems that there's also a way to set the debuglevel for a single handler (by creating urllib.request.HTTPHandler(debuglevel=1) and building an opener with that), but on my installation of Python3 (3.0b3) it's not actually implemented. I imagine that's changed in more recent versions!

Answered By - PAG