I’ve been playing around with trying to pass HTTP connections through a proxy server requiring authentication from Python and after a few hours of hunting around I’ve managed to find enough bits of information to cobble together into a working example.  Since I couldn’t find such an example myself I’ve put this one together.  It looks simple now, which it is (thanks Python!), but unfortunately I couldn’t find any documentation that presented it this clearly.

Just substitute the appropriate variable values and it should run as is.

# urllib2_proxy_handler.py
#
# Author: Wayne Koorts
# Date:   27/10/2008
#
# Example for using urllib2.urlopen() with a proxy server requiring authentication

import urllib2

uri = "http://www.python.org"
http_proxy_server = "someproxyserver.com"
http_proxy_port = "3128"
http_proxy_realm = http_proxy_server # Worked in my (limited) testing environment.
http_proxy_user = "username"
http_proxy_passwd = "password"

# Next line = "http://username:password@someproxyserver.com:3128"
http_proxy_full_auth_string = "http://%s:%s@%s:%s" % (http_proxy_user,
                                                      http_proxy_passwd,
                                                      http_proxy_server,
                                                      http_proxy_port)

def open_url_no_proxy():
    urllib2.urlopen(uri)
    
    print "Apparent success without proxy server!"
        
def open_url_installed_opener():
    proxy_handler = urllib2.ProxyHandler({"http": http_proxy_full_auth_string})
    
    opener = urllib2.build_opener(proxy_handler)
    urllib2.install_opener(opener)
    urllib2.urlopen(uri)
    
    print "Apparent success through proxy server!"
    
if __name__ == "__main__":
    open_url_no_proxy()
    open_url_installed_opener()

Tested using an Ubuntu 8.04 Server Edition proxy server with a mostly out-of-the-box Squid installation using ncsa_auth authentication.

-Wayne