Entry

Specifying a proxy server to be used for httplib?

Aug 2nd, 2000 04:25
unknown unknown, Amit, David Currie

Problem:

The Internet in our office can only be accessed through a proxy server.
I wanted to write a utility in Python that uses the httplib module to
automate some routine searches. But I am stuck since there does not
seem to be a way to access the Internet thru' the proxy.

I am sure there is a very small thing which I am missing. Could someone
please advise on how to channel the request thru' proxy and automate
authentication?

Solution:

I figured it out by reading RFC 2617 on "HTTP Authentication: Basic and
Digest Access Authentication", and by a helpful post by David Currie
dated 12 July, 2000.

The solution - in this case - was as follows:

---------------------------------------------
import httplib
import base64

# 1. connect to the proxy
h1 = httplib.HTTP("154.112.170.19:80")

# 2. give the absolute URL in the request
h1.putrequest('GET', 'http://www.yahoo.com/')

h1.putheader('Accept', 'text/html')
h1.putheader('Accept', 'text/plain')

# 3. set the header with a base64 encoding of user-id:passwd
auth = "Basic " + base64.encodestring("john_doe:glug#123")
h1.putheader('Proxy-Authorization', auth)

h1.endheaders()

# 4. get the page
errcode, errmsg, headers = h1.getreply()

print errcode
print errmsg
print headers

f=h1.getfile()
for line in f.readlines():
    print line
---------------------------------------------