python crawler addition request header information

1. headers by adding the parameters urllib.request.Request

# First to use a dictionary information into 
headers = {} 
headers [ 'the User-- Agent'] = 'the Mozilla / 5.0 (the Windows NT 6.1; Win64; x64-) .......' 
headers [ ' ...... '] =' ........ ' 

# generates a request packet, and where the url data given in advance 
REQ = urllib.request.Request (url, data, headers) 

# use this message to request a web page, then the message with the request to the browser identifies 
html = urllib.request.urlopen (req) .read ( )

2. Add add_header by the method urllib.request.Request

# Generation request message, where the first parameter was not added header 
REQ = urllib.request.Request (URL, reqdata) 

# header added by the request header object returned, here is a neuron progenitor, is not a dictionary 
req.add_header ( 'User-Agent', 'Mozilla / 5.0 (Windows NT 6.1; Win64; x64)') 

# then go with the browser request is identified in the message 
html = urllib.request.urlopen (req) .read ( )

  

Guess you like

Origin www.cnblogs.com/forlive/p/11307946.html