requests Library Learn

Description:
Requests to use Apache2 licensed permit HTTP library
supports HTTP connection remains and connection pooling, support the use of a session cookie to maintain, support file upload, encoding support automatic response content, URL and POST data to support automatic coding internationalization.
Automatically achieve a lasting connection keep-alive

Cons: Can not get information dynamically generated script

Request method (GET PUT POST head Options Delete):
0 requests.request () # a configuration request, the base support of the following methods
. 1 requests.get ( ' https://github.com/timeline.json ') the GET request # The main method (all resources)
2 requests.post ( " http://httpbin.org/post ") # POST request,
3 requests.put ( " http://httpbin.org/put ") # PUT request, submitted PUT request, the storage
4 requests.delete ( " http://httpbin.org/delete ") # dELETE request to delete the
5 requests.head ( " http://httpbin.org/get ") # the hEAD request, the page header information
requests.options. 6 ( " http://httpbin.org/get ") the OPTIONS request #
7 requests.patch () # local modification request submitted

Get request transmitted parameters:
payload = { 'key1': 'VALUE1', 'key2': [ 'value2', 'value3']} # can pass the parameter list
R & lt requests.get = ( " HTTP: // httpbin. ORG / GET ", params = payload) using # pass parameters params
print (r.url)

http://httpbin.org/get?key1=value1&key2=value2&key2=value3

requests.request (Method, URL, kwargs)
Method: mode request
EG: R & lt requests.request = ( "GET", URL)
kwargs access control parameter, optional
params transmission parameters
data dictionary
JSON JSON format
headers dictionary, custom HTTP header
Cookies
auth
files dictionary type, file transfer
timeout set timeout s units
set to access the proxy server proxies dictionary, you can increase the login authentication

allow_redirects Ture/False,默认为Ture，重定向开关
stream  Ture/False,默认为Ture，获取内容立即下载开关
Verify  Ture/False,默认为Ture，认证SSL证书开关
cert    本地SSL证书路径

requests.get(url,params=None,kwargs)
requests.head(url,kwargs)
requests.post(url,data=None,json=None,kwargs)
requests.put(url,data=None,kwargs)
requests.patch(url,data=None,kwargs)
requests.delete(url,kwargs)

Response object attributes:
r.status_code http return request status code
string r.text http response content, i.e. the corresponding page content url
r.headers http response headers
r.encoding guess header from http response content encoding
r.apparent_encoding analyze the content of the response from the content encoding (encoding alternatively)
r.content HTTP response content in binary form, it can be used to download

...
r.status_code
IF 200 -> r.text / r.encoding ...
the else -> Error / Exception

Abnormal:
requests.ConnectionError network connection error exception
requests.HTTPError HTTP error exception
requests.URLRequired URL missing
requests.TooManyRedirects exceed the maximum number of defined
requests.ConnectTimeout connect to a remote server timeout
requests.Timeout request URL timeout
r.raise_for_status (if not 200), produce abnormal requests.HTTPError

eg：

def get_HTML(url):  #获取网页源代码
    try:
        r = requests.get(url,timeout=30)
        r.raise_for_status()
        r.encoding = r.apparent_encoding
        return r.text
    except:
        return "连接失败"

Custom request header:
add HTTP request header, a dict simply pass it to the headers parameter
EG:
URL = ' https://api.github.com/some/endpoint '
headers {= 'User-Agent' : 'My-App / 0.0.1'}
R & lt requests.get = (URL, headers = headers)
Note: custom header of lower priority than certain sources such as:
If a user authentication in .netrc information, use headers = set authorization will not take effect.
If the set auth = parameter .netrcset is invalid.
If you are redirected to another host, the authorization header will be deleted.
Proxy Authorization header will be overwritten URL provided in the proxy identity.
In the case where we can determine the content length, header of Content-Length is overwritten
all header value must be a string, bytestring or unicode.