requests library

Here Insert Picture Description
request library easy to use than urlllib

Examples

import requests
response=requests.get("http://www.baidu.com/")
print(response.status_code)
print(response.txt)
print(type(response.txt))
print(response.cookies)

request library implements various requests

import requests
requests.get("http://www.baidu.com/")
requests.post("http://www.baidu.com/")
requests.delete("http://www.baidu.com/")
requests.head("http://www.baidu.com/")
requests.options("http://www.baidu.com/")

The basic get request

import requests
response=requests.get("http://www.baidu.com/")
print(response.txt)#打印百度的源代码

get request parameters

import requests
response=requests.get("http://httpbin.org/get?name=germey&age=22")
print(response.txt)

Equivalent to

import requests
data={
	''name':'germey'
	'age':22
}
response=requests.get("http://httpbin.org/get?",params=data)
print(response.txt)

Parsing json

import requests
import json
response=requests.get("http://httpbin.org/get")
print(type(response.text))
print(response.json())
print(json.loads(response.text))
print(type(response.json()))

Binary data acquisition

For the text and content of understanding

response.text returns a unicode text data type
response.content returns bytes of binary data type
that is if you want to take the text data can be response.text If you want to take pictures, files, you can response.content

import requests
resopnse=requests.get("https://github.com/favicon.ico")
print(type(resopnse.text),type(resopnse.content))
#  <class 'str'> 
#<class'bytes'>
print(resopnse.text)
print(resopnse.content)

save Picture

import requests
resopnse=requests.get("https://github.com/favicon.ico")
with open("gitub.ico",'wb') as f:
    f.write(resopnse.content)
    f.close()

No header information will be reported 500 errors

import requests
response=requests.get("https://www.zhihu.com/explore")#知乎官网
print(response.text)

The return code is

400 Bad Request

openresty

Add header information

import requests

headers={
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36 Core/1.70.3741.400 QQBrowser/10.5.3863.400'
        }
response=requests.get("https://www.zhihu.com/explore",headers=headers)
print(response.text)

Header information is added, the source code can be printed successfully in almost known

post request

import requests
data={'name':'germey','age':'22'}
headers={
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36 Core/1.70.3741.400 QQBrowser/10.5.3863.400'
        }
response=requests.post('https://www.org/post',data=data,headers=headers)
print(response.json())

response

response property

import requests
response=requests.get("http://www.jianshu.com")
print(type(response.status_code),response.status_code)
print(type(response.headers),response.headers)
print(type(response.cookies),response.cookies)
print(type(response.url),response.url)
print(type(response.history),response.history)#访问历史记录

Status code is determined

import requests
response=requests.get("http://www.jianshu.com")
exit() if not response.status_code==requests.codes.ok else print("request successfully")

Status code commonly used in determining the success of the request, and also provides a built-in requests status code query object requests.codes, by comparing the return code and return code built success, to ensure the normal response to a request to obtain the information output of successful requests, otherwise terminate the program.
Built-in code
Here Insert Picture Description

request Advanced Operations

1. File Upload

import requests
files={‘file’：open('gitub.ico','rb')}
response=requests.post("http://httpbin.org/post",files=files)
print(response.text)

2. Get cookie

import requests
response=requests.get("http://www.jianshu.com")
print(response.cookies)
for key ,value in response.cookies.items():
	print(key+'='+value)

3. session to maintain (for analog login)

import requests
requests.get('http://httpbin.org/cookies/set/number/123456789')
response=requests.get("http://httpbin.org/cookies")
print(response.text)#  得不到，发起了两次请求，每个请求是独立的，是不可以实现会话维持的

Returns the result:
{
"Cookies": {}
}

import requests
s=requests.Session()
s.get('http://httpbin.org/cookies/set/number/123456789')
response=s.get("http://httpbin.org/cookies")
print(response.text)

Session can be done without fear of simulated session cookies problem, he is usually the next step for performing a simulation after a successful login
Returns the result:
{
"Cookies": {
"Number": "123456789"
}
}

Certificate Validation


import requests
response=requests.get("https://www.12306.cn")
print(response.status_code)

import requests
from requests.packages import urllib3
urllib.disable_warmings()#  可以消除警告，你可以把这行代码消除看一下运行结果
response=requests.get("https://www.12306.cn",verfy=False)
print(response.status_code)
#这样就可以避免证书验证了

Timeout settings

import requests
response=requests.get("https://www.12306.cn",timeout=0.01)
print(response.status_code)

Here timeout parameter is the sum of the read and connections, respectively, may want to specify if the incoming tuple timeout, as timeout = (5,11,30)

Authentication Settings

When we visit a Web site, you need to let us Once logged in can access the web site, then we simulate log in, you need to use built-in authentication function requests

import requests
from requests.auth import HTTPBasicAuth
r=requests.get("http://120.27.34.24:9001",auth=HTTPBasicAuth('user','123'))
print(r.status_code)