Before using the requests module to send a request, we have to review the URL (Uniform Resource Locator) we learned before.
After you get the URL address where the data is located, when sending a network request, the requested URL contains two address parameters: query parameters and request parameters .
Python crawler, data analysis, website development and other case tutorial videos are free to watch online
https://space.bilibili.com/523606542
Python learning exchange group: 1039645993
Query parameter
When we crawl some special URLs, there will be some special parameters in the requested URL, such as the following sites:
This is the query parameter of the URL. In front of the URL ? Behind the binary data is query parameters.
URL query string (query string) passed some sort of data. If you are manually constructing the URL, that is, the URL constructed by concatenating strings. Then the data will be placed in the URL in the form of key/value pairs, followed by a question mark.
E.g:
https://image.so.com/i?q=%E9%A3%8E%E6%99%AF&src=srp
params keyword parameters
Requests allows you to use params keyword parameters and provide these parameters in a dictionary.
For example, if you want to pass key1=value1 and key2=value2 to httpbin.org/get, then you can use the following code:
import request
params = {'q' : '风景' , 'src' : 'srp'}
response = requests.get("https: //image.so.com/i"，params=params)
By printing out the URL, you can see that the URL has been correctly encoded:
print(response.url)
# 打印结果
https : / /image.so.com/i?q=%E9%A3%8E%E6%99%AF&src=srp
note
Chinese characters are not supported by default in the URL address, so Chinese characters will be converted into URL encoding form in the request
The same query parameters can be found in the browser capture tool, located in the Query String Parameters under the Headers column , as shown in the following figure:
Request parameter
There is an essential difference between request parameters and query parameters. Request parameters are generally parameters carried when sending a post request and submitting a form data request to the server.
Note: The request parameters will not be displayed in the url address, only the query parameters will be displayed.
The request parameters are in the browser capture tool, and are located in the Form Data under the Headers column as shown in the following data capture:
data keyword parameter
Sending a POST request in the requests module is also a relatively easy operation. To achieve this, simply pass a dictionary to the data parameter . Your data dictionary will be automatically encoded as a form when making a request:
data = {'key1' : 'value1'，'key2 ' : 'value2 '}
response = requests.post("http://httpbin.org/post"，data=data)
Of course, the post method in Requests only has one more data parameter compared to the get method. Other parameters are similar. For example, we can also add query string params parameters to the URL in the post , or add headers parameters like the get method. .