1. Installation
I use python to imitate http requests. I use the requests module in python
1) First install requests
open (cmd) enter pip install request
Then it will automatically download and install. If you are prompted to try over a few times during the download process, you can
enter pip install request again after completion. // Note that I am using requests instead of requests.
If it is displayed as the following figure, the installation is complete
C:\Users\23501>pip install request
Requirement already satisfied: request in c:\users\23501\appdata\local\programs\python\python38\lib\site-packages (2019.4.13)
Requirement already satisfied: get in c:\users\23501\appdata\local\programs\python\python38\lib\site-packages (from request) (2019.4.13)
Requirement already satisfied: post in c:\users\23501\appdata\local\programs\python\python38\lib\site-packages (from request) (2019.4.13)
Requirement already satisfied: setuptools in c:\users\23501\appdata\local\programs\python\python38\lib\site-packages (from request) (41.2.0)
Requirement already satisfied: query_string in c:\users\23501\appdata\local\programs\python\python38\lib\site-packages (from get->request) (2019.4.13)
Requirement already satisfied: public in c:\users\23501\appdata\local\programs\python\python38\lib\site-packages (from query_string->get->request) (2019.4.13)
2) Import requests
in python. Enter python in (cmd)
and import requests
as follows
C:\Users\23501>python
Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>>
2. The method of using requests
1) Add the requests method
response.text 该方式往往出现乱码，出现乱码使用 response.encoding='utf-8'
response.content.decode()把响应的二进制字节流转化为str类型
response.request.url 发送请求的url地址
response.url #respons响应的url
response.request.headers #请求头
response.headers #响应请求
2) Obtain website information
C:\Users\23501>python
Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> url='http://39.104.151.240:7000/get_yiqingdata'
>>> response = requests.get(url)requests.get(url) #发送get请求，请求url对应的响应
>>> response.text
'[{"name": "\\u7701/\\u5730\\u533a", "citytotal": "\\u786e\\u8bca", "citynew": "\\u4eca\\u65e5\\u786e\\u8bca"}, {"name": "\\u5c71\\u4e1c", "citytotal": 768, "citynew": 0}, {"name": "\\u5883\\u5916\\u8f93\\u5165", "citytotal": 9, "citynew": 0}, {"name": "\\u70df\\u53f0", "citytotal": 47, "citynew": 0}, {"name": "\\u9752\\u5c9b", "citytotal": 61, "citynew": 0}, {"name": "\\u6d4e\\u5357", "citytotal": 47, "citynew": 0}, {"name": "\\u5fb7\\u5dde", "citytotal": 37, "citynew": 0}, {"name": "\\u4e34\\u6c82", "citytotal": 49, "citynew": 0}, {"name": "\\u6dc4\\u535a", "citytotal": 30, "citynew": 0}, {"name": "\\u67a3\\u5e84", "citytotal": 24, "citynew": 0}, {"name": "\\u6d4e\\u5b81", "citytotal": 260, "citynew": 0}, {"name": "\\u6f4d\\u574a", "citytotal": 44, "citynew": 0}, {"name": "\\u6cf0\\u5b89", "citytotal": 35, "citynew": 0}, {"name": "\\u5a01\\u6d77", "citytotal": 38, "citynew": 0}, {"name": "\\u65e5\\u7167", "citytotal": 16, "citynew": 0}, {"name": "\\u83cf\\u6cfd", "citytotal": 18, "citynew": 0}, {"name": "\\u6ee8\\u5dde", "citytotal": 15, "citynew": 0}, {"name": "\\u804a\\u57ce", "citytotal": 38, "citynew": 0}]'
>>>
At this time, the ASCII code string needs to be converted into Chinese
3) The garbled code is converted into Chinese.
Use the encode (). Decode ('unicode_escape') method of python to convert the garbled code into Chinese
>>> name=response.text //将抓取的text 给name
//name 调用encode().decode('unicode_escape')方法
>>> name.encode().decode('unicode_escape')
The final result is as follows.
Below is a little white. If there is an inappropriate place, please correct me.
