Table of contents
1. Goal 1: Use seesion to enter the request
1. Goal 1: Use seesion to enter the request
(1) Application:
Request and crawl the returned data packets by creating a session object
Scenario: A website that requires login to crawl
Role: the session object can remain logged in between multiple requests
(2) Steps:
Step 1: Create a session object
Step 2: Simulate login and get session
Step 3: Carry cookies and use session to crawl
Step 4: Processing of data packets
(3) Code part:
import requests
data = {}
headers1={}
#part1:
# 创建一个session对象
session = requests.Session()
#part2:
# 发送POST请求(登录,数据包我就不带了,简写一下)
response = session.post(login_url,json=data)
#part3:
#需要携带cookie
headers2={}
# 发送GET请求进行爬取
response = session.get('https://baidu.com')
# 设置响应的编码方式为UTF-8
response.encoding = 'utf-8'
# 获取返回的数据包
#part4:
data = response.text
print(data)
# 处理数据包
# ...
# 关闭session
session.close()
2. Network Security
GitHub - BLACKxZONE/Treasure_knowledgehttps://github.com/BLACKxZONE/Treasure_knowledge