CSV file, contains a total of 9529 User-Agent, can be directly used to open or import excel database
network link plate: https://pan.baidu.com/s/1NdlLUvLM0nDnrC8Ax1F-QA extraction code: ztte
Content Example:
| id | agent |
|---|---|
| … | … |
| 1411 | Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14931 |
| 1412 | Chrome (AppleWebKit/537.1; Chrome50.0; Windows NT 6.3) AppleWebKit/537.36 (KHTML like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393 |
| 1413 | Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/46.0.2486.0 Safari/537.36 Edge/13.9200 |
| 1414 | Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/46.0.2486.0 Safari/537.36 Edge/13.10586 |
| 1415 | Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246 |
| 1416 | Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533+ (KHTML, like Gecko) Element Browser 5.0 |
| 1417 | ELinks/0.9.3 (textmode; Linux 2.6.9-kanotix-8 i686; 127x41) |
| 1418 | ELinks/0.9.3 (textmode; Linux 2.6.11-auditor-10 i686; 80x24) |
| 1419 | ELinks/0.9.3 (textmode; Linux 2.6.11 i686; 79x24) |
| 1420 | ELinks (0.4pre6; Linux 2.2.19ext3 alpha; 80x25) |
| … | … |
Example of use:
First, CSV table into the database pvAgent.db in AgentTable, I use the built-in python sqlite3 database
import sqlite3
import random
import requests
import time
# 连接数据库pvAgent.db，从数据表AgentTable随机取一条User-Agent并返回
def randomUserAgent():
connection = sqlite3.connect("pvAgent.db")
cursor = connection.cursor()
cursor.execute("CREATE TABLE IF NOT EXISTS AgentTable(_id INTEGER PRIMARY KEY AUTOINCREMENT, agent VARCHAR(1024))")
lastIDCursor = cursor.execute("select * from AgentTable order by _id desc limit 1")
userAgentRandomID = 0
for lastID in lastIDCursor:
userAgentRandomID = random.randint(1, lastID[0])
userAgentRow = cursor.execute("select * from AgentTable where _id=%d" % userAgentRandomID)
for userAgent in userAgentRow:
agentStr = userAgent[1]
connection.close()
return agentStr
if __name__ == '__main__':
while(True):
# 调用randomUserAgent()函数从数据表AgentTable中随机抽一条User-Agent来使用
agent = randomUserAgent()
ip = '119.57.108.53'
headers = {'User-Agent' : agent}
proxies = {'http' : ip}
requests.get('https://www.baidu.com', headers=headers, proxies=proxies, verify=True)
print('Access to success.')
time.sleep(1)
