在爬虫中请求中几乎都要有header请求头、fake_useragent是python中应用于UA的拓展包
安装方式 CMD中运行
pip install fake_useragent
使用方法
from fake_useragent import UserAgent
ua = UserAgent()print(ua.random) #随机
print(ua.ie) #ie
print(ua.chrome) #Google
print(ua.firefox) #火狐
print(ua.safari) #苹果
print(ua.opera) #欧朋
结果如下
Mozilla/5.0 (Windows NT 6.4; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36
Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 7.0; InfoPath.3; .NET CLR 3.1.40767; Trident/6.0; en-IN)
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1468.0 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20130330 Firefox/21.0
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27
Opera/12.0(Windows NT 5.1;U;en)Presto/22.9.168 Version/12.00
实践
from fake_useragent import UserAgent
ua = UserAgent()def html(url):
try:
r = req.get(url,timeout=80,headers={'User-Agent': ua.random})
r.raise_for_status()
r.encoding = r.apparent_encoding
return r.text
except Exception as e:
print(e)