获取不到正确的requests请求结果

​ 爬取过程中一直获取到的结果和抓包获取的不一样,一直以为是页面发生了跳转,添加了allow_redirects=False,还是不对。使用fiddler抓包到的headers和data访问也不对,最后换浏览器,还是不对,多次尝试后,将data=data,居然拿到了正确的json

import json

import requests

headers = {
        'Accept': 'application/json, text/javascript, */*; q=0.01',
        'Accept-Encoding': 'gzip, deflate, br',
        'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8',
        'Connection': 'keep-alive',
        'Content-Length': '110',
        'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
        'Cookies':'hehe',
        'Host': 'www.ll.com',
        'Origin': 'https://www.lagou.com',
        'Referer': 'https://www.lau.com/dd/list_%E6%9D%AD%E5%B7%9E%E7%82%B9%E6%9C%9B%E7%A7%91%E6%8A%80%E6%9C%89%E9%99%90%E5%85%AC%E5%8F%B8?labelWords=&fromSearch=true&suginput=',
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36',
        'X-Anit-Forge-Code': '0',
        'X-Anit-Forge-Token': 'None',
        'X-Requested-With': 'XMLHttpRequest'
}

data = {

    'first': 'false',
    'pn': '5',
    'kd': '杭州点点科技有限公司'


}


url = 'https://www.lau.com/dd/positionAjax.json?needAddtionalResult=false'
#一直请求下来的json和页面看到的不一样
# resp = requests.post(url,headers=headers,allow_redirects=False,data=data,verify=False)
resp = requests.post(url,headers=headers,data=data,verify=False)
print(resp.text)

猜你喜欢

转载自blog.csdn.net/u014229742/article/details/85063713