Ajax crawling Baidu pictures

Python crawls Baidu pictures

# -*- coding: utf-8 -*-
"""
Created on Sat Jan  5 01:38:24 2021

@author: kun
"""
import time
import requests
import json
import urllib

headers = {
    
    'Accept': 'text/plain, */*; q=0.01',
           'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36'}


def get_info(category, num):
    url = 'https://image.baidu.com/search/acjson?tn=resultjson_com&logid=10506596288186331059&ipn=rj&ct=201326592&is=&fp=result&queryWord={}&cl=2&lm=-1&ie=utf-8&oe=utf-8&adpicid=&st=-1&z=&ic=0&hd=&latest=&copyright=&word={}&s=&se=&tab=&width=&height=&face=0&istype=2&qc=&nc=1&fr=&expermode=&force=&pn={}&rn=30&'.format(
        category, category, num)

    html = requests.get(url, headers=headers)
    json_data = json.loads(html.text)
    datas = json_data['data']
    for data in datas[0:30]:
        picture_url = data['thumbURL']
        path = 'C:\\picture\\'
        urllib.request.urlretrieve(picture_url, path + picture_url[-23:])
        print("下载成功", picture_url[-23:])
        time.sleep(1)


if __name__ == '__main__':
    get_info('风景', 20)
    time.sleep(1)

The results are as follows:

Recommended reading:

1. Use xpath to crawl data
2. jupyter notebook use
3. BeautifulSoup crawls the top 250 Douban movies
4. An article takes you to master the requests module
5. Python web crawler basics-BeautifulSoup

This is the end, if it helps you, welcome to like and follow, your likes are very important to me