I found a good-looking mobile wallpaper website, and the hands of the code are already hungry and thirsty.

The text and pictures in this article are from the Internet and are for learning and communication purposes only. They do not have any commercial use. The copyright belongs to the original author. If you have any questions, please contact us for processing.

Recently, there was a bug in the crawler code of a classmate, and asked me how to change it

 

So I found this nice mobile wallpaper website.

 

Seeing so many beautiful wallpapers, my crawlers are already hungry and thirsty

Victim URL

http://m.bcoderss.com/

For benign crawling, this code is incompletely displayed

 

Part of the crawler code

Import tool

from urllib import parse
import requests
import parsel
import re

Analyze the website and crawl the data

for page in range(1, 10):
    print('正在爬取第{}页壁纸'.format(page))
    url = 'http://m.bcoderss.com/tag/{}/page/{}/'.format(name, page)
    headers = {
        'Cookie': 'UM_distinctid=1747c5616688f-0da459aa281e74-3962420d-1fa400-1747c56166982d; CNZZDATA1278590218=744878758-1599811024-%7C1599811024',
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'
    }

		response = requests.get(url=url, headers=headers)
    selector = parsel.Selector(response.text)
    img_url = selector.css('#main a::attr(href)').getall()

Set crawl category

string = input('请输入你要爬取的类目：')
name = parse.unquote(string)

save Picture

        path = 'C:\\Users\\Administrator\\Desktop\\手机壁纸\\' + new_title + '.jpg'
        with open(path, mode='wb') as f:
            f.write(data_response.content)
            print(title)

Run the code, the effect is as shown below