How to make the program download songs in batches like a human? Python crawls paid songs

Preface

The text and pictures in this article are from the Internet and are for learning and communication purposes only. They do not have any commercial use. The copyright belongs to the original author. If you have any questions, please contact us for processing.

Today I will teach you an automated crawler tool selenium

selenium

Selenium is an automated testing tool for the Web. It was originally developed for automated testing of websites. It is like a button wizard for playing games, which can be automatically operated according to specified commands.

The Selenium testing tool directly controls the browser, just like a real user is operating. Selenium can let the browser automatically load the page according to the instructions, obtain the required data, even take a screenshot of the page, or determine whether certain actions on the website have occurred.

Project Objectives

Today’s goal is to crawl paid songs

Victim address

http://tool.liumingye.cn/music/?page=homePage

Let's show you the effect first

 

Crawler code

Import tool

import time
import requests
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

Parse webpage

Open the F12 developer tools, no matter the three or seven twenty one, let’s have a blind analysis first~

Oh, there is actually a data interface in this, or post request, then we will look at its data parameter, there are changes

driver = webdriver.Chrome(executable_path='chromedriver.exe', options=chrome_options)
# key_world = input('请输入歌手名字：')
driver.get('http://tool.liumingye.cn/music/?page=searchPage')
driver.find_element_by_css_selector('#input').send_keys('张杰')
driver.find_element_by_css_selector('#search  button:nth-child(2) i').click()
def download(name, url):
    filename = 'C:\\Users\\Administrator\\Desktop\\音乐\\' + name + '.mp3'
    response = requests.get(url=url)
    with open(filename, mode='wb') as f:
        f.write(response.content)

def drop_down():
    """模拟人去滚动鼠标向下浏览页面"""
    for x in range(1, 20, 10):
        time.sleep(0.5)
        j = x / 10
        js = 'document.documentElement.scrollTop = document.documentElement.scrollHeight * %f' % j
        driver.execute_script(js)


lis = driver.find_elements_by_css_selector('#player li')
f = 0
for li in lis:
    f += 1
    name = li.find_element_by_css_selector('.aplayer-list-title').text
    li.find_element_by_css_selector('.aplayer-list-download').click()
    down_url = driver.find_element_by_css_selector('#m-download > div > div > div.modal-body > div:nth-child(6) > div.input-group-append > a.btn.btn-outline-secondary.download').get_attribute('href')
    driver.find_element_by_css_selector('#m-download > div > div > div.modal-header > button').click()
    # time.sleep(1)
    download(name, down_url)
    print(name, down_url)
    if f % 10 == 0:
        drop_down()

After running the code, the effect is as follows

Did you learn? Click here to get the complete code! ! ! ! !