20 lines of code to crawl the list music of XX Cloud

Today I came to climb music, a little boring

Preparation

Software Environment

• Python3.8
• pycharm

module

requests, re, os three

Among them, requests is a third-party module that needs to be installed manually.
re and os are built-in modules and do not need to be installed

browser developer tools

We need to learn how to use developer tools.

Many friends don’t know this, because the details of the developer tools of each browser are somewhat different. I recommend using Google Chrome. If you can’t understand English, you can switch to Chinese.

Open the developer tools, click the ellipsis, click shortcuts

Click preferences - language in turn, and
pull directly to the bottom to select Chinese.

Code

This implementation step is roughly divided into the following five steps:

1. send request
2. retrieve data
3. Analytical data
4. save data

I won’t talk about how to analyze developer tools here. I will make a detailed tutorial to explain developer tools when I have time (touch fish).

import module

import requests 
import re  
import os  

send request

url = 'https://网址自己打一下/discover/toplist?id=3778678'
headers = {
    
    
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'
}
response = requests.get(url=url, headers=headers)

retrieve data

result = re.findall('<li><a href="/song\?id=(\d+)">(.*?)</a></li>', response.text)
for music, title in result:
    music_url = f'http://music.163.com/song/media/outer/url?id={
    
    music}.mp3'
    music_content = requests.get(url=music_url, headers=headers).content

save data

with open(filename + title + '.mp3', mode='wb') as f:
    f.write(music_content)
    print(title)

Of course, this is just the simplest way to crawl list songs,
comments, lyrics, etc. can be crawled,
as well as making word cloud maps, downloading through search, music downloaders,
and other ways to achieve downloading.

Well, that’s all for today’s sharing, get the business card below the complete code~