Teach you how to use python crawler to take you through the Japanese P station, super detailed teaching

One, page analysis

Let’s talk about some opening remarks today, and go directly to the topic.

First, open the target website: P station
(minors are prohibited from entering the house (manually funny))

 

The structure is similar to Tieba, it must be divided into two steps.
The first step: through the main detail page, get the link of each sub-detail page.

 

Right-click the page to check and find the article node corresponding to each subpage. Click on an article node to observe. It is easy to find that there is a jump link inside.

Goal one, get!

The second step is to get the download link of each picture.
Right-click to check, it is easy to find their download links, but at the same time, they also found "disturbing items"

When obtaining the link, the p-node must be found first. However, the p-node has an interference item with the same name. At this time, some methods need to be used for processing. Look at the code specifically.

Second, the complete code

I still want to recommend the Python learning group I built myself: 705933274. The group is all learning Python. If you want to learn or are learning Python, you are welcome to join. Everyone is a software development party and share dry goods from time to time (only Python software development related), including a copy of the latest Python advanced materials and zero-based teaching compiled by myself in 2021. Welcome friends who are in advanced and interested in Python to join!

3. Results display

Picture name meaning: num1_num2_num3 respectively represents the number of pages in the main page_the number of subpages in the page_the number of pictures in the subpage.

Four, Blogger's speech

I’ve lost my studies, don’t hurry up and try!

If there are any shortcomings, please leave a message in the comment area or privately message me, and I will add.

Thank you for your support, I hope you can like, follow, favorite, three links with one click.

I still want to recommend the Python learning Q group I built by myself : 705933274. The group is all learning Python. If you want to learn or are learning Python, you are welcome to join. Everyone is a software development party and share dry goods from time to time ( Only related to Python software development), including a copy of the latest Python advanced materials and zero-based teaching compiled by myself in 2021. Welcome to the advanced and friends interested in Python to join!

 

Guess you like

Origin blog.csdn.net/pyjishu/article/details/115206224