Tip: After the article is written, the table of contents can be automatically generated. How to generate it can refer to the help document on the right
Article Directory
foreword
Today I will introduce to you the data of an outsourcing platform for Python crawlers. Here I will help those who need it and give some tips.
1. Development tools
Python version: 3.6
Related modules:
import requests
import parsel
import csv
import re
2. Environment construction
Install Python and add it to the environment variable, and pip installs the required related modules.
Complete code and files in the article, leave a message in the comment area
3. Data source query analysis
Open the page we want to capture in the browser
Press F12 to enter the developer tool to view the data of the outsourcing platform we want
Here we need the page data.
4. Code implementation
1. Send request
response = requests.get(url=url, headers=headers)
2. Data acquisition
print(response.text)
3. Parse the data
selectors = parsel.Selector(response.text)
divs = selectors.css('.itemblock')
for div in divs:
title = div.css('div.title a::attr(title)').get()
modelName = div.css('div.modelName::text').get().strip()
num = div.css('div.browser div:nth-child(2) span::text').get().strip()
num_1 = div.css('div.browser div:nth-child(3) span::text').get().strip()
status = div.css('span.status::text').get().strip()
price = div.css('span.price::text').get().strip()
href = div.css('div.title a::attr(href)').get()
4. Save data
csv_writer.writerow(dit)
print(title, modelName, num, num_1, status, price, href)
Summarize
Today's sharing ends here
By the way, I would like to recommend some Python crawler video tutorials, hoping to help you:
A Collection of Python Crawler Practical Case Tutorials
If you have any questions about the article, or have other questions about python, you can discuss it together. If you
think the article I shared is good, you can follow me, or give the article a thumbs up (/≧▽≦)/