This series of blogs strives to include all the content of the introductory stage of crawlers, and requires you to have a good basic knowledge of Python. Of course, you can also watch this series of blogs if you have a complete zero foundation.
The advantage of this tutorial is that there are many crawler cases and many knowledge points involved.
- [Python crawler series tutorial 1- 100] The installation of crawler module and three kinds of databases
- [Python crawler series tutorial 2- 100] Understand the basic principles of HTTP and the basic process of crawling
- [Python crawler series tutorial 3- 100] Web page basics: HTML, CSS and JavaScript.
- [Python crawler series tutorial 4-100] Detailed use of the Requests library, how to grab web pages, grab binary data, add headers and solve POST requests
- [Python crawler series tutorial 5-100] Requests advanced usage: file upload, how to set up SSL certificate verification, processing timeout settings, identity authentication and proxy settings
- [Python crawler series tutorial 6-100] Use regular expressions to obtain useful information from the source code of the web page
- [Python crawler series tutorial 7-100] A small test, the first simple crawler case development, crawling all the course names on the homepage of Mukenet
- [Python crawler series tutorial 8-100] The main points of increasing the speed of crawlers: multithreading
- [Python crawler series tutorial 9-100] Crawler increases speed, takes advantage of multi-core, and uses multi-process
- [Python crawler series tutorial 10-100] Python web crawler actual combat: Use Re analysis to crawl the top 100 list of cat's eye movies
- [Python crawler series tutorial 11-100] Python web crawler combat: the simplest read_html line of code in Pandas crawls web tabular data, you can crawl the information of NBA players in Hupu Sports
- [Python crawler series tutorial 12-100] Python web crawler combat: 20 lines of Python code to crawl Douban popular movies
- [Python crawler series tutorial 13-100] Beautiful Soup, this is a delicious soup
- [Python crawler series tutorial 14-100] XPath, this is a Python crawler must kill
- [Python crawler series tutorial 15-100] JSON and JsonPath, this is a good helper for Python to get the interface (including the interface data instance of the hook)
- [Python crawler series tutorial 16-100] Python web crawler combat: how to make Ajax dynamic request, isn't it just a simple json? Thirty lines of code teach you to crawl the headlines of Python article information
- [Python crawler series tutorial 17-100] Python web crawler actual combat: Miss sister teaches you to use dozens of lines of code to download all the skins of King Glory
- [Python crawler series tutorial 18-100] Python web crawler actual combat: Miss sister teaches you to crawl and download all the hero skins in the League of Legends.
- [Python crawler series tutorial 19-100] Python web crawler actual combat: Miss Sister crawls the beautiful pictures of Baidu Beauty Tieba, and feels that it is time to get in touch with young people
- [Python crawler series tutorial 20-100] It’s time for Miss Sister to take everyone to make a Turing robot chat with automatic reply, no longer worry about being alone in your awkward chat!
- [Python crawler series tutorial 21-100] It’s time for Miss Sister to take everyone to crawl the emoticon pack, and never worry that you don’t have an emoticon pack anymore!
- [Python crawler series tutorial 22-100] Ms. Sister teaches everyone to crawl the audio data of the whole Himalayas and explore the sound of nature in the Himalayas
- [Python crawler series tutorial 23-100] Do my friend a favor and write a crawler for Xiaomi App Store, which mainly extracts the download link of each App.
- [Python crawler series tutorial 24-100] Miss sister teaches you Douban film review analysis, simulated landing to crawl 500 short reviews, using Snownlp and pyplot to perform sentiment analysis and display of short reviews
- [Python crawler series tutorial 25-100] Miss sister teaches you to crawl the number of Baidu keywords included, and tell you what keywords are with traffic!
- [Python crawler series tutorial 26-100] Miss sister teaches you how to obtain ajax data through Selenium, since then dynamic web pages are not terrible
- [Python crawler series tutorial 27-100] Using selenium to crawl the Python literature information in CNKI, Miss Sister suddenly feels that she has become a great god
- [Python crawler series tutorial 28-100] Miss sister takes you to get started with the crawler framework Scrapy, use the Scrapy framework to crawl the embarrassing encyclopedia
Python crawler 100 examples tutorial navigation post (to be completed)