Scrapy distributed crawler ES search engine website | Scrapy distributed crawler tutorial

ES search engine website of Scrapy distributed crawler

Share network disk address - https://pan.baidu.com/s/1smNcos1 Password: wnze

alternate address (Tencent Weiyun): http://url.cn/51n4soD Password: The era of SyRADx


big data is coming, and information is updated very quickly. If all walks of life do not keep pace with the times, they will face the survival of the fittest. Knowledge is constantly updated. Web crawler, or Web Spider, is a very vivid name. At present, the crawler development language is mainly python. This course combines several small crawler cases to help students learn crawler better.

In the latest global database management system rankings, Redis and Elasticsearch have risen steadily. Judging from the growth trend in the past four years, MongoDB, PostgreSQL, Elasticsearch, and Spkunk have the fastest growth rates. This project is to use Elasticsearch as the search theme to conduct a series of Scrapy data acquisition, MongoDB data storage, Django building website, etc.

2. Content Introduction
scrapy distributed crawler builds search engine
Data data can be done (all can use crawler to obtain data):
1) Data analysis service
2) Internet finance
3) Data Modeling
4) Information clustering
5) Natural language processing
6) Medical case analysis
Then build a search engine website through Elasticsearch and Django
Through the study of this course, you can learn to obtain the data you want, as well as have a deep understanding of network knowledge and programming knowledge