In python reptile: Use requests + selenium can solve almost 90% of the reptiles needs, then scrapy is to solve the remaining 10% of it?
This is clearly not the case, scrapy framework is to enable our crawlers more powerful, more efficient presence, so we need a good look at scrapy framework.
scrapy application framework is a website for crawling data, extract structured data written
Framework: Users only need to customize the development of several modules can easily achieve a reptile, used to crawl Web content and a variety of picture
It can be used in many scenarios:
A series of program data mining, information processing, storing historical data, etc., scrapy use twisted the asynchronous network library to handle network traffic, clear structure, and contains a variety of middleware interfaces, the flexibility to complete a variety of needs