scrapy framework crawling IMDb top250

Frame data using scrapy crawling. To top250 Department IMDb data, for example, learning to deepen scrapy framework for understanding.

URL scrapy teaching learning recommendation framework Mu class network: https://www.imooc.com/learn/1017

Destination Address: https://movie.douban.com/top250 

Creating IMDb reptiles, after the successful implementation can be found reptiles projects created in the directory

scrapy startproject doubanMovie

Then enter the reptile project file, use the command to create a file generation crawler, crawling for implementing data page functions

# Into the project file 
cd doubanMovie
 # create crawling data file 
scrapy genspider doubanMovieSpider movie.douban.com

You can see reptiles files created after the successful implementation

 

 So far the project has been created, the next step is to realize crawling data capabilities.

 

To crawling data, first of all to the project up and running, and the command is:

scrapy crawl doubanMovieSpider

 But times and were executed in cmd window, it is troublesome, to create a file in the project, named main.py can. Edits

 After main.py only need to run the file to the project. The entire project is completed this reptile.

 Project code link address: https: //files.cnblogs.com/files/no-end-to-learning/SpiderProject.zip