1, ready crawlers
2, modify the project configuration
Find project configuration file scrapy.cnf, which will be commented url to untie
This code is tied to a database, it is necessary to modify the corresponding configuration database
In fact, the address inside the database to be modified into a remote database server address, the need to ensure scrapy server can connect to the database server
3, the deployment project
Before deploying the project to ensure scrapyd service is turned on.
Enter the directory of the local reptile engineering
Corresponding deployment command execution
4, scheduling reptiles
Scheduling requires curl crawler, the following command may correspond to the project directory dos window, commands can be executed in rows pycharm
4.1, view the remote server above reptile
curl http://node100:6800/listprojects.json
4.2, remove the remote server deployed on top of reptiles engineering
curl http://node100:6800/delproject.json -d project=doubanSpider
4.3, start reptile
curl http://node100:6800/schedule.json -d project=doubanSpider -d spider=douban
4.4 Cancellation reptiles
curl http://node100:6800/cancel.json -d project=doubanSpider -d job=000f47e035e911e9a8e90bb05244259f