scrapy爬虫脚本气动
单一爬虫
if __name__ == '__main__':
cmdline.execute("scrapy crawl baidu".split())
多个爬虫
if __name__ == '__main__':
# 通过方法 get_project_settings() 获取配置信息
process = CrawlerProcess(get_project_settings())
process.crawl(BaiduSpider)
process.start()
可以参考 https://blog.csdn.net/mouday/article/details/89524009