配置Pycharm的Scrapy爬虫Spider子类通用模板

版权声明：本文为博主原创文章，欢迎转载，请注明出处 https://blog.csdn.net/mouday/article/details/84564682

Scrapy爬虫的模板比较单一，每次新建爬虫程序要么重新手敲一遍，要么复制粘贴

• 从头手敲：效率较低，容易出错，浪费时间
• 复制粘贴：老代码需要改动的地方较多，容易漏掉，导致出错

所以，pycharm中配置一个模板文件就很重要了

# -*- encoding: utf-8 -*-

"""
@Date    : ${YEAR}-${MONTH}-${DAY}
@Author  : Peng Shiyu


"""

from scrapy import Spider, Request, cmdline


class XXXSpider(Spider):
    name = "${NAME}"

    start_urls = []
    
    def start_request(self):
        for url in self.start_urls:
            yield Request(url)
        
    def parse(self, response):
        pass

if __name__ == "__main__":
    cmdline.execute("scrapy crawl ${NAME}".split())