Pyinstaller packaged scrapy

surroundings

Windows7

Python3.65

scrapy1.74

PyInstaller3.5

Creating packaging script

Creating start.py same path with scrapy.cfg

# -*- coding: utf-8 -*-
from scrapy.crawler import CrawlerProcess
from scrapy.utils.project import get_project_settings
 

# import robotparser
# 必加的依赖
import scrapy.spiderloader
import scrapy.statscollectors
import scrapy.logformatter
import scrapy.dupefilters
import scrapy.squeues
import scrapy.extensions.spiderstate
import scrapy.extensions.corestats
import scrapy.extensions.telnet
import scrapy.extensions.logstats
import scrapy.extensions.memusage
import scrapy.extensions.memdebug
import scrapy.extensions.feedexport
import scrapy.extensions.closespider
import scrapy.extensions.debug
import scrapy.extensions.httpcache
import scrapy.extensions.statsmailer
import scrapy.extensions.throttle
import scrapy.core.scheduler
import scrapy.core.engine
import scrapy.core.scraper
import scrapy.core.spidermw
import scrapy.core.downloader
import scrapy.downloadermiddlewares.stats
import scrapy.downloadermiddlewares.httpcache
import scrapy.downloadermiddlewares.cookies
import scrapy.downloadermiddlewares.useragent
import scrapy.downloadermiddlewares.httpproxy
import scrapy.downloadermiddlewares.ajaxcrawl
import scrapy.downloadermiddlewares.chunked
import scrapy.downloadermiddlewares.decompression
import scrapy.downloadermiddlewares.defaultheaders
import scrapy.downloadermiddlewares.downloadtimeout
import scrapy.downloadermiddlewares.httpauth
import scrapy.downloadermiddlewares.httpcompression
import scrapy.downloadermiddlewares.redirect
import scrapy.downloadermiddlewares.retry
import scrapy.downloadermiddlewares.robotstxt
import scrapy.spidermiddlewares.depth
import scrapy.spidermiddlewares.httperror
import scrapy.spidermiddlewares.offsite
import scrapy.spidermiddlewares.referer
import scrapy.spidermiddlewares.urllength
import scrapy.pipelines
import scrapy.core.downloader.handlers.http
import scrapy.core.downloader.contextfactory

 
process = CrawlerProcess(get_project_settings())
 
# 参数1：爬虫名 参数2：域名
process.crawl('biqubao_spider',domain='biqubao.com')
process.start()  # the script will block here until the crawling is finished

Epigenetic complete packaged into three files: dist, build (can be deleted), start.spec (can be deleted) 

Question 1: 
About importing robotparser library this problem, this article is to import a reference library, but after a try and found no install the library. 
Scrapy robot will automatically parse the agreement, so try not to import this library can be packaged found success. 

Question 2: 
You may have to use other items of Scrapy libraries, so in this script package you need to import all the libraries you use. 

Question 3: 
not packaged after successfully running the program, since at least two issues, as FIG.

You need to add in the scrapy dist folder, which contains these two files, these two files are installed in the scrapy library. 

Reference article: https: //blog.csdn.net/la_vie_est_belle/article/details/79017358

Pyinstaller packaged scrapy

Guess you like