1, not properly install the corresponding version of pycurl
Cause Analysis:
Pycurl installation errors, install pycurl libraries (pycurl a Python interface
is a multi-protocol file transfer libcurl libraries. Python module similar to the urllib, pycurl can be used to obtain the object identified by the URL from the Python programs)
Solution:
Access https://www.lfd.uci.edu/~gohlke/pythonlibs/#pycurl, download the corresponding file wheel, for example, I was 10 Windows
64-Wei, Python 3.6.5, then download pycurl-7.43.0.3-cp36- cp36m-win_amd64.whl,
then use the command pip install pycurl-7.43.0.3-cp36- cp36m-win_amd64.whl can be installed
2, start pyspider prompt syntax error
Cause Analysis:
Follow the prompts to find the async keyword is wrong, py3.7 above async keyword
Solution:
When prompted to replace the corresponding .py file async keyword into another name
python\lib\site-packages\pyspider\run.py
python\lib\site-packages\pyspider\fetcher\tornado_fetcher.py
3, suggesting that a configuration error
Cause Analysis:
The problem may be too high a problem wsgidav version of leads, reduce what version wsgidav, more fit compatible Python
sudo -m PIP install Python 2.4.1 wsgidav ==
Solution:
Modify webui.py line 209 under Catalog code pyspider
'domaincontroller': NeedAuthController(app),
Read:
'http_authenticator': {'HTTPAuthenticator': NeedAuthController(app)},
4, right after the start pyspider, request that the site prompts 599 SSL error
Solution:
Problems mentioned above, may be added in the certificate verification ignored crawl method of capturing function, validate_cert = False
即：self.crawl('__START_URL__', callback=self.index_page, validate_cert=False)
5, and four-similar to SSL issues
Pyspider - HTTP 599: SSL certificate problem: unable to get local issuer certificate
Cause Analysis:
Has clearly pointed out the reason for the error is not added SSL certificate validation. But even with validate_cert = False will not be matched to validate_cert parameters, because PySpider official release version of the problem. Current latest release is 0.3.6, but the latest version is now 0.4.0 GitHub. So the need to manually download the update.
Solution:
A, GitHub download the latest pyspider, the link below
https://github.com/binux/pyspider
Second, download, unzip, execute the following command into the folder
Open cmd, cd to the unzipped file folder, execute the following command:
python setup.py install
Third, restart pyspider solve the problem!