every blog every motto: a bold attempt is half success
0. Introduction
Installation and solutions encountered in the pit of about pyspider.
Pyspider personal feeling is not too stable, a simple installation should toss for a long time, and binux like a long time not updated. Ado, enter the following text.
Description: The following seems to have missed a step, can not remember, the time to fix on it.
1. text
1. preparation
For version compatibility issues with anconda need to create a virtual environment, ease of management.
1.1 virtual environment
1.1.1 to create a virtual environment
Note: because of the keyword, for python3.7, to edit keywords, so the direct use python3.6 here. Reference may be about 3.7 [1]
conda create -n pys python=3.6
1.1.2 virtual environment is activated (extended) [2]
activate pys
1.1.3 Delete virtual environment (extension application)
conda remove -n pys --all
1.1.4 view existing virtual environment (extension application)
conda env list
1.2 delete windows defender
It does not remove the line, but it seems to turn it off. Because I already have security software (tinder), on the one hand, you do not need defender, on the other hand, defender not closed, sometimes closed, as if will work. It is simply deleted.
Warning: make sure there is security software, and then delete, do not blindly!
Specific reference [3]
2. started
2.1 Installation pyspider
Currently (2020.3.17) version 0.3.10
pip install pyspider
2.2 Configuration PHantomJS
Download
, after downloading the configuration to the next sibling directory python, as shown below (this is when I was in a virtual environment)
2.2 wsgidav version of the problem
a first method: modifying code [5]
Found installation package pys-> Lib-> site-packages-> pyspider-> webui-> webdav.py. FIG follows:
Modify line 209, as follows:
'domaincontroller': NeedAuthController(app),
change into:
'http_authenticator':{
'HTTPAuthenticator':NeedAuthController(app),
},
. A second approach: reduce version
Because it is not compatible with version 3.0, it is replaced with 2.4.1 [4]
pip uninstall wsgidav
pip install wsgidav==2.4.1
2.3 verification
At the command line, enter the following code:
Be sure to quit just re-enter the terminal! ! ! Otherwise, we will have to wait result_worker starting ...! ! !
Note: If you create a virtual environment, the virtual environment to be activated, enter
pyspider
Correct output is as follows:
Input your browser to http: // localhost: 5000 /
2.4 problem
If the card has been result_worker starting ..., as shown in FIG.
-------------------------------------------------- ----- the first terminal
can open a new terminal (the terminal temporarily closed), and then sequentially input pyspider, after the completion of closing the first input terminal.
-------------------------------------------------- ----- second terminal
after running:
references
[1] https://segmentfault.com/q/1010000015188569/a-1020000015213575
[2] https://blog.csdn.net/weixin_39190382/article/details/104113028
[3] http://www.winwin7.com/JC/16052.html
[4] https://blog.csdn.net/weixin_33842328/article/details/94538000
[5] https://blog.csdn.net/qq_37253540/article/details/88196994
