A, pip install pyspider installation error occurs:
Command "python setup.py egg_info" failed with error code 10 in C:\Users\xxx\AppData\Local\Temp\pip-install-tc5uvu7l\pycurl\
After the big online to get experience you need to install wheel, execute the command pip3 install wheel installation, suggesting Successfully installed wheel-0.31.1 on behalf of the installation is successful, then visit https://www.lfd.uci.edu/~gohlke/pythonlibs/ download installation file, as follows:
PycURL, libcurl Library The A interface to.
The pycurl-7.43.1-cp27-cp27m-win32.whl
the pycurl-7.43.1-cp27-cp27m-win_amd64.whl
the pycurl-7.43.1-CP34-cp34m-win32.whl
pycurl- CP34-cp34m--7.43.1 win_amd64.whl
the pycurl-7.43.1-cp35-cp35m-win32.whl
the pycurl-7.43.1-cp35-cp35m-win_amd64.whl
the pycurl-7.43.1-CP36-cp36m-win32.whl
-7.43.1-CP36-pycurl cp36m-win_amd64.whl
pycurl-7.43.1-CP37-cp37m-win32.whl
pycurl-7.43.1-CP37-cp37m-win_amd64.whl
specific version to download depending on the environment you are using to decide, For example, I was python3.7, then python is a 32-bit version, you need to select pycurl-7.43.1-cp37-cp37m- win32.whl download Note: 32 or 64-bit version is not the computer, is to install the python version! ! !
After downloading the installation, the command pip3 install D: \ python \ demo \ pycurl-7.43.1-cp37-cp37m-win_amd64.whl execution, if the error:
pycurl-7.43.1-cp37-cp37m- win_amd64.whl is not a supported wheel on this platform.
represent you installed the wrong version is described above that, my computer is 64-bit, python is 32, so error, then download pycurl-7.43.1-cp37-cp37m- win32.whl then execute the command pip3 install D: \ python \ demo \ pycurl-7.43.1-cp37-cp37m-win32.whl be installed, suggesting Successfully installed pycurl-7.43.1 and installed successfully!
Spent a lot of things, and finally installed pyspider OK, execute the command pip3 install pyspider prompt
Successfully installed Flask-1.0.2 Flask-Login -0.4.1 Jinja2-2.10 MarkupSafe-1.0 PyYAML-3.13 Werkzeug-0.14.1 click-6.7 cssselect-1.0.3 defusedxml-0.5.0 itsdangerous- 0.24 jsmin-2.2.2 lxml-4.2.5 pyquery-1.4.0 pyspider-0.3.10 tblib-1.3.2 tornado-4.5.3 u-msgpack-python-2.5 .0 wsgidav-2.4.1
Second, it will complain PySpider when running on Python3.7
File "/usr/local/lib/python3.7/site-packages/pyspider/run.py", Line 231
the async = True, that get_object = False, no_input = False):
^
SyntaxError: invalid syntax
because the python3.7 async has become a keyword. Therefore this error.
Modify the way it is manually replace
Async to the lower position mark_async
/usr/local/lib/python3.7/site-packages/pyspider/run.py 231 lines, 245 lines (two), 365 line
/usr/local/lib/python3.7 /site-packages/pyspider/webui/app.py 95 rows
81 of /usr/local/lib/python3.7/site-packages/pyspider/fetcher/tornado_fetcher.py row, row 89 (two), the line 95 117 line
three,
ValueError: Invalid configuration:
- Deprecated option 'domaincontroller': use 'http_authenticator.domain_controller' instead.
3.1 final positioning in the wsgidav3.x problem, pyspider installed by default wsgidav3.x, a solution to the first 3.x unloading, reloading 2.x (pip install wsgidav 2.x installed by default I installed the 2.4.1
Uninstall wsgidav PIP
PIP install wsgidav
if installed wsgidav this or 3.x, you can after you uninstall this version, the installation command followed by the specific version number
python -m pip install wsgidav == 2.4.1, this method does not resolve the problem
3.2 The reason is because WsgiDAV released version pre-release 3.x.
Solutions are as follows:
Find pyspider resource package in the installation package, and then locate the file inside the webdav.py webui file is opened, you can modify the first 209 rows.
把 'domaincontroller': NeedAuthController(app),
change into:
'http_authenticator': {
'HTTPAuthenticator': NeedAuthController (App),
},
the first execution pyspider all may get stuck in result_worker starting ... here, need to re-open after Ctrl + C interrupt CMD input pyspider all success.
Fourth, find pyspider the Web preview screen is very small only in the actual debugging
The reason: web preview box is too small because the css property page element is replaced with 60px height
so we need to change the content of the CSS file, but this should not all browsers are common, just testing the Chrome browser is possible
Modify debug.min.css document reads as follows: https://www.jianshu.com/p/7bff6fd4dc1b
