A .window installation scrapy
1. install twisted
download link: https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted
find your system with the corresponding version
I was Twisted-18.9.0-cp36-cp36m -win_amd64.whl downloaded
to install pip3 install + your twisted path of
2. install scrapy
PIP3 install scrapy ( direct installation can cause an error error:. Microsoft Visual C ++ 14.0 is required to see the solution 1 )
error solution:
If there is: ModuleNotFoundError: No module named 'win32api'
installation win32api, download links: http://sourceforge.net/projects/pywin32/files%2Fpywin32/
(download for your version of Python) I installed this pywin32-221.win-amd64-py3 .6.exe
installation error occurs pywin32 --Python version 3.6 required, which was not
found in the registry to solve: create register.py, he executed SYS Import
the try:
from the winreg Import *
except ImportError:
from _winreg import *
# tweak as necessary
version = sys.version[:3]
installpath = sys.prefix
regpath = "SOFTWARE\\Python\\Pythoncore\\{0}\\".format(version)
installkey = "InstallPath"
pythonkey = "PythonPath"
pythonpath = "{0};{1}\\Lib\\;{2}\\DLLs\\".format(
installpath, installpath, installpath)
def RegisterPy():
try:
reg = OpenKey(HKEY_CURRENT_USER, regpath)
except EnvironmentError as e:
try:
reg = CreateKey(HKEY_CURRENT_USER, regpath)
SetValue(reg, installkey, REG_SZ, installpath)
SetValue(reg, pythonkey, REG_SZ, pythonpath)
CloseKey(reg)
except:
print("*** Unable to register!")
return
print("--- Python", version, "is now registered!")
return
if (QueryValue(reg, installkey) == installpath and
QueryValue(reg, pythonkey) == pythonpath):
CloseKey(reg)
print("=== Python", version, "is already registered!")
return
CloseKey(reg)
print("*** Unable to register!")
print("*** You probably have another Python installation!")
if __name__ == "__main__":
RegisterPy()
二.ubuntu安装scrapy
1.Create a virtual environment
mkvirtualenv spider_py3 -p python3
Note: You may have problems, not create virtual environments, blocked living
solve: Upgrade pip, appeared Successfully installed pip-19.0.2 installed successfully re-create the
sudo install --upgrade PIP PIP3
2. scapy installed in a virtual environment
(recommendation 1.6 .0, may be due to version causes crawling data encoded in unicode, no middleware and create a project file )
PIP install scrapy == 1.6.0
More technical information may concern: gzitcast