Win is indeed the worst development platform, and there are often some moths when installing third-party modules.
Today, when I installed scrapy using the pip that comes with python3 in the window, an error occurred. The error code is as follows:
Command "f:\python\python36\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\WONDER~1\\AppData\\Local\\Temp\\pip-build-90gvzc6m\\Twisted\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\WONDER~1\AppData\Local\Temp\pip-2xl1_rnm-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\WONDER~1\AppData\Local\Temp\pip-build-90gvzc6m\Twisted\- 1
The key words are: failed with error code 1
In fact, this error code also appeared when installing pandas before, and it was nothing at that time.
There are many reasons for this error code to be speculated today:
- pip version is too low
- The wheel module is not installed. Although the wheel module should be installed automatically when scrapy is installed.
- The module to be installed does not support the current python version. This problem is especially noticeable on python3. (scrapy just can't support 3 perfectly)
- The name of the computer contains Chinese or other bizarre non-alphanumeric characters.
- The computer's virtual memory is insufficient. This is a bit ridiculous, but when I was on Baidu, many people mentioned this point, and I don't know if it was the conclusion of the same person who reprinted it.
- Mercury retrograde.
(It seems that the module to be installed is missing a dependency package, because I have encountered similar code when configuring the environment on a new CentOS - on 2017.09.22)
The easiest way to fix it is to stop using Windows and go with Linux. Of course, python3 under Windows is not incapable of using these third-party modules, but you need to install these modules manually instead of using pip. The disadvantage is that in many cases, the dependencies between packages are not clear, and packages are often missed.
According to different modules, go to their respective documentation, there will generally be instructions on how to install them. The official documentation like scrapy recommends the installation method: use Mingwpy.
Alternatively, you can directly use an integrated third-party development environment, such as using Anaconda directly when configuring the machine learning environment.
For the installation of scrapy with python3.6 under winx, I looked at the error code carefully and determined that there was a problem when installing twisted. After searching online, I found a solution:
First download the whl file of the corresponding version of twisted from http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted (such as my Twisted‑17.5.0‑cp36‑cp36m‑win_amd64.whl), followed by cp Python version, amd64 stands for 64-bit, run the command:
pip install D:\TDownloads\Twisted-17.5.0-cp36-cp36m-win_amd64.whl- 1
where install is followed by the full path name
Once installed, run again:
pip install Scrapy- 1
to be successful.