Python3安装scrapy的玄学

一不小心把Linux服务器的重要文件给删掉了,不能连接ssh,只能重新做镜像安装了,什么开发环境都没有了。

刚开始一起正常,在使用pip install安装scrapy的时候遇到了经常出现的问题:

Collecting Twisted>=13.3.0 (from scrapy)
  Could not find a version that satisfies the requirement Twisted>=13.3.0 (from scrapy) (from versions: )
No matching distribution found for Twisted>=10.0.0 (from scrapy)

这很正常,之前我安装scrapy也是这样。同样 pip install twisted同样也是失败,不能使用pip安装这个包。

ok,那就在官网下载相应的包安装,我打开的是https://pypi.org/,搜一下twisted,然后直接安装最新版的twisted

wget https://files.pythonhosted.org/packages/5d/0e/a72d85a55761c2c3ff1cb968143a2fd5f360220779ed90e0fadf4106d4f2/Twisted-18.9.0.tar.bz2

这里有个小坑:下载下来包后由于Linux没有带bzip2,所以不能解压这个包,要用

yun install bzip2

下载一下bzip2

下载了bzip2,解压:

tar -xvf Twisted-18.9.0.tar.bz2

  , 然后进入

cd Twisted-18.9.0

,再使用python命令安装

python3 setup.py install

???

安装失败,报错了。。。

ValueError: bad marshal dat File "/usr/local/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 414, in iter_symbols     for name in code.co_names: AttributeError: 'complex' object has no attribute 'co_names'

对象里没有属性,啥玩意???

找了好一会百度谷歌也找不到解决方案(玄学来了...)

正在我准备放弃去睡觉的时候,在试试之前那个pip 安装命令...奇迹出现了

pip3 install twisted
Requirement already satisfied: twisted in /usr/local/lib/python3.7/site-packages/Twisted-18.9.0-py3.7-linux-x86_64.egg (18.9.0)
Collecting zope.interface>=4.4.2 (from twisted)
  Downloading https://files.pythonhosted.org/packages/3a/b5/5cbafbe09bbfc242503b0cd9bcf9dee32458b3ca166f6a63cb7b4f465b60/zope.interface-4.6.0-cp37-cp37m-manylinux1_x86_64.whl (167kB)
    100% |████████████████████████████████| 174kB 1.0MB/s 
Collecting constantly>=15.1 (from twisted)
  Downloading https://files.pythonhosted.org/packages/b9/65/48c1909d0c0aeae6c10213340ce682db01b48ea900a7d9fce7a7910ff318/constantly-15.1.0-py2.py3-none-any.whl
Collecting incremental>=16.10.1 (from twisted)
  Downloading https://files.pythonhosted.org/packages/f5/1d/c98a587dc06e107115cf4a58b49de20b19222c83d75335a192052af4c4b7/incremental-17.5.0-py2.py3-none-any.whl
Collecting Automat>=0.3.0 (from twisted)
  Downloading https://files.pythonhosted.org/packages/a3/86/14c16bb98a5a3542ed8fed5d74fb064a902de3bdd98d6584b34553353c45/Automat-0.7.0-py2.py3-none-any.whl
Collecting hyperlink>=17.1.1 (from twisted)
  Downloading https://files.pythonhosted.org/packages/a7/b6/84d0c863ff81e8e7de87cff3bd8fd8f1054c227ce09af1b679a8b17a9274/hyperlink-18.0.0-py2.py3-none-any.whl
Collecting PyHamcrest>=1.9.0 (from twisted)
  Downloading https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl (52kB)
    100% |████████████████████████████████| 61kB 659kB/s 
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.7/site-packages/attrs-18.2.0-py3.7.egg (from twisted) (18.2.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from zope.interface>=4.4.2->twisted) (33.1.1)
Requirement already satisfied: six in /usr/local/lib/python3.7/site-packages (from Automat>=0.3.0->twisted) (1.12.0)
Requirement already satisfied: idna>=2.5 in /usr/local/lib/python3.7/site-packages (from hyperlink>=17.1.1->twisted) (2.8)
Installing collected packages: zope.interface, constantly, incremental, Automat, hyperlink, PyHamcrest
Successfully installed Automat-0.7.0 PyHamcrest-1.9.0 constantly-15.1.0 hyperlink-18.0.0 incremental-17.5.0 zope.interface-4.6.0

Twisted竟然用pip安装成功了!!!woc,我再试试了pip安装scrapy,结果同样也是安装成功。

pip3 install scrapy
Looking in indexes: http://mirrors.aliyun.com/pypi/simple/
Collecting scrapy
  Downloading http://mirrors.aliyun.com/pypi/packages/3e/45/414e87ac8209d537c91575538c5307c20217a6943f555e0ee39f6db4bb0f/Scrapy-1.6.0-py2.py3-none-any.whl (231kB)
    100% |████████████████████████████████| 235kB 4.0MB/s 
Collecting PyDispatcher>=2.0.5 (from scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/cd/37/39aca520918ce1935bea9c356bcbb7ed7e52ad4e31bff9b943dfc8e7115b/PyDispatcher-2.0.5.tar.gz
Collecting cssselect>=0.9 (from scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/7b/44/25b7283e50585f0b4156960691d951b05d061abf4a714078393e51929b30/cssselect-1.0.3-py2.py3-none-any.whl
Collecting parsel>=1.5 (from scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/96/69/d1d5dba5e4fecd41ffd71345863ed36a45975812c06ba77798fc15db6a64/parsel-1.5.1-py2.py3-none-any.whl
Requirement already satisfied: six>=1.5.2 in /usr/local/lib/python3.7/site-packages (from scrapy) (1.12.0)
Collecting service-identity (from scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/e9/7c/2195b890023e098f9618d43ebc337d83c8b38d414326685339eb024db2f6/service_identity-18.1.0-py2.py3-none-any.whl
Collecting w3lib>=1.17.0 (from scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/81/43/9dcf92a77f5f0afe4f4df2407d7289eea01368a08b64bda00dd318ca62a6/w3lib-1.20.0-py2.py3-none-any.whl
Collecting pyOpenSSL (from scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/01/c8/ceb170d81bd3941cbeb9940fc6cc2ef2ca4288d0ca8929ea4db5905d904d/pyOpenSSL-19.0.0-py2.py3-none-any.whl (53kB)
    100% |████████████████████████████████| 61kB 44.1MB/s 
Collecting queuelib (from scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/4c/85/ae64e9145f39dd6d14f8af3fa809a270ef3729f3b90b3c0cf5aa242ab0d4/queuelib-1.5.0-py2.py3-none-any.whl
Requirement already satisfied: Twisted>=13.1.0 in /usr/local/lib/python3.7/site-packages/Twisted-18.9.0-py3.7-linux-x86_64.egg (from scrapy) (18.9.0)
Collecting lxml (from scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/08/f2/04bf04e42c070f65b64dbde02d2c94851251f19f5e9f803cc8f8bc61ac77/lxml-4.3.1-cp37-cp37m-manylinux1_x86_64.whl (5.7MB)
    100% |████████████████████████████████| 5.7MB 58.1MB/s 
Collecting cryptography (from service-identity->scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/98/71/e632e222f34632e0527dd41799f7847305e701f38f512d81bdf96009bca4/cryptography-2.5-cp34-abi3-manylinux1_x86_64.whl (2.4MB)
    100% |████████████████████████████████| 2.4MB 61.7MB/s 
Requirement already satisfied: attrs>=16.0.0 in /usr/local/lib/python3.7/site-packages/attrs-18.2.0-py3.7.egg (from service-identity->scrapy) (18.2.0)
Collecting pyasn1-modules (from service-identity->scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/da/98/8ddd9fa4d84065926832bcf2255a2b69f1d03330aa4d1c49cc7317ac888e/pyasn1_modules-0.2.4-py2.py3-none-any.whl (66kB)
    100% |████████████████████████████████| 71kB 44.1MB/s 
Collecting pyasn1 (from service-identity->scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/7b/7c/c9386b82a25115cccf1903441bba3cbadcfae7b678a20167347fa8ded34c/pyasn1-0.4.5-py2.py3-none-any.whl (73kB)
    100% |████████████████████████████████| 81kB 44.1MB/s 
Requirement already satisfied: zope.interface>=4.4.2 in /usr/local/lib/python3.7/site-packages (from Twisted>=13.1.0->scrapy) (4.6.0)
Requirement already satisfied: constantly>=15.1 in /usr/local/lib/python3.7/site-packages (from Twisted>=13.1.0->scrapy) (15.1.0)
Requirement already satisfied: incremental>=16.10.1 in /usr/local/lib/python3.7/site-packages (from Twisted>=13.1.0->scrapy) (17.5.0)
Requirement already satisfied: Automat>=0.3.0 in /usr/local/lib/python3.7/site-packages (from Twisted>=13.1.0->scrapy) (0.7.0)
Requirement already satisfied: hyperlink>=17.1.1 in /usr/local/lib/python3.7/site-packages (from Twisted>=13.1.0->scrapy) (18.0.0)
Requirement already satisfied: PyHamcrest>=1.9.0 in /usr/local/lib/python3.7/site-packages (from Twisted>=13.1.0->scrapy) (1.9.0)
Collecting cffi!=1.11.3,>=1.8 (from cryptography->service-identity->scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/34/c2/c207685b3d0c9e87c01386c242b9ec9f5ed9305eefa39d005f40245a114e/cffi-1.12.0-cp37-cp37m-manylinux1_x86_64.whl (430kB)
    100% |████████████████████████████████| 440kB 56.0MB/s 
Collecting asn1crypto>=0.21.0 (from cryptography->service-identity->scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl (101kB)
    100% |████████████████████████████████| 102kB 59.2MB/s 
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from zope.interface>=4.4.2->Twisted>=13.1.0->scrapy) (33.1.1)
Requirement already satisfied: idna>=2.5 in /usr/local/lib/python3.7/site-packages (from hyperlink>=17.1.1->Twisted>=13.1.0->scrapy) (2.8)
Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography->service-identity->scrapy)
  Downloading http://mirrors.aliyun.com/pypi/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
    100% |████████████████████████████████| 163kB 54.2MB/s 
Installing collected packages: PyDispatcher, cssselect, w3lib, lxml, parsel, pycparser, cffi, asn1crypto, cryptography, pyasn1, pyasn1-modules, service-identity, pyOpenSSL, queuelib, scrapy
  Running setup.py install for PyDispatcher ... done
  Running setup.py install for pycparser ... done
Successfully installed PyDispatcher-2.0.5 asn1crypto-0.24.0 cffi-1.12.0 cryptography-2.5 cssselect-1.0.3 lxml-4.3.1 parsel-1.5.1 pyOpenSSL-19.0.0 pyasn1-0.4.5 pyasn1-modules-0.2.4 pycparser-2.19 queuelib-1.5.0 scrapy-1.6.0 service-identity-18.1.0 w3lib-1.20.0

检验:

IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import scrapy                                                                                                                           

In [2]:                                                                                                                                      

scrapy安装成功!!!

总结:

粗略看了看,在使用python setup.py install 命令时把twisted安装上了,至于报错可以暂时不用去理会,所以pip能够在后面顺利安装scrapy。

上面的总结都是瞎编的,下面才是真的[会心一笑]

不要乱动Linux服务器的文件,特别是自己不熟悉文件的作用而且还处在root的权限使用时,否则就只能在情人节凌晨一个人静悄悄地重新安装环境了...一句话:单身狗情人节温馨提醒您:文件千万个,熟悉第一个。删除不规范,凌晨两行泪

猜你喜欢

转载自blog.csdn.net/kingyuan666/article/details/87228585