MAC使用Scrapy遇到的坑

MAC版本：EI Captain

Python版本: 2.7.10

MAC默认没有安装pip，所以首先

sudo easy_install pip

然后安装Scrapy:

sudo pip install Scrapy

安装过程中报错: "libxml/xmlversion.h' file not found"，解决方案如下:

brew install libxml2
brew install libxslt
brew link libxml2 --force
brew link libxslt --force

再次安装，成功。

 

然而，后续执行scrapy startproject tutorial命令，又报错了: "cannot import name xmlrpc_client"，我简直要疯了。解决方案如下:

sudo rm -rf /Library/Python/2.7/site-packages/six*
sudo rm -rf /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six*
sudo pip install six

然而，执行scrapy startproject tutorial命令，再次报错: 'module' object has no attribute 'OP_NO_TLSv1_1'，解决方案如下:

sudo pip install Twisted==16.4.1

参考资料：

http://stackoverflow.com/questions/19548011/cannot-install-lxml-on-mac-os-x-10-9

http://stackoverflow.com/questions/30964836/scrapy-throws-importerror-cannot-import-name-xmlrpc-client

Issue with running scrapy spider from script

转载于:https://www.cnblogs.com/gattaca/p/6743766.html