PyCurl安装过程中的问题记录分析

Pyspider & Pycurl

pyspider是一个大名鼎鼎的爬虫框架，在安装过程中，碰到了pycurl的相关问题，现在记录如下，方便后续参考。
说明： pyspider在windows 7下的安装可以正常，但是运行过程中，会报出不同的错误信息，不建议在windows上运行。

环境说明

Centos 7, Python 3.6.5

分析过程

1. pip install pyspider
   碰到的错误信息如下：

   Looking in indexes: http://mirrors.aliyun.com/pypi/simple/
   Collecting pycurl
   Downloading http://mirrors.aliyun.com/pypi/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz (214kB)
   100% |████████████████████████████████| 215kB 10.5MB/s 
   Complete output from command python setup.py egg_info:
   Traceback (most recent call last):
     File "/tmp/pip-install-ramaxa44/pycurl/setup.py", line 223, in configure_unix
       stdout=subprocess.PIPE, stderr=subprocess.PIPE)
     File "/root/.pyenv/versions/3.6.5/lib/python3.6/subprocess.py", line 709, in __init__
       restore_signals, start_new_session)
     File "/root/.pyenv/versions/3.6.5/lib/python3.6/subprocess.py", line 1344, in _execute_child
       raise child_exception_type(errno_num, err_msg, err_filename)
   FileNotFoundError: [Errno 2] No such file or directory: 'curl-config': 'curl-config'
   
   During handling of the above exception, another exception occurred:
   
   Traceback (most recent call last):
     File "<string>", line 1, in <module>
     File "/tmp/pip-install-ramaxa44/pycurl/setup.py", line 913, in <module>
       ext = get_extension(sys.argv, split_extension_source=split_extension_source)
     File "/tmp/pip-install-ramaxa44/pycurl/setup.py", line 582, in get_extension
       ext_config = ExtensionConfiguration(argv)
     File "/tmp/pip-install-ramaxa44/pycurl/setup.py", line 99, in __init__
       self.configure()
     File "/tmp/pip-install-ramaxa44/pycurl/setup.py", line 227, in configure_unix
       raise ConfigurationError(msg)
   __main__.ConfigurationError: Could not run curl-config: [Errno 2] No such file or directory: 'curl-config': 'curl-config'
   
   ----------------------------------------
   Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-ramaxa44/pycurl/

   经过分析，发现其为pycurl的错误信息。
   于是直接安装pycurl：

   pip install pycurl

得到同样的错误信息，确定是由于安装pycurl出现的问题。
2. 安装libcurl-devel
在网络上搜索之后，发现了需要安装如下类库：

yum install libcurl-devel

安装完成之后，重新尝试安装pycurl。原有的问题，已经解决，但是新的问题出现了。
3. 新的问题 openssl
新的错误信息如下：

Looking in indexes: http://mirrors.aliyun.com/pypi/simple/
Collecting pycurl
  Downloading http://mirrors.aliyun.com/pypi/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz (214kB)
    100% |████████████████████████████████| 215kB 6.4MB/s 
    Complete output from command python setup.py egg_info:
    Using curl-config (libcurl 7.29.0)
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-apf84sq1/pycurl/setup.py", line 913, in <module>
        ext = get_extension(sys.argv, split_extension_source=split_extension_source)
      File "/tmp/pip-install-apf84sq1/pycurl/setup.py", line 582, in get_extension
        ext_config = ExtensionConfiguration(argv)
      File "/tmp/pip-install-apf84sq1/pycurl/setup.py", line 99, in __init__
        self.configure()
      File "/tmp/pip-install-apf84sq1/pycurl/setup.py", line 316, in configure_unix
        specify the SSL backend manually.''')
    __main__.ConfigurationError: Curl is configured to use SSL, but we have not been able to determine which SSL backend it is using. Please see PycURL documentation for how to specify the SSL backend manually.

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-apf84sq1/pycurl/

从字面来分析，是由于openssl的配置问题，但是该如何来解决呢？
4. 解决ssl的问题
尝试进行了各种尝试，思路是按照libcurl、pycurl和openssl的配置问题。于是找到了如下的方式：

yum install libcurl-openssl-devel # 无效的方案

最终的方案如下：

pip uninstall pycurl
将export PYCURL_SSL_LIBRARY=openssl写入~/.bashrc
pip install pycurl

执行上述方案之后，发现问题依然，最终发现，是由于没有使用source命令执行bashrc。
于是执行如下命令：

source ~/.bashrc

问题完美解决。
5.ubuntu的方案

sudo apt-get install libssl-dev libcurl4-openssl-dev python-dev

最后的最后

pip install pyspider

这个最终的目标也被正确安装了…….

总结

问题总是在寻找中，不停地被解决和碰到的，：-）