PySpider HTTP 599: SSL certificate problem错误的解决方法

前言

在用 PySpider 爬取 https 开头的网站的时候遇到了 HTTP 599: SSL certificate problem: self signed certificate in certificate chain 的错误。

[E 160329 11:32:22 base_handler:194] HTTP 599: SSL certificate problem: self signed certificate in certificate chain

    Traceback (most recent call last):
      File "D:\Python27\lib\site-packages\pyspider\libs\base_handler.py", line 187, in run_task
        result = self._run_task(task, response)
      File "D:\Python27\lib\site-packages\pyspider\libs\base_handler.py", line 166, in _run_task
        response.raise_for_status()
      File "D:\Python27\lib\site-packages\pyspider\libs\response.py", line 183, in raise_for_status
        raise http_error
    HTTPError: HTTP 599: SSL certificate problem: self signed certificate in certificate chain

错误原因

这个错误会发生在请求 https 开头的网址，SSL 验证错误，证书有误。

报错如下

解决方案

最简单的解决方法是：

在 crawl 方法中加入忽略证书验证的参数，validate_cert=False，即

self.crawl(url, callback=method_name, validate_cert=False)

以上方法基本可以解决问题。

如果还是出错，重新启动一遍pyspider