【原文链接】https://doc.scrapy.org/en/latest/topics/downloader-middleware.html
Writing your own downloader middleware
Each middleware component is a Python class that defines one or more of the following methods:
class scrapy.downloadermiddlewares.DownloaderMiddleware
Note
Any of the downloader middleware methods may also return a deferred (延迟的).
process_request(request, spider)
This method is called for each request that goes through the download middleware.
process_request() should either: return None, return a Response object, return a Request object, or raise IgnoreRequest.
If it returns None, Scrapy 会继续处理这个请求, 一直执行其他所有的中间件，直到最终合适的下载器 handler 被调用并且请求被处理 (且响应被下载).
If it returns a Response object, Scrapy 不会调用任何其它 process_request() 或 process_exception() 方法, 或 the appropriate 下载函数; 而是会返回 that response. 安装好的中间件的 process_response() 方法对每个响应都会被调用.
If it returns a Request object, Scrapy 会停止调用 process_request() 方法，并且会 reschedule 被返回的请求. 一旦新被返回的请求被执行，合适的中间件 chain will be called on the downloaded response.
If it raises an IgnoreRequest exception, 被安装的下载器中间件的 process_exception() 方法会被调用. 如果他们中没有能够处理异常的, 请求的 errback 方法会被调用 (Request.errback). If no code handles the raised exception, it is ignored and not logged (unlike other exceptions).
| Parameters: |
|---|
process_response(略)
process_exception(request, exception, spider)
当一个下载 handler or a process_request() (from a downloader middleware) 抛出异常 (including an IgnoreRequest exception)时，Scrapy 会调用 process_exception() 方法.
process_exception() 应该返回: either None, a Response object, or a Request object.
If it returns None, Scrapy 会继续处理这个异常, 执行安装的中间件的其他 process_exception() 方法, 直到没有中间件被剩下，然后默认的异常处理 kicks in.
If it returns a Response object, 已安装中间件的 process_response() method chain is started, and Scrapy won’t bother calling any other process_exception() methods of middleware.
If it returns a Request object, 被返回的请求 is rescheduled to 下载 in the future. 这会停止中间件的 process_exception() 方法的执行 the same as returning a response would.
| Parameters: |
|---|
from_crawler(cls, crawler)，
如果有此方法，该类方法会被调用创建一个中间件实例 from a Crawler. 它必须返回中间件的一个新实例. Crawler 对象对所有 Scrapy 核心组件，比如 settings 和 signals 提供 access; 它是中间件 access them and hook its functionality into Scrapy 的一种方式.
| Parameters: | crawler ( |
|---|