python scrapy uses Redis persistent storage

Install Redis  Releases · microsoftarchive/redis · GitHub

This is a windows application

Just fool-proof installation

Enter the Redis installation directory and right-click to open the Shell window.

Enter the command: redis-server to start the service

redis-cli opens the client

keys * View all keys

Using python:

Install Redis

pip install redis==2.10.6

Create link

from redis import Redis

 

Store data in pipeline