How scrapy links have a password set scrapy-redis redis redis password scrapy-redis redis how to configure password

# Scrapy_redis using scheduler 
SCHEDULER = "scrapy_redis.scheduler.Scheduler" 
# scrapy_redis mechanism used to weight the 
DUPEFILTER_CLASS = "scrapy_redis.dupefilter.RFPDupeFilter" 
# Add the redis conduit ITEM_PIPELINES 
# 'scrapy_redis.pipelines.RedisPipeline': 200 is 
# defined redis host address and port number 



REDIS_HOST = '176.140.7.197' 
REDIS_PORT = 6379 

# set password 
REDIS_PARAMS {= 
    'password': 'ADMIN', 
}

windows at how to configure a password scrapy-redis redis
1. Environmental
System: Win7
scrapy-redis
redis 3.0.5
Python 3.6.1
2. Configure the password for the redis-server and start
redis installation and configuration in the windows, please refer to this article: HTTP: //blog.csdn.net/zwq912318834/article/details/78770209
. 2.1 modify the configuration files
to locate the file redis.windows.conf, be set as follows:

################################## SECURITY ###################################

# Require clients to issue AUTH <PASSWORD> before processing any other
# commands. This might be useful in environments in which you do not trust
# others with access to the host running redis-server.
#
# This should stay commented out for backward compatibility and because most
# people do not need auth (e.g. they run their own servers).
#
# Warning: since Redis is pretty fast an outside user can try up to
# 150k passwords per second against a good box. This means that you should
# use a very strong password otherwise it will be very easy to break.
#
# requirepass foobared
requirepass redisPasswordTest666666

一定要注意：requirepass redisPasswordTest666666 前面不能有任何空格和字符。
2.2. Redis service starts
when you start redis-server, be sure to add redis.windows.conf this parameter, otherwise redis uses the default boot parameters, configuration information will not take effect ......
redis-server.exe redis.windows .conf
1

2.3. Redis Desktop Manager to authenticate
without a password, can not log in

Using a password, successfully log

3. scrapy-redis redis distributed crawler connection with a password 
on the principles of interpretation, please refer to the article: HTTP: //blog.csdn.net/zwq912318834/article/details/78904192 
# password using redis 
class MySpider (RedisSpider): 
"" "Spider URLs that the reads from Redis Queue (myspider: start_urls)." "" 
name = 'XXXX' 
redis_key = 'XXXX: start_urls' 

# ...... 
custom_settings = { 
', LOG_LEVEL,': 'the DEBUG', 
'DOWNLOAD_DELAY': 0, 

# connection parameter specifies redis database 
'REDIS_HOST': '192.168.1.99', 
'REDIS_PORT': 6379, 

# redis link specified password, and use of a database which 
'REDIS_PARAMS': { 
'password': 'redisPasswordTest666666', 
}, 
}

4. Note
in windows, installed in two ways: the compression and decompression installation. The method of the above-mentioned manner is compressed decompression. The installed version of Redis service is self-starting, directly through redis-server.exe start, however, and not loaded on startup configuration file redis.windows.conf Redis, thus leading to password redis does not take effect.

How scrapy links have a password set scrapy-redis redis redis password scrapy-redis redis how to configure password

Guess you like