[Django] The use of Redis [original]

0. Reference

Chinese documentation for django-redis

There are two ways to use Redis with Django:

• custom mode
• Use third-party components [recommended]

1. Custom Mode

In this way, not only Django can be used, other frameworks or native Python can also be used

A. Install dependencies

pip install redis

B. Redis configuration

Encapsulated into modules, which can be used globally

import redis

# 抽取封装成模块，全局使用（单例模式，redis_pool.py）
POOL = redis.ConnectionPool(host='xx.xx.xx.xx', port=6379, password='xxx', max_connections=1000)

c. use

# 引用全局连接池
from redis_pool import POOL

conn = redis.Redis(connection_pool=POOL)
conn.set(key, value)

2. Using third-party components

If it is Django, it is recommended to use this method

A. Install dependencies

pip install django-redis

B. Redis configuration (settings.py)

CACHES = {
    
    
    "default": {
    
    
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379",
        "OPTIONS": {
    
    
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "CONNECTION_POOL_KWARGS": {
    
    "max_connections": 100, 'decode_responses': True},
            # "PASSWORD": "密码",
        },
        # 前缀
        "KEY_PREFIX": "test"
    }
}

Note: max_connections is a connection pool, if you don’t need it, you can remove it
. Note: The prefix depends on personal needs.
-8, if you add this setting, you don't need to decode it when you read it. However, if decode_responses is set to True, then using the second method below to read the cache will report an error. For
example:

>>> from django_redis import get_redis_connection
>>> conn = get_redis_connection()
>>> conn.set('redis_test3', 22)
True
>>> conn.get('redis_test3')
b'22'

Note: If there are multiple Redis, you can add a new one in CACHE, which is different from default

c. use

There are two ways:

• get_redis_connection
• cache

The first method: get_redis_connection
recommends this method

from django_redis import get_redis_connection
 
# 这里可以传使用哪个redis，不传默认是default
redis = get_redis_connection()
redis.get(key)
redis.set(key, value)

Note: If decode_responses is not set to True, the data read using get_redis_connection is bytes, which needs to be decoded to utf-8. If you add this setting, you don't need to decode it yourself. For
example:

>>> from django_redis import get_redis_connection
>>> conn = get_redis_connection()
>>> conn.set('redis_test3', 22)
True
>>> conn.get('redis_test3')
b'22'

Note: If this method is used, the prefix in caches will not take effect, that is, if it is set to redis, the keys are what they are, and the prefix will not be automatically added.

The second method: cache

from django.core.cache import cache
 
cache.get(key)
cache.set(key, value)

Note: If decode_responses in caches is set to True, then reading will report an error

>>> cache.get('redis_test3')
Traceback (most recent call last):
  File "c:\users\jiand\appdata\local\programs\python\python37\Lib\code.py", line 90, in runcode
    exec(code, self.locals)
  File "<console>", line 1, in <module>
  File "D:\Python\Envs\Django2Demo\lib\site-packages\django_redis\cache.py", line 87, in get
    value = self._get(key, default, version, client)
  File "D:\Python\Envs\Django2Demo\lib\site-packages\django_redis\cache.py", line 27, in _decorator
    return method(self, *args, **kwargs)
  File "D:\Python\Envs\Django2Demo\lib\site-packages\django_redis\cache.py", line 94, in _get
    return self.client.get(key, default=default, version=version, client=client)
  File "D:\Python\Envs\Django2Demo\lib\site-packages\django_redis\client\default.py", line 220, in get
    value = client.get(key)
  File "D:\Python\Envs\Django2Demo\lib\site-packages\redis\client.py", line 1579, in get
    return self.execute_command('GET', name)
  File "D:\Python\Envs\Django2Demo\lib\site-packages\redis\client.py", line 878, in execute_command
    return self.parse_response(conn, command_name, **options)
  File "D:\Python\Envs\Django2Demo\lib\site-packages\redis\client.py", line 892, in parse_response
    response = connection.read_response()
  File "D:\Python\Envs\Django2Demo\lib\site-packages\redis\connection.py", line 734, in read_response
    response = self._parser.read_response()
  File "D:\Python\Envs\Django2Demo\lib\site-packages\redis\connection.py", line 358, in read_response
    response = self.encoder.decode(response)
  File "D:\Python\Envs\Django2Demo\lib\site-packages\redis\connection.py", line 129, in decode
    value = value.decode(self.encoding, self.encoding_errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

Summary:
You can only choose one of these two methods. It is best to use one. It is recommended to use get_redis_connection and set the decode_responses of the cache to True.
If you need to use both methods, remove the configuration of decode_responses from the cache, and then use get_redis_connection to read When converting bytes to strings:

redis_value = self.conn.get(lock_name)

# 注意：如果是get_redis_connection的话，从redis里面读取的是bytes字符的
redis_value = redis_value.decode('utf-8') if isinstance(redis_value, bytes) else redis_value

sample code

from django_redis import get_redis_connection

def get_private_ips(self, subnet_id, is_ignore_cache=False):
    """
    查询私有IP列表
    使用Redis缓存，保存1小时
    :param subnet_id:
    :param is_ignore_cache: 是否忽略缓存 默认是否
    :return:
    """
    key = 'HW_private_ips_area_%s_subnetId_%s' % (self.area, subnet_id)
    cache = get_redis_connection()

    # 保存1小时
    expired_time = 60 * 60 * 1
    res = cache.get(key, None)

    if not res or is_ignore_cache:
        sig = self.sig
        uri = "/v1/{0}/subnets/{1}/privateips".format(sig.ProjectId, subnet_id)
        res = self._hw_request(uri)

        if res and not res.get('code') and res.get('privateips'):
            cache.set(key, res, expired_time)

    return res

Simple Redis lock implementation:

from django_redis import get_redis_connection


class LockService:
    def __init__(self, conn=None):
        """
        如果不传连接池的话，默认读取配置的Redis作为连接池
        :param conn:
        """
        self.conn = conn if conn else get_redis_connection()

    def acquire_lock(self, lock_name, value, expire_time=60):
        """
        加锁
        如果不存在lock_name，则创建，并且设置过期时间，避免死锁
        如果存在lock_name，则刷新过期时间

        插入成功：返回True
        已存在：返回False并且刷新过期时间
        :param lock_name:
        :param value:
        :param expire_time:
        :return:
        """
        if self.conn.setnx(lock_name, value):
            # 注意：Todo 这里可能会有问题，如果程序在写入redis之后但未设置有效期之前突然崩溃，则无法设置过期时间，将发生死锁
            self.conn.expire(lock_name, expire_time)
            return True
        elif self.conn.ttl(lock_name):
            self.conn.expire(lock_name, expire_time)
        return False

    def release_lock(self, lock_name, value):
        """
        释放锁
        注意：只有value值一致才会删除，避免在并发下删除了其他进程/线程设置的锁
        :param lock_name:
        :param value:
        :return:
        """
        redis_value = self.conn.get(lock_name)

        # 注意：如果是get_redis_connection的话，从redis里面读取的是bytes字符的
        redis_value = redis_value.decode('utf-8') if isinstance(redis_value, bytes) else redis_value
        if str(redis_value) == str(value):
            self.conn.delete(lock_name)