Redis is currently the most comprehensive knowledge points and face questions Summary

Redis (full name REmote DIctionary Server) is written by Salvatore Sanfilippo of key-value storage systems. Redis is an open source use written in ANSI C, BSD comply with the agreement and support network, based on the persistence of memory can log type, Key-Value database, and provides multi-lingual API. It is often called a data structure of the server, since the value (value) can be a string (String), a hash (the Map), list (List), set (sets), and an ordered collection (sorted sets) and other types.

Redis has two biggest feature, it is to use a multiplexer IO port model, greatly improved the ability to handle IO; the other is said above can support multiple types. Redis overall architecture shown in Figure 1, a plurality of event handlers that can monitor and distributed to specific business processing unit when there is an event.

Redis is currently the most comprehensive knowledge points and face questions Summary

 

Redis is already the hottest KV database, and is widely used in various fields. To facilitate learning, the number compiled some knowledge of Redis and online issues, we hope to daily development, operation and maintenance or quit help.

1 What is Redis?

Is the Key-Value Redis essentially a type of memory database, the memcached like, operates the entire database in memory to load all of them, the asynchronous operation by periodically flush the database data to be stored on the hard disk. Because it is pure memory operation, Redis performance is very good, can handle more than 100,000 times per second read and write operations, it is known to the fastest performance of Key-Value DB.

Redis not only excellence of performance, Redis greatest charm is to support a variety of data structures stored, in addition to the maximum limit of a single value is 512MB, unlike memcached data can only be saved 1MB, so Redis can be used for many useful Features.

The main characteristics of 2 Redis

high speed

Redis written in standard C implementation, and all data is loaded into memory, it is very fast. Official data show, on a regular Linux machine, read and write speeds reach the Redis 81000 / s and 110000 / s.

2) Data Structure

Redis can be seen as a "data structure server." Currently, Redis supports five data structures.

3) endurance of

Since all data is kept in memory, so updates to the data will be saved to disk asynchronously, Redis provides some strategies to save data, such as the number of times according to the time or date. Exceed the memory data, the use of swap, to ensure that transactions;

memcacache not persistent, mongo part in memory;

4) automatic operation

Redis operation on different data types is automated, thus increasing the setting or key values, to add or remove an element from a safe operation can be set.

5) Support for multiple languages

Redis supports multiple languages, such as Ruby, Python, Twisted Python, PHP, Erlang, Tcl, Perl, Lua, Java, Scala, Clojure and so on.

6) the master - slave replication

Redis supports simple and fast master - slave replication.

Official provides a data, Slave in 21 seconds to complete the duplication of the Amazon site 10Gkey set.

7)Sharding

It is easy to Redis data across multiple instances, but this is mainly to see if the language support. Sharding function currently supports only the language of PHP, Ruby and Scala.

3 Redis which supports several types of data?

1) String (String)

redis string is the most basic type, you can be understood as type of exactly the same with Memcached, a key corresponding to a value. The string type is binary safe, that redis the string can contain any data. Such as jpg image or serialized object. The maximum value of the string type can store 512MB.

2) Hash (hash)

Redis hash is a key (key => value) pairs. Redis hash field is a string type and the value of the mapping table, hash is particularly suited for storing objects.

3) List (List)

Redis list is a simple list of strings, sort insertion order. You can add an element to the head of the list (on the left) or tails (to the right).

4) Set (collection)

Redis is a string Set the type of unordered collection. Collection is achieved through a hash table, so add, delete, search complexity is O (1).

5) zset (sorted set: an ordered collection)

Redis zset and is set as a collection of string type elements, and does not allow duplicate members. The difference is that a double score will be associated with each type of element. It is to redis from small to large order of collection of member passing score.

4 Redis memcached What are the advantages compared to?

All values ​​1) memcached are simple strings, redis as its replacement, support richer data types

2) redis much faster than memcached

3) redis can persist its data

What are the appropriate scene 5 Redis there?

1) session cache (Session Cache)

The most common scenario is using Redis session cache (session cache). Redis caching session with than other storage (such as Memcached) has the advantage of: Redis provide persistence. When not strictly required to maintain a cache coherency, if the user's shopping cart information lost, most people will be unhappy, and now, they will do so?

Fortunately, with the improvement of Redis these years, it's easy to find the right document how to use Redis caching session. Even well-known business platform Magento also offers Redis plug-ins.

2) full-page cache (FPC)

In addition to the basic session token, Redis also provides a very convenient platform for FPC. Back consistency, even restart Redis instance, because of the persistent disk, users will not see a decrease page loading speed, which is a great improvement, similar to PHP local FPC.

Magento again as an example, Magento offers a plug-in to use Redis as a full-page cache backend.

In addition, the WordPress user, Pantheon has a great plugin wp-redis, this plugin can help you with the fastest speed of loading pages you've visited.

3) Queue

Reids a big advantage in the field memory storage engine is to provide a list and set operations, which makes Redis can serve as a good platform to use the message queue. Redis used as an operation queue, similar to native language (e.g., Python) push on the list / pop operations.

If you do a quick search for "Redis queues" in Google, you'll be able to find a large number of open source projects, the purpose of these projects is to use Redis create a very good back-end tools to meet the needs of various queues. For example, Celery has a background is to use Redis as a broker, you can go to see from here.

4) Ranking / counter

Redis digital up or down in memory operations are implemented very well. Collection (Set) and an ordered set (Sorted Set) also allows us the time to perform these operations become very simple, Redis just provide just these two data structures. So, we need to sort the collection to get the highest ranked 10 users - what we call "user_scores", we just need to execute something like the following:

Of course, this assumes you are doing increasing user score based on your sort. If you want to return the user and the user's score, you need to perform:

ZRANGE user_scores 0 10 WITHSCORES

Agora Games is a good example, implemented in Ruby, its ranking is to use Redis to store data, you can see here.

5) publish / subscribe

Last (but certainly not least) is Redis publish / subscribe functionality. Publish / Subscribe usage scenario is indeed very much. I have seen people use social networking connections, but also to build a chat system based publish / subscribe Script Triggers, and even publish Redis / subscribe functionality! (No, it's true, you can go to verify).

What kinds of policies 6 Redis data out of there?

noeviction: returns an error when the memory limit is reached and the client attempts to execute commands allow more memory to be used (most of the write command, but DEL and with a few exceptions)

allkeys-lru: key recovery attempts (LRU) the least used, so that the newly added data storage space.

volatile-lru: key recovery attempts (LRU) the least used, but only in the key set expired, so that the newly added data storage space.

allkeys-random: the random key so that the recovered data are newly added storage space.

volatile-random: the random key so that the recovery of the newly added data storage space, but only in the set of key expiration.

volatile-ttl: recovered in expired key set, and the survival time (TTL) shorter recovery key priority, so that the newly added data storage space.

7 Redis official Why not offer Windows version?

Currently the software mostly operating data center server on the Linux operating system, Windows application scenarios much; another Linux version has been fairly stable, small windows environment of users, not the need to develop, but will bring compatibility issues .

8 a value of type string can store the maximum capacity is how much?

Value is the maximum supported size 512M, the maximum length of the additional key (Key) is also 512MB.

9 Why Redis need to put all the data into memory?

We know that based on the performance of the pyramid is a million times the memory performance of mechanical disk, if the data in memory, disk I / O speed to seriously affect the performance of redis. In memory getting cheaper today, redis will become increasingly popular.

If you set the maximum memory used, the data has been recorded can not continue to insert a new value after the number reached memory limit.

Redis is currently the most comprehensive knowledge points and face questions Summary

 

10 Redis cluster program should be how to do? What are the solutions?

1.twemproxy, about the concept is that it is similar to a proxy approach, using methods and general redis without distinction of any kind, set up multiple instances redis after its subordinate, when used in this connection required redis place or move to an twemproxy, it will be an agent of identity receives the request and uses a consistent hash algorithm, the request is forwarded to a specific redis, the results back twemproxy. Easy to use (relatively redis simply modify the connection port), extension of the old project of choice. Problems: twemproxy pressure itself single port instance, the hash consistency after use, the number of nodes redis change when the calculated value changes, the data can not be automatically moved to the new node.

2.codis, currently the largest clustering solutions with basic and consistent twemproxy effect, but it supports the change in the number of nodes, the node data to restore the old to the new hash node.

Redis is currently the most comprehensive knowledge points and face questions Summary

 

3.redis cluster3.0自带的集群,特点在于他的分布式算法不是一致性hash,而是hash槽的概念,以及自身支持节点设置从节点。具体看官方文档介绍。

4.在业务代码层实现,起几个毫无关联的redis实例,在代码层,对key 进行hash计算,然后去对应的redis实例操作数据。 这种方式对hash层代码要求比较高,考虑部分包括,节点失效后的替代算法方案,数据震荡后的自动脚本恢复,实例的监控,等等。

11 Redis集群方案什么情况下会导致整个集群不可用?

有A,B,C三个节点的集群,在没有复制模型的情况下,如果节点B失败了,那么整个集群就会以为缺少5501-11000这个范围的槽而不可用。

12 MySQL里有2000w数据,redis中只存20w的数据,如何保证redis中的数据都是热点数据?

redis内存数据集大小上升到一定大小的时候,就会施行数据淘汰策略。

13 Redis支持的Java客户端都有哪些?官方推荐用哪个?

Redisson、Jedis、lettuce等等,官方推荐使用Redisson。

14 Redis和Redisson有什么关系?

Redisson是架设在Redis基础上的一个Java驻内存数据网格(In-Memory Data Grid)。充分的利用了Redis键值数据库提供的一系列优势,基于Java实用工具包中常用接口,为使用者提供了一系列具有分布式特性的常用工具类。使得原本作为协调单机多线程并发程序的工具包获得了协调分布式多机多线程并发系统的能力,大大降低了设计和研发大规模分布式系统的难度。同时结合各富特色的分布式服务,更进一步简化了分布式环境中程序相互之间的协作。

15 Jedis与Redisson对比有什么优缺点?

Jedis是Redis的Java实现的客户端,其API提供了比较全面的Redis命令的支持;Redisson实现了分布式和可扩展的Java数据结构,和Jedis相比,功能较为简单,不支持字符串操作,不支持排序、事务、管道、分区等Redis特性。Redisson的宗旨是促进使用者对Redis的关注分离,从而让使用者能够将精力更集中地放在处理业务逻辑上。

16 Redis如何设置密码及验证密码?

设置密码:config set requirepass 123456

授权密码:auth 123456

17 说说Redis哈希槽的概念?

Redis集群没有使用一致性hash,而是引入了哈希槽的概念,Redis集群有16384个哈希槽,每个key通过CRC16校验后对16384取模来决定放置哪个槽,集群的每个节点负责一部分hash槽。

18 Redis集群的主从复制模型是怎样的?

为了使在部分节点失败或者大部分节点无法通信的情况下集群仍然可用,所以集群使用了主从复制模型,每个节点都会有N-1个复制品.

19 Redis集群会有写操作丢失吗?为什么?

Redis并不能保证数据的强一致性,这意味这在实际中集群在特定的条件下可能会丢失写操作。

20 Redis集群节点之间是如何复制的?

节点之间是采用的异步复制(asynchronous replication),因此Redis集群是无法保证数据的强一致性的。

Redis is currently the most comprehensive knowledge points and face questions Summary

 

21 Redis集群最大节点个数是多少?

16384个。

22 Redis集群如何选择数据库?

Redis集群目前无法做数据库选择,默认在0数据库。

23 怎么测试Redis的连通性?

ping

24 Redis中的管道有什么用?

一次请求/响应服务器能实现处理新的请求即使旧的请求还未被响应。这样就可以将多个命令发送到服务器,而不用等待回复,最后在一个步骤中读取该答复。

这就是管道(pipelining),是一种几十年来广泛使用的技术。例如许多POP3协议已经实现支持这个功能,大大加快了从服务器下载新邮件的过程。

25 怎么理解Redis事务?

事务是一个单独的隔离操作:事务中的所有命令都会序列化、按顺序地执行。事务在执行的过程中,不会被其他客户端发送来的命令请求所打断。

事务是一个原子操作:事务中的命令要么全部被执行,要么全部都不执行。

26 Redis事务相关的命令有哪几个?

MULTI、EXEC、DISCARD、WATCH

27 Redis key的过期时间和永久有效分别怎么设置?

EXPIRE和PERSIST命令。

28 Redis如何做内存优化?

尽可能使用散列表(hashes),散列表(是说散列表里面存储的数少)使用的内存非常小,所以你应该尽可能的将你的数据模型抽象到一个散列表里面。比如你的web系统中有一个用户对象,不要为这个用户的名称,姓氏,邮箱,密码设置单独的key,而是应该把这个用户的所有信息存储到一张散列表里面.

29 Redis回收进程如何工作的?

一个客户端运行了新的命令,添加了新的数据。

Redi检查内存使用情况,如果大于maxmemory的限制, 则根据设定好的策略进行回收。

一个新的命令被执行,等等。

所以我们不断地穿越内存限制的边界,通过不断达到边界然后不断地回收回到边界以下。

如果一个命令的结果导致大量内存被使用(例如很大的集合的交集保存到一个新的键),不用多久内存限制就会被这个内存使用量超越。

30 Redis如何做大量数据插入?

Redis2.6开始redis-cli支持一种新的被称之为pipe mode的新模式用于执行大量数据插入工作。

31 为什么要做Redis分区?

分区可以让Redis管理更大的内存,Redis将可以使用所有机器的内存。如果没有分区,你最多只能使用一台机器的内存。分区使Redis的计算能力通过简单地增加计算机得到成倍提升,Redis的网络带宽也会随着计算机和网卡的增加而成倍增长。

32 你知道有哪些Redis分区实现方案?

客户端分区就是在客户端就已经决定数据会被存储到哪个redis节点或者从哪个redis节点读取。大多数客户端已经实现了客户端分区。

代理分区 意味着客户端将请求发送给代理,然后代理决定去哪个节点写数据或者读数据。代理根据分区规则决定请求哪些Redis实例,然后根据Redis的响应结果返回给客户端。redis和memcached的一种代理实现就是Twemproxy

查询路由(Query routing) 的意思是客户端随机地请求任意一个redis实例,然后由Redis将请求转发给正确的Redis节点。Redis Cluster实现了一种混合形式的查询路由,但并不是直接将请求从一个redis节点转发到另一个redis节点,而是在客户端的帮助下直接redirected到正确的redis节点。

33 Redis分区有什么缺点?

涉及多个key的操作通常不会被支持。例如你不能对两个集合求交集,因为他们可能被存储到不同的Redis实例(实际上这种情况也有办法,但是不能直接使用交集指令)。

同时操作多个key,则不能使用Redis事务.

分区使用的粒度是key,不能使用一个非常长的排序key存储一个数据集(The

partitioning granularity is the key, so it is not possible to shard a

dataset with a single huge key like a very big sorted set).

当使用分区的时候,数据处理会非常复杂,例如为了备份你必须从不同的Redis实例和主机同时收集RDB / AOF文件。

分区时动态扩容或缩容可能非常复杂。Redis集群在运行时增加或者删除Redis节点,能做到最大程度对用户透明地数据再平衡,但其他一些客户端分区或者代理分区方法则不支持这种特性。然而,有一种预分片的技术也可以较好的解决这个问题。

34 Redis持久化数据和缓存怎么做扩容?

如果Redis被当做缓存使用,使用一致性哈希实现动态扩容缩容。

如果Redis被当做一个持久化存储使用,必须使用固定的keys-to-nodes映射关系,节点的数量一旦确定不能变化。否则的话(即Redis节点需要动态变化的情况),必须使用可以在运行时进行数据再平衡的一套系统,而当前只有Redis集群可以做到这样。

35 分布式Redis是前期做还是后期规模上来了再做好?为什么?

既然Redis是如此的轻量(单实例只使用1M内存),为防止以后的扩容,最好的办法就是一开始就启动较多实例。即便你只有一台服务器,你也可以一开始就让Redis以分布式的方式运行,使用分区,在同一台服务器上启动多个实例。

一开始就多设置几个Redis实例,例如32或者64个实例,对大多数用户来说这操作起来可能比较麻烦,但是从长久来看做这点牺牲是值得的。

这样的话,当你的数据不断增长,需要更多的Redis服务器时,你需要做的就是仅仅将Redis实例从一台服务迁移到另外一台服务器而已(而不用考虑重新分区的问题)。一旦你添加了另一台服务器,你需要将你一半的Redis实例从第一台机器迁移到第二台机器。

36 Twemproxy是什么?

Twemproxy是Twitter维护的(缓存)代理系统,代理Memcached的ASCII协议和Redis协议。它是单线程程序,使用c语言编写,运行起来非常快。它是采用Apache 2.0 license的开源软件。

Twemproxy支持自动分区,如果其代理的其中一个Redis节点不可用时,会自动将该节点排除(这将改变原来的keys-instances的映射关系,所以你应该仅在把Redis当缓存时使用Twemproxy)。

Twemproxy本身不存在单点问题,因为你可以启动多个Twemproxy实例,然后让你的客户端去连接任意一个Twemproxy实例。

Twemproxy是Redis客户端和服务器端的一个中间层,由它来处理分区功能应该不算复杂,并且应该算比较可靠的。

37 支持一致性哈希的客户端有哪些?

Redis-rb、Predis等。

38 Redis与其他key-value存储有什么不同?

Redis有着更为复杂的数据结构并且提供对他们的原子性操作,这是一个不同于其他数据库的进化路径。Redis的数据类型都是基于基本数据结构的同时对程序员透明,无需进行额外的抽象。

Redis运行在内存中但是可以持久化到磁盘,所以在对不同数据集进行高速读写时需要权衡内存,应为数据量不能大于硬件内存。在内存数据库方面的另一个优点是,

相比在磁盘上相同的复杂的数据结构,在内存中操作起来非常简单,这样Redis可以做很多内部复杂性很强的事情。

同时,在磁盘格式方面他们是紧凑的以追加的方式产生的,因为他们并不需要进行随机访问。

39 Redis的内存占用情况怎么样?

给你举个例子: 100万个键值对(键是0到999999值是字符串“hello world”)在我的32位的Mac笔记本上 用了100MB。同样的数据放到一个key里只需要16MB, 这是因为键值有一个很大的开销。 在Memcached上执行也是类似的结果,但是相对Redis的开销要小一点点,因为Redis会记录类型信息引用计数等等。

当然,大键值对时两者的比例要好很多。

64位的系统比32位的需要更多的内存开销,尤其是键值对都较小时,这是因为64位的系统里指针占用了8个字节。 但是,当然,64位系统支持更大的内存,所以为了运行大型的Redis服务器或多或少的需要使用64位的系统。

40 都有哪些办法可以降低Redis的内存使用情况呢?

如果你使用的是32位的Redis实例,可以好好利用Hash,list,sorted set,set等集合类型数据,因为通常情况下很多小的Key-Value可以用更紧凑的方式存放到一起。

41 查看Redis使用情况及状态信息用什么命令?

info

42 Redis的内存用完了会发生什么?

如果达到设置的上限,Redis的写命令会返回错误信息(但是读命令还可以正常返回。)或者你可以将Redis当缓存来使用配置淘汰机制,当Redis达到内存上限时会冲刷掉旧的内容。

43 Redis是单线程的,如何提高多核CPU的利用率?

可以在同一个服务器部署多个Redis的实例,并把他们当作不同的服务器来使用,在某些时候,无论如何一个服务器是不够的,

所以,如果你想使用多个CPU,你可以考虑一下分片(shard)。

44 Redis常见性能问题和解决方案?

1) Master最好不要做任何持久化工作,如RDB内存快照和AOF日志文件

2) 如果数据比较重要,某个Slave开启AOF备份数据,策略设置为每秒同步一次

3) 为了主从复制的速度和连接的稳定性,Master和Slave最好在同一个局域网内

4) 尽量避免在压力很大的主库上增加从库

5) 主从复制不要用图状结构,用单向链表结构更为稳定,即:Master <- Slave1 <- Slave2 <- Slave3...

这样的结构方便解决单点故障问题,实现Slave对Master的替换。如果Master挂了,可以立刻启用Slave1做Master,其他不变。

45 Redis提供了哪几种持久化方式?

RDB持久化方式能够在指定的时间间隔能对你的数据进行快照存储.

AOF持久化方式记录每次对服务器写的操作,当服务器重启的时候会重新执行这些命令来恢复原始的数据,AOF命令以redis协议追加保存每次写的操作到文件末尾.Redis还能对AOF文件进行后台重写,使得AOF文件的体积不至于过大.

如果你只希望你的数据在服务器运行的时候存在,你也可以不使用任何持久化方式.

你也可以同时开启两种持久化方式, 在这种情况下, 当redis重启的时候会优先载入AOF文件来恢复原始的数据,因为在通常情况下AOF文件保存的数据集要比RDB文件保存的数据集要完整.

最重要的事情是了解RDB和AOF持久化方式的不同,让我们以RDB持久化方式开始。

Redis is currently the most comprehensive knowledge points and face questions Summary

 

46 如何选择合适的持久化方式?

一般来说, 如果想达到足以媲美PostgreSQL的数据安全性, 你应该同时使用两种持久化功能。如果你非常关心你的数据, 但仍然可以承受数分钟以内的数据丢失,那么你可以只使用RDB持久化。

有很多用户都只使用AOF持久化,但并不推荐这种方式:因为定时生成RDB快照(snapshot)非常便于进行数据库备份, 并且 RDB 恢复数据集的速度也要比AOF恢复的速度要快,除此之外, 使用RDB还可以避免之前提到的AOF程序的bug

47 触发RDB快照

1)在指定的时间间隔内,执行指定次数的写操作

2)执行save(阻塞, 只管保存快照,其他的等待) 或者是bgsave (异步)命令

3) 执行flushall 命令,清空数据库所有数据,意义不大。

4) 执行shutdown 命令,保证服务器正常关闭且不丢失任何数据,意义...也不大。

48 通过RDB文件恢复数据

将dump.rdb 文件拷贝到redis的安装目录的bin目录下,重启redis服务即可。在实际开发中,一般会考虑到物理机硬盘损坏情况,选择备份dump.rdb 。可以从下面的操作演示中可以体会到。

49 RDB 的优缺点

优点:

1 适合大规模的数据恢复。

2 如果业务对数据完整性和一致性要求不高,RDB是很好的选择。

缺点:

1 数据的完整性和一致性不高,因为RDB可能在最后一次备份时宕机了。

2 备份时占用内存,因为Redis 在备份时会独立创建一个子进程,将数据写入到一个临时文件(此时内存中的数据是原来的两倍哦),最后再将临时文件替换之前的备份文件。

所以Redis 的持久化和数据的恢复要选择在夜深人静的时候执行是比较合理的。

50 修改配置不重启Redis会实时生效吗?

For running instance, there are many configuration options can be modified through the CONFIG SET command, without any form of execution restart.

Guess you like

Origin blog.csdn.net/shuningzhang/article/details/90667395