Worship! This Redis in-depth adventure note full of technical points is worth millions!

Preface

Redis is the most widely used middleware of Internet technology architecture in storage systems. It is also one of the most popular skills for engineers in technical interviews, especially those excellent and highly competitive large Internet companies (such as Twitter, Sina Weibo) , Alibaba Cloud, Tencent Cloud, Taobao, Zhihu, etc.), interviewers are usually required to not only master the basic use of Redis, but also require a deep understanding of the detailed principles of Redis internal implementation. It is no exaggeration to say that if you can fully understand the knowledge of Redis, half of your foot has entered the door of your favorite company!

So the editor compiled this "Deep Adventures of Redis" document for everyone, and will explain this document from the three parts of the catalog, the foreword, and the main content. At the same time, I hope it will be useful to all the big brothers and friends, and I hope you will like! Finally, friends who need this "Deep Adventures of Redis" document only need to pay attention + forward, scan the QR code of the editor to get free contact information~

 

Let's take a look at this Redis catalog first:

 

 

main content

This "Deep Adventures of Redis: Core Principles and Application Practices" is mainly divided into five parts, including application, principle, cluster, expansion, and source code. So next, the editor will carefully expand each article. Explain in detail the knowledge points of this book!

Opening and Basics

• Opening: Teaching people how to fish is worse than teaching people how to fish-what can Redis be used for?
• Basics: building on the ground from the ground up-Redis basic data structure

 

Opening

 

basis

 

Application

• Application 1: Qianfan Competition-Distributed Lock
• Application 2: Delaying strategy-delay queue
• Application 3: Save food and clothing-bitmap
• Application 4: Four or two dialing a thousand pounds-HyperLogLog
• Application 5: Layers of mountains and mountains-Bloom filter
• Application 6: Tail docking to survive-simple current limit
• Application 7: No hair is drawn-funnel current limit
• Application 8: Near the water platform-GeoHash
• Application 9: Finding a Needle in a Haystack-Scan

 

 

Some knowledge points:

Application 1: Qianfan Competition-Distributed Lock

 

Application 4: Four or two dialing a thousand pounds-HyperLogLog

In this section, we can learn to use the HyperLogLog data structure to estimate. It is very valuable and can solve many statistical requirements with low accuracy.

 

Application 5: Layers of mountains and mountains-Bloom filter

 

Application 7: No hair is drawn-funnel current limit

Funnel current limiting is one of the most commonly used current limiting methods. As the name suggests, this algorithm is inspired by the structure of the funnel.

 

Application 8: Buildings near the water-GeoHash

Redis has added the geographic location GEO module after version 3.2, which means that we can use Redis to implement functions such as Mobike "Nearby Mobike", Meituan and Ele.me "Nearby Restaurants".

 

Principles

• Principle 1: Whip into the inside-thread IO model
• Principle 2: Talking to each other-communication protocol
• Principle 3: Plan ahead-persistence
• Principle 4: Vigorously and Resolutely-Pipelines
• Principle 5: Together in the same boat-affairs
• Principle 6: Grapevine-PubSub
• Principle 7: open source and reduce expenditure-small object compression
• Principle 8: Be prepared without danger-master-slave synchronization

 

 

Some knowledge points:

Principle 3: Plan ahead-persistence

Redis data is all in memory. If there is a sudden downtime, all data will be lost. Therefore, there must be a mechanism to ensure that Redis data will not be lost due to failures. This mechanism is Redis's persistence mechanism.

 

Principle 4: Vigorously and vigorously-pipeline

Everyone has always misunderstood the Redis pipeline. They think this is a special technology provided by the Redis server. With this technology, Redis access efficiency can be accelerated. But in fact, the Redis pipeline (Pipelinc) itself is not a technology directly provided by the Redis server. This technology is essentially provided by the client and has no direct relationship with the server. Let's do an in-depth exploration of this area.

 

Principle 6: Grapevine-PubSub

Earlier we talked about the use of Redis message queues, but did not mention the shortcomings of Redis message queues, that is, it does not support the mechanism of multiple messages.

 

Principle 8: Be prepared and not in danger-master-slave synchronization

Many companies have not used Redis clusters, but at least they have become master-slave. With the master-slave, when the master hangs up, the operation and maintenance allows the slave library to take over, and the service can continue, otherwise the master needs to go through the process of data recovery and restart, which may delay a long time and affect online business Continuous service.

 

Cluster

• Cluster 1: Li Daitao Zheng-Sentinel
• Cluster 2: Divide and conquer-Codis
• Cluster 3: Unite as one-Cluster

 

Some knowledge points:

Cluster 2: Divide and conquer-Codis

 

Cluster 3: Unite as one-Cluster

 

Expansion

• Expansion 1: listen to all directions-Stream
• Extension 2: omniscient-Info command
• Extension 3: Picking up omissions-talk about distributed locks again
• Tuo Zhen 4: Life and Death-Expiration Strategy
• Expansion 5: Survival of the Fittest-LRU
• Topic 6: Smooth progress - lazy deletion
• Expansion 7: Good hands and benevolence-use Jedis gracefully
• Expansion 8: Be prepared for danger in times of peace-protect Redis
• Extension 9: Walls have ears-Redis secure communication

 

 

Some knowledge points:

Expansion 1: listen to all directions-Stream

Redis5.0 was suddenly released by the author Antirez, adding many new features. The biggest new feature of Redis5.0 is the addition of a data structure Stream, which is a new powerful and durable message queue that supports multicast. The author admits that Redis Strcam has borrowed heavily from Kafka's design.

 

Extension 3: Picking up omissions-talk about distributed locks again

In the third section, we explained the principle of distributed lock in detail, its use is very simple, one instruction can complete the lock operation. However, in a cluster environment, this method is flawed, and it is not absolutely safe.

 

Expansion 6: Smooth progress-lazy deletion

We have always believed that Redis is single-threaded, and single-threading brings simplicity of code and rich and diverse data structures to Redis. However, there is actually not only one main thread inside Redis, it also has several asynchronous threads dedicated to processing some time-consuming operations.

 

Expansion 9: Walls have ears-Redis secure communication

 

Source code

• Source code 1: Extremely deep cold-exploring the internal structure of ``strings''
• Source code 2: Extremely deep cold-exploring the interior of the ``dictionary''
• Source code 3: Extremely deep cold-exploring the inside of the ``compressed list''
• Source code 4: Extremely deep cold-explore the inside of the ``quick list''
• Source code 5: Extremely deep cold-exploring the internal structure of the ``jump list''
• Source code 6: Extremely deep cold-exploring the inside of the ``compact list''
• Source code 7: Extremely deep cold-exploring the inside of the ``cardinal tree''

 

Source code 1: Extreme class cold-exploring the internal structure of "string"

A string in Redis is a string that can be modified, and it exists in the form of a byte array in memory. We know that the standard form of strings in the C language uses NULL as the terminator, but strings are not represented in Redis. Because the length of the NULL-terminated string is obtained using the strlen standard library function, the algorithm complexity of this function is O(n), it needs to traverse the byte array and cannot afford it as a single-threaded Redis representation.

 

Source code 3: Extreme class cold-exploring the inside of the "compressed list"

In order to save memory space usage in Redis, set and hash container objects are stored in a compressed list (ziplist) when the number of elements is small. The compressed list is a continuous memory space, and the elements are stored next to each other without any redundant gaps.

 

Source code 5: Extremely deep cold-exploring the internal structure of the "jump list"

Redis's zset is a composite structure. On the one hand, it needs a hash structure to store the corresponding relationship between value and score. On the other hand, it needs to provide the function of sorting according to seore. It also needs to be able to specify the range of score to obtain the function of valuc list. This requires another structure "jump list"-

 

Source code 7: Extremely deep cold-exploring the inside of ``even number tree''

Rax is a special data structure inside Redis. It is an ordered dictionary tree (RadixTrec), arranged in lexicographic order of keys, and supports fast positioning, insertion, and deletion operations.

 

method of obtaining

Like + follow + forward, scan the QR code of the editor below to get this "Redis Deep Adventure Core Principles and Application Practice" document for free! ! !