1. The background generated by Redis
1.1 History of data storage
1.1.1 Disk Era
In ancient times, our data was stored on disks, and each disk had a track. Each track has many sectors, and one sector is close to 512Byte.
The addressing speed of the disk is in milliseconds, and the bandwidth is in GB/M. The memory is ns level, and the bandwidth is several orders of magnitude larger than that of the disk. Overall, disks are nearly 10W times slower in addressing than memory.
During this difficult history, we faced serious I/O problems. Therefore, when reading and writing files, we often face a large I/O cost problem. The initial solution is to add a buffer (buffer). The purpose is to perform a temporary storage in the buffer, and then uniformly transport it to the memory, which slightly improves the I/O performance.
1.1.2 Generation of database
In the course of history, no technology is produced for no reason.
Our database technology is to solve the disk I / O bottleneck. In order to solve this problem, we divide the disk sector into 4K small partitions to form an index. With these index values, we can search more conveniently through the index. For faster lookup, we use B+ tree to store the index.
What is a B+ tree?
Our ordinary 1-0 tree, also known as binary search tree (binary sorting tree), binary search tree and sorting tree are the same kind of tree, which is a simple 1-0 tree. to large (or large to small) for storage. The binary search tree is prone to disadvantages during insertion and deletion. For example, it may be deleted as a linked list during the deletion process, so the search efficiency will still become very low. Therefore, we use rotation to convert this "linked list" tree into a left-right balanced tree by rotating left and right, which is the so-called B-tree. (Pay attention to the misunderstanding, are B-tree and B-tree the same tree? Beginners think that the latter is B minus tree, but it is not, it is a separator added when it is translated. B means Balance balance) Of course, our The database file is not binary, nor is the file system, so our multi-fork search tree is the so-called 1 B+ tree. The + sign means that each node is more than binary. For details, please check my article B+ tree .
In our database lookup, we have a problem? That's the byte width problem. When we build a database, we must provide a schema (mode). Our row-level storage, even if the column is empty, still occupies a space. Then, when the amount of data is huge, a lot of storage space will be wasted. But the advantage of this is that we don't need to move the location of the data when updating.
1.1.3 Generation of key-value database
No technology happens for nothing.
We have developed the database to the extreme and produced a HANA database similar to SAP. This kind of database requires a lot of hardware, with a memory of about 2T, and a package of about 200 million hardware and software services.
With the development of the Internet, we are faced with a new problem. How can we resist high concurrency and slow search caused by big data? (Note that the larger amount of data only affects multi-data lookup, and single-data lookup does not affect performance. Our business logic is usually multi-data lookup, so there will be bottlenecks)
So our kv database was created, which relies on two infrastructures. Von Neumann hardware, ethernet, and tcp/ip networks.
What is a von Neumann system? Our operating system teacher is a very old professor, and here are his original words in class. The von Neumann system has not yet been clearly defined. Some books say that there are 5, and some books say that there are 7. However, I will standardize the von Neumann system according to the postgraduate entrance examination questions of a certain year. The von Neumann system consists of five parts, controller, arithmetic unit, memory, bus, hard disk and I/O interface. (How to memorize the composition of the von Neumann system: CPU is divided into controllers and arithmetic units, and the others serve the CPU. Operations require memory, connections require buses, and we need I/O interfaces to read and write. Everything can be considered with the CPU remember all 6)
