The main content of this chapter
The similarities and differences between Redis and other software
The usage of Redis
Simple interaction with Redis using Python sample code
Using Redis to solve practical problems
The following knowledge points analysis comes from part of the "Redis Actual Combat" document! Since it does not affect viewing, only a part of it is selected for display.
If you need the full version, click here! ! Code CSDN, get it!
Redis is a remote memory database. It not only has strong performance, but also has replication characteristics and
a unique data model for solving problems . Redis offers five different types of data structures, all kinds of problems can be naturally
mapped to these data structures: Redis data structures dedicated to helping users solve the problem, but not like other databases that
kind of request Users distort the problem to fit the database. In addition, through
features such as replication, persistence, and client-side sharding, users can easily expand Redis into a system that can contain hundreds of gigabytes of data and
process millions per second. The requested system.
I first use Redis is a company which, a company needs to save the 60,000 customer contacts
the way relational database search, the search can be done by name, email address, location and telephone number of each
search It takes 10-15 seconds. After spending a week learning the basics of Redis, I use Redis
rewrite a new search engine, and then spent several weeks to carefully test the new system, to bring it to production-level
do not, eventually with this new search The system can not only filter
and sort customer contact information based on information such as name, e-mail address, location, and phone number , but each operation can be completed within 50 milliseconds, which is
200 times faster than the original search system . Reading this book allows you to learn a lot of tips, tricks, and methods to solve some common problems with Redis
.
This chapter will introduce the scope of Redis and the methods of using Redis in different environments (such as how to communicate with different groups
Software and programming language to communicate, etc.); and the following chapters will show a variety of problems, and the use of Redis to solve
these problems.
Chapter 1. Getting to Know Redis
1.1 Introduction to Redis
Installing Redis and Python Appendix A introduces how to quickly install Redis and Python.
Using Redis in other programming languages This book only shows sample codes written in Python language. Sample codes written in Ruby, Java and JavaScript (Node.js) can be found here: https://github.com/josiahcarlson/ redis-in-action. Readers who use the Spring framework can learn how to use Redis in the Spring framework by viewing http://www.springsource.org/spring-data/redis.
The previous description of the Redis database only tells part of the truth. Redis is a very fast
non-relational database (non-relational database), it can store the key (key) and 5 different types of values (value) between the
mapping (mapping), can be stored in the memory Key-value pair data is persisted to the hard disk, you can use the replication feature to expand the
read performance, and you can also use client sharding
1.1.1 Comparison of Redis with other databases and software
To extend the write performance, the following sections will introduce these features of Redis respectively.
If you are familiar with relational databases, then you must have written SQL queries that associate the data of two tables. Redis is a NoSQL database or non-relational database that people often say: Redis does not use tables, and its database does not predefine or force users to associate different data stored in Redis.
The high-performance key-value cache server memcached is often compared with Redis: both can be used to store key-value mappings, and their performance is almost the same, but Redis can automatically write data to the hard disk in two different ways And Redis can store 4 other data structures in addition to ordinary string keys, while memcached can only store ordinary string keys. These differences make Redis can be used to solve a wider range of problems, and can be used as a primary database (primary database), but also as an auxiliary database (auxiliary database) of other storage systems.
Subsequent chapters of this book will introduce the usage and query mode when Redis is used as primary storage and secondary storage. Generally speaking, many users will only store data in Redis when the performance or function of Redis is necessary: if the program has low performance requirements, or because of cost reasons, there is no way to store a large amount of data In the memory, then users may choose to use relational databases, or other non-relational databases. In practice, readers should decide whether to use Redis according to their needs, and consider whether to use Redis as primary storage or secondary
① Fragmentation is a method of dividing data into multiple parts. The division of data can be based on the ID contained in the key, the hash value of the key, or
some combination of the two. By slicing the data, users can store data in multiple machines, or
obtain data from multiple machines. This method can achieve linear performance improvements when solving certain problems.
Help storage, and how to ensure data integrity through replication, persistence, and transactions.
Table 1-1 shows some of the database servers and cache servers that overlap with Redis in function. From this table, we can
see the difference between Redis and these databases and software.
1.1.2 Additional features
When using an in-memory database like Redis, one of the first questions to consider is "When the server is shut down, where will the data stored by the server go?" Redis has two different forms of persistence methods, they The data stored in the memory can be written to the hard disk in a small and compact format: the first persistence method is point-in-time dump, and the dump operation can be performed within the specified time period. It is executed when the condition of “the specified number of write operations is executed” is met, and it can be executed by calling any one of the two dump-to-disk commands; the second method of persistence will modify all All database commands are written into an append-only file. The user can set the append-only write to never sync, sync once per second, or write a command every time according to the importance of the data. Just sync once.
We will discuss these persistence options in more depth in Chapter 4. In addition, although the performance of Redis is very good, it is limited by the memory storage design of Redis, and sometimes only one Redis server may not be able to handle all requests. Therefore, in order to extend the read performance of Redis and provide failover for Redis
5 (failover) support, Redis implements the master-slave replication feature: the slave server performing the replication will connect to the master server and receive the initial copy of the entire database (copy) sent by the master server; subsequent write commands executed by the master server will be sent Perform execution for all connected slave servers to update the data set of the slave server in real time. Because the data contained in the slave server is constantly updated, the client can send a read request to any slave server to avoid centralized access to the master server. We will discuss Redis slave servers in more depth in Chapter 4.
1.1.3 Reasons for using Redis
Readers who have experience using memcached may know that users can only use the APPEND command to add data to the end of an existing string. The memcached documentation states that the APPEND command can be used to manage the element list. This is good! Users can append elements to the end of a string and use that string as a list. But then how to delete these elements? The method memcached uses is to hide the elements in the list through a blacklist, thereby avoiding operations such as reading, updating, and writing to the elements (including memcached writes performed after a database query). On the contrary, Redis's LIST and SET allow users to directly add or delete elements. Using Redis instead of memcached to solve the problem can not only make the code shorter, easier to understand, and easier to maintain, but also make the code run faster (because the user does not need to read the database to update the data). In addition, in many other cases, Redis's efficiency and ease of use are much better than relational databases.
A common usage of the database is to store long-term report data and use these report data as aggregated data within a fixed time range (aggregates). The common method of collecting aggregated data is: first insert each row into a report table, then collect aggregated data by scanning these rows, and update those rows that already exist in the aggregated table based on the collected aggregated data. The reason for using the insert row method for storage is that for most databases, the insert row operation is very fast (the insert row will only be written at the end of the hard disk file). However, updating the rows in the table is a rather slow operation, because this update may cause a random write (random write) in addition to a random read. In Redis, users can directly use atomic INCR commands and their variants to calculate aggregate data, and because Redis stores data in memory①
① Objectively speaking, memcached can also be used in this simple scenario, but using Redis to store aggregated data has the following three advantages:
1. First of all, using Redis can put aggregated data related to each other in the same structure for access Aggregating data will become easier;
2. Secondly, using Redis can put aggregated data into an ordered collection to build a real-time ranking;
3. Finally, Redis aggregate data can be integer or floating point numbers. The aggregate data of memcached can only be integers. Moreover, the command request sent to Redis does not need to be processed by a typical query analyzer (parser) or query optimizer (optimizer), so the speed of random writing to the data stored in Redis is always very fast.
Using Redis instead of relational databases or other hard disk storage databases can avoid writing unnecessary temporary data, and also eliminate the trouble of scanning or deleting temporary data, and ultimately improve the performance of the program. Although the examples listed above are simple examples, they are a good proof that "tools can greatly change the way people solve problems."
1.2 Introduction to Redis data structure
As, the Redis may store the mapping between keys and 5 different types of data structures prior to Table 1-1, the number of these five
data structure types are STRING (a string), the LIST (list), the SET (set) , HASH (hash) and ZSET (ordered set). Some Redis commands are common to these five structures, such as DEL, TYPE, RENAME, etc.; but there are also some Redis commands that can only be used for one or two specific structures. Chapter 3 will discuss the commands provided by Redis. More in-depth introduction.
Most programmers should not have Redis of STRING, LIST, HASH structure of these three feel strange, because
for them, and many built-in programming language strings, lists and hash and other structures (semantics) in terms of implementation and semantics Both are
very similar. Some programming languages also have a collection data structure, which is similar to Redis SET in implementation and semantics. ZSET
is a structure unique to Redis to some extent, but when you are familiar with it, you will find that it is also a very
useful structure. Table 1-2 compares the five structures provided by Redis, explains the values stored in these structures, and briefly introduces
their semantics.
Command List In this section, when introducing each data type, a small part of the commands for processing these data structures will be displayed in a table. The third part will show a more detailed (but still incomplete) command list, complete A list of Redis commands can be found at http://redis.io/commands.
This section will introduce how to represent these five structures of Redis, and will also introduce how to use Redis commands, so as to lay a good foundation for the rest of this book. All the sample codes shown in this book are written in Python. If readers have installed Redis according to the method described in Appendix A, then they should have installed Python and the client libraries required to use Redis in Python. . As long as readers have installed Redis, Python, and redis-py libraries on the computer, they can try to execute the sample code shown in the book while reading this book.
Please install Redis and Python. Before reading the follow-up content, readers are asked to install Redis and Python according to the method introduced in Appendix A. If the reader feels that the installation method described in Appendix A is too complicated, then there is a simpler method, but this method can only be used for the Debian system (or a derivative system of this system): download from http://redis.io/download Redis compressed package, unzip the compressed package, execute make&& sudo make install, and then execute sudo python -m easy_install redis hiredis (hiredis is optional, it is a high-performance Redis client written in C language).
If readers are familiar with procedural programming languages or object-oriented programming languages, they should be
able to understand Python code even if they have not used Python before . On the other hand, if the reader decides to use another programming language to operate Redis, then he needs
to translate the Python code of this book into the code of the language being used.
Although sample code written in other languages is not included in the book, the Python sample code shown in this book has been translated into Ruby code, Java code and JavaScript code. These translated codes can be found at https://github.com/josiahcarlson /redis-in-action is downloaded to. Like the sample code written in Python, these translated codes also include corresponding comments for readers' reference.
In order for the sample code as simple as possible, this book will try to avoid the use of the advanced features of Python, and use the function instead of
a class or something else Redis to perform the operation, in order to focus on the use Redis to solve the above problem, without having to Too much
attention to Python syntax. This section will use the redis-cli console to interact with Redis. First, let's take a look at
the simplest structure in Redis: STRING.
1.2.1 Strings in Redis
Redis strings are very similar to strings provided by other programming languages or other key-value stores. When using
pictures to represent keys and values in this book , the key name and value type are usually placed at the top of the box, and the value is
placed inside the box. Figure 1-1 takes the string with the key as hello and the value as world as an example, and each
part of the box is marked separately .
Strings have commands similar to other key-value stores, such as GET (get value), SET (set value) and DEL (delete value). If the reader has installed Redis according to the method given in Appendix A, then you can try to use redis-cli to execute SET, GET and DEL according to the example shown in Listing 1-1. Table 1-3 describes the three commands Basic usage.
Using redis-cli In order to allow readers to interact with Redis conveniently at the beginning, this chapter will use redis-cli, an interactive client, to introduce
Redis commands.
In addition to being able to GET, SET and DEL string values, Redis also provides some commands that can read and write part of the contents of the string, as well as some commands that can auto-increase or decrement the value stored in the string The command for the operation. Chapter 3
will introduce these commands, but before that, we still have a lot of basic knowledge to understand. Let's take a look at the list of Redis and its functions.
1.2.2 List in
Redis Redis's support for linked-list structure makes it unique in the world of key-value storage. A list structure can store multiple strings in an orderly manner. As with the method used to represent strings, this section uses labeled boxes to represent lists, and puts the elements contained in the list in the boxes. Figure 1-2 shows an example of this.
The operations that Redis lists can perform are very similar to list operations in many programming languages: LPUSH and RPUSH commands are used to push elements into the left end and right end of the list, respectively; LPOP and RPOP commands are used respectively To pop elements from the left and right ends of the list; the LINDEX command is used to get an element of the list at a given position; the LRANGE command is used to get all the elements of the list in a given range. Code Listing 1-2 shows some examples of using list commands, and Table 1-4 briefly introduces each command used in the example.
Even Redis list only supports a few commands mentioned above, it can already be used to solve many problems, but Redis did not
have to stop there - in addition to the above-mentioned commands, Redis list also has shifted from a list inside Commands for removing elements, commands for inserting elements in the
middle of the list, commands for trimming the list to a specified length (equivalent to removing elements from one or both ends of the list),
and other commands. Chapter 3 will introduce many list commands, but before that, let us first understand Redis collections.
1.2.3 Redis collections
Redis collections and lists can store multiple strings, the difference between them is that the list can store multiple identical strings, while the
collection uses a hash table to ensure that each of its storage The strings are all different (these hash tables have only keys, but no values associated with the keys). The method of representing a collection in this book is basically the same as the method of representing a list. Figure 1-3 shows an example collection with 3 elements.
Because Redis collections store elements in an unordered manner, users cannot push elements into
one end of the collection or pop elements from one end of the collection like using lists . However, the user can use the SADD command to add elements to the collection, or use the SREM command to remove elements from the collection. In addition, you can use the SISMEMBER command to quickly check whether an element already exists in the collection, or use the SMEMBERS command to get all the elements contained in the collection (if the collection contains a lot of elements, the execution speed of the SMEMBERS command may be very slow, so please Use this command with caution). Code Listing 1-3 shows some examples of the use of collective commands. Table 1-5 briefly introduces each command used in the code listing.
Like strings and lists, in addition to the basic set of operations to add and remove operations, also supports many other operations than
as SINTER, SUNION, SDIFF these three commands on a common intersection calculation can be performed separately, and set computing and Difference
calculation. Chapter 3 will give a more detailed introduction to the related commands of collections, and Chapter 7 will also show how to use collections to solve multiple
problems. But don't worry, because among the five data structures provided by Redis, there are two that we have not yet understood. Let's
take a look at Redis's hash first .
1.2.4 Redis hash
Redis hash can store the mapping between multiple key-value pairs. Like character strings, the value stored in the hash can be either a character
string or a numeric value, and the user can also perform increment or decrement operations on the numeric value stored in the hash. Figure 1-4
shows a hash containing two key-value pairs.
In many ways, hashing is like a miniature version of Redis, and many string commands have corresponding hashed versions. Code clear
Lists 1-4 show how to perform operations such as inserting elements, obtaining elements, and removing elements on the hash. Table 1-6 briefly introduces the
commands used in the code list.
Readers familiar with document databases can regard Redis hashes as documents in a document database, while
readers familiar with relational databases can regard Redis hashes as rows in a relational database, because hashes, documents, and rows All three allow users to access
or modify one or more fields at the same time . Finally, let's take a look at the last of the five data structures of Redis: ordered collections.
1.2.5 Redis ordered collections
, like hashes, are used to store key-value pairs: the keys of ordered collections are called members, and each member is different; and ordered The value of the set is called a score, and the score must be a floating point number. Ordered set is the only structure in Redis that can access elements based on members (this is the same as hashing), and can access elements based on scores and the order of scores. Figure 1-5 shows an example of an ordered set of two elements.
Like other structures of Redis, users can perform operations such as adding, removing, and obtaining ordered collections. Code Listing 1-5
shows examples of the execution of these operations, and Table 1-7 briefly introduces each of the code listings. command.
Readers should now know what an ordered set is and what it can do. So far, we have basically understood the
five structures provided by Redis . The next section will show how
to solve a common problem by combining the data storage capabilities of hashes with the built-in sorting capabilities of ordered sets .
The analysis of the above knowledge points comes from part of the "Redis Actual Combat" document! Since it does not affect viewing, only a part of it is selected for display.
If you need the full version, click here! ! The secret number CSDN, find the owner to get it!