Two social platforms, Weibo and Twitter, rely heavily on Redis to carry massive user access. This article describes how to use Redis to design a social system, and how to scale Redis so that it can host hundreds of millions of users.
Although a single Redis has excellent performance, as the scale of the system increases, the problem that a single server cannot store all data and cannot handle all read and write requests will appear sooner or later. At this time, we need to expand Redis. make it meet the needs.
Before introducing how to expand, let's take a look at how to use Redis to build a social platform.
Use Redis to build a social platform
To build a social platform with Redis, you need to consider the following core functions first.
1. Published Weibo
You can use Redis hash to save published tweets.
A Weibo usually includes multiple fields, such as posting time, posting user, body content, etc. Usually, the Weibo id is used as the key to save multiple key-value pairs as hashes in Redis.
2. Information flow
When a user visits its homepage information stream, he can see the latest information of all the users he follows. The key is the uid of the current user, the content of the information stream is stored in the zset in the form of id/timestamp, and the timestamp is used for sorting, so that the returned list is arranged in chronological order. The id of the microblog is used for the next step of the business to obtain the relevant information of the microblog.
3. Follow and Followers
We can also store the follower and fan base in zset, and still use timestamp to sort. key is the current user uid.
After understanding the above structure, let's continue to see how to use Redis to expand the entire system and have the ability to handle hundreds of millions of users.
The first thing we need to do is to make Redis' read query processing capability exceed that provided by a single Redis server, under the condition that Redis can store all data and can process write queries normally.
Extended read performance
Suppose we use Redis to build a social networking site with the same features and functions as Weibo or Twitter. One of the features of the site is to allow users to view their own profile pages and personal homepage information streams. About 30 pieces of content are obtained in the information flow.
Because a Redis server dedicated to fetching feeds can simultaneously fetch feeds for at least 3,000 to 10,000 users per second, this operation is not a problem for smaller social networking sites.
However, for larger social networking sites, the number of information flow messages that the program needs to obtain per second will far exceed the upper limit that a single Redis server can handle. Therefore, we must find a way to increase the information flow messages that Redis can obtain per second. quantity.
Below we will discuss how to use a read-only slave server to improve the performance of the system to process read queries, so that the overall read performance of the system can exceed the upper limit of read query performance provided by a single Redis server.
Before scaling the performance of read queries and using additional servers as slaves to improve the system's performance for processing read queries, let's review a few ways Redis can improve performance.
When using short structures, make sure that the maximum length of the compressed list is not so large that it affects performance.
Depending on the type of query your program needs to perform, choose the structure that provides the best performance for that query. For example, don't use LIST as SET; don't get the entire HASH and sort it in the client, but use ZSET directly; and so on.
Before caching large objects in Redis, consider compressing it to reduce the network bandwidth required to read and write objects. Compare the compression algorithms lz4, gzip, and bzip2 to see which algorithm provides the best compression and the best performance for the data being stored.
Use a pipeline (whether a pipeline enables transactional nature is determined by a specific program) and connection pooling.
With everything in place to ensure that read and write queries execute quickly, the next thing to consider is how to actually solve the real problem of "how can we handle more read requests".
The easiest way to improve the read capability of Redis is to add a slave server that provides read capability.
Users can run some additional servers, have them connect to the main server, and then receive copies of the data sent by the main server and make near real-time updates over the network (the specific update speed depends on the network bandwidth). By distributing read requests to different slave servers for processing, users can obtain additional read query processing capabilities from the newly added slave servers.
Remember: only write to the master
When using read-only slaves, remember to only write to the Redis master. By default, attempting to write to a Redis server configured as a slave will raise an error (even if the slave is the master of other slaves).
Simply put, to turn a Redis server into a slave server, we only need to add a slaveof host port statement to the Redis configuration file, and replace the values of the host and port parameters with the IP address of the master server respectively. and the port number will do. In addition to this, we can configure a running Redis server as a slave by sending the SLAVEOF host port command to it. One thing to note is that when a slave server connects to the master server, all the data originally stored by the slave server will be cleared. Finally, by sending the SLAVEOF no one command to the slave, we can make the slave disconnect from the master.
The most thorny problem you may encounter when using multiple Redis slaves to process read queries is the temporary or permanent offline of the master. Whenever a slave server tries to establish a connection with the master server, the master server will create a snapshot of the slave server. If multiple slave servers try to connect to the master server before the snapshot is created, these slave servers will received the same snapshot. This is great from an efficiency standpoint, as it avoids creating multiple snapshots.
However, sending multiple copies of a snapshot to multiple slaves at the same time can consume much of the bandwidth available to the master. Makes the master's latency high, and even causes the slaves to which the master has established a connection to disconnect.
One of the ways to solve the problem of resyncing from the server (resync) is to reduce the amount of data that the master server needs to transmit to the slave server, which can be done by building a tree-like replication middle layer.
(Figure: An example of a Redis master-slave replication tree, the bottom layer of the tree consists of 9 slave servers, and the middle tier consists of 3 replication secondary servers)
Slave trees are very useful, and are even necessary when replicating between different data centers: resyncing over slow wide area network (WAN) connections is quite resource-intensive, This kind of work should be done by the slave servers in the middle tier without bothering the topmost master server. But on the other hand, building a slave server tree also brings complex network topology, which increases the difficulty of manual and automatic handling of failover.
In addition to building a tree-like group of slave servers, another way to solve the problem of slave server resynchronization is to compress the network connection, thereby reducing the amount of data that needs to be transmitted. Some Redis users have found that using a compressed SSH tunnel (tunnel) for connection can significantly reduce bandwidth usage. For example, a company once used this method to reduce the bandwidth required for replicating a single slave server from the original 21Mbit to 1.8 Mbit (http://mng.bz/2ivv). If the reader intends to use this method, please remember to use the option provided by SSH to make the SSH connection automatically reconnect after disconnection.
Encryption and compression overhead
In general, the encryption overhead of using an SSH tunnel is not a huge burden on the server, as the 2.6 GHz Intel Core 2 Duo single-core processor is capable of using AES per second using only a single processing core The -128 algorithm encrypts 180MB of data, while with the RC4 algorithm, approximately 350MB of data can be encrypted per second. With a powerful enough processor and a gigabit network connection, the program can fully utilize the entire network connection even with encryption.
The only place that could go wrong is compression—since SSH uses the gzip compression algorithm by default. SSH provides configuration options that allow users to select a specified compression level (for details, please refer to the SSH documentation). Its level 1 compression can be used at the beginning of the replication when using the 2.6GHz processor mentioned earlier. Compress the Redis RDB file at a speed of 24-52MB per second; and compress the Redis AOF file at a speed of 60-80MB per second after the replication enters the continuous update phase.
Using Redis Sentinel
Redis Sentinel can be used in conjunction with Redis's replication capabilities to failover the primary server that goes offline. Redis Sentinel is a Redis server running in a special mode, but it does not behave the same as a normal Redis server.
Sentinel monitors a series of master servers and the slave servers of these master servers. By sending PUBLISH commands and SUBSCRIBE commands to the master server, and sending PING commands to the master server and slave servers, each Sentinel process can autonomously identify available slave servers and other Sentinel servers .
When the master server fails, all Sentinels monitoring the master server will elect a Sentinel based on the information they have in common with each other, and elect a new master server from the existing slave servers. When the selected slave becomes the master, that selected Sentinel will let the remaining slaves replicate the new master (by default, Sentinel will migrate slaves one by one, but this The number can be modified via configuration options).
Generally speaking, the purpose of using Redis Sentinel is to provide automatic failover services to the slave servers under the master server. In addition, Redis Sentinel provides an optional failover notification feature, which can perform operations such as configuration updates by invoking user-provided scripts.
For a deeper understanding of Redis Sentinel you can read http://redis.io/topics/sentinel
After understanding how to scale read performance, it's time to consider how to scale write performance.
Expand write performance and memory capacity
As more and more data is cached, when the data cannot be stored on a single machine, we need to find a way to split the data and store it in a cluster composed of multiple machines.
Extended write capacity
Although this section discusses how to use sharding to increase the total amount of memory available, these methods can also improve Redis write throughput when a Redis server's write performance is at its limit.
Before scaling write performance, we first need to confirm whether we have exhausted all means to reduce memory usage and have reduced the amount of data that needs to be written as much as possible.
All methods written by myself have been checked to reduce the amount of data that the program needs to read as much as possible.
Migrate unrelated functionality to other servers.
Before writing to Redis, try to aggregate the data to be written in local memory. This practice can be applied to all analytical and statistical calculation methods.
Use locks to replace WATCH/MULTI/EXEC transactions that may limit speed, or use Lua scripts.
In the case of using AOF persistence, the hard disk of the machine must store all the data written by the program, which takes a certain amount of time. For 400,000 short commands, the hard disk may only need to write a few MB per second; but for 100,000 1KB commands, the hard disk will need to write 100 MB per second.
If the problem remains unresolved after exhausting all methods to reduce memory usage and improve performance as much as possible, then we have encountered the bottleneck caused by only using a single machine, and it is time to shard the data to multiple machines. .
The data sharding method described in this article requires users to use a fixed number of Redis servers. For example, if writes are expected to increase by a factor of 4 every 6 months, then we can preshard the data into 256 shards to have one that will last for the next 2 years. A sharding scheme that meets the expected increase in write volume (how long you need to plan is up to you).
Pre-sharding for growth
When pre-sharding the system to cope with possible traffic growth in the future, we may be in a situation where we currently have too little data, and store this data according to the number of machines calculated by the pre-sharding method It will only be worth the loss. To be able to split the data as usual, we can run multiple Redis servers on a single machine and use each server as a shard.
Note that when running multiple Redis servers on the same machine, remember to make each server listen on a different port, and make sure that all servers write to different snapshot files or AOF files.
Run multiple Redis servers on a single machine
The above describes how to shard write commands to multiple servers for execution, thereby increasing the total amount of available memory in the system and improving the system's ability to process write operations. However, if you are performing complex queries such as searching and sorting, you feel that the performance of the system is limited by the single-threaded design of Redis, and your machine has more computing cores, more communication network resources, and more Hard disk I/O is mostly used to store snapshot files and AOF files, then you can consider running multiple Redis servers on a single machine. All you need to do is configure all the servers on the same machine to listen on different ports, and make sure they have different snapshot configurations or AOF configurations.
Expand complex business scenarios
When expanding various Redis services, we often encounter such a situation: because the query executed by the service is not as simple as reading and writing, simply sharding the data is not enough to satisfy complex business. the needs of the scene.
Extend social networking sites
The following describes how to extend social networking sites like Weibo or Twitter. The purpose of the introduction is to give us a better understanding of what data structures and methods are used to build a large social network. These methods can be used almost unlimitedly. - We can scale a social networking site to any size as long as funding allows.
The first step in scaling a social networking site is to find out what data is often read and what data is often written, and consider whether it is possible to separate frequently used data from infrequently used data.
First, assume that we have put the user's published Weibo on a separate Redis server, and use a read-only slave server to process a large number of read operations on these data. Then there are mainly two types of data that need to be expanded on a social networking site: information flow, following and fan lists.
Expand the database of published tweets
When your social networking site gets a certain amount of traffic, we need to further expand the database for storing published microblogs, not just adding slave servers.
Because each microblog is completely stored in a separate hash, the program can easily shard each microblog hash into a cluster consisting of multiple Redis servers based on the key where the hash is located.
Because it is not difficult to shard each microblog hash, the work of sharding should not be difficult to complete. Another way to expand the Weibo database is to use Redis as a cache and store the latest published messages in Redis, while older (that is, less read) messages are stored in hard disk storage. Inside the server, like PostgreSQL, MySQL, Riak, MongoDB, etc.
On a social networking site, there are three main information flows: the information flow of the user's home page, the profile information flow and the grouped information flow. Each information flow itself is similar, so we will use the same treatment.
Let's see how the two core systems in the social system can be extended through different sharding strategies.
1. Shard the stream list
The title says "slicing the stream" is actually a bit of a misnomer, because the home page stream and the group list stream are usually relatively short (the maximum is usually only 1,000, the actual number is determined by zset-max-ziplist-size The value of the option is determined), so it is not actually necessary to shard the content of the information stream; what we really need to do is to store different information streams on different shards according to the key name.
On the other hand, the information flow of each user profile on social networking sites is usually infinitely growing. Although the vast majority of users only post a few microblogs a day at most, there are also talkative users who post a large amount of information at a rate significantly higher than this frequency. Taking Twitter as an example, the top 1,000 users on the site posted more than 150,000 tweets each, and the top 15 users posted millions of tweets each.
From a practicality standpoint, a reasonable approach would be to limit each user's published tweets to a maximum of about 20,000 posts, and delete or hide the oldest posts - this is sufficient for 99.999% of Twitter users, and we also use this scheme to expand the personal information flow of the social networking site. Another way to expand the flow of personal information is to use the expansion technique of the follow library described later in this section.
2. Follow and follower list expansion through sharding
While the methods for extending the feed are fairly straightforward, extending the "lists" of ordered collections such as follow and follower lists is not so easy. The vast majority of these ordered sets are short (eg 99.99% of users on Twitter have less than 1,000 followers), but there are also a small number of users with very large lists who follow a very large number of people or have a large number of followers. fan.
From a practical point of view, a reasonable approach is to set an upper limit on the number of users and groups that can follow (for example, ordinary users of Sina Weibo are allowed to follow a maximum of 2,000 users). However, although this method can control the number of followers of a user, it still cannot solve the problem of too many followers of a single user.
In order to deal with the situation where the list of followers and fans becomes very large, we need to divide the ordered set that implements these lists into multiple shards. Multiple parts, in multiple zsets. To do this, we need to implement specific shard versions for the ZADD command, the ZREM command, and the ZRANGEBYSCORE command.
The difference from information flow sharding is that the object of this sharding is data instead of keys. In addition, in order to reduce the number of connections created and called by the program, it would be a very meaningful practice to place the data of followers and followers in the same shard. So this time we will use a new method to shard the data.
In order to store both data in the same shard when the follower and follower data are sharded, the program will use the IDs of both followers and followers as one of the parameters to find the shard key.
Summarize
This chapter reviews a variety of programs and describes some ways to scale them to handle more read and write traffic and get more memory available, including using read-only slave servers, using slave servers that can execute write queries. Servers, using sharding, and using classes and functions that support sharding. Although these methods may not fully cover all the problems a reader may encounter when extending a particular program, each of the techniques shown in these examples can be broadly applied to other scenarios.
This article hopes to convey to the reader the concept that scaling any system is a challenging task. But with Redis, we can scale the platform to the scale we want using a number of different methods.
http://it.dataguru.cn/article-9284-1.html