Java performance optimization to create a billion-level traffic spike system: 4. Query performance optimization: multi-machine caching

4. Query performance optimization: multi-machine caching

1. Learning objectives:

In the previous section, through the horizontal expansion of the server, deploying the application on multiple machines to process requests greatly improved the TPS, but these multiple databases still use the same machine's database.
Next, some query optimization techniques will be used to complete the query optimization solution for the product detail page.

• Grasp the definition of multi-level cache
• Master redis cache, local cache
• Master the hot nginx lux cache

2. Overview of cache design principles

• Cache to use fast access equipment, memory
• Push the cache to the nearest place to the user to reduce network latency
• Dirty cache cleaning, after the database is changed, the old data in the cache is 脏缓存how to clean up, the cleaning strategy

** Multi-level cache **

• redis cache
• Hot data, local memory cache
• nginx proxy cache cache
• nginx lua cache

Three, Redis centralized cache introduction

See above for installation, principle, session management, and token management.
Database in key-value format.
memorycatch is a complete memory database.
Redis can flush data to disk, allowing a certain amount of loss. It is generally regarded as volatile storage.

Why centralized caching?

After the application server is horizontally expanded, it is connected to the same redis server.

1. Stand-alone version

Failure bottleneck, upper capacity limit

2.sentinal sentinel mode

redis2 is the slave backup redis of redis1, and the changes on redis are synchronized to redis2 for backup.
But when the party redis1 hangs, it should be that the application server should automatically request services from redis2, but because the concurrent environment is too complicated, it is difficult for the application server to perceive that redis2 is down. Hence the sentinel mode.

When using it for the first time, the application server first asks sentinel, which redis should be used, sentinel replies and informs, and the application server then requests redis.
The sentinel tool establishes a heartbeat mechanism with redis. Any situation that causes the heartbeat to be disconnected, sentinel considers it to be a failure of redis1. Send a command to change redis2 to master and redis1 to slave, and notify the application server to change (as shown in the figure below).

3. Cluster cluster mode

The shortcomings of the sentinel mode are also obvious. At the same time, only one redis provides services to the outside world.
Before cluster, use fragmentation and read-write separation:

But when redis5 is added, various complicated data migrations are required. The scalability is still poor.
Therefore, the cluster mode appeared:
redis 2 reads and 2 writes. And automatically elect master and slave. And every redis knows all the relationships. And send this relational routing table to the application server, and the application server maintains the fragmentation information in its own memory.
When redis3 fails, the status is automatically redistributed through the Parker mechanism, and the hash fragments will be automatically readjusted.
But there is still an incorrect fragment routing list in the application server. When the application server finds some data requested by redis2 according to the error list, redis2 finds that the key requested by the server does not belong to the management category after adjustment, and returns a reask request to the server to let the server re-pull the latest fragment routing table.


These three models have been jedisrealized.

Fourth, Redis centralized cache product details page

At the controller layer, the detailed information is cached, and no downstream service calls are made. Reduce dependence on the database.

But the garbled code after being serialized by redis itself

The default key value serialization method has been changed to make it easier to read.

Five, local hotspot cache

That is, the local of the JVM server

• Hot data
• Dirty reads are very insensitive. Once the database is changed, a lot of dirty cache data will be read without knowing

It can be updated through MQ, but the gains outweigh the losses.

• The memory is controllable and precious

It is only used as an instantaneous access, so the effective time of the cache is very short.

Must support concurrent reading and writing. A simple hashmap cannot be satisfied. It can also be eliminated by strategies such as LRU, and the KEY is automatically invalidated according to time.

achieve:

Guava cache

• Controllable only size and timeout
• Configurable lru strategy
• Thread safe

@Service
public class CacheServiceImpl implements CacheService {
    
    

    private Cache<String,Object> commonCache = null;

    @PostConstruct
    public void init(){
    
    
        commonCache = CacheBuilder.newBuilder()
                //设置缓存容器的初始容量为10
                .initialCapacity(10)
                //设置缓存中最大可以存储100个KEY,超过100个之后会按照LRU的策略移除缓存项
                .maximumSize(100)
                //设置写缓存后多少秒过期
                .expireAfterWrite(60, TimeUnit.SECONDS).build();
    }

    @Override
    public void setCommonCache(String key, Object value) {
    
    
            commonCache.put(key,value);
    }

    @Override
    public Object getFromCommonCache(String key) {
    
    
        return commonCache.getIfPresent(key);
    }
}

Increase 1000TPS.

六、Nginx proxy catch

• Proxy catch can only be used when nginx reverse proxy is pre-enabled
• Rely on files at the index level of the file system to complete the cache operation
• Rely on memory to cache file addresses

But because it stores the cache in the local disk system instead of in the memory, the QPS has not been significantly improved, and it is even lower than the performance of the jvm local cache. The average delay is longer.

Seven, Nginx Lua principle

• Lua coroutine mechanism

In the thread space station, relying on the thread, according to the user simulation

• Nginx coroutine mechanism

No need to consider the asynchronous programming model

• nginx lua plug-in point
• OpenResty

1. Coroutine mechanism

• Depends on the thread memory model, low switching overhead
• In case of blockage, the right of execution will be returned in time, and the code will be synchronized
• No need to lock

2.Nginx coroutine

• Each worker process of nginx is encapsulated into a coroutine on top of the event model of epoll or kqueue.

when

• Every request is processed by a coroutine
• Even if ngx_lua runs lua, it has a certain overhead relative to c, but it can still guarantee high concurrency.

3. Nginx coroutine mechanism

• Each worker process of nginx creates a lua virtual machine. Used to run lua files
• All coroutines in the worker process share the same vm
• Each external request is processed by a lua coroutine, with data isolation between them
• When lua code calls asynchronous interfaces such as io, the coroutine is suspended and the context data remains unchanged
• Automatically save, without blocking the work process
• After the io asynchronous operation is completed, the coroutine context is restored, and the code continues to execute. That is, the code is synchronous programming, which is relatively simple.

That is, after receiving the HTTP request, a Lua coroutine is allocated to process it, but when the reverse proxy needs to wait for the back-end server to return data, for example, it puts the socket handle in the epoll listening queue and hangs itself. Continue to execute other coroutines. When the back-end server returns, epoll will listen, and then the coroutine wakes up to process the subsequent response operations.
For this reason, the data between each coroutine is isolated.
Nginx is a worker is a thread, and there are many coroutines underneath. Serial processing is completely based on the coroutine.
When a java servlet processes an http request, it is a request and a thread is allocated to handle it.

4. Nginx processing stage

5. Nginx lua mount point

6.OpenResty

• With the help of Nginx's event-driven model and non-blocking IO, high-performance web applications can be realized.
• A large number of components such as Mysql, redis, memcache, etc. are provided to make it easier to develop web applications on nginx.

(1) hello world

That is, when visiting helloworld, the result of visiting /item/get?id=6 is returned.

(2) shared dic

Shared memory dictionary, visible to all worker processes

Memory dictionary similar to guaua catch, and supports lru elimination

But it has memory limitations.
and so

(3) Openresty redis support

Nginx is definitely slower to read from redis than from shared dic.

The update mechanism is omitted at nginx, allowing downstream application servers to update, nginx is read-only and does not write.
Nginx can perceive the data in redis in real time and can avoid dirty reading.

Therefore, nginx can only store hot data in its own memory cache, and non-hot data reads the redis slave.

8. Summary

The closer to the upstream, the more expensive the system resources occupied by the cache, the more difficult the corresponding update mechanism, and the higher the performance.
Therefore, there is no unified and universal caching scheme. It is necessary to determine the caching scheme for dynamic requests according to the actual business scenario, the degree of tolerance for dirty reads, and the degree of hot data.