Hello everyone, I am Yi An! Today let's talk about how the cache should be designed.
what is cache
Caching is a way of temporarily storing data. When a user queries data, the system will first search in the cache. If the data already exists in the cache, it will be used directly, otherwise the system will search for the original location of the data. Therefore, caching is essentially a technology that trades space for time, and improves the speed of data access through the repetition of data in space.
However, with the continuous development of distributed and cloud computing technologies, data storage technologies are also constantly changing. Different storage technologies vary widely in price and performance. Therefore, when designing software, if a multi-level cache system is not properly designed, not only will it waste money, but it will also be difficult to achieve the desired performance benefits.
when to use cache
Let's start with two questions to understand when the cache design should be done and how to make a comparative analysis based on the characteristics of different data types to achieve an excellent cache design.
-
In Internet application services, is the purpose of cache technology only to improve access speed?
In fact, cache technology plays an important role in system-level performance design in distributed systems. For example, when the load of a database is close to the system bottleneck, we can distribute the load to other databases through caching technology to achieve load balancing. Therefore, the purpose of using caching is not only to improve access speed.
-
In a large system, is it necessary to design a caching mechanism for each data type?
Actually not needed. In actual business scenarios, there are many types of business data contained in the system, and it is impossible to design and implement a caching mechanism for each data type. Therefore, we need to identify which data accesses have a greater impact on performance to determine the data types that need to use the caching mechanism.
How to identify the data types that need to use the caching mechanism? Generally, we need to design a method for performance modeling, that is, identify which data access operations have the most critical impact on performance through analysis and evaluation methods, and then perform cache design.
After determining the data types that need to use the cache mechanism, you may find that the characteristics of these data types are very different. If you use the same cache design, it is difficult to achieve the best software performance.
Therefore, here, I summarize three types of data that need to use the caching mechanism: immutable data, weakly consistent data, and strongly consistent data. Understanding the differences between these three data types and the corresponding cache mechanism design methods can help you achieve an excellent cache design.
immutable data
The first is immutable data, which means that data will never change, or will not change for an extended period of time. Therefore, we can use this type of data as a data type that gives priority to caching technology, which is also very common in actual business scenarios. For example, static web pages and static resources in Web services, the mapping relationship between column data and keys in database tables, and business startup configurations can all be regarded as immutable data.
In addition, immutable data also means that it is very easy to achieve distributed consistency. We can choose any data storage method for these data, and we can also choose any storage node location. Therefore, when implementing the caching mechanism, we can adopt a flexible and simple way. For example, in the Java language, you can directly use in-memory Caffeine or built-in data structures as caches.
It is worth noting that when designing a cache for immutable data, you can choose permanent non-expiration or time-based invalidation as the cache invalidation mechanism. If you use the time-based invalidation method, you also need to make a trade-off between cache capacity and access speed according to specific business needs.
weakly consistent data
The second data type is weakly consistent data, which means that the data changes frequently, but the business does not require high data consistency. In other words, the data seen by different users at the same point in time may not be completely consistent, but this is acceptable.
Since this type of data has low requirements for consistency, when designing a caching mechanism, we only need to achieve eventual consistency. In actual business, this kind of data is very common, such as the historical analysis data of the business and the data returned by some searches. Even if some recent data is not recorded, it will not affect the business.
In addition, there is another way to quickly identify this type of data, and that is to use the data read from the database Replica (copy) node. The data of most replica nodes of the database does not meet the strong consistency requirements, so these data can be considered as weakly consistent data.
For weakly consistent data, we usually use a time-based cache invalidation mechanism. Due to the nature of weak consistency, you can choose various data storage technologies, such as memory cache or distributed database cache. You can even design a multi-level caching mechanism based on load balancing scheduling.
strongly consistent data
The third cache data type is strongly consistent data, which refers to data that changes frequently, and the business requires very high consistency of the database. In other words, the data any user sees anywhere in the system should be up to date.
我不建议在针对这种类型的数据使用缓存机制，因为这会引入新的问题并增加复杂度。例如，用户可以直接提交和修改数据，如果没有同步修改缓存中的数据，就会导致数据不一致性，从而导致严重的业务故障。
但在某些特殊的业务场景中，如果有个别数据访问频率非常高，我们仍需要通过缓存机制来提升性能。在设计缓存机制时，需要注意以下两点：
-
缓存的实现方式必须 采用修改同步。换句话说，所有的数据修改都必须同步更新缓存和数据库中的数据。 -
需要确定特定业务流程中缓存数据可以 使用多长时间。例如，某些缓存数据只能在单个业务流程中使用，不能跨业务流程使用。
需要注意的是，使用缓存目的是为了性能优化，因此需要使用评估模型来分析缓存是否达到了性能优化的目标。
评估模型公式为：AMAT = Thit + MR * MP，其中：
-
AMAT（Average Memory Access Time）代表平均内存访问时间； -
Thit指的是命中缓存后的数据访问时间； -
MR指访问缓存的失效率； -
MP指缓存失效后，系统访问缓存的时间与访问原始数据请求的时间之和。
注意，AMAT与原始数据访问之间的差异代表使用缓存所带来的访问速度提升。在一些不当的缓存使用场景中，增加的缓存机制可能会降低数据访问速度。因此，接下来我将通过真实的缓存设计案例，带你理解如何正确使用缓存，以提升系统性能。
典型场景
好，在开始介绍之前呢，我还想给你说明一下，在真实的业务中，缓存设计的场景其实有很多，这里我的目的主要是让你明确缓存设计的方法。因此，我会从两个比较典型的案例场景入手，来带你理解缓存的使用。
如何做好静态页面的缓存设计？
在Web应用服务中，一个重要的应用场景就是静态页面的缓存使用。这里的静态页面是指一个网站内，所有用户看到的都是一样的页面，除非重新部署否则一般不发生变更，比如大部分公司官网的首页封面等。
通常静态页面的访问并发量是比较大的，如果你不使用缓存技术， 不仅会造成用户响应时延比较长，而且会对后端服务造成很大的负载压力。
那么针对静态页面，我们在使用缓存技术时，可以通过将静态缓存放到距离用户近的地方，来减少页面数据在网络上的传输时延。现在，我们来看一个针对静态页面使用缓存设计的示意图：
如图上所示，针对静态网页，首先你就可以在软件后端服务的实例中使用缓存技术，从而避免每次都要重新生成页面信息。然后，由于静态网页属于不变性数据，所以你可以使用内存或文件级缓存。另外，针对访问量非常大的静态页面，为了进一步减少对后端服务的压力，你还可以将静态页面放在网关处，然后利用OpenResty等第三方框架增加缓存机制，来保存静态页面。
除此之外，在网页中很多的静态页面或静态资源文件，还需要使用浏览器的缓存，来进一步提升性能。
注意，这里我并不是建议你针对所有的静态页面，你都需要设计三层的缓存机制，而是你要知道，在软件设计阶段，一般就需要考虑如何做静态页面的缓存设计了。
设计数据库多级缓存
还有一个典型的缓存场景是针对数据库的缓存。现在的数据库通常都是分布式存储的，而且规模都比较大，在针对大规模数据进行查询与分析计算时，都需要花费一定的时间周期。
因此，我们可以先识别出这些计算结果中可以使用缓存机制的数据，然后就可以使用缓存来提升访问速度了。下面是一张针对数据库缓存机制的原理图：
从图上我们可以看到，内存级Cache、分布式Cache都可以作为数据计算分析结果的缓存。而且，不同级的缓存访问速度是不一样的，内存级的Cache访问速度可以到微秒级别，甚至更好；分布式Cache访问速度通常可以小于毫秒级别；而针对原生数据库的查询与分析，通常是大于毫秒级别的。
因此，在具体设计缓存机制的时候，你就需要依据前面我介绍的缓存使用原理，识别出数据类型，然后选择并设计缓存实现机制。
另外，在使用缓存机制实现访问速度优化的过程中，我们的 主要关注点是不同层级缓存所带来的访问速度提升，而在这里，不同层级缓存也是可以在一个数据库中的。比如，在我参与设计的一个性能优化项目中，其Cache策略就是，使用MongoDB中的另外一个Collection（集合），来作为缓存查询分析，以此优化性能。
所以，你在做缓存设计时，关注点应该放到不同的数据种类，以及不同层级缓存的性能评估模型上，而不是只关注数据库。只有这样，你才能设计出更好、更优的性能缓存方案。
总结
本文重点介绍了缓存技术的使用原理和典型应用场景，当你在进行软件业务系统性能设计时，可以结合本文内容，识别出系统中各种可缓存的数据类型，然后有针对性地设计缓存方案，并且还可以根据评估模型，来进行前期的性能验证分析。
另外，在具体的缓存技术实现中，比如缓存替换算法、缓存失效策略等，通常这部分能力是内置于缓存库的配置选项当中的，或者选用第三方库即可，需要自定义设计算法的实现场景很少。所以这里你需要重点做的，就是选择合适的配置和策略即可。
本文由 mdnice 多平台发布