Ali sister REVIEW: Cache is one of the essential means to build high-performance concurrent systems, often used to resolve performance bottlenecks is essential knowledge programmer, but also interview the necessary test sites.
Although, product manager for high probability not concerned about system performance, but the programmer must think about the amount of concurrent users and the amount of system hosted in the realization of the demand. Cache is mainly used to solve the problem of performance bottlenecks, once misused it would lead to a system crash. Today, we summarize the cache relevant theoretical knowledge by means of systematic 4W.
With the rapid iteration of users and Internet traffic surge, application architecture need to constantly adjust or even reconstructed to adapt to the rapid development of this business. When the amount of data growing rapidly, the more complex business logic, increasing service link and so on a range of issues, will lead to RT too long, gradually increase the service performance needs to meet better user experience. In optimizing the system architecture of the two methods commonly used for scale up and scale out, scale out is known as horizontal scaling, application service designed to statelessness, you can easily expand the level of decomposition pressure by increasing access to hardware. Scale up the single service lifting sucked link performance, and to enhance system throughput QPS. In the pursuit of better performance, most of the business scenario is read write many cases less, usually resolved through the introduction of cache.
1. What-- What is cache?
As can be seen from the definition of so-called cache must exist for a copy of the existing data, can also be seen using cached for quick access to the scene to solve the data (read data). In the existing Internet applications, the use of cache is a key technology which can enhance the service to respond quickly, but also product manager to attend to the non-functional requirements, it is necessary in the design of technology solutions to the business scene, has a forward-looking assessment of the , decided to introduce the need for technical architecture caching solution that this non-functional requirements.
The actual presence of the cache in the computer field many cases, such as a CPU cache is to solve the data read operation speed of the CPU and memory imbalance problem, the CPU operation speed much faster read and write speed of the memory, the CPU waits for data to reduce read time, the introduction of L1 / L2 / L3 multi-level cache in the CPU.
Another example is the Linux file cache, in fact we are in programming, will talk to the data memory address, but we are in contact with the virtual address rather than real physical address, the computer's memory management unit (MMU) and page table It places the virtual address into a physical address. In the field of computer hardware in case you have a lot of application caching, in fact, software architecture design on the cache will be borrowed from many traditions of thought and mature design of computer hardware cache.
Why 2. Why-- need to use caching?
Software services can get the trust of users, and will bring the value of products users can solve the target user's pain point it was decided to issue a beginning user will decide to use, or "growth hacker" referred to a product can bring to user "aha moment", and decided to use as well as high-frequency users will continue to use, the user experience is considered to be the key factors of software products to enhance user stickiness.
2.1 What is the user experience
The user experience is to define and promote professional needs to be extended to the 1990s, popularized by Donald Norman sermon. User experience is being taken seriously in the field of human-computer interaction, and once and three traditional usability metrics (ie efficiency, effectiveness and basic satisfaction) par.
ISO 9241-210 standard user experience as the official definition: people are using or expect to use the product, cognitive impression and response systems or services. It can be seen on the subjective perception of the user experience is a software product for users, particularly contains the user before use and after use, the use of emotions, preferences, cognitive impression, psychological reactions and emotional expression, and so many subjective feelings, every a user perspective on subjective feelings of different products, different concerns, will result in software products allow most users are able to get a good user experience itself is a very challenging thing.
Most in the industry, the user experience is divided into three categories: user status, software products and environmental performance, user status and environment (user environment as well as products in similar products in the product environment) of these two factors need to fields of professional interaction design and research with other students to overcome, software developers will need to address the problem of system performance. For users, the most basic needs is in the use of software services, software products provide timely services, which is commonly referred to in the course of ongoing Loading (turn chrysanthemum) will lead to poor user experience, timeliness is also a minimum requirement of system performance content.
The problem of system performance, is the product manager to attend to a point, but also non-functional requirements, require developers to put in some effort to think of places. Indicators to assess system performance are many, in the case of the premise to enhance the user experience, we need to focus on performance metrics, what does?
2.2 common performance indicators
Several common indicators in the design of software architecture needs attention: response time, latency, throughput, number of concurrent users and resource utilization.
1) System Response time: the response time is the time to make a request to the user system in response to the different functions of different length links, the same function and result in a different response times in different amounts of data such as these will. Therefore, when measuring the system response time, average response time is usually concerned about all the features of software products and the maximum response time.
2) a delay time: When discussing the system response time, more fine-grained partitioning can be divided into:
Client content rendered in receiving data "present time";
The server receiving a user request to the server, and the server returns the data to the client both processes involved: the network transmission time and the delay time of the application. Application delay time that is the time when the server executes the entire service link it takes, but also performance optimization is important to reduce this time.
3) Throughput: Throughput refers to the number of requests can be processed per unit of time, for non-concurrent applications, the throughput and response time is inversely proportional to the request, a service delay is longer the lower system throughput.
4) number of concurrent users: the number of concurrent users refers to the number of users the system can simultaneously carry normal use of system functions, compared to the throughput, this indicator is more general but for people in the field of non-software is easier to understand.
5) resource utilization: resource utilization reflects the resource occupied for some time.
2.3 caching benefits of
In the pursuit of better optimize the experience, the objective is the need to constantly improve the performance of these indexes above, the optimal solution system experience closer and closer. Cache in the end what kind of advantage, we should spend a lot of effort to design a structure are well suited to the current business scenario cache it?
1) greatly enhance the user experience software
软件产品主要围绕两个核心问题，一是解决目标用户的痛点问题，二是提升产品黏性。在提供软件服务时，抽象的来看是解决数据在整个链路上的流转问题，如何让数据流转更加高效、更加顺畅是在实现时着重关注的地方，事实上，无论是浏览器、负载均衡、应用服务器还是数据库等等各个环节都会应用到缓存，当数据离用户“更近”，比如数据副本在客户端上，也就意味着请求能够很快的进行响应，相应的给用户进行数据呈现的耗时就更短。现如今用户爸爸们“日理万机”，如果一个软件产品不能在很短时间就获取用户的注意力，很大可能性就意味着失败。因此，使用缓存能够让用户从主观上获取更优的用户体验。
2）提升吞吐量
试想，如果在服务链路上，请求能够在缓存中获取服务数据的话，也就意味着很多数据并不需要从源应用服务器进行获取，降低了源服务器网络传输的频率，在一定IDC带宽下，系统能够降低网络传输时间以及应用延迟时间，从而支撑更多的系统访问以提升系统整体吞吐量以及并发用户数，硬件的使用效率也会明显提升。
从实际场景下，在系统性能优化时大概率会优先选择使用缓存进行系统优化，也是一种被证明有效的手段，缓存也被认为是一种“空间换时间”的艺术。
3. Where——缓存存在链路中的哪些地方？
3.1 缓存分类
从一个请求到最终获取响应，会经过很多环节，缓存可以几乎存在整个链路的每个节点。缓存按照不同的维度可以有如下分类：
1）缓存所处链路节点的位置：
客户端缓存
网络缓存
服务端缓存
2）缓存架构部署方式：
单机缓存
缓存集群
分布式缓存
3）缓存的内存区域
本地缓存/进程内缓存
进程间缓存
远程缓存
按照缓存在服务链路上的位置来划分，可以系统性的梳理下缓存的不同应用。
3.2 客户端缓存
客户端缓存是离用户“最近”的一种存储介质，经常和网络测和服务端缓存一起配合使用，常见的客户端缓存有如下几种：
1）页面缓存：页面缓存是指将静态页面获取页面中的部分元素缓存到本地，以便下次请求不需要重复资源文件，h5很好的支持的离线缓存的功能，具体实现可通过页面指定manifest文件，当浏览器访问一个带有manifest属性的文件时，会先从应用缓存中获取加载页面的资源文件，并通过检查机制处理缓存更新的问题。
2）浏览器缓存：浏览器缓存通常会专门开辟内存空间以存储资源副本，当用户后退或者返回上一步操作时可以通过浏览器缓存快速的获取数据，在HTTP 1.1中通过引入e-tag标签并结合expire、cache-control两个特性能够很好的支持浏览器缓存，关于浏览器缓存更为细节的知识可以查看该文章。
3）APP缓存：APP可以将内容缓存到内存或者本地数据库中，例如在一些开源的图片库中都具备缓存的技术特性，当图片等资源文件从远程服务器获取后会进行缓存，以便下一次不再进行重复请求，并可以减少用户的流量费用。
客户端缓存是前端性能优化的一个重要方向，毕竟客户端是距离“用户”最近的地方，是一个可以充分挖掘优化潜力的地方。
3.3 网络缓存
网络缓存位于客户端以及服务端中间，通过通过代理的方式解决数据请求的响应，降低数据请求的回源率。通常具有如下几种形式的网路缓存：
1）web代理缓存：常见的代理形式分为分为：正向代理、反向代理以及透明代理。web代理缓存通常是指正向代理，会将资源文件和热点数据放在代理服务器上，当新的请求到来时，如果在代理服务器上能获取数据，则不需要重复请求到应用服务器上；
2）边缘缓存：和正向代理一样，反向代理同样可以用于缓存，例如nginx就提供了缓存的功能。进一步，如果这些反向代理服务器能够做到和用户请求来自同一个网络，那么获取资源的速度进一步提升，这类的反向代理服务器可以称之为边缘缓存。常见的边缘缓存就是CDN（Content Delivery Network），可以将图片等静态资源文件放到CDN上。
3.4 服务端缓存
服务端缓存是后端开发中进行性能优化的发力点，常见的后端性能优化也是通过引入缓存来进行解决，常见的有数据库的查询缓存、缓存框架以及引入应用级缓存。
3.4.1 数据库查询缓存
例如，MySQL的缓存机制是通过将SELECT语句以及相应的ResultSet进行缓存，当后续接受到SELECT请求后，如果MySQL已经开启了Query Cache功能，会将SELECT语句以字符串的方式进行hash，然后去从缓存中进行查询，如果查询出数据，则直接进行返回，省去了后续的优化器以及存储引擎IO的操作，能够极大的提升响应时效。如何优化Query Cache需要从如下几个指标上进行考虑：
query_cache_size：设置能够缓存ResultSet的内存区域大小
query_cache_type：表示使用缓存的场景。0表示任何场景下都不使用Query Cache，1表示显式指定不使用Query Cache的查询都可以使用，2（DEMAND）表示只有明确指示使用Query Cache才会生效；
Qcache hits：表示多少次查询命中Query Cache
Qcache inserts：表示多少次没有命中Query Cache而插入数据
Qcahce lowmem prunes：表示多少条Query引入空间不足而被清除
Qcache free memory：表示剩余内存大小
Qcache free blocks：该值很大表示内存碎片很多，需要及时清理
在进行Qcache优化时，可以对以上指标综合进行分析，比如了解Qcache的缓存命中率 = Qcache hits/ Qcache hits + Qcache inserts，来判断当前Qcache的效率。也可以结合Qcahce lowmem prunes、Qcache free memory以及Qcache free blocks来判断当前Qcache的内存使用效率。
另外，如果使用Innodb存储引擎的话，也需要着重关注innodb_buffer_pool_size参数，该参数决定了innodb的索引以及数据是否有足够大的空间放入到缓存中。table_cache决定了能够缓存表的最大数量，也是需要关注的一个参数。
3.4.2 缓存框架
在功能开发时，会常用提供缓存特性的缓存框架或者实现缓存功能的类库来高效的完成开发，常见的缓存框架有Ehcache、Guava等，这些缓存框架配置简单，能够简单灵活的使用。这些开源的缓存框架不仅支持单机的本地缓存还能配置集群的方式达到灵活伸缩。
3.4.3 应用级缓存
当缓存框架不能满足需求的时候，就需要引入应用级缓存，比如Redis、MongoDB等NoSQL数据库，应用级缓存具备高可用性以及伸缩性的分布式架构能够支撑业务需求，当然，做好一款应用级缓存产品其中的挑战也是巨大。
4. When——什么时候需要使用缓存？
缓存不是架构设计的必选项，也不是业务开发中的必要功能点，只有在业务出现性能瓶颈，进行优化性能的时候才需要考虑使用缓存来提升系统性能。也不是所有的业务场景都适合使用缓存，读多写少且数据时效要求越低的场景越适合使用缓存，缓存并不是所有性能问题的灵丹妙药，如果滥用缓存反而会成为毒药，并且会引入维护缓存的操作成本，使得系统复杂度更高不利于维护。
另外把缓存当做存储来使用是一件极其致命的做法，这种错误的认识，将缓存引入系统的那一刻起就意味着已经让系统走上了危险的局面，对缓存的使用边界要有深刻的理解，才能尽可能保证做出引入缓存才是一个正确的决定。
When making a cache structure design, there are many points to consider:
1) the order of traffic and application scale: For applications where low concurrent low flow, the introduction of cache and will not result in significant performance improvements, but will bring the complexity of the applications, and high operation and maintenance costs. Nor is any need to use cached data, such as images and video files using a distributed file system cache instead of more appropriate. Therefore, before the introduction of the cache, you need to assess the current flow of traffic, the introduction of a large cache of high concurrent flow of business scenarios relatively earnings will be higher;
Select 2) cache applications: There are a lot of cache application such as Redis, Memcached and tair, etc., as well as the scope for the advantages and disadvantages of each application distributed cache memory efficiency, operation and maintenance costs and even knowledge structure team developers are required understanding, in order to better technology selection;
3) properly assess the impact of factors cache: Before the introduction of the cache, you need to focus on assessing value size, cache memory, the peak QPS, the expiration time, cache hit rates, read and write update policy, consistent key value distribution routing policy, expiration policies, and data programs, and so on a number of factors to be aware of;
4) high availability cache architecture: distributed cache for high availability, which is a distributed system, the pursuit of three high indicators, the cache cluster design, master-slave synchronization scheme design, etc., only the cache reliable enough to serve business system, bring value to the business;
5) improve the monitoring platform: When the cache into production environment, you need to have a monitoring system capable of observing the operation of the explicit caching system in order to identify problems earlier, while the forecast for the unexpected lack of hot data, also need hot unexpected discovery system to solve the problem of hot data cache.
6) cache recently principles: the cached data on the core idea of the nearest place users will no doubt greatly enhance the speed of response, which is multi-level cache design.
Original release time: 2019-11-25
Author: Ali technical
article from Yunqi community partners, " Ali technology " for information may concern "Ali technology."