I talked roughly about caching in the article "Thinking about Performance Tuning". Here is another article and combined with an example written by myself to talk about the cache writing and elimination strategy. There are many usage scenarios for the cache. Here you can combine the idea of computer hardware layering from the register -> Cache (L1, L2, L3) -> main memory -> disk to know the purpose of the cache. The most basic scenario of general web applications is to store and retrieve data. For example, the storage medium is a database. The database is not all disk IO. The database itself will also open up a space in the memory to store some hot data. In a word: the memory cache is to use the characteristics of the memory itself to speed up access to data. However, a very fatal part of the memory is its small capacity and it is impossible to put everything in the memory. In this way, there is a caching strategy. In the final analysis, it is to improve memory utilization (hit rate). Revolving around the basic fact of limited memory, if we need to improve memory utilization, the general memory cache will have an elimination strategy.
There are generally two elimination strategies:
one. Access the bottom out of
this strategy is based on such a hypothesis: The more data being accessed is reason to believe a greater probability that it is next accessed.
two. Time out of
time is out of the data clean-up within a fixed period of time. This is also to improve memory utilization, because the last elimination strategy is to eliminate the old data only when new data comes in, and sometimes most of the data in the memory is actually useless data, and timely cleaning is also possible. Improve the utilization of the entire memory.
Here are two application scenarios, and then analyze the corresponding strategies in each application scenario.
One. Data average scene
this scene does not tell the whole memory access is very concentrated in a piece, like the cache hit rate under this scenario will be relatively low, in fact, not very good memory cache to play the role. This is better understood. If there are a total of 1 million data, the cache can only hold 10,000 data, so if 1 million data will be accessed on average, the probability of each data being hit is only 1/100. This scenario may be better with time elimination strategy. Because the assumption of the last elimination is no longer valid here.
two. Hot data scene
this scene is actually very common, and this scene is the best play-memory cache performance. Generally, as long as the elimination strategy is good in this scenario, the overall application performance can be well improved.
To sum up, the two elimination strategies here will basically be mixed and matched. Okay, let's discuss it with an example I wrote specifically.
Here briefly describe my overall thoughts:
First: with ordinary HashMap as a container
with a time out of the policy, and supports hot data front: second.
Third: The idea of "bucketing", the first two points are easy to understand. It is estimated that some people will ask questions. I will elaborate on
a). The so-called idea of bucketing is actually that a large container will be divided into multiple smaller ones. container. Then use a certain strategy to distribute to different "buckets". In fact, some features of distributed caching are applied here. The core idea is to allow each small bucket to operate independently without interfering with each other. When I talk about the code below, I will specifically talk about why bucketing is needed, what problems are being solved, and some drawbacks of
bucketing b). I used the simplest modulo operation for the bucket allocation strategy.
Start to code,
/**
* @author sanxing 2012-11-3 下午10:33:30 本缓存旨在解决数据在内存区块缓存策略
*
* <p>
* 利用HashMap作为最基本容器进行存储
* <p>
* <p>
* 采用时间淘汰策略，并支持热点数据前置(更新其时间)。
* </p>
* <p>
* "分桶"的思想
* <i>
* 所谓分桶的思想其实就是一个大容器中会切分成多个小容器。然后用运用一定的策略进行分发到不同的
* "桶"中，其实这里应用了分布式缓存的一些特性，其核心思想旨在让每个小桶能够独立运作
* ，互不干扰。这样当桶清理(@see remove())的时候会保证达到影响最小。
* </i>
* </p>
*
* <p>
* 本缓存策略是建立在数据均匀分配的一个假设上，也就是说海量的数据会平均分配到各个桶上，这样才能保证每个桶的利用率达到最高，性能最好。
* 如果出现集中式缓存也就是数据集约到其中几个桶上，而其他桶的拿不到数据，不能使用此缓存或者需要改变数据的分发策略。
* </p>
*
*/
public class LRUCache {
/**
* 缓存过期时间
*/
private long aliveTime;
/**
* 桶的最大容量
*/
private int maxSize;
/**
* 每个小桶的容量
*/
private int everyPoolSize;
/**
* 命中次数
*/
private int hit;
/**
* 丢失次数
*/
private int missHit;
private Map<Integer, Map<String, CacheObject>> cacheMap;
/**
* 小桶的个数
*/
private int mod = 32;
public int getHit() {
return hit;
}
public int getMissHit() {
return missHit;
}
public void setMod(int mod) {
this.mod = mod;
}
public int getMod() {
return mod;
}
public int getEveryPoolSize() {
return everyPoolSize;
}
public LRUCache(int maxSize, long aliveTime) {
this.maxSize = maxSize;
this.aliveTime = aliveTime;
cacheMap = new HashMap<Integer, Map<String, CacheObject>>();
for (int i = 0; i < mod; i++) {
Map<String, CacheObject> sonMap = new HashMap<String, CacheObject>();
cacheMap.put(i, sonMap);
}
/**
* 这里并不是准确的，因为当不是mod的倍数的时候会多出<mod的个数，举一个简单的例子：假如maxSize=33,mod=32,那么
* 每个小桶的poosize=2,这样加起来是64，所以并不精准。 <br />
* 如果需要准确的可以寻求其他策略
*/
this.everyPoolSize = (this.maxSize + mod - 1) / mod;
}
public void put(String key, Object val) {
// 如果小于最大数量
int keyMod = getKeyMod(key);
Map<String, CacheObject> sonMap = null;
synchronized (sonMap = cacheMap.get(keyMod)) {
if (sonMap.size() >= everyPoolSize) {
/**
* 如果发现桶已经满了时候，触发remove对整个桶进行清理，有木有很像JVM中的FGC啊
* 其实基本思想就是这样，但是由于HashMap并非线程安全的，所以必须对桶进行加锁
* 而本身清理是有一定耗时的，这样势必降低缓存的访问。所以必须降低remove事件的
* 发生，也就像JVM需要减少FGC一样。这里会有许多策略。
* 而分桶的作用就出现了，这样触发remove事件，只需要锁住小桶，而其他小桶并不会
* 因此受到影响仍然可以访问。这里也是遵循锁运用基本原则：只在需要的地方加锁。
*/
remove(keyMod);
}
if (sonMap.size() < everyPoolSize) {
sonMap.put(key,
new CacheObject(val, System.currentTimeMillis()));
}
}
}
/**
* 最简单的取模数，进行桶的分发策略
*
* @param key
* @return
*/
private int getKeyMod(String key) {
return (key.hashCode() & Integer.MAX_VALUE) % mod;
}
/**
* 负责对桶进行清理工作，把所有已经过期的数据全部移除出缓存区，这项工作还是比较耗时的。
*
* @param keyMod
*/
public void remove(int keyMod) {
synchronized (cacheMap.get(keyMod)) {
Map<String, CacheObject> oldMap = cacheMap.get(keyMod);
Map<String, CacheObject> newMap = new HashMap<String, CacheObject>();
for (Map.Entry<String, CacheObject> entry : oldMap.entrySet()) {
if ((System.currentTimeMillis() - entry.getValue().getTime()) < aliveTime) {
newMap.put(entry.getKey(), entry.getValue());
}
}
oldMap = null;
cacheMap.put(keyMod, newMap);
}
}
/**
* 拿到缓存区的数据，这里的策略会先判断数据是否过期，过期则移除
*
* @param key
* @return
*/
public Object get(String key) {
int keyMod = getKeyMod(key);
Map<String, CacheObject> map = null;
synchronized (map = cacheMap.get(keyMod)) {
CacheObject co = map.get(key);
if (co != null) {
if ((System.currentTimeMillis() - co.getTime()) > aliveTime) {
// 过期则移除，这里类似于懒加载，和JVM里的YGC有点类似
map.remove(key);
missHit++;
} else {
// 没有过期
hit++;
/**
* 热点数据策略，当数据被访问时更新它的时间以延长它在内存中驻留的时间。 保证下次能够顺利命中。
*/
co.setTime(System.currentTimeMillis());
return co.getObj();
}
}
}
return null;
}
/**
* 获取整个数据区的数据容量
*
* @return
*/
public int size() {
int allSize = 0;
// 这里不加锁，不保证得到的size一定准确
for (int i = 0; i < mod; i++) {
allSize += cacheMap.get(i).size();
}
return allSize;
}
/**
* 拿到每个桶的容量。
*
* @param mod
* @return
*/
public int poolSize(int mod) {
if (mod < 0 || mod > this.mod) {
throw new IllegalArgumentException("mod 参数不在桶的区域内,mod=" + mod);
}
return cacheMap.get(mod).size();
}
/**
* 缓存数据结构
*
* @author sanxing 2012-11-4 下午01:48:47
*
*/
private class CacheObject {
private Object obj;
private long time;
public CacheObject(Object obj, long time) {
this.obj = obj;
this.time = time;
}
public Object getObj() {
return obj;
}
public void setTime(long time) {
this.time = time;
}
public long getTime() {
return time;
}
}
} The above is the entire code and some comments. The most difficult thing is that the data is distributed to each bucket evenly, which is the distribution strategy to ensure the maximum utilization of the cache. This requires actual testing for testing. I did the simplest modulo operation here. Statistically speaking, when the data is massive and disorderly, it should basically conform to the principle of equal distribution.
The whole strategy and thinking is over. After a period of thinking, I found that the entire computer system and even other non-computer scenarios will follow a similar strategy. This seems to be the charm of thinking. It is very interesting to be able to integrate the whole computer to think and diverge, and sometimes it even feels awkward.