In the present application development, substantially cache is standard, and is used more redis, memercache the like, there are some local cache, such hashmap, google guava bag, spring cache, etc., or a little better as a local level cache, Redis like as a secondary cache, the local cache data amount smaller
@Override
public OrderEntity selectById(Long id) {
Object object = redisUtils.get(String.valueOf(id));
if(object!=null){
return (OrderEntity) object;
}else {
return orderMapper.selectByPrimaryKey(id);
}
}The code starting with redis inquiry, if there is no query from the database, which is the normal practice.
If the amount of concurrency is very high, redis there is no data directly query the database, resulting in pressure on the database is very large. It might cause the database is very dangerous, even unusable. Once a buffer penetrate the underlying database will collapse. General mysql server 8 core 16G can withstand concurrent 2W +, 16 nuclear 96G can withstand concurrent 4.5W +, have to guard against.
1, verify the parameters in the query time, in accordance with the parameters specified actual business rule validation, not on the direct return match
2, the matching, redis not, or will query the database, the database even if the query is no data, the same cache, the cache is null, may be added an expiration time or regular cleaning is null cache, set the expiration time to ensure atomicity, preventing redis way hang.
3, frequent data query cache in advance
After transformation, the code:
@PostConstruct
public void init(){
//初始化数据，遍历所有数据将id放入redis中
}
@Override
public OrderEntity selectById(Long id) {
Object object = redisUtils.get(String.valueOf(id));
if(object!=null){
return (OrderEntity) object;
}else {
OrderEntity orderEntity = orderMapper.selectByPrimaryKey(id);
redisUtils.set(String.valueOf(id), orderEntity, 5);
return orderEntity;
}
}Initialization execution code block constructor annotated Spring provides @PostConstruct
The above code is the idea, actually certainly not write
Ahead cache data After initialization, the amount of data will be very large, for example, an 8-byte ID, if the amount of data at 1.6 billion, then take up memory is 32G. This is just a cache ID, although the official said redis query speed 12W + / s, if coupled with other data caches, memory footprint and high, and query speed is not high
Bloom filter (Bloom Filter)
The algorithm lot of information online, here is not telling. The purpose is to save memory, redis are some related algorithms, bitmap, you must verify the validity of the key.
bitmap length of the 32 th power of 2, is a binary data, the length of a full length of the 4 billion +, and occupy the highest 512M of memory, compared to 1.6 billion the amount of data occupies much better. It element of storage is 1/0. Binary storage, can also be understood that a hashmap
The disadvantage is generated hash collision, the data may not be redis, but few, even if not, the database can be requested again. Because very few such requests, even under high concurrent database can fully cope with, but also solve the problem of huge footprint, can save a lot of costs, cost consciousness is essential to consider factors architect.
Fake code:
@PostConstruct
public void init(){
//初始化数据,可以是从数据库或者其他地方获取
List<String> ids=new ArrayList<String>();
//遍历放入redis
ids.stream().forEach(id ->{
int hash = id.hashCode();
//获取bitmap映射位置(hashmap)
long index = (long)Math.abs(hash%Math.pow(2,32));
redisUtils.setBit("bloom_filter_ids",index,true);
});
}
@Override
public OrderEntity selectById(Long id) {
int hash = id.hashCode();
long index = (long)Math.abs(hash%Math.pow(2,32));
boolean b = redisUtils.getBit("bloom_filter_ids", index);
if(b){
Object object = redisUtils.get(String.valueOf(id));
if(object!=null){
return (OrderEntity) object;
}else {
OrderEntity orderEntity = orderMapper.selectByPrimaryKey(id);
redisUtils.set(String.valueOf(id), orderEntity, 5);
return orderEntity;
}
}else {
System.out.println("请求ID被拦截");
return null;
}
}When ensure key (represented here with my id) effectiveness, but also may produce countless id / key in redis not, queries to a database, synchronization can lock or lock and other programs can be distributed
RedisUtils:
package com.chwl.cn.config.redis;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.stereotype.Component;
import org.springframework.util.CollectionUtils;
/**
* @author ypp 创建时间：2018年11月19日 下午4:58:04
* @Description: TODO(redis缓存工具类)
*/
@Component
public class RedisUtils {
@Autowired
@Qualifier("redisTemplate")
private RedisTemplate<String, Object> redisTemplate;
// =============================common============================
/**
* 指定缓存失效时间
* @param key 键
* @param time 时间(秒)
* @return
*/
public boolean expire(String key, long time) {
try {
if (time > 0) {
redisTemplate.expire(key, time, TimeUnit.SECONDS);
}
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 根据key 获取过期时间
* @param key 键 不能为null
* @return 时间(秒) 返回0代表为永久有效
*/
public long getExpire(String key) {
return redisTemplate.getExpire(key, TimeUnit.SECONDS);
}
/**
* 判断key是否存在
* @param key 键
* @return true 存在 false不存在
*/
public boolean hasKey(String key) {
try {
return redisTemplate.hasKey(key);
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 删除缓存
* @param key 可以传一个值 或多个
*/
@SuppressWarnings("unchecked")
public void del(String... key) {
if (key != null && key.length > 0) {
if (key.length == 1) {
redisTemplate.delete(key[0]);
} else {
redisTemplate.delete(CollectionUtils.arrayToList(key));
}
}
}
// ============================String=============================
/**
* 普通缓存获取
* @param key 键
* @return 值
*/
public Object get(String key) {
return key == null ? null : redisTemplate.opsForValue().get(key);
}
/**
* 普通缓存放入
* @param key 键
* @param value 值
* @return true成功 false失败
*/
public boolean set(String key, Object value) {
try {
redisTemplate.opsForValue().set(key, value);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 普通缓存放入并设置时间
*
* @param key 键
* @param value 值
* @param time 时间(秒) time要大于0 如果time小于等于0 将设置无限期
* @return true成功 false 失败
*/
public boolean set(String key, Object value, long time) {
try {
if (time > 0) {
redisTemplate.opsForValue().set(key, value, time, TimeUnit.SECONDS);
} else {
set(key, value);
}
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 递增
* @param key 键
* @param by 要增加几(大于0)
* @return
*/
public long incr(String key, long delta) {
if (delta < 0) {
throw new RuntimeException("递增因子必须大于0");
}
return redisTemplate.opsForValue().increment(key, delta);
}
/**
* 递减
* @param key 键
* @param by 要减少几(小于0)
* @return
*/
public long decr(String key, long delta) {
if (delta < 0) {
throw new RuntimeException("递减因子必须大于0");
}
return redisTemplate.opsForValue().increment(key, -delta);
}
// ================================Map=================================
/**
* HashGet
* @param key 键 不能为null
* @param item 项 不能为null
* @return 值
*/
public Object hget(String key, String item) {
return redisTemplate.opsForHash().get(key, item);
}
/**
* 获取hashKey对应的所有键值
* @param key 键
* @return 对应的多个键值
*/
public Map<Object, Object> hmget(String key) {
return redisTemplate.opsForHash().entries(key);
}
/**
* HashSet
* @param key 键
* @param map 对应多个键值
* @return true 成功 false 失败
*/
public boolean hmset(String key, Map<String, Object> map) {
try {
redisTemplate.opsForHash().putAll(key, map);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* HashSet 并设置时间
* @param key 键
* @param map 对应多个键值
* @param time 时间(秒)
* @return true成功 false失败
*/
public boolean hmset(String key, Map<String, Object> map, long time) {
try {
redisTemplate.opsForHash().putAll(key, map);
if (time > 0) {
expire(key, time);
}
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 向一张hash表中放入数据,如果不存在将创建
* @param key 键
* @param item 项
* @param value 值
* @return true 成功 false失败
*/
public boolean hset(String key, String item, Object value) {
try {
redisTemplate.opsForHash().put(key, item, value);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 向一张hash表中放入数据,如果不存在将创建
* @param key 键
* @param item 项
* @param value 值
* @param time
* 时间(秒) 注意:如果已存在的hash表有时间,这里将会替换原有的时间
* @return true 成功 false失败
*/
public boolean hset(String key, String item, Object value, long time) {
try {
redisTemplate.opsForHash().put(key, item, value);
if (time > 0) {
expire(key, time);
}
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 删除hash表中的值
* @param key 键 不能为null
* @param item 项 可以使多个 不能为null
*/
public void hdel(String key, Object... item) {
redisTemplate.opsForHash().delete(key, item);
}
/**
* 判断hash表中是否有该项的值
* @param key 键 不能为null
* @param item 项 不能为null
* @return true 存在 false不存在
*/
public boolean hHasKey(String key, String item) {
return redisTemplate.opsForHash().hasKey(key, item);
}
/**
* hash递增 如果不存在,就会创建一个 并把新增后的值返回
* @param key 键
* @param item 项
* @param by 要增加几(大于0)
* @return
*/
public double hincr(String key, String item, double by) {
return redisTemplate.opsForHash().increment(key, item, by);
}
/**
* hash递减
* @param key 键
* @param item 项
* @param by 要减少记(小于0)
* @return
*/
public double hdecr(String key, String item, double by) {
return redisTemplate.opsForHash().increment(key, item, -by);
}
// ============================set=============================
/**
* 根据key获取Set中的所有值
* @param key 键
* @return
*/
public Set<Object> sGet(String key) {
try {
return redisTemplate.opsForSet().members(key);
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
/**
* 根据value从一个set中查询,是否存在
* @param key 键
* @param value 值
* @return true 存在 false不存在
*/
public boolean sHasKey(String key, Object value) {
try {
return redisTemplate.opsForSet().isMember(key, value);
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 将数据放入set缓存
* @param key 键
* @param values 值 可以是多个
* @return 成功个数
*/
public long sSet(String key, Object... values) {
try {
return redisTemplate.opsForSet().add(key, values);
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
/**
* 将set数据放入缓存
* @param key 键
* @param time 时间(秒)
* @param values 值 可以是多个
* @return 成功个数
*/
public long sSetAndTime(String key, long time, Object... values) {
try {
Long count = redisTemplate.opsForSet().add(key, values);
if (time > 0)
expire(key, time);
return count;
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
/**
* 获取set缓存的长度
* @param key 键
* @return
*/
public long sGetSetSize(String key) {
try {
return redisTemplate.opsForSet().size(key);
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
/**
* 移除值为value的
* @param key 键
* @param values 值 可以是多个
* @return 移除的个数
*/
public long setRemove(String key, Object... values) {
try {
Long count = redisTemplate.opsForSet().remove(key, values);
return count;
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
// ===============================list=================================
/**
* 获取list缓存的内容
*
* @param key 键
* @param start 开始
* @param end 结束 0 到 -1代表所有值
* @return
*/
public List<Object> lGet(String key, long start, long end) {
try {
return redisTemplate.opsForList().range(key, start, end);
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
/**
* 获取list缓存的长度
* @param key 键
* @return
*/
public long lGetListSize(String key) {
try {
return redisTemplate.opsForList().size(key);
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
/**
* 通过索引 获取list中的值
* @param key 键
* @param index 索引 index>=0时， 0 表头，1 第二个元素，依次类推；index<0时，-1，表尾，-2倒数第二个元素，依次类推
* @return
*/
public Object lGetIndex(String key, long index) {
try {
return redisTemplate.opsForList().index(key, index);
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
/**
* 将list放入缓存
* @param key 键
* @param value 值
* @param time 时间(秒)
* @return
*/
public boolean lSet(String key, Object value) {
try {
redisTemplate.opsForList().rightPush(key, value);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 将list放入缓存
* @param key 键
* @param value 值
* @param time 时间(秒)
* @return
*/
public boolean lSet(String key, Object value, long time) {
try {
redisTemplate.opsForList().rightPush(key, value);
if (time > 0)
expire(key, time);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 将list放入缓存
* @param key 键
* @param value 值
* @param time 时间(秒)
* @return
*/
public boolean lSet(String key, List<Object> value) {
try {
redisTemplate.opsForList().rightPushAll(key, value);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 将list放入缓存
* @param key 键
* @param value 值
* @param time 时间(秒)
* @return
*/
public boolean lSet(String key, List<Object> value, long time) {
try {
redisTemplate.opsForList().rightPushAll(key, value);
if (time > 0)
expire(key, time);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 根据索引修改list中的某条数据
* @param key 键
* @param index 索引
* @param value 值
* @return
*/
public boolean lUpdateIndex(String key, long index, Object value) {
try {
redisTemplate.opsForList().set(key, index, value);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 移除N个值为value
* @param key 键
* @param count 移除多少个
* @param value 值
* @return 移除的个数
*/
public long lRemove(String key, long count, Object value) {
try {
Long remove = redisTemplate.opsForList().remove(key, count, value);
return remove;
} catch (Exception e) {
e.printStackTrace();
return 0;
}
}
/**
* 添加一个元素, zset与set最大的区别就是每个元素都有一个score，因此有个排序的辅助功能; zadd
*
* @param key
* @param value
* @param score
*/
public boolean zsetAdd(String key, Object value, Double score) {
try {
redisTemplate.opsForZSet().add(key, value,score);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 删除元素 zrem
*
* @param key
* @param value
*/
public void remove(String key, String value) {
redisTemplate.opsForZSet().remove(key, value);
}
/**
* @param key
* @param value
* @param score
*/
public Double incrScore(String key, String value, double score) {
return redisTemplate.opsForZSet().incrementScore(key, value, score);
}
/**
* 查询value对应的score zscore
*
* @param key
* @param value
* @return
*/
public Double score(String key, String value) {
return redisTemplate.opsForZSet().score(key, value);
}
/**
* 判断value在zset中的排名 zrank
*
* @param key
* @param value
* @return
*/
public Long rank(String key, String value) {
return redisTemplate.opsForZSet().rank(key, value);
}
/**
* 返回集合的长度
*
* @param key
* @return
*/
public Long size(String key) {
return redisTemplate.opsForZSet().zCard(key);
}
/**
* 查询集合中指定顺序的值， 0 -1 表示获取全部的集合内容 zrange
*
* 返回有序的集合，score小的在前面
*
* @param key
* @param start
* @param end
* @return
*/
public Set<Object> range(String key, int start, int end) {
return redisTemplate.opsForZSet().range(key, start, end);
}
/**
* setNX
* @param key
* @param value
* @return
*/
public boolean setNx(String key,Object value){
return redisTemplate.opsForValue().setIfAbsent(key, value);
}
/**
* 设置setnx的过期时间 毫秒单位
* @param key
* @param value
* @param milliseconds
* @return
*/
public boolean setNx(String key,Object value,Long milliseconds){
return redisTemplate.opsForValue().setIfAbsent(key, value,milliseconds,TimeUnit.SECONDS);
}
public boolean setBit(String key,Long index,boolean value) {
return redisTemplate.opsForValue().setBit(key, index, value);
}
public boolean getBit(String key,Long index) {
return redisTemplate.opsForValue().getBit(key, index);
}
}
These are some ideas of their own summary, I hope to help some small partners, deficiencies or more program ideas and comments are welcome
