I. Introduction
About repeat request, it refers to our server receives more of the same content within a very short time to repeat the request. And if such a repeat request is idempotent (the results of each request are the same, such as queries), and that in fact, for us there is no impact, but if non-idempotent (every request will affect critical data, such as delete relationships, build relationships, etc.), it will produce ranging from dirty data, while a system error.
Therefore, in the case of the current universal distributed services, how to avoid and resolve duplicate request data brought to us by the abnormal has become a serious problem. While avoiding duplication request, the best practice is to do a common front and rear ends.
1. The front-end or client on a non-idempotent operation button done directly Disallow repeat requests.
2. Upon receiving the rear end of the lock request, the unlock.
This blog is mainly about the back end based on the concept of distributed lock to a common solution on the settlement of a repeat request.
Second, the text
Why use distributed lock to solve it? Because we are currently widely distributed server architecture, the request forwarded by the gateway layer distal to the rear end, as shown below, do so only if the limits on a single server, it can not be completed in a distributed service high frequency response to repeated requests.
The basic idea
Idea is basically to do to prevent the interface coupled distributed lock repeat requests, the following steps:
- Upon receiving the request, according to the method name + md5 parameter takes the value of the acquisition method and uniquely identifies the parameter;
- Distributed Lock disposed after the identifier, and set an expiration time;
- After the request, release distributed lock.
Repeated request to complete the current request is prohibited. If you want universal solution, it would need to make a small step above function out, because I am on java, spring framework are more familiar with, so they took this to be an example.
Achieve spring section, redis-based
Presumably some familiar spring the students already know in what way I want to, and do general-purpose, be sure to use spring aop characteristics of annotations + cut + md5key + reflection + redis achieve, as follows:
- Define a distributed lock a comment to include expiration time, ignore the parameters;
- The definition of a cut, cut-off point for the Distributed Lock notes, requires the use of distributed lock acquisition method name, parameters, time expired in section, and the method name and parameters do not ignore md5 take unique identification;
- Distributed Lock redsis then set based on the unique identifier;
- After unlocking method.
code show as below:
annotation
Define a comment RepeatOperationLock, the parameter lock expiration time and ignore property (ie not participating in the distributed computing MD5 lock identification of property).
@Documented
@Inherited
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD})
@Component
public @interface RepeatOperationLock {
/**
* 锁时长，默认500ms
* @return
*/
long timeOut() default 500;
/**
* 忽略上锁参数位置，从0开始
* @return
*/
int[] ignoreIndex();
}
复制代码section
The Notes are for the cut-off point, cut a few things to do to get the method name, get annotation attributes (expiration time, ignore the property), calculated + md5 value of the property, method calls an external distributed lock.
@Aspect
@Slf4j
@Component
public class LockAspect {
@Autowired
RepeatLockService repeatLockService;
@Pointcut("@annotation(com.ls.javabase.aspect.annotation.RepeatOperationLock)")
public void serviceAspect() {
}
@Before("serviceAspect()")
public void setLock(JoinPoint point) {
log.info("防止方法重复调用接口锁,上锁,point:{}", point);
Method method = ((MethodSignature) point.getSignature()).getMethod();
RepeatOperationLock repeatOperationLock = method.getAnnotation(RepeatOperationLock.class);
if (Objects.isNull(repeatOperationLock)) {
log.warn("---repeatOperationLock is null---");
return;
}
long timeOut = repeatOperationLock.timeOut();
int [] ignoreIndex = repeatOperationLock.ignoreIndex();
log.info("lockTime——{}", timeOut);
if (Objects.isNull(timeOut)) {
log.warn("---timeOut is null");
return;
}
String methodName = method.getName();
Object[] args = point.getArgs();
repeatLockService.setRepeatLock(methodName, args, timeOut);
}
@After("serviceAspect()")
public void removeLock(JoinPoint point) {
log.info("防止方法重复调用接口锁,解锁,point:{}",point);
Method method = ((MethodSignature) point.getSignature()).getMethod();
RepeatOperationLock repeatOperationLock = method.getAnnotation(RepeatOperationLock.class);
if (Objects.isNull(repeatOperationLock)) {
log.warn("---repeatOperationLock is null---");
return;
}
long timeOut = repeatOperationLock.timeOut();
log.info("lockTime——{}", timeOut);
if (Objects.isNull(timeOut)) {
log.warn("---timeOut is null");
return;
}
String methodName = method.getName();
Object[] args = point.getArgs();
repeatLockService.removeRepeatLock(methodName, args);
}
/**
*
* @param args
* @param ignoreIndex
* @return
*/
private Object [] getEffectiveArgs(Object[] args,int [] ignoreIndex) {
for (int i:ignoreIndex){
args[i] = null;
}
for (Object obj:args){
if (obj==null){
}
}
return args;
}
}
复制代码md5 method
public class Md5Encode {
/**
* constructors
*/
private Md5Encode() {
}
/**
* @param s 需要hash的字符串
* @return hash之后的字符串
*/
public static final String md5(final String s) {
char[] hexDigits = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'};
try {
byte[] btInput = s.getBytes(Charset.defaultCharset());
// 获得MD5摘要算法的 MessageDigest 对象
MessageDigest mdInst = MessageDigest.getInstance("MD5");
// 使用指定的字节更新摘要
mdInst.update(btInput);
// 获得密文
byte[] md = mdInst.digest();
// 把密文转换成十六进制的字符串形式
int j = md.length;
char[] str = new char[j * 2];
int k = 0;
for (int i = 0; i < j; i++) {
byte byte0 = md[i];
str[k++] = hexDigits[byte0 >>> 4 & 0xf];
str[k++] = hexDigits[byte0 & 0xf];
}
return new String(str);
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
}
复制代码Distributed Lock
Here's distributed lock using redis, but a little problem, is when you set up the lock and lock expiration time is not atomic, follow-up will make improvements to achieve a complete distributed lock program, wrote in the blog.
@Slf4j
@Service
public class RepeatLockService {
@Autowired
RepeatRedisUtil repeatRedisUtil;
public void setRepeatLock(String methodName, Object[] args, Long expireTime) {
for (Object obj : args) {
log.info("方法名：{},对象:{}，对象hashcode:{}", methodName, obj, obj.hashCode());
}
Boolean lock = repeatRedisUtil.setRepeatLock(methodName, args, expireTime);
if (!lock) {
log.info("已有相同请求");
}
}
public void removeRepeatLock(String methodName, Object[] args) {
repeatRedisUtil.removeRepeatLock(methodName, args);
}
}
@Component
public class RepeatRedisUtil {
@Autowired
RedisTemplate redisTemplate;
private static final String repeatLockPrefix = "repeat_lock_";
/**
* 设置重复请求锁，这一块的分布式锁的加与释放有问题，后续会专门出个文章总结redis分布式锁
* @param methodName
* @param args
* @param expireTime 过期时间ms
* @return
*/
public boolean setRepeatLock(String methodName, Object[] args,long expireTime) {
String key = getRepeatLockKey(methodName, args);
ValueOperations val = redisTemplate.opsForValue();
Long flag = val.increment(key, 1);
if (flag != 1) {
return false;
}
redisTemplate.expire(key, expireTime, TimeUnit.MILLISECONDS);
return true;
}
/**
* 删除重复请求锁
* @param methodName
* @param args
*/
public void removeRepeatLock(String methodName, Object[] args){
String key = getRepeatLockKey(methodName, args);
redisTemplate.delete(key);
}
/**
* 获取重复请求锁Key
*
* @param methodName
* @param args
* @return
*/
public String getRepeatLockKey(String methodName, Object[] args) {
String repeatLockKey = repeatLockPrefix + methodName;
for (Object obj : args) {
repeatLockKey = repeatLockKey+"_"+ obj.hashCode();
}
return repeatLockKey;
}
}
复制代码Testing service interfaces
That is the method used in the notes to, on behalf of the expiration time for the 200000ms, omit the second argument.
@Slf4j
@Service
public class TestLockService {
@RepeatOperationLock(timeOut = 200000, ignoreIndex = 1)
public void testLock(UserDto userDto,int i){
log.info("service中属性:{},{}",userDto,i);
log.info("service中hashcode,userDto:{},i:{}",userDto.hashCode(),i);
}
}
复制代码Epilogue
Such a spring-based general-purpose distributed lock solutions to share finished, indeed, there are still some flaws in the implementation of a distributed lock Shanghai there are some problems, such as when the lock is not atomic, no judge whether it will be unlocked misunderstandings, etc., will be dedicated to the follow-up summary distributed lock and improved, just above proposed the idea of a solution based on a distributed lock repeat request, also we hope to conduct more exchanges.