First, the concept
Idempotency, popular to say that an interface with a request to initiate multiple times, must ensure that the operation can be performed only once
, such as:
-
Orders interfaces, you can not create multiple orders
-
Payment interface, double payments for the same order only once deducted money
-
Alipay callback interface, multiple callbacks may be, have to deal with repeated callbacks
-
Common form submission interfaces, network timeouts and other reasons because click submit multiple times, only once successfully
, etc.
Second, the common solution
-
The only index - prevent new dirty data
-
token mechanisms - to prevent duplicate submission page
-
Pessimistic locking - time access to data lock (lock or lock table rows)
-
Optimistic locking - based version version realized, at that moment the update data verification data
-
Distributed Lock - redis (jedis, redisson) or zookeeper achieve
-
State Machine - state changes, status update data is determined
Third, the paper implements
In this paper, two kinds of ways, i.e., the interface implemented by checking idempotent redis + token mechanism
Fourth, the realization of ideas
Idempotency need to ensure that every request to create a unique token identifier, to obtain the token, and the token is stored redis this, when the request interface, this token into the request header parameter or a request interface, backend interface determines redis whether there is this token:
-
If so, normal processing business logic, and delete the token from redis, then, if the request is repeated, since the token has been removed, you can not check, do not repeat the operation returns tips
-
If not, explain the parameters illegal or repeat request, return the prompts to
V. Project
-
springboot
-
repeat
-
+ @ApiIdempotent annotation request interceptor intercepts
-
@ControllerAdvice global exception handler
-
Pressure measurement tools: jmeter
Description:
This article focuses on idempotency core implementation on how springboot integrated within the scope of this redis, ServerResponse, ResponseCode and other minor details are not described, we are interested in small partner can see my Github project:
https://github.com/wangzaiplus/springboot/tree/wxw
Sixth, code implementation
pom
<!-- Redis-Jedis -->
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>2.9.0</version>
</dependency>
<!--lombok 本文用到@Slf4j注解, 也可不引用, 自定义log即可-->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.16.10</version>
</dependency>
JedisUtil
package com.wangzaiplus.test.util;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
@Component
@Slf4j
public class JedisUtil {
@Autowired
private JedisPool jedisPool;
private Jedis getJedis() {
return jedisPool.getResource();
}
/**
* 设值
*
* @param key
* @param value
* @return
*/
public String set(String key, String value) {
Jedis jedis = null;
try {
jedis = getJedis();
return jedis.set(key, value);
} catch (Exception e) {
log.error("set key:{} value:{} error", key, value, e);
return null;
} finally {
close(jedis);
}
}
/**
* 设值
*
* @param key
* @param value
* @param expireTime 过期时间, 单位: s
* @return
*/
public String set(String key, String value, int expireTime) {
Jedis jedis = null;
try {
jedis = getJedis();
return jedis.setex(key, expireTime, value);
} catch (Exception e) {
log.error("set key:{} value:{} expireTime:{} error", key, value, expireTime, e);
return null;
} finally {
close(jedis);
}
}
/**
* 取值
*
* @param key
* @return
*/
public String get(String key) {
Jedis jedis = null;
try {
jedis = getJedis();
return jedis.get(key);
} catch (Exception e) {
log.error("get key:{} error", key, e);
return null;
} finally {
close(jedis);
}
}
/**
* 删除key
*
* @param key
* @return
*/
public Long del(String key) {
Jedis jedis = null;
try {
jedis = getJedis();
return jedis.del(key.getBytes());
} catch (Exception e) {
log.error("del key:{} error", key, e);
return null;
} finally {
close(jedis);
}
}
/**
* 判断key是否存在
*
* @param key
* @return
*/
public Boolean exists(String key) {
Jedis jedis = null;
try {
jedis = getJedis();
return jedis.exists(key.getBytes());
} catch (Exception e) {
log.error("exists key:{} error", key, e);
return null;
} finally {
close(jedis);
}
}
/**
* 设值key过期时间
*
* @param key
* @param expireTime 过期时间, 单位: s
* @return
*/
public Long expire(String key, int expireTime) {
Jedis jedis = null;
try {
jedis = getJedis();
return jedis.expire(key.getBytes(), expireTime);
} catch (Exception e) {
log.error("expire key:{} error", key, e);
return null;
} finally {
close(jedis);
}
}
/**
* 获取剩余时间
*
* @param key
* @return
*/
public Long ttl(String key) {
Jedis jedis = null;
try {
jedis = getJedis();
return jedis.ttl(key);
} catch (Exception e) {
log.error("ttl key:{} error", key, e);
return null;
} finally {
close(jedis);
}
}
private void close(Jedis jedis) {
if (null != jedis) {
jedis.close();
}
}
}
Custom annotation @ApiIdempotent
package com.wangzaiplus.test.annotation;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
/**
* 在需要保证 接口幂等性 的Controller的方法上使用此注解
*/
@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
public @interface ApiIdempotent {
}
ApiIdempotentInterceptor interceptor
package com.wangzaiplus.test.interceptor;
import com.wangzaiplus.test.annotation.ApiIdempotent;
import com.wangzaiplus.test.service.TokenService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.method.HandlerMethod;
import org.springframework.web.servlet.HandlerInterceptor;
import org.springframework.web.servlet.ModelAndView;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.lang.reflect.Method;
/**
* 接口幂等性拦截器
*/
public class ApiIdempotentInterceptor implements HandlerInterceptor {
@Autowired
private TokenService tokenService;
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
if (!(handler instanceof HandlerMethod)) {
return true;
}
HandlerMethod handlerMethod = (HandlerMethod) handler;
Method method = handlerMethod.getMethod();
ApiIdempotent methodAnnotation = method.getAnnotation(ApiIdempotent.class);
if (methodAnnotation != null) {
check(request);// 幂等性校验, 校验通过则放行, 校验失败则抛出异常, 并通过统一异常处理返回友好提示
}
return true;
}
private void check(HttpServletRequest request) {
tokenService.checkToken(request);
}
@Override
public void postHandle(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse, Object o, ModelAndView modelAndView) throws Exception {
}
@Override
public void afterCompletion(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse, Object o, Exception e) throws Exception {
}
}
TokenServiceImpl
package com.wangzaiplus.test.service.impl;
import com.wangzaiplus.test.common.Constant;
import com.wangzaiplus.test.common.ResponseCode;
import com.wangzaiplus.test.common.ServerResponse;
import com.wangzaiplus.test.exception.ServiceException;
import com.wangzaiplus.test.service.TokenService;
import com.wangzaiplus.test.util.JedisUtil;
import com.wangzaiplus.test.util.RandomUtil;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.text.StrBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import javax.servlet.http.HttpServletRequest;
@Service
public class TokenServiceImpl implements TokenService {
private static final String TOKEN_NAME = "token";
@Autowired
private JedisUtil jedisUtil;
@Override
public ServerResponse createToken() {
String str = RandomUtil.UUID32();
StrBuilder token = new StrBuilder();
token.append(Constant.Redis.TOKEN_PREFIX).append(str);
jedisUtil.set(token.toString(), token.toString(), Constant.Redis.EXPIRE_TIME_MINUTE);
return ServerResponse.success(token.toString());
}
@Override
public void checkToken(HttpServletRequest request) {
String token = request.getHeader(TOKEN_NAME);
if (StringUtils.isBlank(token)) {// header中不存在token
token = request.getParameter(TOKEN_NAME);
if (StringUtils.isBlank(token)) {// parameter中也不存在token
throw new ServiceException(ResponseCode.ILLEGAL_ARGUMENT.getMsg());
}
}
if (!jedisUtil.exists(token)) {
throw new ServiceException(ResponseCode.REPETITIVE_OPERATION.getMsg());
}
Long del = jedisUtil.del(token);
if (del <= 0) {
throw new ServiceException(ResponseCode.REPETITIVE_OPERATION.getMsg());
}
}
}
test Application
package com.wangzaiplus.test;
import com.wangzaiplus.test.interceptor.ApiIdempotentInterceptor;
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.web.cors.CorsConfiguration;
import org.springframework.web.cors.UrlBasedCorsConfigurationSource;
import org.springframework.web.filter.CorsFilter;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;
@SpringBootApplication
@MapperScan("com.wangzaiplus.test.mapper")
public class TestApplication extends WebMvcConfigurerAdapter {
public static void main(String[] args) {
SpringApplication.run(TestApplication.class, args);
}
/**
* 跨域
* @return
*/
@Bean
public CorsFilter corsFilter() {
final UrlBasedCorsConfigurationSource urlBasedCorsConfigurationSource = new UrlBasedCorsConfigurationSource();
final CorsConfiguration corsConfiguration = new CorsConfiguration();
corsConfiguration.setAllowCredentials(true);
corsConfiguration.addAllowedOrigin("*");
corsConfiguration.addAllowedHeader("*");
corsConfiguration.addAllowedMethod("*");
urlBasedCorsConfigurationSource.registerCorsConfiguration("/**", corsConfiguration);
return new CorsFilter(urlBasedCorsConfigurationSource);
}
@Override
public void addInterceptors(InterceptorRegistry registry) {
// 接口幂等性拦截器
registry.addInterceptor(apiIdempotentInterceptor());
super.addInterceptors(registry);
}
@Bean
public ApiIdempotentInterceptor apiIdempotentInterceptor() {
return new ApiIdempotentInterceptor();
}
}
OK, so far, check the code is ready, the next test validation
Seven, testing and certification
1, access token controller TokenController
package com.wangzaiplus.test.controller;
import com.wangzaiplus.test.common.ServerResponse;
import com.wangzaiplus.test.service.TokenService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequestMapping("/token")
public class TokenController {
@Autowired
private TokenService tokenService;
@GetMapping
public ServerResponse token() {
return tokenService.createToken();
}
}
2, TestController, attention @ApiIdempotent notes, notes to this statement on the method requires idempotency check, the check is not required no effect
package com.wangzaiplus.test.controller;
import com.wangzaiplus.test.annotation.ApiIdempotent;
import com.wangzaiplus.test.common.ServerResponse;
import com.wangzaiplus.test.service.TestService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequestMapping("/test")
@Slf4j
public class TestController {
@Autowired
private TestService testService;
@ApiIdempotent
@PostMapping("testIdempotence")
public ServerResponse testIdempotence() {
return testService.testIdempotence();
}
}
3, access token
View redis
4, Test Interface Security: 50 jmeter testing tools simulate concurrent requests, the upper step of the token acquired as a parameter
5, header or parameters not pass token or token is empty, or chaos filled token value, they were unable to check, such as the token value "abcd"
Eight Points of Attention (very important)
The figure above, can not simply delete token without checking whether the deletion is successful, there will be complicated by security problems, because there are multiple threads may come on line 46, token has not yet been deleted at this time, so continue to under execution, if you do not check jedisUtil.del (token) to delete the results directly released, then there will be duplicate or submit questions, even though in fact only one actually delete operation, to reproduce it below
Slightly modified the code below:
Request again
Look at the console
Although only a truly deleted token, but because there is no check to delete the results, they still have concurrency issues, therefore, must check
Nine, summary
In fact, the idea is very simple, it is guaranteed to be unique for each request, so as to ensure idempotency by interceptor + notes, would not every request to write duplicate code, in fact, can also use spring aop realize, does not matter.
Github
https://github.com/wangzaiplus/springboot/tree/wxw