MDC implements log tracking

Annotating log context with MDC

background

Recently, I am studying ELK, and I want to use ELK to manage logs in a unified way, and simply analyze some functions of the system, such as: the transaction volume under the institution, the ratio of transaction success/failure, the number of certain transactions per unit time, and the top 50 IPs accessing the system. ..., but due to the inability to establish a unified analysis standard and implement it, the idea is to print some business parameters into the log for analysis and statistics.

Introduction

　　MDC (Mapped Diagnostic Context) is a function provided by log4j and logback to facilitate logging in multi-threaded conditions . Some applications use multiple threads to process requests from multiple users. During the use of a user, there may be multiple different threads for processing. A typical example is a web application server. When a user visits a page, the application server may create a new thread to process the request, or it may reuse an existing thread from the thread pool. During the existence of a user's session, multiple threads may process the user's request. This makes it difficult to distinguish logs corresponding to different users. It becomes cumbersome when it is necessary to keep track of the relevant log records of a user in the system.

　　One solution is to use a custom log format that encodes the user's information in the log record in some way . The problem with this approach is that it requires access to user-related information in every class that uses the logger. This makes it possible to use it when logging. Such conditions are usually more difficult to satisfy. The role of MDC is to solve this problem.

　　MDC can be seen as a hash table bound to the current thread , to which key-value pairs can be added. Content contained in the MDC can be accessed by code executing in the same thread. Child threads of the current thread inherit the contents of the MDC in their parent thread. When logging is required, just get the required information from the MDC. The contents of the MDC are saved by the program when appropriate. For a Web application, this data is usually stored at the very beginning of the request being processed.

benefit

• If your system is online, suddenly one day the boss said that we should add some user data to the log for analysis. Without MDC I guess you should be in an avalanche at this point. MDC is just right to allow you to implement some of the sudden requirements on the log
• If you are a code cleaner, encapsulate the operation of the company LOG, and encapsulate the processing thread tracking log number, but only the part that uses your encapsulated log tool can print the tracking log number, other parts (such as hibernate, mybatis, httpclient) etc.) logs will not reflect the tracking number. Of course, we can bypass these troubles through linux commands.
• Make code concise and log style consistent

use

• spirng: import org.slf4j.MDC;
• springboot: import org.apache.log4j.MDC;

accomplish

• Filter (Filter)

• Encountered a problem: how to read the return value of the controller in the filter ( because the direct Response does not provide a method to get the return value directly. So the return value must be obtained through the proxy )

@WebFilter(urlPatterns = "/*")
@Order(1)
public class MyFilter implements Filter {

    private static Logger logger = LoggerFactory.getLogger(MyFilter.class);

    private static ObjectMapper mapper = new ObjectMapper();

    @Override
    public void init(FilterConfig filterConfig) throws ServletException {

    }

    @SuppressWarnings("unchecked")
    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
            throws IOException, ServletException {

        HttpServletRequest req = (HttpServletRequest) request;

        String body = IOUtils.toString(request.getInputStream());
        Map<String, Object> reqMap = mapper.readValue(body, Map.class);

        // 可以在此处解析请求报文，然后将业务参数放入日志打印
        MDC.put("service_name", "my_service1");
        MDC.put("trcode", "123456");
        MDC.put("chid", "999999");

        ResponseWrapper wrapperResponse = new ResponseWrapper((HttpServletResponse) response);// 转换成代理类

        ParameterRequestWrapper wrapRequest = new ParameterRequestWrapper(req, reqMap);
        chain.doFilter(wrapRequest, wrapperResponse);

        byte[] content = wrapperResponse.getContent();// 获取返回值
        String str = new String(content, "UTF-8");

        Map<String, Object> rspMap = JSONUtil.jsonToObj(str, Map.class);
        Map<String, String> rspHeadMap = (Map<String, String>) rspMap.get("rspHead");
        MDC.put("rspcode", rspHeadMap.get("rspcode"));

        MDC.clear();// 清除数据
    }

    /*
     * (non-Javadoc)
     * 
     * @see javax.servlet.Filter#destroy()
     */
    @Override
    public void destroy() {

    }

}

/**
 * 返回值输出代理类
 */
public class ResponseWrapper extends HttpServletResponseWrapper {

    private ByteArrayOutputStream buffer;

    private ServletOutputStream out;

    public ResponseWrapper(HttpServletResponse httpServletResponse) {
        super(httpServletResponse);
        buffer = new ByteArrayOutputStream();
        out = new WrapperOutputStream(buffer);
    }

    @Override
    public ServletOutputStream getOutputStream() throws IOException {
        return out;
    }

    @Override
    public void flushBuffer() throws IOException {
        if (out != null) {
            out.flush();
        }
    }

    public byte[] getContent() throws IOException {
        flushBuffer();
        return buffer.toByteArray();
    }

    class WrapperOutputStream extends ServletOutputStream {
        private ByteArrayOutputStream bos;

        public WrapperOutputStream(ByteArrayOutputStream bos) {
            this.bos = bos;
        }

        @Override
        public void write(int b) throws IOException {
            bos.write(b);
        }

        @Override
        public boolean isReady() {

            return false;
        }

        @Override
        public void setWriteListener(WriteListener arg0) {

        }
    }

}

• Interceptor ( not yet implemented to get the return value in the controller )

/**
 * 拦截器
 * 
 * @author Fan.W
 * @since 1.8
 */
public class MDCInterceptor implements HandlerInterceptor {

    private static final Logger logger = LoggerFactory.getLogger(MDCInterceptor.class);

    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler)
            throws Exception {

        MDC.put("user_name", "fan wei");
        MDC.put("user_id", "123456");
        return true;
    }

    @Override
    public void postHandle(
            HttpServletRequest request,
            HttpServletResponse response,
            Object handler,
            ModelAndView modelAndView) throws Exception {

    }

    /**
    *这个方法的主要作用是用于清理资源的，当然这个方法也只能在当前这个Interceptor的preHandle方法的返回值为true时才会执行。
    */
    @Override
    public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex)
            throws Exception {
        MDC.clear();// 清除
    }

}

• Binding and Registration

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tx="http://www.springframework.org/schema/tx"
    xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:context="http://www.springframework.org/schema/context"
    xmlns:aop="http://www.springframework.org/schema/aop"
    xsi:schemaLocation="
        http://www.springframework.org/schema/beans 
        http://www.springframework.org/schema/beans/spring-beans.xsd 
        http://www.springframework.org/schema/tx 
        http://www.springframework.org/schema/tx/spring-tx.xsd 
        http://www.springframework.org/schema/mvc
         http://www.springframework.org/schema/mvc/spring-mvc.xsd
        http://www.springframework.org/schema/context 
        http://www.springframework.org/schema/context/spring-context.xsd
        http://www.springframework.org/schema/aop
        http://www.springframework.org/schema/aop/spring-aop.xsd"
    default-lazy-init="false">

    <!--拦截器 -->  
    <mvc:interceptors> 
        <bean class="com.seeker.interceptor.MDCInterceptor"/> 
    </mvc:interceptors>  

</beans>

• spring aop implementation (recommended)

Exception notification must catch exception before ExceptionHandler in controller! ! ! ! !

@Aspect
@Component
public class MyInterceptor {

    private static Logger logger = LoggerFactory.getLogger(MyInterceptor.class);

    /**
     * 拦截类的入口--拦截所有controller类
     */
    @Pointcut("execution(public * com.seeker.controller..*.*(..)) ")
    public void pointCut() {
    }

    /**
     * 方法调用之前调用
     * 
     * @param joinPoint
     */
    @Before(value = "pointCut()")
    public void doBefore(JoinPoint joinPoint) {

    }

    /**
     * 环绕通知
     * 
     * @param pjp
     * @return
     * @throws Throwable
     */
    @SuppressWarnings({ "rawtypes", "unchecked" })
    @Around("pointCut()")
    public Object doBasicProfiling(ProceedingJoinPoint pjp) throws Throwable {

        String className = pjp.getTarget().getClass().getName(); // 拦截类
        String methodName = pjp.getSignature().getName() + "()";
        Object[] args = pjp.getArgs();// 获取请求参数，可以校验属性

        // 解析请求报文，使用MDC,打印日志，也可在前置通知中获取
        MDC.put("service_name", "my_service1");
        MDC.put("trcode", "123456");
        MDC.put("chid", "999999");

        // 此处返回的是拦截的方法的返回值，如果不执行此方法，则不会执行被拦截的方法，利用环绕通知可以很好的做权限管理
        Object obj = pjp.proceed();

        MDC.put("rspcode", "0000");
        return obj;
    }

    /**
     * 异常通知：pjp.proceed();跑出异常即捕获，先于@ExceptionHandler中捕获到
     * 
     * @param joinPoint
     * @param e
     */
    @AfterThrowing(pointcut = "pointCut()", throwing = "e")
    public void doAfterThrowing(JoinPoint joinPoint, Throwable e) {
        if (e instanceof BizException) {
            MDC.put("rspcode", ((BizException) e).messageCode);
        } else {
            MDC.put("rspcode", "9999");
        }
    }
}

• Constrain and enable aop

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:mvc="http://www.springframework.org/schema/mvc"
    xmlns:aop="http://www.springframework.org/schema/aop" 
    xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd  
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
        http://www.springframework.org/schema/aop 
                        http://www.springframework.org/schema/aop/spring-aop.xsd">

        <!--通知spring使用cglib而不是jdk的来生成代理方法 AOP可以拦截到Controller  -->
    <aop:aspectj-autoproxy expose-proxy="true"></aop:aspectj-autoproxy>

</beans>

• logback.xml configuration

 <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> 
    <pattern>[%X{service_name}] %d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] [%X{chid}] [%X{trcode}] [%X{rspcode}] - %m%n</pattern>   
</encoder>