logback implementation of distributed link tracking system logs

The need to log tracking link

I Radisson Jeter who log query whether in a test environment or a production environment, is often used as a developer to go, things to see. A business appear BUG, then how quickly locate what we are looking for logs and other logs related? Related to testing and development personnel positioning of efficiency, it is often based on the keywords of their own print log to query log, but in concurrent capacity, or business process long logs do not lead to a coherent scene often you can not pass once, two log query logs to see the entire line of business. And even then only by to see real-time log to reproduce, in order to see the problem. However, in many cases the production environment which will give you the chance to reproduce, while vigorously now distributed micro-services, once invoked other services out of the BUG is not going to re-check the logs to find keywords?
The question is, why so many people do not bother to optimize? We get the print log Jingdong like shopping, like, all the logs get hold of the order number for this request are put on the order number, the background every time response logs are put on order number, or throw an exception as long as the order number on the tape log return to the front. Then look for the log directly check the order number can not it do? Testers holding a log ID to the developer which also need to reproduce? 10 times more efficient there? With perfect program let's buckle down!

New Filter

The first step of course is to intercept all HTTP request logs for each request are generated plus a unique identification.

Introduced maven dependence

<dependency>
	<groupId>org.slf4j</groupId>
	<artifactId>slf4j-api</artifactId>
	<version>1.7.13</version>
	<scope>provided</scope>
</dependency>
<dependency>
	<groupId>ch.qos.logback</groupId>
	<artifactId>logback-classic</artifactId>
	<scope>provided</scope>
	<version>1.1.7</version>
</dependency>

New Filter MdcFilter inherited MDCInsertingServletFilter

Logback comes ch.qos.logback.classic.helpers.MDCInsertingServletFilter HTTP request can be hostname, request URI, user-agent information loaded MDC

package com.test.trace.web.filter;

import java.io.IOException;
import javax.servlet.FilterChain;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;
import org.slf4j.MDC;
import com.alibaba.dubbo.common.utils.StringUtils;
import com.test.trace.support.AbstractMyThreadContext;
import com.test.trace.support.AbstractUUIDShort;
import ch.qos.logback.classic.helpers.MDCInsertingServletFilter;

public class MdcFilter extends MDCInsertingServletFilter {
    private static final String HEARDER_FOR_TRACE_ID = "X-TRACE-ID";
    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
            throws IOException, ServletException {
        if (!(request instanceof HttpServletRequest)) {
            return;
        }
        /**如前端传递了唯一标识ID 拿出来直接用 根据业务这段也可以删除 然后去掉if判断*/
        String traceId =
                ((HttpServletRequest) request).getHeader(HEARDER_FOR_TRACE_ID);
        if (StringUtils.isEmpty(traceId)) {
            traceId = AbstractUUIDShort.generate();
        }
        AbstractMyThreadContext.setTraceId(traceId);
        MDC.put(AbstractMyThreadContext.MDC_TRACE_ID,traceId);
        try {
            //从新调动父类的doFilter方法
            super.doFilter(request, response, chain);
        } finally {
            //资源回收
            MDC.remove(AbstractMyThreadContext.MDC_TRACE_ID);
            AbstractMyThreadContext.removeTraceId();
        }
    }
}

package com.test.trace.web.filter;

import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@ConditionalOnClass(name = "org.springframework.web.servlet.DispatcherServlet")
public class MdcFilterAutoConfigure {

    @Bean
    @ConditionalOnMissingBean(MdcFilter.class)
    public MdcFilter mdcFilter() {
        return new MdcFilter();
    }
}

spring annotation small classroom commenced friends

• @Configuration class is represented spring arranged class
• @ConditionalOnClass determine whether a class exists in the classpath is present together with two org.springframework.web.servlet.DispatcherServlet when loaded on the classpath MdcFilterAutoConfigure
• @Bean generate a Bean, and then to the Spring container management. Use @Configuration and @Bean combination. @Configuration understood as a spring when the inside xml <beans> tag, @ Bean understood as a spring when the inside xml <bean> tag
• Returns true if @ConditionalOnMissingBean Bena specified does not exist, that is, if used in conjunction @Bean vessel MdcFilter absent we generated a spring to MdcFilter

Log link ID generation tools

package com.test.trace.support;

import java.util.Base64;
import java.util.UUID;

public abstract class AbstractUUIDShort {

    /**生成唯一ID*/
    public static String generate() {
        UUID uuid = UUID.randomUUID();
        return compressedUUID(uuid);
    }

    protected static String compressedUUID(UUID uuid) {
        byte[] byUuid = new byte[16];
        long least = uuid.getLeastSignificantBits();
        long most = uuid.getMostSignificantBits();
        long2bytes(most, byUuid, 0);
        long2bytes(least, byUuid, 8);
        return Base64.getEncoder().encodeToString(byUuid);
    }

    protected static void long2bytes(long value, byte[] bytes, int offset) {
        for (int i = 7; i > -1; i--) {
            bytes[offset++] = (byte) ((value >> 8 * i) & 0xFF);
        }
    }
}

Log link ID is stored, and destruction tools

Mainly applied to the interface when the call dubbo out TraceId and made to ensure the child thread can get TraceId, in fact, does not involve a cross-system interface call this class is no longer necessary in.

package com.test.trace.support;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.HashMap;
import java.util.Map;

public abstract class AbstractMyThreadContext {

    private static final Logger log = LoggerFactory.getLogger(AbstractMyThreadContext.class);
    /**
     * logback模板对应的变量
     */
    public final static String MDC_TRACE_ID = "traceId";

    public static final String TRACE_ID = "com.test.trace.MyThreadContext_TRACE_ID_KEY";
    //确保子线程能够继承父线程的数据
    private static final ThreadLocal<Map<Object, Object>> RESOURCES =
            new InheritableThreadLocalMap<Map<Object, Object>>();


    protected AbstractMyThreadContext() {}


    public static Map<Object, Object> getResources() {
        return RESOURCES != null ? new HashMap<Object, Object>(RESOURCES.get()) : null;
    }

    public static void setResources(Map<Object, Object> newResources) {
        if (newResources == null || newResources.isEmpty()) {
            return;
        }
        RESOURCES.get().clear();
        RESOURCES.get().putAll(newResources);
    }


    private static Object getValue(Object key) {
        return RESOURCES.get().get(key);
    }


    public static Object get(Object key) {
        if (log.isTraceEnabled()) {
            log.trace("get() - in thread [{}]",Thread.currentThread().getName());
        }

        Object value = getValue(key);
        if ((value != null) && log.isTraceEnabled()) {
            log.trace("Retrieved value of type [{}]  for key [{}] bound to thread [{}]",
                    value.getClass().getName(), key, Thread.currentThread().getName());
        }
        return value;
    }

    public static void put(Object key, Object value) {
        if (key == null) {
            throw new IllegalArgumentException("key cannot be null");
        }

        if (value == null) {
            remove(key);
            return;
        }

        RESOURCES.get().put(key, value);

        if (log.isTraceEnabled()) {
            log.trace("Bound value of type [{}]  for key [{}]  to thread [{}]",
                    value.getClass().getName(), key, Thread.currentThread().getName());
        }
    }

    public static Object remove(Object key) {
        Object value = RESOURCES.get().remove(key);

        if ((value != null) && log.isTraceEnabled()) {
            log.trace("Retrieved value of type [{}]  for key [{}] from thread [{}]",
                    value.getClass().getName(), key, Thread.currentThread().getName());
        }

        return value;
    }

    public static void remove() {
        RESOURCES.remove();
    }

    //从线程局部变量中获取TraceId
    public static String getTraceId() {
        return (String) get(TRACE_ID);
    }

    //将TraceId摄入线程局部变量中
    public static void setTraceId(String xid) {
        put(TRACE_ID, xid);
    }

    //清除线程局部变量中的TraceId
    public static void removeTraceId() {
        remove(TRACE_ID);
    }

    private static final class InheritableThreadLocalMap<T extends Map<Object, Object>>
            extends InheritableThreadLocal<Map<Object, Object>> {
        @Override
        protected Map<Object, Object> initialValue() {
            return new HashMap<Object, Object>(1 << 4);
        }

        @SuppressWarnings({"unchecked"})
        @Override
        protected Map<Object, Object> childValue(Map<Object, Object> parentValue) {
            if (parentValue != null) {
                return (Map<Object, Object>) ((HashMap<Object, Object>) parentValue).clone();
            } else {
                return null;
            }
        }
    }
}

logback.xml log output format set

<pattern> Format attribute configured :( posted [% X {traceId}] is the thread ID of the configuration I)
[-5level%] [% contextName can]% {D the MM-dd-YYYY HH: mm: ss.SSS } [% thread] [% X {req.remoteHost}] [% X {req.requestURI}] [% X {traceId}]% logger -% msg% n

Here the link to the entire log to track completed half can already do all HTTP request gave plus traceId, but if we call dubbo interfaces, MdcFilter can not be intercepted, resulting in logback not find traceId, so we need coupled with a com.alibaba.dubbo.rpc.Filter keep the door for consumers and service providers dubbo, consumers need to give his traceId, service provider receives a request to traceId intake.
See my previous blog: the Filter filter Dubbo's (interceptor) using
lazy I will not do too much to describe sorry.

When you complete this step after the entire log link system basically 90% complete if you work hard enough, but to get up EFK query logs efficiency directly asking for the moon

• Term analysis EFK

Real-time transmission logs collected by filebeat nginx access log ▷ filebeat collected by kibana to elasticsearch cluster ▷ display the log.

However, students will be asked to think, time to perform the task of being given the old lady is not the same day the dog inside traceId log is null ah.
Plainly the above method is still only get a request for external timing launched its own task Zezheng it?
This problem often occurs when the interview AOP surfaced.
AOP has a need to know the students can enter Portal: easy aop function in three ways

New interceptor

package com.test.trace.interceptor;

import com.test.trace.support.AbstractMyThreadContext;
import com.test.trace.support.AbstractUUIDShort;
import org.aopalliance.intercept.MethodInterceptor;
import org.aopalliance.intercept.MethodInvocation;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.MDC;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;

/**
 * @description logback全局日志交易id拦截器
 */
@Order(Ordered.HIGHEST_PRECEDENCE)
public class MdcTraceIdMethodInteceptor implements MethodInterceptor {
    protected final Logger log = LoggerFactory.getLogger(this.getClass());

    @Override
    public Object invoke(MethodInvocation invocation) throws Throwable {
        if (MDC.get(AbstractMyThreadContext.MDC_TRACE_ID) != null) {
            return invocation.proceed();
        }
        try {
            String traceId = AbstractUUIDShort.generate();
            AbstractMyThreadContext.setTraceId(traceId);
            MDC.put(AbstractMyThreadContext.MDC_TRACE_ID,traceId);
            return invocation.proceed();
        } catch (Throwable e) {
            log.warn("MdcTraceIdMethodInteceptor error", e.getMessage());
            throw e;
        } finally {
            MDC.remove(AbstractMyThreadContext.MDC_TRACE_ID);
            AbstractMyThreadContext.removeTraceId();
        }
    }
}

package com.test.trace.interceptor;

import org.springframework.aop.Advisor;
import org.springframework.aop.aspectj.AspectJExpressionPointcut;
import org.springframework.aop.support.DefaultPointcutAdvisor;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * @descriptionlogback全局日志交易id拦截器配置 <br/>
 *  主要针对例如 定时任务 MQ等 非HTTP发起的请求没有traceId 配置需要拦截过滤的地址使用 <br/>
 *  配置demo ：<br/>
 *  log.traceId.pointcutExpression=execution(* com.test.service.rabbitmq..*.*(..)) || execution(* com.test.job..*.*(..))
 */
@Configuration
@ConditionalOnProperty(name = "log.traceId.pointcutExpression")
public class MdcTraceIdConfiguration {
    @Value("${log.traceId.pointcutExpression}")
    private String POINTCUT_EXPRESSION;

    @Bean("MdcTraceIdMethodInteceptor")
    public MdcTraceIdMethodInteceptor mdcTraceIdMethodInteceptor() {
        return new MdcTraceIdMethodInteceptor();
    }

    @Bean("MdcTraceIdAdvisor")
    public Advisor MdcTraceIdAdvisor(MdcTraceIdMethodInteceptor mdcTraceIdMethodInteceptor) {
        AspectJExpressionPointcut cut = new AspectJExpressionPointcut();
        cut.setExpression(POINTCUT_EXPRESSION);
        Advisor advisor = new DefaultPointcutAdvisor(cut, mdcTraceIdMethodInteceptor);
        return advisor;
    }
}

Starting small classroom notes and friends: the error log can not be found, mostly lazy, and beat just fine

• When @ConditionalOnProperty is true when you can read the configuration specified name, if you have configured the property value, and it would be better than the specified value as the value of the configuration it is true.

Here it is completely finished path will slip through the net configuration bit, hit a jar package each item just depend on it. If you need to configure each log.traceId.pointcutExpression to get.