摘要

在我们的系统中需要记录日志，包括接口、方法的调用用户信息、用时、参数等。分布式环境中通过dubbo调用rpc服务，需要提供全局traceId追踪完整调用链路。

解决方案

日志中心独立部署，提供rpc服务，日志统一记录统一管理，可以记录到数据库或者log文件中
request入口添加拦截器，采用slf4j提供的MDC记录用户信息
自定义注解和aspect，添加环绕切面，调用日志中心的rpc服务记录日志
添加dubbo拦截器，使用户信息,全局traceId可以跨服务传输
mysql数据库添加Interceptors，将mysql日志记录到ThreadLocal中
log-chain-spring-boot-starter

实现

代码结构

日志中心
提供RPC服务记录日志，demo中采用的dubbo服务，日志记录到数据库中

@Component
@Service(interfaceClass = LogApi.class, timeout = 10000)
public class LogApiImpl implements LogApi{
    @Autowired
    private LogServiceImpl logService;
    public void save(LogDTO dto){
        LogEntity entity=new LogEntity();
        BeanUtils.copyProperties(dto,entity);
        logService.save(entity);
    }
}

日志内容

@Data
public class LogDTO implements Serializable{
    private static final long serialVersionUID = 4069882290787051188L;
    private Integer id;
    private String traceId;
    private String appName;
    private String userId;
    private String userName;
    private String methodName;
    private String type;
    private String param;
    private String result;
    private String description;
    private Long spendTime;
    private Date optTime;
}

log-chain-spring-boot-start

类名	作用
LogChainConfiguration	starter配置类
LogChainProperties	配置文件类，配置appName
LogRecord	注解,配置日志描述以及方法是否记录Mysql日志
LogRecordAspect	从MDC和ThreadLocal中读取信息并记录日志
LogRecordManager	ThreadLocal记录日志链
MysqlLogManager	ThreadLocal记录Mysql日志信息
TraceFilter	dubbo过滤器
MySQLStatementInterceptor	mysql拦截器
LogRecordVO	日志VO，记录方法唯一标识以及是否记录mysql
MysqlLogVO	Mysql日志信息

在这里插入图片描述

servlet拦截器
拦截请求，生成全局TraceId、记录用户信息至MDC中

public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
            throws IOException, ServletException {
        String userId= SysUtil.getCurUserId((HttpServletRequest)request);
        if(!StringUtils.isEmpty(userId)){
            UserDTO user= userService.getUserById(userId);
            if(user!=null){
                MDCUtil.setUserName(user.getUserName());
            }
        }
        MDCUtil.setUserId(userId);
        MDCUtil.setTraceId();
        chain.doFilter(request, response);
    }

自定义注解添加日志描述

@Documented
@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
public @interface LogRecord {
    String description() default "" ;
    Boolean recordMysql() default true;
}

需要记录日志的方法添加注解

@LogRecord(description = "根据用户id查询用户角色列表")
public List<RoleDTO> listUserRole(String userId){
    UserDTO user=getUserById(userId);
    if(user==null){
        return new ArrayList<>();
    }
    //获取用户关联的角色id
    List<String> roleIds=getUserService().listUserRoleIds(userId);
    //根据角色id查询角色详情
    return getUserService().listRoleByRoleId(roleIds);
}

切面记录日志信息
Aspect为所有添加了@LogRecord注解的方法添加环绕切面

@Aspect
@Order(-6)
@Component
public class LogRecordAspect {
    @Reference
    private LogApi logApi;
    private String appName;
    public LogRecordAspect(String appName){
        this.appName=appName;
    }
    private final String MYSQL="-->mysql";
    private final String START="开始";
    private final String END="结束";
    @Pointcut("@annotation(com.ym.logchain.aop.LogRecord)")
    public void logAspect(){
        //记录日志
    }
    @Around("logAspect()")
    public Object recordLog(ProceedingJoinPoint joinPoint) throws Throwable {
        //记录前序sql日志
        recordPreLog();
        //获取方法信息、注解信息
        MethodSignature signature = (MethodSignature)joinPoint.getSignature();
        Method method = signature.getMethod();
        Map<String,Object> param=formatParam(joinPoint);
        LogRecord logRecord = method.getAnnotation(LogRecord.class);
        String guid= UUID.randomUUID().toString();
        String description= logRecord.description();
      	addLogRecord(guid,logRecord.recordMysql(),method.getName(),description);
        //记录开始日志
        saveLog(JSONObject.toJSONString(param),"",method.getName(), OptTypeEnum.start.name(),
                description+START,new Long(-1),new Date());
        //执行方法，记录用时
        Long startTime=System.currentTimeMillis();
        Object proceed = joinPoint.proceed();
        Long endTime=System.currentTimeMillis();
        String result="void";
        if(!signature.getReturnType().equals(void.class)){
            result=JSONObject.toJSONString(proceed);
        }
        //记录当前方法的mysql日志
        recordCurLog(guid,method.getName(),description);
        //记录结束日志
        saveLog(JSONObject.toJSONString(param),result,method.getName(), OptTypeEnum.end.name(),
                description+END,endTime-startTime,new Date());
        return proceed;
    }

    /**
     * 保存日志
     * @param param
     * @param result
     * @param methodName
     * @param type
     * @param description
     * @param spendTime
     * @param optTime
     */
    private void saveLog(String param,String result,String methodName,String type,String description,Long spendTime,Date optTime){
        LogDTO log = new LogDTO();
        log.setAppName(appName);
        log.setUserId(MDCUtil.getUserId());
        log.setUserName(MDCUtil.getUserName());
        log.setMethodName(methodName);
        log.setType(type);
        log.setDescription(description);
        log.setTraceId(MDCUtil.getTraceId());
        log.setSpendTime(spendTime);
        log.setParam(param);
        log.setResult(result);
        log.setOptTime(optTime);
        logApi.save(log);
    }

    /**
     * 格式化参数
     * @param joinPoint
     * @return
     */
    private Map<String,Object> formatParam(JoinPoint joinPoint){
        Map<String,Object> res=new HashMap();
        MethodSignature signature = (MethodSignature) joinPoint.getSignature();
        String[] paramNames = signature.getParameterNames();
        if(paramNames==null || paramNames.length==0){
            return res;
        }
        Object[] params=joinPoint.getArgs();
        for (int i = 0; i < paramNames.length; i++) {
            res.put(paramNames[i],params[i]);
        }
        return res;
    }
}

保存前序方法中存储的mysql日志记录

/**
* 保存之前未保存的日志
*/
private void recordPreLog(){
   List<LogRecordVO> preLogs = LogRecordManager.getLogRecord();
   for (LogRecordVO preLog : preLogs) {
       List<MysqlLogVO> preMysqlLogs = MysqlLogManager.getMysqlLog(preLog.getId());
       for (MysqlLogVO preMysqlLog : preMysqlLogs) {
           String sqlMethodName=preLog.getMethodName()+MYSQL;
           String sqlDesc=preLog.getDescription()+MYSQL;
           saveLog(preMysqlLog.getSql(),preMysqlLog.getResult(),sqlMethodName,OptTypeEnum.mysql.name(),sqlDesc,preMysqlLog.getSpendTime(),preMysqlLog.getOptTime());
       }
       MysqlLogManager.remove(preLog.getId());
   }
}

添加是否记录mysql以及生成guid至ThreadLocal中
方便后续mysql检查是否记录日志以及对应归属那个方法

/**
* 添加日志信息
* @param guid
* @param recordMysql
* @param methodName
* @param description
*/
private void addLogRecord(String guid,Boolean recordMysql,String methodName,String description){
   LogRecordVO vo=new LogRecordVO();
   vo.setId(guid);
   vo.setRecordSql(recordMysql);
   vo.setMethodName(methodName);
   vo.setDescription(description);
   LogRecordManager.addLogRecord(vo);
}

保存当前方法mysql日志
该日志由mysql拦截器保存至mysql中并通过guid关联对应方法

/**
* 记录当前方法的数据库日子
* @param guid
* @param methodName
* @param description
*/
private void recordCurLog(String guid,String methodName,String description){
   List<MysqlLogVO> logs=MysqlLogManager.getMysqlLog(guid);
   for (MysqlLogVO log : logs) {
       String sqlMethodName=methodName+MYSQL;
       String sqlDesc=description+MYSQL;
       saveLog(log.getSql(),log.getResult(),sqlMethodName,OptTypeEnum.mysql.name(),sqlDesc,log.getSpendTime(),log.getOptTime());
   }
   MysqlLogManager.remove(guid);
   LogRecordManager.remove();
}

dubbo拦截器
通过dubbo拦截器
资源文件夹下创建 META-INF/dubbo 文件夹，创建com.alibaba.dubbo.rpc.Filter 文件，并编辑文件内容traceIdFilter=com.ym.filter.TraceIdFilter
调用服务时将traceId及用户信息添加至RpcContext中
服务被调用时从RpcContext中获取traceId及用户信息转存至MDC中

@Activate(group = {Constants.CONSUMER, Constants.PROVIDER} , order = -9999)
public class TraceIdFilter implements Filter {
    @Override
    public Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException {
        String traceId = RpcContext.getContext().getAttachment("traceId");
        if (!StringUtils.isEmpty(traceId) ) {
            MDCUtil.setTraceId(traceId);
            MDCUtil.setUserId(RpcContext.getContext().getAttachment("userId"));
            MDCUtil.setUserName(RpcContext.getContext().getAttachment("userName"));
        } else {
            RpcContext.getContext().setAttachment("traceId", MDCUtil.getTraceId());
            RpcContext.getContext().setAttachment("userId", MDCUtil.getUserId());
            RpcContext.getContext().setAttachment("userName", MDCUtil.getUserName());
        }
        return invoker.invoke(invocation);
    }
}

Mysql拦截器
记录sql以及执行时间
数据库url中添加拦截器

spring.datasource.url=jdbc:mysql://127.0.0.1:3306/test_ym?characterEncoding=UTF-8&useSSL=false&statementInterceptors=com.ym.logchain.interceptor.MySQLStatementInterceptor

public class MySQLStatementInterceptor implements StatementInterceptorV2 {
    private ThreadLocal<Long> timeHolder = new ThreadLocal<Long>();

    @Override
    public void destroy() {
    }

    @Override
    public boolean executeTopLevelOnly()
    {
        return true;
    }

    @Override
    public void init(Connection arg0, Properties arg1) throws SQLException {
    }

    @Override
    public ResultSetInternalMethods postProcess(String sql, Statement statement, ResultSetInternalMethods methods,
                                                Connection connection, int warningCount, boolean noIndexUsed, boolean noGoodIndexUsed, SQLException statementException) throws SQLException {
        LogRecordVO vo = LogRecordManager.getCurrentLogRecord();
        if(vo!=null && vo.getRecordSql()){
            String exeSql=getSql(statement);
            if (StringUtils.isNotBlank(exeSql)){
                Long useTime = System.currentTimeMillis() - timeHolder.get();
                MysqlLogVO log=new MysqlLogVO();
                log.setSql(exeSql);
                log.setSpendTime(useTime);
                log.setResult("");
                log.setOptTime(new Date());
                MysqlLogManager.addMysqlLog(vo.getId(),log);
            }
        }
        return null;
    }

    @Override
    public ResultSetInternalMethods preProcess(String sql, Statement statement, Connection connection) throws SQLException {
        timeHolder.set(System.currentTimeMillis());
        return null;
    }

    private String getSql(Statement statement) {
        String sql = null;
        if (statement instanceof PreparedStatement)
        {
            try
            {
                sql = ((PreparedStatement)statement).asSql();
            }catch (Exception e){
                e.printStackTrace();
            }
        }
        return sql;
    }
}

效果

单个应用内部调用链
应用间调用链

一张小羊毛

发布了25 篇原创文章 · 获赞 28 · 访问量 5万+

私信关注

日志链路追踪

摘要

解决方案

实现

效果

猜你喜欢