Micro Business Services log collection program

background

Complex and diverse contents of the log, log how to gather valuable is our focus. Log in fact, depends on the value of business operations, the value of the same type under different business scenarios logs would be completely different.
Based on past business practices, combined with a number of enterprise-level business requirements, we focus on the following categories selected log.

• trace log [] trace.log debug log Server engine running problem for system maintenance personnel positioning system.
• System log] [system.log large size of the engine running import, export of logs for the call stack analysis, performance analysis can be used.
• deployment log recording system] [deploy.log start, stop, log information component package deployment, a cluster notification.
• engine logs] [engine.log fine-grained engine running log can be printed contextual data for locating business problems.
• Packet Log [member] member contribution.log package recorded traffic logs (using the underlying component library log output API to write the log)

Here we discuss several special collection for system log collection program

Program: log component was collected by

Here refers to the output file by logback, log4j and other logging component, and then output through the file to logstash, kibana such as logging component to visualize statistics and analysis by the logging component, there is need to harmonize the key log output format for future statistical search .

advantage

• Simple, easy to collect
• Reduce dependence Business
• Fine granularity

Shortcoming

• It depends on logstash, kibana
• Can only be satisfied simply log operation, elaborate or individual needs to operate more complex

Option Two: use aop to intercept controller

Controller layer interception, by the method name is included in the controller insert, update, delete and other key business journal to record.

advantage

• Simple, easy to collect

Shortcoming

• Only simple log record
• Different people are not the same naming convention, the log may not be accurate

Option Three: use annotations in a little accurate service logging

The particle size of the program vary, the code invasive relatively small, operability strong, if desired parameter information of return values ​​or information can be acquired annotation configuration, it can be assembled in jpath fastjson acquires a parameter value below several pseudo-code for reference

@Slf4j
@Aspect
public class SysLogAspect {

	@Around("@annotation(sysLog)")
	@SneakyThrows
	public Object around(ProceedingJoinPoint point, SysLog sysLog) {
		// 根据系统上下文获取相关数据
		Operation logVo = SysLogUtils.getOperationModel();
		Object obj=null;
		try{
			// 操作方式
			logVo.setOperationName(sysLog.value());
			// 操作时间
			logVo.setOperationTime(new Date());
			logVo.setObjectType(sysLog.objectType().name());
			logVo.setAppName(StringUtils.defaultString(sysLog.appName(),"TSP"));
			// 发送异步日志事件
			obj = point.proceed();
			if(StringUtils.isNotBlank(sysLog.objectIdKey())&&StringUtils.isNotBlank(sysLog.objectNameKey())){
				Map<String,Object> params=Maps.newHashMap();
				params.put("args",point.getArgs());
				params.put("response",obj);
				logVo.setObjectId(getKeyValue(sysLog.objectIdKey(),params));
				logVo.setObjectName(getKeyValue(sysLog.objectNameKey(),params));
			}
		}catch (BusinessException e){
			logVo.setException(e.getMessage());
			throw new Exception(e);
		}catch (RuntimeException e){
			logVo.setException(e.getMessage());
			throw new Exception(e);
		}catch (Exception e){
			logVo.setException(e.getMessage());
			throw new Exception(e);
		}finally {
			ApplicationContextUtils.publishEvent(new OperationEvent(logVo));
		}
		return obj;
	}

	private String getKeyValue(String key,Map<String,Object> params){
		try{
			Object mm= JSONPath.eval(params,key);
			if(mm!=null){
				return mm.toString();
			}
		}catch (Exception e){
			log.error("JSONPath.eval:",e);
		}
		return null;
	}


}

``

@SysLog(value = "xxxxxx",objectType = LogObjectType.OFFLINE_THRONG,
        objectIdKey = "$['args'][0][0]['id']",objectNameKey = "$['response'][0][0]['throngName']")

advantage

• easy to use
• More flexible and vary in size

Shortcoming

• There invade Code
• Does not meet the individual needs

Option 4: auditing requirements for complex scenes or manually record, invasive strong

If some public business method, requires a smaller particle size, service data to be recorded or recording the change, then you can only choose a stronger invasive manner to the log, and then direct business code log

logClient.logObject(LogObjectType.XXX,id,UserContext.getUserName(),"编辑xxx","xxx变更为xxxxx");

Program five: the log records sql

You can use mybatis interceptors to, if not used mybatis, you can do a general preparestatement and proxy statement.

Of course, now there are components to satisfy the need to monitor, for example jeager, javamelody, druid, etc.

to sum up

Under normal circumstances we will use a variety of ways to record traffic log, this is to be assessed in several ways with which you can better meet product demand according to specific needs.