(Adversity gives people precious opportunities for tempering. Only those who can stand the test of the environment can be regarded as the real strong. Since ancient times, most of the great men have struggled from adversity with an indomitable spirit. ——Panasonic Fortunately)
Service Link Tracking
Service link tracking means that we can quickly check the entire business process from start to finish through a mark. Quickly locate information and exceptions in the entire business link.
Common business scenarios are front-end and back-end communication, and communication between back-end services.
The link id industry is usually calledTraceId
Front-end and back-end communication
scene one
The front-end generates a "request-id" before each request to the back-end to unify the link tracking id of the front-end and back-end. The front-end can use this id to track the link of its own business, and the back-end can also directly retrieve it from the request headers. use.
advantage
- The front-end and back-end link tracking ids are unified, making it easy to troubleshoot business problems
- No need to generate link id at the back end, reducing business performance loss
shortcoming
- The link id is generated by the front end, and there are security issues
- Front-end and back-end business coupling is too high
scene two
When the front-end does not maintain the front-end log or there is no log, the link id is usually managed by the back-end itself. Before receiving the request, the multi-thread feature is used to store the link id into the thread context and obtain it in the subsequent business.
advantage
- The link id is managed by the backend and decoupled from the frontend
shortcoming
- It is impossible to unify the link id with the front-end, which increases the difficulty of unified troubleshooting of the front-end and front-end
Inter-service communication
In back-end multi-service communication or third-party communication, in order to quickly locate the problem, it is often necessary to unify the link id of the logs between multiple services. If A->B->C, A service will put the generated traceId in the request header and hand it over For service B, service B obtains the id in the request header for business processing, and service C also performs the same processing, so the log ids of the three services are the same, which can greatly reduce the difficulty of troubleshooting in log troubleshooting
Difficulties in NodeJs service link tracking
For multi-threaded services such as java, each thread has its own context that can be used for link tracking, but the main thread of nodejs is single-threaded, and cannot perform multi-threaded processing for each request.
In the past, the link tracking of the node service usually made the ctx context variable similar to the koa framework, and maintained the context through the transmission of each layer to achieve the purpose of link tracking. But this approach has some redundancy, which makes the code bloated and difficult to maintain.
Therefore, async_hooks asynchronous function hooks are provided in subsequent versions of nodejs, which can save the context in a request link and make link tracking possible
Best Practices
Use the nestjs service to demonstrate link tracking in three different ways
- async_hooks
NodeJs Official - cls-rtracer
cls-rtracer best practice
link tracing performance - nestjs-cls
nestjs official
async_hooks
async_hooks log printing, using lazy loading form, providing static method printing
import {
AsyncLocalStorage } from "async_hooks";
import * as dayjs from "dayjs";
let asyncLocalStorage: AsyncLocalStorage<unknown>;
export class LogAsyncLocalStorage {
static init (next) {
this.getInstance().run(Math.random(), () => next());
}
static getInstance () {
if (!asyncLocalStorage) {
asyncLocalStorage = new AsyncLocalStorage();
}
return asyncLocalStorage;
}
static info (message: string) {
const nowTime = dayjs().format('YYYY-MM-DD HH:mm:ss');
const requlestId = asyncLocalStorage.getStore();
console.log(`requestId：${
requlestId} time：${
nowTime} info：${
message}`);
}
}
cls-rtracer
Compared with native async_hooks, this component is more in line with the middleware loading mode, and the code is less intrusive.
Two components are provided. TraceIdMiddleware uses an automatically generated uuid as traceId, and TraceIdRandomMiddleware uses a custom random number as traceId.
Random numbers are used here for demonstration purposes. In actual business, they are usually unique strings or numbers such as uuid
import * as rTracer from 'cls-rtracer';
export const TraceIdMiddleware = rTracer.expressMiddleware({
useHeader: true,
headerName: 'traceId',
});
export const TraceIdRandomMiddleware = rTracer.expressMiddleware({
requestIdFactory: () => Math.random(),
});
nestjs-cls
It is optimized for nestjs, and on the basis of modularization, ts inspection and other customized apis are done. Among them, userId is a custom traceId. It is initialized when the package is imported, which will be mentioned below.
import {
Injectable } from "@nestjs/common/decorators";
import * as dayjs from "dayjs";
import {
ClsService } from "nestjs-cls/dist/src/lib/cls.service";
@Injectable()
export class LogNestCls {
constructor(
private readonly clsService: ClsService
) {
}
info (message: string) {
const nowTime = dayjs().format('YYYY-MM-DD HH:mm:ss');
const requlestId = this.clsService.get('userId');
console.log(`requestId：${
requlestId} time：${
nowTime} info：${
message}`);
}
}
control layer
import {
Controller, Get } from '@nestjs/common';
import {
AppService } from './app.service';
import {
LogAsyncLocalStorage } from './log.asyncLocalStorage';
import {
LogNestCls } from './log.nest.cls';
import {
LogRtracer } from './log.rTracer';
@Controller()
export class AppController {
constructor(
private readonly appService: AppService,
private readonly LogNestCls: LogNestCls
) {
}
@Get('asyncLocalStorage')
getHeeloAsyncLocalStorage () {
LogAsyncLocalStorage.info('收到前端请求，即将进入service');
return this.appService.getHeeloAsyncLocalStorage();
}
@Get('rTracer')
getRTracer () {
LogRtracer.info('收到前端请求，即将进入service');
return this.appService.getRTracer();
}
@Get('nestCliId')
getNestCli () {
this.LogNestCls.info('收到前端请求，即将进入service');
return this.appService.getNestCli();
}
}
Business Layer
import {
Injectable } from "@nestjs/common";
import {
LogRtracer } from "./log.rTracer";
import {
LogAsyncLocalStorage } from './log.asyncLocalStorage';
import {
LogNestCls } from "./log.nest.cls";
@Injectable()
export class AppService {
constructor(
private readonly LogNestCls: LogNestCls
) {
}
getRTracer () {
LogRtracer.info('getHello 进入service，即将处理业务');
}
getHeeloAsyncLocalStorage () {
LogAsyncLocalStorage.info('getHeeloAsyncLocalStorage 进入service，即将处理业务');
}
getNestCli () {
this.LogNestCls.info('getNestCli 进入service，即将处理业务');
}
}
module loading
async_hooks
import {
MiddlewareConsumer, Module, NestModule } from "@nestjs/common";
import {
AppController } from "./app.controller";
import {
AppService } from "./app.service";
import {
LogAsyncLocalStorage } from "./log.asyncLocalStorage";
@Module({
imports: [],
controllers: [AppController],
providers: [AppService],
})
export class AppModule implements NestModule {
configure (consumer: MiddlewareConsumer) {
consumer
.apply((req, res, next) => LogAsyncLocalStorage.init(next))
.forRoutes('*')
}
}
cls-rtracer
import {
MiddlewareConsumer, Module, NestModule } from "@nestjs/common";
import {
AppController } from "./app.controller";
import {
AppService } from "./app.service";
import {
TraceIdMiddleware } from "./rTracer";
@Module({
imports: [],
controllers: [AppController],
providers: [AppService],
})
export class AppModule implements NestModule {
configure (consumer: MiddlewareConsumer) {
consumer
.apply(TraceIdMiddleware)
// .apply(TraceIdRandomMiddleware)
.forRoutes('*')
}
}
nestjs-cls
import {
Module } from "@nestjs/common";
import {
AppController } from "./app.controller";
import {
AppService } from "./app.service";
import {
ClsModule } from "nestjs-cls";
import {
LogNestCls } from "./log.nest.cls";
@Module({
imports: [
ClsModule.forRoot({
global: true,
middleware: {
mount: true,
setup: (cls, req) => {
cls.set('userId', Math.random());
}
}
})
],
controllers: [AppController],
providers: [AppService, LogNestCls],
})
export class AppModule {
}
operation result
async_hooks
requestId：0.8233251518769558 time：2023-05-27 22:40:27 info：收到前端
请求，即将进入service
requestId：0.8233251518769558 time：2023-05-27 22:40:27 info：getHeeloAsyncLocalStorage 进入service，即将处理业务
cls-rtracer
requestId：354b79f0-fc9c-11ed-beac-f57b0395da38 time：2023-05-27 22:38:55 info：收到前端请求，即将进入service
requestId：354b79f0-fc9c-11ed-beac-f57b0395da38 time：2023-05-27 22:38:55 info：getHello 进入service，即将处理业务
nestjs-cls
requestId：0.43237919752342924 time：2023-05-27 22:41:04 info：收到前端请求，即将进入service
requestId：0.43237919752342924 time：2023-05-27 22:41:04 info：getNestCli 进入service，即将处理业务