Author: a push application platform infrastructure senior research engineer hooligan
01 Business Background
With the popularity of micro-services architecture, systems become more complex, the monomer system is split into a number of modules, each module communicate via a lightweight communication protocol, mutual cooperation and common system functions.
Monomer architecture, a very clear link call request, generally by the load balancer forwards the user request to the back-end service, the service processing performed by the back-end services needed data acquired from an external storage, after the request has been handled and then through the load balancer to the user.
In the micro-service architecture, a request is often requires multiple processing modules to work together, the different modules may also be dependent on different external memory, each module further implementation techniques vary, how a request is performed between the different modules of the entire system circulation, how to call the relationship between the various modules across the entire call chain, the length of time of each micro-processing services, processing results are correct, it is difficult to track, and this information for the entire system operation and maintenance, performance analysis, fault track are particularly helpful, but also because of this, only the various distributed link tracking technology.
02 Distributed Link Tracking Status
Distributed link tracking technology there are many, there are also open source and closed source. There are open source Jaeger, PinPoint, Zipkin, SkyWalking, CAT, etc., are closed source Google Dapper, Taobao Hawkeye Tracing, Sina Watchman, MTrace the United States and other groups. CNCF (Cloud Native Computing Foundation) in order to solve the industry's distributed cross-platform compatibility problem tracking system designed standard trace of the proposed unified paradigm -OpenTracing distributed tracking system products, Zipkin also partially supported OpenTracing standard.
03 reasons for choosing the Zipkin
During the practice, the selection is based on the following reasons Zipkin link to track:
• the open source, community activists
• Support for multiple languages, Nodejs, Lua, Java has open source implementation, and our services are primarily three language the
• provide query API, to facilitate secondary development
04Zipkin architecture introduced
Zipkin overall structure as shown below:
Zipkin's overall architecture
(Zipkin quoted from the official website: https://zipkin.io/pages/architecture.html )
Where:
• Instrumented Instrumented server and Client specific services need to be integrated in a distributed system, collect trace information, call Transport, Zipkin to send tracking information to the Server.
• Transport Interface Zipkin is provided externally, support HTTP, Kafka, Scribe other means of communication.
• Zipkin That Zipkin server, includes four modules:
Collector: receiving tracking information for each application service delivery;
Storage: Zipkin back-end storage, support for In-Memory, MySql, Elasticsearch and Cassandra;
API: provide external inquiry interface;
the UI: provide external Web interface. 
Http Tracing timing diagram
(Zipkin quoted from the official website: https://zipkin.io/pages/architecture.html )
以上是Http Tracing的时序图，用户的请求/foo首先被Trace Instrumentationlan拦截，记录下Tags，时间戳，同时在Header里增加Trace信息，然后再流转到后端服务进行处理，处理完成后，正常响应，Trace Instrumentationlan拦截响应，记录处理延时后，将Response正常返回给调用方，同时异步地将Trace的Span发送给Zipkin Server。Span中的traceId是在整个调用链路上唯一的ID，用于唯一标识一条调用链。
05个推的Zipkin实践
个推的微服务是基于Kubernetes和Docker进行部署的，每个微服务对应于Kubernetes中的一组Pod。
在整个微服务体系中，API网关是基于Openresty开发的，主要使用Lua进行开发；后端服务主要使用Node.js和Java进行开发实现。在对接Zipkin时，不同的微服务采用不同的方式进行实现。
API网关主要通过增加网关插件（主要参考了Kong的Zipkin插件实现）来实现与Zipkin的对接；Node.js实现的服务主要使用了中间件实现与Zipkin的对接；Java服务使用了spring-cloud-sleuth来与Zipkin对接。 整体的架构如下图所示：
个推基于Zipkin的分布式链路追踪系统的整体架构
其中，Zipkin也容器化部署在Kubernetes集群中，简化了Zipkin的搭建和部署。如下图所示，通过Zipkin可以很方便地追踪请求的调用链路，整个调用链上各个服务的处理耗时，响应状态，服务间的调用关系都可以方便地在Zipkin中进行查询。Zipkin对于分析整个系统的性能瓶颈，定位故障也都有很大的帮助。
Zipkin的Web界面
06总结
Zipkin as a distributed link tracking system, the application has invade smaller, higher community activity, supports multiple languages and other advantages, the general open source implementation based on slightly modified can be achieved with the butt of Zipkin. So a push in the micro-services architecture also introduces Zipkin, with Zipkin to track calling relationships micro services, micro-service performance analysis and troubleshooting. The future will be based on a push Zipkin do secondary development, to provide a more friendly interface.