SkyWalking-Open source application performance monitoring system

Table of contents

I. Overview

1.1 Features

1.2 Usage Scenarios

1.3 Functions

1.4 Overall Architecture

2. Installation and configuration

2.1 Build SkyWalking

2.2 Build SkyWalking OAP service

2.3 SkyWalking UI Construction

2.4 skywalking-oap-server port introduction

2.5 SkyWalking Agent

2.6 skywalking-tomcat

---

I. Overview

SkyWalking is a distributed tracing, service grid and application performance management system, which can help users gain insight into application performance bottlenecks and provide performance indicators. SkyWalking supports multiple languages, including Java, .NET, NodeJS, Go, etc. With SkyWalking, you can monitor inter-service calls in the microservice architecture and provide performance indicators.

1.1 Features

SkyWalking has the following features:

• Distributed application tracing: SkyWalking can trace request chains in distributed applications to understand the flow path and latency of requests.
• Application performance measurement: SkyWalking can measure the performance of applications and provide performance indicators to users.
• Service Mesh Tracking: SkyWalking can track inter-service communication in a service mesh and provide performance metrics about inter-service communication.
• Cross-language support: SkyWalking supports multiple languages, including Java, .NET, NodeJS, Go, etc.

1.2 Usage Scenarios

SkyWalking is suitable for the following scenarios:

• Microservice architecture: SkyWalking can help users monitor inter-service calls in the microservice architecture.
• Distributed applications: SkyWalking can help users trace request chains in distributed applications and understand the flow path and latency of requests.
• Service Mesh: SkyWalking can help users track inter-service communication in a service mesh and provide performance metrics about inter-service communication.

1.3 Functions

1.  Performance metrics analysis of services, service instances and endpoints (e.g. individual HTTP URIs)
2.  Root Cause Analysis
3.  Service Topology Analysis
4.  Service and endpoint dependency analysis
5.  Slow service and slow endpoint detection
6.  Distributed Tracing and Context Propagation
7. Overall architecture

1.4 Overall Architecture

1. collector: link data collector, the data can land in MySQL, ElasticSearch, TiDB, H2, etc.
2. agent: probe, use Javaagent for bytecode implantation, non-intrusive collection, and send data collector via HTTP or gRPC
3. web: web visualization platform, used to display landing data

2. Installation and configuration

This article takes v8.7.0 as an example to deploy and use, and we use the version of Elasticsearch 7.x for storage

Environment build

$ mkdir skywalking

$ cd skywalking

# Download the application installation package

wget https://archive.apache.org/dist/skywalking/8.7.0/apache-skywalking-apm

-es7- 8.7.0.tar.gz

 Extract to current directory

tar -zxvf apache-skywalking-apm-es7-8.7.0.tar.gz

cd apache-skywalking-apm-bin-es7

 The specific file directory structure is as follows

1. The agent directory will be copied to the machine where each service is located in the future for use as a probe
2. The bin directory is the service startup script
3. The config directory is the configuration file
4. The oap-libs directory is the jar package required for the operation of the oap service
5. The webapp directory is the jar package required for the web service to run
6. The configuration file location of SkyWalking OAP service is config/application.yml
7. The configuration file location of SkyWalking UI service: webapp/webapp.yml.
8. Before deployment, you first need to prepare the Elasticsearch service. If there is a ready-made one in the project, you can use it directly. The deployment process is as follows:

2.1 Build SkyWalking

Change setting:

<1>, set cluster.name

Modify the configuration file config\application.yml of the collector, in which the configuration namespace is consistent with the value of es-cluster.name; the view method of es-cluster.name curl ip:9200 is as follows:

cd /data/skywalking/apache-skywalking-apm-bin-es7

vi config/application.yml

2.2 Build SkyWalking OAP service

First modify the OAP configuration file to set Elasticsearch as the storage database

cd /data/skywalking/apache-skywalking-apm-bin-es7

vi config/application.yml

# The H2 data source is selected by default in the configuration file, switch to elasticsearch7, and configure elasticsearch7 as the information installed by yourself

# Focus on modifying the storage configuration item, and set the specific storage used by the storage.selector configuration item.

# You can mainly modify the two configurations of nameSpace and clusterNodes, and set the cluster and namespace of Elasticsearch used.

# storage.elasticsearch7 configuration item, set to use Elasticsearch7.X version as storage.

The storage.selector configuration item is used to set the specific storage, and the default is {SW_STORAGE:elasticsearch7}. Then modify nameSpace and clusterNodes in the storage.elasticsearch7 configuration item. If elasticsearch has enabled authentication, you need to modify user and password 

Start the SkyWalking service

  

Start the skywalking oap service

bin/oapServiceInit.sh

return

SkyWalking OAP started successfully!

Open the logs/skywalking-oap-server.log log file to check whether there is an error log and whether the startup is successful. When starting for the first time, because SkyWalking OAP will create a large number of Elasticsearch indexes, it may be slow. The SkyWalking backend supports multiple storage implementors. Most of them can automatically initialize storage when the backend starts, such as Elasticsearch, Database.

2.3 SkyWalking UI Construction

1. Since the default address of SkyWalking UI is 8080, which conflicts with many middleware, you can modify it

# Modify webapp/webapp.yml

server:

  port: 18080

2. Start the SkyWalking UI service 

$ bin/webappService.sh

SkyWalking Web Application started successfully!

Open the logs/logs/webapp.log log file to check whether there is an error log and whether the startup is successful. If you need to modify the parameters of the SkyWalking UI service, you can edit the webapp/webapp.yml configuration file. for example:

server.port: SkyWalking UI service port.

spring.cloud.discovery.client.simple.instances.oap-service: SkyWalking OAP service address array, because the data of SkyWalking UI interface is obtained by requesting SkyWalking OAP service. Note that the 8.7.0 version of the UI background introduced zuul as a proxy for static resources, so the OAP address is no longer set through collector.ribbon.listOfServers.

2.4 skywalking-oap-server port introduction

Start directly:

You can also use the direct startup method: bin/startup.sh to start the skywalking service

After successful startup, two services will be started, one is skywalking-oap-server and the other is skywalking-web-ui

After the skywalking-oap-server service is started, two ports 11800 and 12800 will be exposed, which are port 11800 for collecting monitoring data and port 12800 for accepting front-end requests. To modify the port, you can modify config/applicaiton.yml

Access the UI interface:

The browser opens http://IP:18080, as shown below

The lower right corner of the page can switch between Chinese and English, and you can switch to select the tracking data of the time interval to be displayed

2.5 SkyWalking Agent

The agent probe allows us to dynamically monitor the components used in the java application without modifying the code, obtain the running data and send it to OAP for statistics and storage

The agent probe is implemented in java using java agent technology, without any code changes, the java agent will change the code during runtime through the virtual machine (VM) interface

All files of the agent probe are under the agent folder of Skywalking.

1> Modify the default configuration of the probe

vi agent/config/agent.config

collector.backend_service=${SW_AGENT_COLLECTOR_BACKEND_SERVICES:192.ip:11800}

agent.sample_n_per_3_secs=${SW_AGENT_SAMPLE:1}

2> Sampling rate modification

Agent.sample_n_per_3_secs configuration instructions:

When the number of visits is small, the full collection of links will not bring too much burden to the system, and the operating status of the system can be completely observed. However, when there is a large amount of visits, the full amount of link collection will bring a large performance overhead to the client (agent probe), server (SkyWalking OAP), and storage (such as Elastcsearch) of link collection, and even It will affect the normal operation of the application. In the case of a large number of visits, sampling is often selected to collect only part of the link information. SkyWalking Agent defines the agent.sample_n_per_3_secs configuration item in the agent/config/agent.config configuration file, which sets the number of link data that can be collected every 3 seconds.

3> Place the probe:

We need to copy the agent directory to the server where the Java application is located. In this way, the Java application can be configured to use the SkyWalking Agent. The system is detected by setting startup parameters, and there is no code intrusion.

# Add parameters javaagent, Dskywalking.agent.service_name to the startup command of the service

nohup java -javaagent:agent/skywalking-agent.jar -Dskywalking.agent.service_name=app_name -jar app.jar > app.log 2>&1 &

Take the track-et-ext application as an example

nohup /data/ppm/opt/jdk1.8/bin/java \

-javaagent:/data/skywalking/agent/skywalking-agent.jar \

-Dskywalking.agent.service_name=springboot -Dskywalking.collector.backend_service=192.168.136.123:11800

-jar springboot > springboot.log 2>&1 &0

#There is no need to add the Dskywalking.collector.backend_service parameter under the same host; but this parameter must be added when deploying applications under different hosts

 #After the service starts successfully, visit skywalking to view

 #Dashboard: You can view service performance, the overall time-consuming interface, database sql execution time-consuming ranking, etc.

Topology map: You can view the detected services and the relationship between services

#Tracking: You can see the time-consuming situation of each step in the entire link, where you can see the time-consuming execution of each sql in the method and the corresponding sql, and you can optimize sql in a targeted manner

2.6 skywalking-tomcat

#For application services deployed using tomcat, the configuration is as follows

Package the agent directory under skywalking to the target server when configuring tomcat startup projects on different servers

And specify Dskywalking.collector.backend_service when configuring parameters, the specific configuration is as follows

vi tomcat/webapps/bin/catalina.sh

#Add in the first line, which has the same meaning as the jar startup configuration

CATALINA_OPTS="$CATALINA_OPTS \

-javaagent:/data/skywalking/agent/skywalking-agent.jar \

-Dskywalking.agent.service_name=Tomcat \

-Dskywalking.collector.backend_service=192.168.136.123:11800"; export CATALINA_OPTS