Author: David Hope
In the fast-paced world of software development, especially in the cloud-native world, DevOps and SRE teams are increasingly important partners for application stability and growth.
DevOps engineers continuously optimize software delivery, while SRE teams act as stewards of application reliability, scalability, and top performance. challenge? These teams need a cutting-edge observability solution that includes full-stack insights that allow them to quickly manage, monitor, and correct potential disruptions before they ultimately lead to operational challenges.
Observability in a modern distributed software ecosystem is more than just monitoring — it requires unlimited data collection, processing precision, and correlating that data with actionable insights. However, the road to achieving this holistic view is fraught with obstacles, from resolving version incompatibilities to battling restrictive proprietary code.
OpenTelemetry (OTel) will bring the following benefits to users who adopt it:
- Free yourself from vendor lock-in and ensure best-in-class observability with OTel.
- View unified logs, metrics, and traces harmoniously unified to provide a complete system view.
- Improve your application oversight with richer and enhanced tools.
- Protect your previous detection investments by taking advantage of backward compatibility.
- Embark on your OpenTelemetry journey with a simple learning curve that simplifies onboarding and scalability.
- Rely on proven, future-proof standards to increase your confidence in every investment.
In this blog, we will explore how to use auto-instrumentation in Java applications using Docker without having to refactor any part of the application code. We will use an application called Elastiflix which helps to highlight automatic detection in a simple way.
Applications, prerequisites and configuration
The app we're using in this blog is called Elastiflix , a movie streaming app. It consists of multiple microservices written in .NET, NodeJS, Go, and Python.
Before we instrument the sample application, we first need to understand how Elastic receives telemetry data.
All APM capabilities of Elastic Observability are available through OTel data. Some of these include:
- Service map
- Service details (latency, throughput, failed transactions)
- Dependencies between services, distributed tracing
- Transactions (Tracking)
- Machine Learning (ML) Relevance
- Log correlation
In addition to Elastic's unified view of APM and telemetry data, you can use Elastic's powerful machine learning capabilities to reduce analytics and generate alerts to help reduce MTTR.
prerequisites
- Elastic Cloud account— Sign up now .
- Clone the Elastiflix demo application , or your own Java application
- Basic understanding of Docker—possibly installing Docker Desktop
- Have basic understanding of Java
View sample source code
The complete source code, including the Dockerfile used in this blog, can be found on GitHub . This repository also contains the same application, but without instrumentation . This allows you to compare each file and see the differences.
The following steps will show you how to implement this application and run it from the command line or Docker. If you're interested in a more complete OTel example, check out the docker-compose file here which will show the complete project.
Step-by-step guide
Step 0: Sign in to your Elastic Cloud account
This blog assumes you have an Elastic Cloud account - if not, follow the instructions to get started with Elastic Cloud .
Step 1: Configure automatic detection for Java services
We will use automatic detection with a Java service in the Elastiflix demo application .
We will use the following services from Elastiflix:
Elastiflix/java-favorite-otel-autoAccording to the OpenTelemetry Automatic Instrumentation for Java documentation and documentation, you just need to install the appropriate Java package.
Create a local OTel directory to download the OpenTelemetry Java agent. Download opentelemetry-javaagent.jar.
>mkdir /otel
>curl -L https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar –output /otel/opentelemetry-javaagent.jarIf you want to run the service on the command line, you can use the following command:
java -javaagent:/otel/opentelemetry-javaagent.jar \
-jar /usr/src/app/target/favorite-0.0.1-SNAPSHOT.jar --server.port=5000For our application, we will execute it as part of a Dockerfile.
Dockerfile
Start with a base image containing Java runtime
FROM maven:3.8.2-openjdk-17-slim as build
# Make port 8080 available to the world outside this container
EXPOSE 5000
# Change to the app directory
WORKDIR /usr/src/app
# Copy the local code to the container
COPY . .
# Build the application
RUN mvn clean install
USER root
RUN apt-get update && apt-get install -y zip curl
RUN mkdir /otel
RUN curl -L -o /otel/opentelemetry-javaagent.jar https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v1.28.0/opentelemetry-javaagent.jar
COPY start.sh /start.sh
RUN chmod +x /start.sh
ENTRYPOINT ["/start.sh"]Step 2: Run the Docker image using environment variables
As specified in the OTEL Java documentation , we will use environment variables and pass in configuration values to enable it to connect to Elastic Observability's APM server .
Since Elastic natively accepts OTLP, we only need to provide the endpoint and authentication that the OTEL Exporter needs to send data, as well as some other environment variables.
Get Elastic Cloud variables
You can copy the endpoint and token from Kibana under the path "/app/home#/tutorial/apm".
You will need to copy the following environment variables:
OTEL_EXPORTER_OTLP_ENDPOINT
OTEL_EXPORTER_OTLP_HEADERSBuild Docker image
docker build -t java-otel-auto-image .Run the Docker image
docker run \
-e OTEL_EXPORTER_OTLP_ENDPOINT="REPLACE WITH OTEL_EXPORTER_OTLP_ENDPOINT" \
-e OTEL_EXPORTER_OTLP_HEADERS="REPLACE WITH OTEL_EXPORTER_OTLP_HEADERS" \
-e OTEL_RESOURCE_ATTRIBUTES="service.version=1.0,deployment.environment=production" \
-e OTEL_SERVICE_NAME="java-favorite-otel-auto" \
-p 5000:5000 \
java-otel-auto-imageYou can now make some requests to generate tracking data. Note that these requests are expected to return errors because this service relies on a Redis connection that you are not currently running. As mentioned before, you can find a more complete example using docker-compose here .
curl localhost:5000/favorites
# or alternatively issue a request every second
while true; do curl "localhost:5000/favorites"; sleep 1; done;Step 3: Explore tracing and logs in Elastic APM
Once you have this up and running, you can ping the instrumentation service's endpoint (/favorites in our example) and you should see the application appear in Elastic APM, as shown below:
It will first track the key metrics of throughput and latency that the SRE needs to focus on.
Digging deeper, we can see an overview of all transactions.
Let’s take a look at the specific transactions:
Click Logs and we see that the logs are also brought here. OTel Agent will automatically pull in the logs and associate them with your traces:
This gives you complete visibility into logs, metrics, and traces!
Basic concepts: How APM works with Java
Before we continue, let's first understand some basic concepts and terminology.
- Java Agent : This is a tool that can be used to detect (or modify) the bytecode of class files in a Java Virtual Machine (JVM). Java agents are used for various purposes such as performance monitoring, logging, security, etc.
- Bytecode : This is the intermediate code generated by the Java compiler based on the Java source code. This code is interpreted or compiled on the fly by the JVM to produce machine code that can be executed.
- Byte Buddy : Byte Buddy is a Java code generation and manipulation library. It is used to create, modify or adapt Java classes at runtime. In the context of a Java Agent, Byte Buddy provides a powerful and flexible way to modify bytecode. Both the Elastic APM agent and the OpenTelemetry agent use Byte Buddy behind the scenes.
Now, let's talk about how auto-detection works with Byte Buddy :
Auto-instrumentation is the process by which an agent modifies the bytecode of an application class, usually to insert monitoring code. The agent does not directly modify the source code, but rather the bytecode loaded into the JVM. This is done when the JVM loads the class, so the modifications are effective at runtime.
Here's a simplified explanation of the process:
- Starting the JVM using an agent : When starting a Java application, you can specify a Java agent using the -javaagent command line option. This instructs the JVM to load the agent before calling the application's main method. At this point, the agent has the opportunity to set class transformers.
- Register the class file converter with Byte Buddy : Your agent will register the class file converter with Byte Buddy. A converter is a piece of code that is called every time a class is loaded into the JVM. This converter receives the bytecode of a class and can modify that bytecode before actually using the class.
- Convert bytecode : When the converter is called, it will use Byte Buddy's API to modify the bytecode. Byte Buddy allows you to specify transformations in a high-level, expressive way instead of manually writing complex bytecode. For example, you can specify a class and a method in that class that you want to instrument, and provide an "interceptor" to add new behavior to the method.
- For example, suppose you want to measure the execution time of a method. You'll instruct Byte Buddy to locate specific classes and methods, and then provide an interceptor that wraps the method calls with timing code. Each time this method is called, first the interceptor is called and the start time is measured, then the original method is called, and finally the end time is measured and the duration is printed.
- Using the converted class : Once the agent sets its converter, the JVM continues loading the class as usual. Converters are called every time a class is loaded, allowing them to modify the bytecode. Your application will then use these transformed classes as if they were the original classes, but they now have the extra behavior you injected via the interceptor.
In essence, Byte Buddy's automatic detection is to modify the behavior of Java classes at runtime without directly changing the source code. This is particularly useful for cross-cutting concerns such as logging, monitoring, or security, because it allows you to centralize this code in the Java agent rather than spreading it throughout the application.
Summarize
With this Dockerfile, you have transformed a simple Java application into an application that uses OpenTelemetry auto-detection. This will greatly help in understanding application performance, tracking errors, and gaining insight into how users interact with the software.
Remember, observability is an important aspect of modern application development, especially in distributed systems. With tools like OpenTelemetry, understanding complex systems becomes easier.
In this blog we discuss the following:
- How to use OpenTelemetry to automatically detect Java.
- Using standard commands in Docker files, automatic detection can be done efficiently and without adding code in multiple places, allowing for manageability.
- Using OpenTelemetry and its support for multiple languages, DevOps and SRE teams can automatically instrument their applications, easily gain immediate visibility into the health of the entire application stack, and reduce mean time to resolution (MTTR).
Because Elastic can support multiple methods of ingesting data, whether automatic detection using open source OpenTelemetry or manual detection using its native APM agent, you can focus on a few applications first and use them later to best suit your business needs. way to turn on OpenTelemetry in your application.
原文：Auto-instrumentation of Java applications with OpenTelemetry | Elastic Blog