Java Agent has such a problem that although the application and the Agent are executed as one, the Agent is actually loaded by the AppClassLoader at the JVM level, but the application code is not necessarily. Therefore, when there are application enhancement codes in the Agent, various problems are likely to occur. OpenTelemetry Agent introduces a special mechanism muzzle to solve these problems. This article will explain how muzzle solves similar problems.
The role of Muzzle
Muzzle is a safety feature of the Java agent that prevents applying instrumentation when a mismatch between the instrumentation code and the instrumented application code is detected.
Simply put, muzzle is a mechanism for class and classloader validation at compile time and runtime. There is a separate part of code in the Agent to realize this complex capability.
As for how it takes effect, we will talk about it later.
Why do you need Muzzle
Muzzle is a mechanism to check if classes match at runtime, so why do we need this mechanism?
Imagine such a scenario: the sdk of otel is referenced in the application, the version is 1.14.0, and the sdk of otel is also referenced in the Agent, but the version is 1.15.0, so how to deal with the actual conflict?
Let’s imagine another scenario: If the user code is enhanced in the Agent, but this part refers to a third-party SDK, and this SDK is also used in the application, and the version may be different, how to solve it?
Of course, the above two scenarios can be solved by using shadow sdk, or by using BootstrapClassLoader to load classes. But this will also encounter other various problems. So Opentelemetry Java Agent provides a muzzle mechanism to solve this problem once and for all.
How Muzzle works
Muzzle is divided into two parts:
- At compile time, muzzle will collect references to the used helper classes and third-party symbols (including classes, methods, variables, etc.)
- At runtime, he will check whether these references are consistent with the classes actually referenced on the classpath
Collect at compile time
Compile-time collection is realized with the help of the gradle plug-in muzzle-generation.
Opentelemetry Java Agent provides such an interface InstrumentationModuleMuzzle:
java复制代码public interface InstrumentationModuleMuzzle {
Map<String, ClassRef> getMuzzleReferences();
static Map<String, ClassRef> getMuzzleReferences(InstrumentationModule module) {
if (module instanceof InstrumentationModuleMuzzle) {
return ((InstrumentationModuleMuzzle) module).getMuzzleReferences();
} else {
return Collections.emptyMap();
}
}
void registerMuzzleVirtualFields(VirtualFieldMappingsBuilder builder);
List<String> getMuzzleHelperClassNames();
static List<String> getHelperClassNames(InstrumentationModule module) {
List<String> muzzleHelperClassNames =
module instanceof InstrumentationModuleMuzzle
? ((InstrumentationModuleMuzzle) module).getMuzzleHelperClassNames()
: Collections.emptyList();
List<String> additionalHelperClassNames = module.getAdditionalHelperClassNames();
if (additionalHelperClassNames.isEmpty()) {
return muzzleHelperClassNames;
}
if (muzzleHelperClassNames.isEmpty()) {
return additionalHelperClassNames;
}
List<String> result = new ArrayList<>(muzzleHelperClassNames);
result.addAll(additionalHelperClassNames);
return result;
}
}
This interface provides some methods for obtaining reference information of helper classes and third-party classes, etc. For all InstrumentationModule, this interface will be applied again. But this interface is very special, he does not implement the class!
InstrumentationModuleMuzzle does not directly implement this interface in the code, but constructs an implementation through ByteBuddy.
Agent implements AsmVisitorWrapper by constructing MuzzleCodeGenerator to completely construct the implementation method of InstrumentationModuleMuzzle. Therefore, although this interface is useless on the surface, it is useful through the construction of dynamic bytecode.
runtime check
The runtime check is also implemented based on ByteBuddy, and the Agent constructs the matching class MuzzleMatcher by implementing AgentBuilder.RawMatcher.
The class implements the matches method, and builds doesMatch to use the data collected at compile time for runtime verification:
java复制代码private boolean doesMatch(ClassLoader classLoader) {
ReferenceMatcher muzzle = getReferenceMatcher();
boolean isMatch = muzzle.matches(classLoader);
if (!isMatch) {
MuzzleFailureCounter.inc();
if (muzzleLogger.isLoggable(WARNING)) {
muzzleLogger.log(
WARNING,
"Instrumentation skipped, mismatched references were found: {0} [class {1}] on {2}",
new Object[] {
instrumentationModule.instrumentationName(),
instrumentationModule.getClass().getName(),
classLoader
});
List<Mismatch> mismatches = muzzle.getMismatchedReferenceSources(classLoader);
for (Mismatch mismatch : mismatches) {
muzzleLogger.log(WARNING, "-- {0}", mismatch);
}
}
} else {
if (logger.isLoggable(FINE)) {
logger.log(
FINE,
"Applying instrumentation: {0} [class {1}] on {2}",
new Object[] {
instrumentationModule.instrumentationName(),
instrumentationModule.getClass().getName(),
classLoader
});
}
}
return isMatch;
}
It is worth noting that since the muzzle check is expensive, it is only executed after the InstrumentationModule#classLoaderMatcher() and TypeInstrumentation#typeMatcher() matchers have done a match. The results of the muzzle matcher are cached per classloader, so it is only executed once for the entire detection module.
Summarize
Otel Agent has spent a lot of energy building a muzzle system to resolve class conflicts between Agents and applications. Although it is very complicated, this part of the implementation is hidden from users, so users will feel very friendly when using it. If you are interested, you can study the code implementation of muzzle by yourself, and you may have different gains.