overview
Recently, online monitoring found that OOM has increased significantly, so I tried to locate and fix this problem. After fixing some memory leaks and large object occupation problems, OOM still did not reach the normal standard. In these newly reported hprof files, it was found that almost There is an object called FinalizerReference in all cases, the number is huge, and the memory usage ranks first, so it is judged that it is the culprit that caused the OOM rise this time.
ReferenceQueue
First of all, understand what the ReferenceQueue reference queue is. In short, it is a queue used to store Reference objects. When the object referenced by the Reference object is recycled by the GC, the Reference object will be added to the reference queue ReferenceQueue . Right now:
valqueue= ReferenceQueue<Bean>()
valbean= Bean()
valreference= SoftReference(bean, queue)
复制代码In the above code, a bean strong reference and a reference soft reference are created. When the bean is recycled, the soft reference reference object will be placed in the queue queue, and the subsequent reference object needs to be processed by the developer (such as polling from the queue) wait).
The principle of Leakcanary to detect memory leaks is the application's ReferenceQueue reference queue, taking Activity as an example:
Monitor the life cycle of Activity.
At the time of onDestory, the Refrence reference object corresponding to the Actitity is created through the ReferenceQueue.
After a period of time, read from the ReferenceQueue, if there is a Refrence of this Activity, it means that the Refrence of this Activity has been recycled; if the ReferenceQueue does not have a Refrence of this Activity, it means that there is a memory leak.
Regarding Reference citations, I searched many articles on the Internet, and everyone probably knows it clearly, so I won’t repeat them. I have made some related notes before. If you are interested, you can take a look at JVM Garbage Collection-Reference .
FinalizerReference
First introduce the Finalizer object, which refers to the object whose finalize() method is overridden in its Java class, and the method is not empty. Some places call this class f class, and we also call it that. During the class loading process, the loaded Java class will be marked as class f or not.
FinalizerReference overview
Now let's see what FinalizerReference is. Only by knowing yourself and the enemy can you know how to fix it. This FinalizerReference is used to assist the FinalizerDaemon thread to process the finalize() of the object . There are three nouns at one time. Let's start with FinalizerReference.
Take a look at part of its code:
// java/lang/ref/FinalizerReference.javapublicfinalclassFinalizerReference<T> extendsReference<T> {
// This queue contains those objects eligible for finalization.publicstaticfinal ReferenceQueue<Object> queue = newReferenceQueue<Object>();
// This list contains a FinalizerReference for every finalizable object in the heap.// Objects in this list may or may not be eligible for finalization yet.privatestatic FinalizerReference<?> head = null;
// The links used to construct the list.private FinalizerReference<?> prev;
private FinalizerReference<?> next;
publicstaticvoidadd(Object referent) {
FinalizerReference<?> reference = newFinalizerReference<Object>(referent, queue);
synchronized (LIST_LOCK) {
reference.prev = null;
reference.next = head;
if (head != null) {
head.prev = reference;
}
head = reference;
}
}
publicstaticvoidremove(FinalizerReference<?> reference) {
synchronized (LIST_LOCK) {
FinalizerReference<?> next = reference.next;
FinalizerReference<?> prev = reference.prev;
reference.next = null;
reference.prev = null;
if (prev != null) {
prev.next = next;
} else {
head = next;
}
if (next != null) {
next.prev = prev;
}
}
}
}
复制代码There are two static methods in FinalizerReference: add and remove methods. It creates a linked list of FinalizerReference type (the header is represented by the static variable FinalizerReference.head), in which each FinalizerReference object is created using the static variable queue of the ReferenceQueue type, so that when the corresponding Object referent is recycled, the FinalizerReference will is put into the ReferenceQueue.
FinalizerReference.add
The above FinalizerReference.add method is invoked by the virtual machine. When creating an object, if the class is found to be class f, it will call the FinalizerReference.add method to create a FinalizerRefence object and add it to the head list.
// art/runtime/mirror/class-alloc-inl.htemplate<bool kIsInstrumented, Class::AddFinalizer kAddFinalizer, bool kCheckAddFinalizer>
inline ObjPtr<Object> Class::Alloc(Thread* self, gc::AllocatorType allocator_type){
CheckObjectAlloc();
gc::Heap* heap = Runtime::Current()->GetHeap();
bool add_finalizer;
switch (kAddFinalizer) {
case Class::AddFinalizer::kUseClassTag:
// 这里判断是不是 f 类
add_finalizer = IsFinalizable();
break;
case Class::AddFinalizer::kNoAddFinalizer:
add_finalizer = false;
DCHECK_IMPLIES(kCheckAddFinalizer, !IsFinalizable());
break;
}
// Note that the `this` pointer may be invalidated after the allocation.
ObjPtr<Object> obj = heap->AllocObjectWithAllocator<kIsInstrumented, /*kCheckLargeObject=*/false>(self, this, this->object_size_, allocator_type, VoidFunctor());
if (add_finalizer && LIKELY(obj != nullptr)) {
// 如果是 f 类，则调用 AddFinalizerReference 方法
heap->AddFinalizerReference(self, &obj);
// ...
}
return obj;
}
// art/runtime/gc/heap.ccvoidHeap::AddFinalizerReference(Thread* self, ObjPtr<mirror::Object>* object){
ScopedObjectAccess soa(self);
ScopedLocalRef<jobject> arg(self->GetJniEnv(), soa.AddLocalReference<jobject>(*object));
jvalue args[1];
args[0].l = arg.get();
InvokeWithJValues(soa, nullptr, WellKnownClasses::java_lang_ref_FinalizerReference_add, args);
// Restore object in case it gets moved.
*object = soa.Decode<mirror::Object>(arg.get());
}
复制代码The AddFinalizerReference method above will call the FinalizerReference.add() static method of the Java layer to complete the add operation.
FinalizerReference.remove
We mentioned earlier that when the object referenced by the Reference object is reclaimed by GC, the Reference object will be added to the reference queue ReferenceQueue, so when the object of class f is gc, its corresponding FinalizerReference object will be added to FinalizerReference .queue in the queue. The timing of remove is related to the FinalizerDaemon daemon thread. Let's look at the source code of this thread:
// libcore/libart/src/main/java/java/lang/Daemons.java// private static abstract class Daemon implements RunnableprivatestaticclassFinalizerDaemonextendsDaemon {
privatestaticfinalFinalizerDaemonINSTANCE=newFinalizerDaemon();
privatefinal ReferenceQueue<Object> queue = FinalizerReference.queue;
@OverridepublicvoidrunInternal() {
// ...while (isRunning()) {
try {
// Use non-blocking poll to avoid FinalizerWatchdogDaemon communication when busy.
FinalizerReference<?> finalizingReference = (FinalizerReference<?>)queue.poll();
// ...
doFinalize(finalizingReference);
} catch (InterruptedException ignored) {
} catch (OutOfMemoryError ignored) {
}
}
}
}
复制代码In the FinalizerDaemon.runInternal method, the Reference reference in the queue will be obtained through the poll/remove method of FinalizerReference.queue (ReferenceQueue reference queue), and then the doFinalize() method will be executed, which will call the finalize() method of the Finalizer object:
privatevoiddoFinalize(FinalizerReference<?> reference) {
FinalizerReference.remove(reference);
Objectobject= reference.get();
reference.clear();
try {
object.finalize();
} catch (Throwable ex) {
// The RI silently swallows these, but Android has always logged.
System.logE("Uncaught exception thrown by finalizer", ex);
} finally {
// Done finalizing, stop holding the object as live.
finalizingObject = null;
}
}
复制代码First remove the obtained FinalizerReference object from the FinalizerReference.head linked list, then obtain the Java object through the reference.get() method, and execute its finalize() method.
summary
The main role of FinalizerReference is to assist the FinalizerDaemon daemon thread to execute the finalize() method of the Finalizer object .
When creating an object in class f, call the FinalizerReference.add() method to create a FinalizerReference object (using the ReferenceQueue queue) and add it to the head list.
After the object of class f is recycled, its corresponding FinalizerReference object will be added to the ReferenceQueue queue (FinalizerReference.queue).
The FinalizerDaemon daemon thread takes out the FinalizerReference object from the FinalizerReference.queue queue, and executes the finalize() method of its corresponding f class object.
ReferenceQueueDaemon
After reading the FinalizerDaemon daemon thread above, here is a look at the ReferenceQueueDaemon daemon thread. It is mentioned above that a ReferenceQueue queue can be associated when creating a reference object Reference. When the referenced object is recycled by gc, the reference object will be added to its creation When going to the associated queue, the operation of adding to the queue is completed by the ReferenceQueueDaemon daemon thread .
privatestaticclassReferenceQueueDaemonextendsDaemon {
@OverridepublicvoidrunInternal() {
while (isRunning()) {
Reference<?> list;
try {
synchronized (ReferenceQueue.class) {
if (ReferenceQueue.unenqueued == null) {
progressCounter.incrementAndGet();
do {
ReferenceQueue.class.wait();
} while (ReferenceQueue.unenqueued == null);
progressCounter.incrementAndGet();
}
list = ReferenceQueue.unenqueued;
ReferenceQueue.unenqueued = null;
}
} catch (InterruptedException e) {
continue;
} catch (OutOfMemoryError e) {
continue;
}
ReferenceQueue.enqueuePending(list, progressCounter);
}
}
}
复制代码If you are interested in the specific source code, you can go and see it yourself, so I won’t post too much source code here.
FinalizerWatchdogDaemon
Add here the FinalizerWatchdogDaemon daemon thread, which starts together with the FinalizerDaemon and ReferenceQueueDaemon threads. Let's take a look at the runInternal() method of the FinalizerDaemon and ReferenceQueueDaemon threads above:
// ReferenceQueueDaemon@OverridepublicvoidrunInternal() {
FinalizerWatchdogDaemon.INSTANCE.monitoringNeeded(FinalizerWatchdogDaemon.RQ_DAEMON);
while (isRunning()) {
Reference<?> list;
try {
synchronized (ReferenceQueue.class) {
if (ReferenceQueue.unenqueued == null) {
progressCounter.incrementAndGet();
FinalizerWatchdogDaemon.INSTANCE.monitoringNotNeeded(FinalizerWatchdogDaemon.RQ_DAEMON);
do {
ReferenceQueue.class.wait();
} while (ReferenceQueue.unenqueued == null);
progressCounter.incrementAndGet();
FinalizerWatchdogDaemon.INSTANCE.monitoringNeeded(FinalizerWatchdogDaemon.RQ_DAEMON);
}
list = ReferenceQueue.unenqueued;
ReferenceQueue.unenqueued = null;
}
} catch (InterruptedException e) {
continue;
} catch (OutOfMemoryError e) {
continue;
}
ReferenceQueue.enqueuePending(list, progressCounter);
FinalizerWatchdogDaemon.INSTANCE.resetTimeouts();
}
}
// FinalizerDaemon@OverridepublicvoidrunInternal() {
FinalizerWatchdogDaemon.INSTANCE.monitoringNeeded(FinalizerWatchdogDaemon.FINALIZER_DAEMON);
while (isRunning()) {
try {
FinalizerReference<?> finalizingReference = (FinalizerReference<?>)queue.poll();
if (finalizingReference != null) {
finalizingObject = finalizingReference.get();
} else {
finalizingObject = null;
FinalizerWatchdogDaemon.INSTANCE.monitoringNotNeeded(FinalizerWatchdogDaemon.FINALIZER_DAEMON);
finalizingReference = (FinalizerReference<?>)queue.remove();
finalizingObject = finalizingReference.get();
FinalizerWatchdogDaemon.INSTANCE.monitoringNeeded(FinalizerWatchdogDaemon.FINALIZER_DAEMON);
}
doFinalize(finalizingReference);
} catch (InterruptedException ignored) {
} catch (OutOfMemoryError ignored) {
}
}
}
复制代码The above monitoringNotNeeded method will sleep the thread, stop the timeout timing, and monitoringNotNeeded will wake up the FinalizerWatchdogDaemon daemon thread. Looking at the source code of FinalizerWatchdogDaemon, I added some comments:
privatestaticclassFinalizerWatchdogDaemonextendsDaemon {
// Single bit values to identify daemon to be watched.// 监控的两种类型staticfinalintFINALIZER_DAEMON=1;
staticfinalintRQ_DAEMON=2;
@OverridepublicvoidrunInternal() {
while (isRunning()) {
// sleepUntilNeeded 内部会调用 wait() 方法，不贴了if (!sleepUntilNeeded()) {
continue;
}
finalTimeoutExceptionexception= waitForProgress();
if (exception != null && !VMDebug.isDebuggerConnected()) {
// 抛出异常 Thread.currentThread().dispatchUncaughtException(exception)
timedOut(exception);
break;
}
}
}
// 去掉 whichDaemon 的监控，进入 waitprivatesynchronizedvoidmonitoringNotNeeded(int whichDaemon) {
activeWatchees &= ~whichDaemon;
}
// 添加 whichDaemon 的监控, notify 线程privatesynchronizedvoidmonitoringNeeded(int whichDaemon) {
intoldWatchees= activeWatchees;
activeWatchees |= whichDaemon;
if (oldWatchees == 0) {
// 调用 notify 唤醒
notify();
}
}
// 判断是否添加了指定 whichDaemon 类型的监控privatesynchronizedbooleanisActive(int whichDaemon) {
return (activeWatchees & whichDaemon) != 0;
}
// 调用 isActive 判断开启监控的类型，通过 Thread.sleep() 方法休眠指定时间// 休眠结束后返回 TimeoutException 异常，若途中监控的逻辑执行完成则表示未超时，返回 null// 等待时间: VMRuntime.getFinalizerTimeoutMs// If the FinalizerDaemon took essentially the whole time processing a single reference,// or the ReferenceQueueDaemon failed to make visible progress during that time, return an exception.private TimeoutException waitForProgress() {
finalizerTimeoutNs = NANOS_PER_MILLI * VMRuntime.getRuntime().getFinalizerTimeoutMs();
// ...
}
}
复制代码Some detailed codes above will not be posted. FinalizerWatchdogDaemon is mainly used to monitor the execution time of two types: FINALIZER_DAEMON and RQ_DAEMON. TimeOutException will be thrown when the execution times out, so don't do time-consuming operations in the finalize() method.
This timeout period is defined as 10s in AOSP. Domestic manufacturers may modify this value. It is said that some have changed it to 60 s. Interested students can unpack it by themselves:
RUNTIME_OPTIONS_KEY (unsigned int, FinalizerTimeoutMs, 10000u)
复制代码OOM troubleshooting
The possibility of large objects and memory leaks was ruled out through the online hprof file, and then it was found in hprof that there were a large number of X (using X to represent a certain object in the business) object accumulation, and the Java class corresponding to these objects was related to the Native layer. The class, which overrides the finalize() method, cannot reproduce the path of accumulating a large number of X objects offline.
There may be an improper use of code logic in a certain business scenario, which caused the crazy creation of X objects, which caused the FinalizerDaemon thread to be too late to recycle
Since there are many creation scenarios of this X object, it is impossible to locate the problematic creation code through code review. After taking some monitoring measures, there is still no result.
Explicitly call the system gc and runFinalization methods in a way that treats the symptoms but not the root cause
After locating the specific problem scenario, it is guessed that a large number of X objects were created and the FinalizerDaemon thread was not recycled in time, so the system gc and runFinalization methods were explicitly called in the main thread and the child thread to manually recycle the X objects.
Result: The sub-thread call has no effect, and OOM does not drop; the main thread call has ANR.
After checking the ANR stack, it is found that the position of ANR is a line of Native code called in the finalize() method of another f object, so the problem is clear, because this line of code is called in a finalize() method Stuck (logic problems lead to deadlock), thus blocking the execution of the FinalizerDaemon thread, which in turn leads to object accumulation. In addition, the TimeoutException that should have occurred was caught by our exception handling framework, so it was never exposed.
Summarize
There is no destructor in Java. finalize() implements a concept similar to a destructor, which can perform some recycling operations before the object is recycled. It may be used to release Native memory in JNI development. Regarding the impact of improper use of class f:
The priority of the FinalizerDaemon daemon thread is not high, and the scheduling may be affected when the CPU is tight, so it may not be possible to reclaim the f-class object in time.
The finalize object becomes a strong reference due to the reference of finalize(). Even if there are no other explicit strong references, it still cannot be recycled immediately.
Therefore, the f object needs at least 2 GCs before it can be recycled. The f object will be added to the ReferenceQueue during the first GC, and it will not be recycled until the next GC after the finalize() method is executed. Due to the low priority of the daemon thread, GC may have occurred many times during this period. If it is not recycled for a long time, it may cause class f to enter the old age when resources are tight, thus causing GC in the old age or even full gc.
So try not to overload the finalize() method, but to release memory through some logical interfaces. If you must overload it, don't let some frequently created objects or large objects be released through finalize(), otherwise There will always be a problem.
There are four related daemon threads. Interested students can take a deep look at the relevant source code:
// libcore/libart/src/main/java/java/lang/Daemons.javaprivatestaticfinal Daemon[] DAEMONS = newDaemon[] {
HeapTaskDaemon.INSTANCE,
ReferenceQueueDaemon.INSTANCE,
FinalizerDaemon.INSTANCE,
FinalizerWatchdogDaemon.INSTANCE,
};Some time ago, I collected the knowledge brain map summary and study manual documents about Android performance optimization! It can not only consolidate the core technical points such as underlying principles and performance tuning, but also master the architecture design methodology that is difficult for ordinary developers to touch . Then you will have core competitiveness that is difficult for your peers to copy when you are at work, in a team, or during interviews. Friends who need the full version can private message [Performance Optimization] in the background! !
Android performance analysis and optimization practical advanced manual
Android performance analysis and optimization practical advanced manual catalog and content display
Chapter 1: Overview of Android Performance Optimization
Chapter 2: Caton optimization
Chapter 3: Startup Speed Optimization
Chapter 4: Memory Optimization
Chapter 5: Layout Optimization
Chapter 6: Thread Optimization
Chapter 7: Power Optimization
Chapter 8: Stability Optimization
