Author: Chang'an Refuge Hometown
Hello everyone, recently, due to project reasons, I conducted some research and in-depth understanding of the monitoring of IO resource leakage, and found that the implementation cost of the IO leakage monitoring framework is relatively low, and the effect is very significant; at the same time, because IO monitoring involves reflection, I also learned that through a A clever way to access non-public APIs above Android P.
Next, this article will first take you to understand some pre-knowledge, and then will lead you from 0 to 1 to teach you how to build an IO leak monitoring framework.
1. Why do IO leak detection?
IO generally refers to common file stream reading and writing, database reading and writing. I believe everyone knows that after reading and writing, you should manually call the stream close() method to close. Once you forget, it will cause IO leaks .
If there are many such problem scenarios in the project, it will lead to uncontrolled increase of fd, resulting in tight application memory, serious and even OOM , which will greatly affect the user experience.
In order to avoid forgetting to close after operating the read and write streams, the Java and Kotlin programming languages respectively provide us with the following syntactic sugar:
1. Realize AutoCloseablethe combination of javatry-with-resource
Look at a common piece of code:
public static void main(String[] args) {
try (FileInputStream fis = new FileInputStream(new File("test.txt"))) {
byte[] data = new byte[1024];
int read = fis.read(data);
//执行其他操纵
} catch (Exception e) {
e.printStackTrace();
}
}
FileInputStreamImplement AutoCloseablethe interface and rewrite the interface close()method. Through the above try-with-resourcesyntax, we don't need to call the close method to close the io . Java will automatically help us complete this operation:
Common interfaces InputStream、OutputStream 、Scanner 、PrintWriterare implemented AutoCloseable, so it is very convenient to use the above syntactic sugar when reading and writing files.
use()2. Using extensions in kotlin
Kotlin Closeable（实现了AutoCloseable）provides the following extensions for interfaces:
Our common InputStream、OutputStream 、Scanner 、PrintWriterones all support this extension function:
override fun create(context: Context) {
FileInputStream("").use {
//执行某些操作
}
}
Although both kotlin and java help us read and write io streams to achieve safe closure from the language level, but we really forgot when we wrote the code; and there may be historical codes in the project that also forgot to close the stream, find It also looks clueless.
In the face of the above situation, a detection mechanism for io leakage is needed. Whether it is for the historical code of the project or the newly written code, it can detect whether the file stream is closed. If it is not closed, obtain the stack created by the stream and report it to help development and positioning. Question , let's realize this ability step by step.
2. Implementation ideas of IO leak detection
Brainstorm, if you want to detect whether the stream is closed, the key is to detect whether FileInputStreamthe class close method such as operating the file stream has been called; then when should it be detected, when the FileInputStreametc. stream class is ready to be destroyed, it can be detected , and the method will be called when the class is destroyed finalize()(PS: I don’t consider the performance of finalize() in special scenarios for the time being, I think it will be executed normally), so the best time to detect is when the method of the stream class finalize() is executed .
After the above analysis, we can write the following code:
public class FileInputStream {
private Object flag = null;
public void open() {
//打开文件流时赋值
flag = "open";
}
public void close() throws Exception {
//关闭文件流置空
flag = null;
}
@Override
protected void finalize() throws Throwable {
super.finalize();
//flag等于null，说明忘记执行close方法关闭流，io泄漏
if (flag != null) {
Throwable throwable = new Throwable("io leak");
//执行异常日志的打印，或者回调给外部。
//兜底流的关闭
close();
}
}
}
There are very detailed comments in the code, so I won't describe them one by one here.
So if we can also add the above code in our common FileInputStream, FileOutputStream, and other stream classes, io leak monitoring will become a thing! RandomAccessFile!
The Android official can naturally think of it, and it has done it. Common official streams FileInputStream , FileOutputStream , RandomAccessFile , CursorWindowetc. have added similar monitoring logic to the above . Next, we FileInputStreamwill analyze it as an example.
Take a look at the official FileInputStream source code
Let’s say in advance here that the official monitoring of stream class leaks does not directly add logic codes in it. Think about it, so many stream classes, adding one by one leads to too much template code. It is better to encapsulate a tool class for each stream Class usage, the tool class here isCloseGuard .
After clarifying the above, let's look at FileInputStreamthe source code:
1. Get tool classCloseGuard
Since CloseGuardthe source code cannot be directly viewed in AS, here we use aospxref.com/android-12.…the website to view the source code of this class:
CloseGuard.get()The method is to create an CloseGuardobject.
2. Open the file stream
FileInputStreamThe constructor mainly does two things:
-
Open the file stream by calling the incoming file path
IoBridge.open()(this bottom layer will eventually be calledopen(const char *pathname,int flags,mode_t mode), and generally need to hook this method when doing io monitoring). -
Also calls
CloseGuard.open()the method:
The main thing this method does is to create an Throwableobject, get the stack created by the current flow, and assign it to CloseGuardthe closerNameOrAllocationInfofield.
3. Close the file stream
FileInputStreamThe close()method mainly does two things:
CloseGuardThe method calledclose():
It's very simple, just reset the fields assigned above to empty.closerNameOrAllocationInfo
- close the file stream;
4. Destruction of overridden finalize()monitoringFileInputStream
FileInputStreamThe finalize()method mainly does two things:
CloseGuardThe method calledwarnIfOpen():
If closerNameOrAllocationInfothe field is not empty, the method of closing the file stream described FileInputStreamhas close() missed the call, and an io leak has occurred. Call reporter.report() the method and pass in closerNameOrAllocationInfoparameters (this parameter says: the stack when the stream was created is saved, and once we get it, we can quickly Quickly know where the created stream is leaking).
- Close the stream at the bottom;
From the above analysis, we can know that once an io leak occurs, it will be reporter.report() reported. This is the key for us to monitor the overall io leak of the application.
Look at reporterwhat it is:
reporterIt is a static variable, essentially a default implementation class that implements Reporterthe interface , and prints the system log of io leaks DefaultReporter by default .report()
At the same time, the outside can inject a custom Reporterclass that implements the interface:
Speaking of this, do you understand that if we implement io leak detection at the application layer, as long as we proxy this static variable through dynamic proxy + reflection reporter, replace it with the class of our custom implemented Reporterinterface, and implement io in the custom class Isn’t the logic of leakage exception reporting perfect for monitoring! !
The imagination is beautiful, but the reality is cruel. CloseGuardIt is a system class and is @hidehidden. At the same time, the above setReporter()methods are @UnsupportedAppUsageannotated, so this is an official non-public API. Under Android P, it can naturally be called through reflection, but if using reflection on Android P and above, an error will be reported, so it is necessary to explore a method that can successfully reflect the non-public API of the system in a higher version.
4. Implementation of Android P and above non-public API access
If you want to access the non-public API of the system, only the system API can be called. Generally, there are two ways:
- Convert the classloader of our own class to the classloader of the system to call the non-public API of the system;
- Use the system class method to call the system's non-public API, that is, the double reflection implementation mechanism;
Here we use the second double-reflection implementation, and weishu provides a github library for us to use:
dependencies {
implementation 'com.github.tiann:FreeReflection:3.1.0'
}
Then Application.attachBaseContext()call it in the method;
@Override
protected void attachBaseContext(Context base) {
super.attachBaseContext(base);
Reflection.unseal(base);
}
5. Build an IO leak monitoring framework from 0 to 1
The above preparatory knowledge has been explained. Next, let's start our io leak detection framework construction journey from 0 to 1.
1. Create ResourceLeakCanarya module named as , and introduce the following two dependencies
dependencies {
implementation 'com.github.tiann:FreeReflection:3.1.0'
implementation("androidx.startup:startup-runtime:1.1.1")
}
2. Realize the automatic initialization of the SDK through startup, and use FreeReflectionthe library to lift the system's non-public API access restrictions
class IOLeakCanaryInstall : Initializer<Unit> {
override fun create(context: Context) {
//android p及以上非公开api允许调用
Reflection.unseal(context)
//初始化核心io泄漏监测
IOLeakCanaryCore().init(context.applicationContext)
Log.i(IOLeakCanaryCore.TAG, "IOLeakCanaryInstall install success!")
}
override fun dependencies(): MutableList<Class<out Initializer<*>>> = mutableListOf()
}
3. Create , which implements the logic of IOLeakCanaryCorethe core hookCloseGuard#Reporter
class IOLeakCanaryCore {
companion object {
const val TAG = "IOLeakCanary"
lateinit var APPLICATION: Context
}
/**
* CloseGuard原始的Reporter接口实现类DefaultReporter
*/
private var mOriginalReporter: Any? = null
fun init(application: Context) {
APPLICATION = application
val hookResult = tryHook()
Log.i(TAG, "init: hookResult = $hookResult")
}
@SuppressLint("SoonBlockedPrivateApi")
private fun tryHook(): Boolean {
try {
val closeGuardCls = Class.forName("dalvik.system.CloseGuard")
val closeGuardReporterCls = Class.forName("dalvik.system.CloseGuard$Reporter")
//拿到CloseGuard原始的Reporter接口实现类DefaultReporter
val methodGetReporter = closeGuardCls.getDeclaredMethod("getReporter")
mOriginalReporter = methodGetReporter.invoke(null)
//获取setReporter的Method实例，便于后续反射该方法注入我们自定义的Report对象
val methodSetReporter =
closeGuardCls.getDeclaredMethod("setReporter", closeGuardReporterCls)
//将CloseGuard的stackAndTrackingEnabled字段置为true，否则为false将不会调用自定义的Reporter对象
val methodSetEnabled =
closeGuardCls.getDeclaredMethod("setEnabled", Boolean::class.java)
methodSetEnabled.invoke(null, true)
//借助动态代理+反射注入我们自定义的Report对象
val classLoader = closeGuardReporterCls.classLoader ?: return false
methodSetReporter.invoke(
null,
Proxy.newProxyInstance(
classLoader,
arrayOf(closeGuardReporterCls),
IOLeakReporter()
)
)
return true
} catch (e: Throwable) {
Log.e(TAG, "tryHook error: message = ${e.message}")
}
return false
}
/**
* 拦截report并收集堆栈
*/
inner class IOLeakReporter : InvocationHandler {
override fun invoke(proxy: Any?, method: Method?, args: Array<out Any>?): Any? {
if (method?.name == "report") {
//io泄漏，收集堆栈并上报，其中args[1]就代表着上面的
//CloseGuard#closerNameOrAllocationInfo字段，保存了流打开时的堆栈详细
val stack = args?.get(1) as? Throwable ?: return null
val stackTraceToString = stackTraceToString(stack.stackTrace)
//这里只是通过日志进行打印，有需要的可以定制这块逻辑，比如加入异常上报机制
Log.i(TAG, "IOLeakReporter: invoke report = $stackTraceToString")
return null
}
return method?.invoke(mOriginalReporter, args)
}
/**
* 处理堆栈
*/
private fun stackTraceToString(arr: Array<StackTraceElement>?): String {
val stacks = arr?.toMutableList()?.take(8) ?: return ""
val sb = StringBuffer(stacks.size)
for (stackTraceElement in stacks) {
sb.append(stackTraceElement.toString()).appendLine()
}
return sb.toString()
}
}
}
There are very rich comments on the class, I will not explain them one by one here, you will naturally understand after reading the above code carefully.
The above is all the code, about 100 lines in total. We can access the alarm mechanism for io leaks in the above IOLeakReportermethod invoke, which is very suitable for a comprehensive io leak detection of the project in the debug environment. Now that the code is written, let's do a test next.
4. io leak detection test
We write a piece of test code to obtain details about the cpu, and deliberately do not release the file stream:
Run the next project and view the logcat log output:
You can see that there are warning logs printed, and the abnormal logic can be directly located through the logs: the FileInputStreamstream created in line 35 of the code is not closed after use, so we can quickly fix it.
6. Summary
In fact, if you know the source code of matrix-io-canary, you should be able to find it soon. Isn't this the source code of the implementation of io leak monitoring in matrix-io-canary! After reading matrix-io-canary, the author explained it in a more popular way by sorting out the relevant knowledge points involved. I hope this article can be helpful to you.
However, please note that the above CloseGuardanalysis is based on the source code of Android 12. Different system versions, such as Android 8, have different implementations; and the access to the non-public APIs of the system is also implemented with the help of Android. The official Android itself prohibits the use of FreeReflectionthese non-public APIs. API, so for the stability of the application, it is recommended that you only use the above logic in the debug environment .
Android core knowledge points
Android Performance Optimization: https://qr18.cn/FVlo89
Android Cars: https://qr18.cn/F05ZCM
Android Framework Basic Principles: https://qr18.cn/AQpN4J
Android Audio and Video: https://qr18.cn/Ei3VPD
Jetpack (including Compose): https://qr18.cn/A0gajp
Kotlin: https://qr18.cn/CdjtAF
Gradle: https://qr18.cn/DzrmMB
OkHttp Source Code Analysis Notes: https://qr18.cn/Cw0pBD
Flutter: https://qr18.cn/DIvKma
Android Eight Knowledge Bodies: https://qr18.cn/CyxarU
Android Core notes: https://qr21.cn/CaZQLo
Android interview questions from previous years: https://qr18.cn/CKV8OZ
The latest collection of Android interview questions in 2023 https://qr18.cn/CgxrRy
Audio and video interview questions:https://qr18.cn/AcV6Ap