Use of Android WeChat high-performance log storage library Xlog

What is XLog

There is an XLOG log library in Tencent's open source Mars project.

XLog is a high-performance text storage solution that has withstood the test of hundreds of millions of WeChat users in real environments and has very good stability.
Because it is implemented using C language, it has the advantages of performance, small memory, and fast storage speed. It supports the
use of multi-threads and even multi-processes, supports regular deletion of logs,
and has a specific algorithm to compress files. , you can even configure file encryption.

how to use

1. Rely on XLOG

implementation "com.tencent.mars:mars-xlog:1.2.3"

2. Depend on so library

Create a new jniLibsfolder and copy the so in the demo project to this folder

Insert image description here
Then, configure in gradleabiFilters

3. Add permissions

Add storage permissions in Manifest

 <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

Then, apply for permission in MainActivity

ActivityCompat.requestPermissions(this,new String[]{
    
    Manifest.permission.WRITE_EXTERNAL_STORAGE},123);

InitializeXLog

System.loadLibrary("c++_shared")
System.loadLibrary("marsxlog")

val logDir = getExternalFilesDir("xlog")
val cacheLogDir = File(logDir, "cache")
Xlog.setConsoleLogOpen(true); //是否把日志打印到控制台
Xlog.appenderOpen(
    Xlog.LEVEL_ALL,
    Xlog.AppednerModeAsync,
    cacheLogDir?.path,
    logDir?.path,
    "XLOG",
    10,
    ""
)
com.tencent.mars.xlog.Log.setLogImp(Xlog())

Which need to pay attention to

If your program uses multiple processes, do not output the logs of multiple processes to the same file. Ensure that each process has its own log file.
Please use a separate directory to save the log. Do not store any other files to prevent accidental deletion by the xlog automatic cleaning function.
For the debug version, it is recommended to turn on the console log and set the log level to Verbose or Debug. For the release version, it is recommended to turn off the console log and set the log level to Info.
The cachePath parameter must be passed, and the mmap file will be placed in this directory. If an empty string is passed, a SIGBUS crash may occur.

Print log

TextView tv = (TextView) findViewById(R.id.sample_text);
tv.setOnClickListener(new View.OnClickListener() {
    
    
    @Override
    public void onClick(View v) {
    
    
        Toast.makeText(MainActivity.this, "Hello", Toast.LENGTH_SHORT).show();
        com.tencent.mars.xlog.Log.i("z-test","hello world!");
    }
});

Then, you can log at the appropriate timeFlush

com.tencent.mars.xlog.Log.appenderFlush(true);

deinitialization

Deinitialize when the program exits, for example, it can be placed in the onDestory method of MainActivity

com.tencent.mars.xlog.Log.appenderClose();

Run program

Then, we can run the program and print the log.
You can see that there is an .xlog file in the specified log directory

Decode xlog file

First, we need to install Python on the computer . The current version requires 2.7.12.

Then, we copy decode_mars_nocrypt_log_file.py in the source code to the same folder as xlog.

Run CMD, enter the changed folder, and execute Python decode_mars_nocrypt_log_file.py 文件名.xlogdecoding.

You can see that a file has been generated in this folder .xlog.log, and you can open it with Notepad to view the log.

Problems encountered

No module named zstandard

When executing the command, No module named zstandardan error is reported

Traceback (most recent call last):
  File "decode_mars_nocrypt_log_file.py", line 10, in <module>
    import zstandard as zstd
ImportError: No module named zstandard

This bug found a solution on Github https://github.com/Tencent/mars/issues/903

The specific operation is to download pip from the Pip official website, decompress it and execute CMDthe following command

pip install pip==20.3.4

The pip version used here is 20.3.4, which corresponds to python2.7.18.

Then execute it again pip install zstandardand you can see that the installation is successful.

Insert image description here
Then you can execute Python decode_mars_nocrypt_log_file.py 文件名.xlogthe decoding normally.

No module named pyelliptic

If you use Python decode_mars_crypt_log_file.py 文件名.xlogit to decode, you may get No module named pyelliptican error. Similarly, we need to use it pipto installpyelliptic

It should be noted that version 1.5.10 needs to be installed pyplliptic, and the latest version cannot be installed, otherwise the following error will be reported

Traceback (most recent call last):
  File "decode_mars_crypt_log_file.py", line 9, in <module>
    import pyelliptic
  File "C:\Developer\Python2.7.18\lib\site-packages\pyelliptic\__init__.py", line 43, in <module>
    from .openssl import OpenSSL
  File "C:\Developer\Python2.7.18\lib\site-packages\pyelliptic\openssl.py", line 309, in <module>
    raise Exception("Couldn't load OpenSSL lib ...")
Exception: Couldn't load OpenSSL lib ...

Execute the following code to install (github needs to be accessible)

pip install https://github.com/mfranciszkiewicz/pyelliptic/archive/1.5.10.tar.gz#egg=pyelliptic

Insert image description here
After the installation is successful, Python decode_mars_crypt_log_file.py 文件名.xlogyou can decode normally by executing it again.

Advanced

These are the basics of XLog. If you want to customize the log format and make some modifications to the source code, you need to compile it after modification and use the generated so dependency to the project. For details, see Compilation of Android Mars XLog
Android
Mars XLog custom storage format

For details on the Android Mars XLog Demo sample code, please see https://download.csdn.net/download/EthanCo/12168054

other

Official Document
GitHub
Mars Android Access Guide