I wrote a blog before: Teach you how to use bsdiff in Android to implement incremental file update .
Since Android Studio comes with NDK environment, it is relatively simple to implement JNI.
But as mentioned in the blog, the function of file difference must be performed on the server side. The environment where the server runs may be in window, mac or linux. Therefore, we need to compile the source code of bsdiff into a native library file corresponding to the environment, so that it can be called to Java.
Since most projects basically run on linux, here I will only demonstrate how to compile the bsdiff source code into an so file, and then call it in the java code.
Without further ado, let's get started!
Clion builds the compilation environment
Here I use Clion in the Jetbrains family bucket. Needless to say, the IDE of Jetbrains understands everything.
First of all, you need to prepare a Linux system. For Windows, it is recommended to install ubuntu directly with WSL for local debugging. The installation method is also relatively simple, just follow the official documentation. WSL installation documentation:
If it is a MAC, it is recommended to use docker to install linux for local debugging. When using Docker as the compilation environment of Clion, be sure to install the image that has already configured the compilation environment such as cmake and gcc, otherwise Clion will not recognize it. I installed ubuntu first before installing cmake, gdb and other environments. As a result, Clion couldn't find it, and kept saying that the installation package was not found. I was stuck in this place for almost a day, and I doubted my life. Later, it was successful according to the official document. Official document: Using Docker with CLion
Jetbrains also provides several written Dockerfiles on github, you can choose according to your needs. clion-remote
The other is a more recommended way. Go to Alibaba Cloud to buy a cloud host, compile it remotely, and then publish the written code to the remote end for testing. Both windows and mac can be used, which is more convenient.
After installing the linux environment, you need to install cmake, gdb, etc., and there are not many BBs here. After installation, go to Clion to configure the tool chain, and configure it according to your needs, as follows.
Then configure cmake
So far the environment is configured
Compile C to so on Linux
In the previous blog about Android using bsdiff, we have already processed bsdiff, and we can also type so on Android.
It is the same step in the java web project, prepare the native code declared by java, change the cpp code, and configure cmake.
It should be noted that the NDK in the Android project can directly import jni.h. In Clion, we need to manually copy jni.h and jni_md.h in jdk to the project.
jni.h is in the include directory in jdk , jni_md.h is in the win32 directory.
The structure of the code is as follows
and then you can compile the so file. Click build->Rebuild Project.
Wait for the project to be built, as shown below
, you can see that it is completed, and then go to the directory given in the red box above to look at the file.
As you can see, there is source code on linux, which is equivalent to Clion uploading your local code to linux, and then compiling it through the linux environment.
Cut to the specified final directory to see
the so file in the red box above is what we need.
So far, compiling the so file on linux is done.
Java Web Jni implementation
Then it's simple, just a normal JNi implementation. Paste the project directory first.
Note that you need to add java code to access the configuration of the so file in the resources directory.
The following are some key codes. Here is just the demo code, mainly to realize the function. In the production environment, pay attention to the robustness of the code.
package com.yzq.bsdiffserver.utils;
import org.springframework.core.io.ClassPathResource;
import org.springframework.util.FileCopyUtils;
import java.io.File;
import java.io.FileOutputStream;
import java.io.InputStream;
/**
* @description: 用来load so文件，先将项目的so拷贝到linux中，然后加载
* @author : yuzhiqiang ([email protected])
* @date : 2021/12/18
* @time : 14:33
*/
public class LibLoader {
public static void loadLib(String libName, String resourcePath) {
System.out.println("libName = " + libName);
System.out.println("resourcePath = " + resourcePath);
/*获取当前项目所在的linux路径 示例：/home/admin/webapp */
final String projectPath = System.getProperty("user.dir");
System.out.println("projectPath = " + projectPath);
/*创建一个目录 用来放so*/
String nativeLibPath = projectPath + File.separator + "tmp" + File.separator + "bsdiffdemo" + File.separator + "lib" + File.separator;
File nativeLibFolder = new File(nativeLibPath);
if (!nativeLibFolder.exists()) {
nativeLibFolder.mkdirs();
}
/*用来存放临时文件的目录*/
String filePath = projectPath + File.separator + "tmp" + File.separator + "bsdiffdemo" + File.separator + "file" + File.separator;
File fileFolder = new File(filePath);
if (!fileFolder.exists()) {
fileFolder.mkdirs();
}
File libFile = new File(nativeLibFolder, libName);
System.out.println("libFile.getAbsolutePath() = " + libFile.getAbsolutePath());
if (libFile.exists()) {
System.out.println("libFile 文件存在 libFile.getAbsolutePath() = " + libFile.getAbsolutePath());
System.load(libFile.getAbsolutePath());
} else {
try {
final InputStream inputStream = new ClassPathResource(resourcePath).getInputStream();
// InputStream in = LibLoader.class.getResourceAsStream(resourcePath);
final FileOutputStream fileOutputStream = new FileOutputStream(libFile);
/*把文件存到临时目录里*/
final int copy = FileCopyUtils.copy(inputStream, fileOutputStream);
System.out.println("copy result= " + copy);
inputStream.close();
fileOutputStream.close();
System.out.println("libFile.getAbsolutePath() = " + libFile.getAbsolutePath());
System.load(libFile.getPath());
} catch (Exception e) {
e.printStackTrace();
throw new RuntimeException("Failed to load required lib", e);
}
}
}
}
The tool class code is similar to that on Android.
package com.yzq.bsdiffserver.utils;
import java.io.File;
/**
* @author : yuzhiqiang ([email protected])
* @description: BsDiffUtil
* @date : 2021/12/18
* @time : 14:33
*/
public class BsDiffUtil {
static {
String systemType = System.getProperty("os.name");
System.out.println("systemType = " + systemType);
try {
/*获取当前项目所在的linux路径 示例：/home/admin/webapp */
// final String projectPath = System.getProperty("user.dir");
String libPath = "lib" + File.separator + "libxeon_bsdiff.so";
String libName = "libxeon_bsdiff.so";
/*mac上使用dylib*/
// String libPath = "lib" + File.separator + "libxeon_bsdiff.dylib";
// String libName = "libxeon_bsdiff.dylib";
LibLoader.loadLib(libName, libPath);
System.out.println("load so success");
} catch (Exception e) {
System.out.println("e = " + e.getMessage());
e.printStackTrace();
}
}
/**
* 生成补丁文件
*
* @param newFilePath
* @param oldFilePath
* @param patchFilePath
* @return
*/
public static native int fileDiff(String newFilePath, String oldFilePath, String patchFilePath);
/**
* 合并文件
*
* @param oldFilePath
* @param patchFilePath
* @param combineFilePath
* @return
*/
public static native int filePatch(String oldFilePath, String patchFilePath, String combineFilePath);
}
Test Methods
package com.yzq.bsdiffserver.service;
import com.yzq.bsdiffserver.utils.BsDiffUtil;
import java.io.File;
/**
* @description: 测试类
* @author : yuzhiqiang ([email protected])
* @date : 2021/12/18
* @time : 14:55
*/
public class BsDiffService {
public static void testBsDiff() {
try {
/*测试load so*/
// BsDiffUtil.test();
final String projectPath = System.getProperty("user.dir");
String filePath = projectPath + File.separator + "tmp" + File.separator + "bsdiffdemo" + File.separator + "file" + File.separator;
/*旧文件路径*/
String oldFilePath = filePath + "old.txt";
/*新文件*/
String newFilePath = filePath + "new.txt";
/*补丁文件*/
String patchFilePath = filePath + "patch.txt";
/*合并后的文件*/
String combineFilePath = filePath + "combine.txt";
System.out.println("oldFilePath = " + oldFilePath);
System.out.println("newFilePath = " + newFilePath);
System.out.println("patchFilePath = " + patchFilePath);
System.out.println("combineFilePath = " + combineFilePath);
/*生成补丁文件*/
final int diffResult = BsDiffUtil.fileDiff(newFilePath, oldFilePath, patchFilePath);
System.out.println("diffResult = " + diffResult);
/*合并补丁文件*/
final int patchResult = BsDiffUtil.filePatch(oldFilePath, patchFilePath, combineFilePath);
System.out.println("patchResult = " + patchResult);
} catch (Exception e) {
e.printStackTrace();
}
}
}
After the java code is ready, it can be directly deployed to the remote server for testing.
Here are the test results directly.
The directory where the so files are stored
Directory for testing merged diff files
It can be seen that the size of the merged file is the same as that of the new file, and the content is exactly the same after I read it.
At this point, the bsdiff-based incremental update function technology from Android to the back end has been opened up. The specific application depends on your needs.
It is not easy to code in winter, and my hands are frozen. If you think it is helpful, click a like to support it...
If you think this article is helpful to you, please give it a thumbs up. It can help more developers. If there are any mistakes in the article, please correct me. For reprinting, please indicate that you are reposting from Yu Zhiqiang’s blog, thank you !