foreword
This article is in response to a reader's request, and there should be quite a lot of such needs: newcomers who have just entered the job or just entered the laboratory, take over a set of C++ algorithm projects, and now the boss asks you to port it to Android. It is of course unrealistic to rewrite all of them in Java. This article describes a migration method that is as simple as possible.
The software versions used in this paper are as follows:
- VisualStudio 2019 16.3.5
- AndroidStudio 3.5
- GDR-r16B
- Android SDK 21
- Opencv 4.1.1(android & windows)
Prepare the OpencvWindows project for testing
First of all, please refer to the C++ development experience of VS2017 (8) DLL dynamic link - the use of Opencv to establish a simple OpenCV VS solution, as follows:
A simple cv::sum usage.
Start porting to Android
1. First create a VS C++ Android .so project, refer to VS2019 C++ cross-platform development - Android .so development
2. Some basic porting concepts
The so-called transplantation is to remove the dependence of OpenCV on the bottom layer of Windows in the past.
When you hear the underlying dependencies of Windows, do you feel that you are using the C++ standard library for programming, and you don't use any Windows interfaces, so you shouldn't have any dependencies. To give a simple example, for example, when your project uses Opencvxxx.dll, it will use the function LoadLibrary, and this function is implemented by user32.dll of Windows.
If you have all the source code, you can use the Android NDK to recompile the source code, and then replace all the third-party library files .dll and .lib with the .so and .a on the Android side.
3. Use NDK for project compilation
The first honest approach is to copy all the .cpp and .h files under the Android project created in step 1. according to the original project structure.
The second trick is to directly modify "Opencv411Template.vcxproj" and "Opencv411Template.sln" as Android projects.
(The third not-recommended method is to import all .cpp and .h in JNI of AndroidStudio for compilation. It is not recommended to develop C++ with AndroidStudio)
Here is a brief introduction to the second approach, please back up the entire project before trying.
First open "Opencv411Template.vcxproj" (the prepared Opencv Windows project) and "SharedObject7.vcxproj" (the Android project created in step 1.) with the text tool.
First look at the Windows project file:
The files in the red box above are the files that need to be compiled. Except for this part, just replace all the contents in "SharedObject7.vcxproj" (below), isn't it very simple:
Merge the contents of the two red boxes, and remember to copy the three files "SharedObject7.cpp", "SharedObject7.h" and "pch.h" to the corresponding directory of Opencv411Template. The merged results are as follows:
Next, modify the .sln file and replace it with the content in the red box on the right to the left:
Re-open the Opencv411Template project to see:
It has become an Android project, and the next job is to replace the Opencv4.1.1 of the Windows project with the opencv-4.1.1-android-sdk of Android. The main difference between here and Windows is that the loaded library is different.
Let’s briefly talk about the header file directory and library file directory of the Android library:
The header files are located at: \opencv-4.1.1-android-sdk\OpenCV-android-sdk\sdk\native\jni\include
The library files are located at:
\opencv-4.1.1-android-sdk\OpenCV-android-sdk\sdk\native\3rdparty\libs
\opencv-4.1.1-android-sdk\OpenCV-android-sdk\sdk\native\libs
\opencv-4.1.1-android-sdk\OpenCV-android-sdk\sdk\native\staticlibs
The header file is very simple, just add the directory directly, there is no change.
The key is the library file, and the Windows project only links one dll (or statically compiles several .libs). Here you must first link a libopencv_java4.so, and then add multiple static libraries for compilation according to your needs. If you understand the link of the static library, you will naturally understand it. If you don’t understand it, just add all .a to it, and everyone can accept more than ten M space.
The following is the link operation of the Android library in VS2019. First add the above three paths to the additional library directory of the linker, as follows, pay attention to the selection of the red box:
The last thing is to add the additional library:
Because the library name of Opencv Android has no difference between Debug and Release, and there is no platform difference, but the folder name is used to distinguish the library. This is a better naming method, which is convenient for using macros to define paths.
Complete list of libraries:
-lopencv_java4
-lopencv_calib3d
-lopencv_core
-lopencv_dnn
-lopencv_features2d
-lopencv_flann
-lopencv_highgui
-lopencv_imgcodecs
-lopencv_imgproc
-lopencv_ml
-lopencv_objdetect
-lopencv_photo
-lopencv_stitching
-lopencv_video
-lopencv_videoio
-lcpufeatures
-lIlmImf
-littnotify
-llibjasper
-llibjpeg-turbo
-llibpng
-llibprotobuf
-llibtiff
-llibwebp
-lquirc
-ltbb
-ltegra_hal
-lz
-ldl
-lm
-llog
Then compile with confidence, a bunch of errors:
Don't be afraid, it's just a few C++ settings.
Compile the project again:
It is successful. Note that from here on I will only demonstrate the Android loading process of compiling .so under Debug and ARM64 configurations.
4. Modify the function export of .so so that Java can call it
Modify the export file SharedObject7.cpp as follows (this file is used to call the function interface under the previous Windows project, and then export it to JNI and Java classes in two ways):
#include "SharedObject7.h"
#define LOGI(...) ((void)__android_log_print(ANDROID_LOG_INFO, "SharedObject7", __VA_ARGS__))
#define LOGW(...) ((void)__android_log_print(ANDROID_LOG_WARN, "SharedObject7", __VA_ARGS__))
extern float TestOpencv(float* buf, int len); //假设这是过去Opencv工程的导出接口
extern "C" {
float ExternTestOpencv(float* buf, int len)//这个用来导出给Android JNI使用
{
return TestOpencv(buf,len);
}
//C++导出给Java类使用的命名规范
//Java_packagename_classname_functionname
//第一个传参总是JNIEnv* env
//第二个传参 如果是static成员函数就是jclass type,
// 如果是非static成员函数就是jobject thiz,
//第三个传参才是真正的参数
JNIEXPORT jfloat JNICALL
Java_com_jniexample_JNIInterface_CVTestSum(JNIEnv* env, jclass type, jfloatArray buf) //这个用来导出给Java使用
{
auto len = env->GetArrayLength(buf);
jboolean notcopy = JNI_FALSE;
float* fptr = env->GetFloatArrayElements(buf, ¬copy);//从Java内存转换到native指针
return TestOpencv(fptr, len);
}
JNIEXPORT jfloat JNICALL
Java_com_jniexample_JNIInterface_TestSum(JNIEnv* env, jclass type, jfloatArray buf)//这个用来导出给Java使用
{
auto len = env->GetArrayLength(buf);
jboolean notcopy = JNI_FALSE;
float* fptr = env->GetFloatArrayElements(buf, ¬copy);
float sum = 0;
for (size_t i = 0; i < len; i++)
{
sum += fptr[i];
}
return sum;
}
/*此简单函数返回平台 ABI，此动态本地库为此平台 ABI 进行编译。*/
const char * SharedObject7::getPlatformABI()
{
#if defined(__arm__)
#if defined(__ARM_ARCH_7A__)
#if defined(__ARM_NEON__)
#define ABI "armeabi-v7a/NEON"
#else
#define ABI "armeabi-v7a"
#endif
#else
#define ABI "armeabi"
#endif
#elif defined(__i386__)
#define ABI "x86"
#else
#define ABI "unknown"
#endif
LOGI("This dynamic shared library is compiled with ABI: %s", ABI);
return "This native library is compiled with ABI: %s" ABI ".";
}
void SharedObject7()
{
}
SharedObject7::SharedObject7()
{
}
SharedObject7::~SharedObject7()
{
}
}
At this point, the preparation for the C++ side is complete.
Create a JNI project under Android Studio
I am not an expert in AS, so I simply posted the steps below:
After the creation is complete, switch to the project view:
The first thing is to add a Java class in the blue folder java in the above picture.
This class is used to interface with two exported functions in .so: Java_com_jniexample_JNIInterface_CVTestSum and Java_com_jniexample_JNIInterface_TestSum.
So, the next step is to create these two functions:
Then put the generated.so under the project:
Please store the .so according to the Opencv folder structure (I only demonstrate the arm64-v8a version here, and refer to the figure below for other architecture file names):
At this point, the part of Java importing .so is over. Add another method of importing .so in JNI of AndroidStudio.
Importing in JNI means using the export function in .so in the .cpp file of AndroidStudio.
First modify native-lib.cpp (automatically generated file):
Then modify CMakeLists.txt (AS uses Cmake, so you need to know more or less the syntax of Cmake):
The above two steps are the use and linking of jni to the external .so library.
This step is very important, modify build.gradle (whether it is a JNI call or a Java call, you need to modify this step)
The boring work is finally over, let’s add some elements on the interface to display the calculation results of .so:
First double-click activity_main.xml to edit the UI:
Add two buttons to call two exported Java functions, and add a Textview to display the calculation results:
Modify the backend code of the UI:
package cn.com.inxpar.nativeproject;
import androidx.appcompat.app.AppCompatActivity;
import android.os.Bundle;
import android.view.View;
import android.widget.TextView;
import com.jniexample.JNIInterface;
import org.w3c.dom.Text;
public class MainActivity extends AppCompatActivity implements View.OnClickListener {
// Used to load the 'native-lib' library on application startup.
static {
System.loadLibrary("native-lib");
System.loadLibrary("SharedObject7");
}
private TextView sumText;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
// Example of a call to a native method
TextView tv = findViewById(R.id.sample_text);
tv.setText(stringFromJNI());
sumText=findViewById(R.id.textView);
findViewById(R.id.button).setOnClickListener(this);
findViewById(R.id.button2).setOnClickListener(this);
}
float[] a=new float[]{1,2,2,3,3,4};
float sum=0;
@Override
public void onClick(View v) {
switch (v.getId())
{
case R.id.button:
sum= JNIInterface.CVTestSum(a);
sumText.setText("OpenCV Sum:"+Float.toString(sum));
break;
case R.id.button2:
sum= JNIInterface.TestSum(a);
sumText.setText("Raw Sum:"+Float.toString(sum));
break;
default:
break;
}
}
/**
* A native method that is implemented by the 'native-lib' native library,
* which is packaged with this application.
*/
public native String stringFromJNI();
}
So far, you're done.
Compile apk, analyze apk:
If there are these .so libraries in the analyzed apk, then it is correct:
Finally run on an arm64 Android device:
See, isn't it very simple.
hint
1. The virtual machine in AndroidStudio uses the x86 Android system by default, so the .so file compiled by x86 should be used.
2. After the apk is installed, it will prompt that xxx has stopped working (this is the crash in Android). Generally, the .so cannot be found, and you need to use logcat to troubleshoot the problem yourself.
3. After the app is running, click a button and prompt that xxx has stopped working. The logcat shows that the crash is due to the fact that the xxxxx function has not been implemented. The general error is that the names of the two functions exported to Android are incorrect (whether it is static or not, and whether the parameters are passed correctly), check carefully.
4. After VS2019 uses the Android native sdk of Opencv4.1.1, if the llvm-libc++ static library is selected in the project properties, then a compilation error will occur: undefined reference to `strtof_l'. I don't know the specific reason, but due to Opencv Use libc++_shared, so it is not reasonable to use static itself here. After changing to llvm-libc++ shared library, it can be compiled successfully.
5. When transplanting a large project, please first establish the smallest opencv project and test it successfully before starting.
6. Be sure to use logcat
7. This is the end of the matter, please calm down and learn a little Java and Android development knowledge, don't go directly to Baidu for everything, and finally piece together a project that just works.
Android's official documentation: https://developer.android.com/studio/projects/add-native-code.html