我的第一个caffe Android程序

在上一篇文章《我的第一个caffe C++程序》中，说明了如何编写一个最简单的caffe C++程序，但我的最终目的是希望在Android app中使用caffe框架。所以接下来我就将模型测试程序testXOR移植到Android中，让Android app也能使用caffe深度学习框架。

caffe框架采用C++语言编写，具有良好的可移植性，不过官方代码并没有Android平台的移植。好在民间大神也很多，网上搜索了一把，发现已经有人做了caffe的Android平台移植，项目地址为：https://github.com/sh1r0/caffe-android-lib。

编译caffe-android-lib

caffe-android-lib的编译，建议还是采取docker的方式进行构建。我尝试过使用本地的android NDK进行编译，发现NDK版本不同，编译存在一定的问题。caffe-android-lib项目主页上有docker构建方法：

git clone --recursive https://github.com/sh1r0/caffe-android-lib.git
cd caffe-android-lib
# build image
docker build -t caffe-android-lib .
# run a container for building your own caffe-android-lib, e.g.,
docker run --rm --name caffe-android-builder \
    -e ANDROID_ABI=x86_64 \
    -e N_JOBS=2 \
    -v $(pwd)/android_lib/x86_64:/caffe-android-lib/android_lib \
    caffe-android-lib ./build.sh

注意：指导上指定的ANDROID_ABI为x86_64，对于大多数手机而言，请修改为armeabi-v7a

编译好之后，caffe及关联库的头文件和库文件都位于android_lib目录下。在后面的步骤中，需要将这些头文件复制到示例工程下。

Android项目

在Android Studio中新建一个Android项目，跟着向导一步步来即可。然后将上一步骤的android_lib目录下的头文件和库文件复制过来，我把它们放在了cpp/third_party目录下了。caffe库是build为一个so，所以我将它放在了jniLibs目录下。

完整的项目源码可参考：https://gitee.com/mogoweb/dpexamples.git

在XORusingCAFFE-android目录下即为android版本的caffe示例程序。C++端的代码如下：

extern "C" JNIEXPORT jstring

JNICALL
Java_com_mogoweb_caffe_xorusingcaffe_MainActivity_startXORTest(
        JNIEnv *env,
        jobject /* this */,
        jstring modelProto,
        jstring caffeModel) {
    // caffe is using google logging (aka 'glog') as its logging module, and hence this module must be initialized once when running caffe.
    // Therefore the following line
    ::google::InitGoogleLogging("");

    // load the trained weights cached inside XOR_iter_5000000.caffemodel
    shared_ptr<Net<float> > testnet;

    std::string modelproto = jstring2string(env, modelProto);
    std::string caffemodel = jstring2string(env, caffeModel);
    __android_log_print(ANDROID_LOG_INFO, "XOR", "modelproto:%s, caffemodel:%s", modelproto.c_str(), caffemodel.c_str());
    testnet.reset(new Net<float>(modelproto, TEST));
    testnet->CopyTrainedLayersFrom(caffemodel);

    // obtain the input MemoryData layer and pass the input to it for testing
    float testab[] = {0, 0, 0, 1, 1, 0, 1, 1};
    float testc[] = {0, 1, 1, 0};

    MemoryDataLayer<float> *dataLayer_testnet = (MemoryDataLayer<float> *)(testnet->layer_by_name("test_inputdata").get());

    dataLayer_testnet->Reset(testab, testc, 4);

    // calculate the neural network output
    testnet->Forward();

    // access blobs to display results
    boost::shared_ptr<Blob<float> > output_layer = testnet->blob_by_name("output");

    const float* begin = output_layer->cpu_data();
    const float* end = begin + 4;

    // We know the output size is 4, and we save the outputs into the result vector
    vector<float> result(begin, end);

    // display the result
    char buf[512] = {0};
    string s;
    for (int i = 0; i < result.size(); i++) {
        sprintf(buf, "input: %d xor %d, truth: %d, result by NN: %f\n",
                (int)testab[i * 2 + 0], (int)testab[i * 2 + 1], (int)testc[i],result[i]);
        s += buf;
    }

    __android_log_print(ANDROID_LOG_INFO, "XOR", "test result:%s", s.c_str());
    return string2jstring(env, s);
}

相比linux版本的代码，有几处修改：

1. 模型文件路径从Java层传递过来，也就是model.prototxt和XOR_iter_5000000.caffemodel文件的完整路径
2. 输出结果通过JNI返回到Java层。

Java端的代码如下：

public class MainActivity extends AppCompatActivity {
    File sdcard = Environment.getExternalStorageDirectory();
    String modelDir = sdcard.getAbsolutePath();
    String modelProto = modelDir + "/model.prototxt";
    String modelBinary = modelDir + "/XOR_iter_5000000.caffemodel";

    // Used to load the 'native-lib' library on application startup.
    static {
        System.loadLibrary("native-lib");
        System.loadLibrary("caffe");
    }

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        final TextView tv = findViewById(R.id.result);
        Button btn = findViewById(R.id.start_test);
        btn.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                tv.setText(MainActivity.this.startXORTest(modelProto, modelBinary));
            }
        });
    }

    /**
     * A native method that is implemented by the 'native-lib' native library,
     * which is packaged with this application.
     */
    public native String startXORTest(String modelProto, String caffeModel);
}

代码中假设模型文件位于sdcard中，所以在运行代码之前，需要将model.prototxt和XOR_iter_5000000.caffemodel文件push到/sdcard/中。

adb push app/src/main/model/model.prototxt /sdcard/
adb push app/src/main/model/XOR_iter_5000000.caffemodel /sdcard/

代码编译与运行

使用android studio进行build，关键在于CMakeLists.txt的编写：

# For more information about using CMake with Android Studio, read the
# documentation: https://d.android.com/studio/projects/add-native-code.html

# Sets the minimum version of CMake required to build the native library.

cmake_minimum_required(VERSION 3.4.1)

# Creates and names a library, sets it as either STATIC
# or SHARED, and provides the relative paths to its source code.
# You can define multiple libraries, and CMake builds them for you.
# Gradle automatically packages shared libraries with your APK.

add_library( # Sets the name of the library.
             native-lib

             # Sets the library as a shared library.
             SHARED

             # Provides a relative path to your source file(s).
             src/main/cpp/native-lib.cpp )

add_definitions(-DCPU_ONLY)

add_compile_options(-std=c++11)

include_directories(${CMAKE_SOURCE_DIR}/src/main/cpp/third_party/caffe/include
                    ${CMAKE_SOURCE_DIR}/src/main/cpp/third_party/boost/include
                    ${CMAKE_SOURCE_DIR}/src/main/cpp/third_party/gflags/include
                    ${CMAKE_SOURCE_DIR}/src/main/cpp/third_party/glog/include
                    ${CMAKE_SOURCE_DIR}/src/main/cpp/third_party/protobuf/include
                    ${CMAKE_SOURCE_DIR}/src/main/cpp/third_party/openblas/include)

add_library(boost_system STATIC IMPORTED)
set_target_properties(boost_system
                           PROPERTIES
                           IMPORTED_LOCATION
                           ${CMAKE_SOURCE_DIR}/src/main/cpp/third_party/boost/lib/libboost_system.a )

add_library(caffe SHARED IMPORTED)
set_target_properties(caffe  PROPERTIES IMPORTED_LOCATION
                            ${CMAKE_SOURCE_DIR}/src/main/jniLibs/armeabi-v7a/libcaffe.so)


# Searches for a specified prebuilt library and stores the path as a
# variable. Because CMake includes system libraries in the search path by
# default, you only need to specify the name of the public NDK library
# you want to add. CMake verifies that the library exists before
# completing its build.

find_library( # Sets the name of the path variable.
              log-lib

              # Specifies the name of the NDK library that
              # you want CMake to locate.
              log )

# Specifies libraries CMake should link to your target library. You
# can link multiple libraries, such as libraries you define in this
# build script, prebuilt third-party libraries, or system libraries.

target_link_libraries( # Specifies the target library.
                       native-lib

                       # Links the target library to the log library
                       # included in the NDK.
                       ${log-lib}
                       caffe
                       boost_system)

需要注意的是：
1. 需要添加-DCPU_ONLY的定义，因为在Android app中，暂时还无法使用GPU
2. 需要加上caffe及相关库的头文件路径
3. 需要链接caffe及相关库

最后运行的结果和linux PC环境下的一致：

参考

1. 我的第一个caffe C++程序
2. caffe-android-lib