text

However, this optimization still fails to use the entire set of VoE, because VoE not only includes audio preprocessing, but also integrates the audio encoding module and transmission module into the engine, and the project needs to use the existing encoding and transmission layers, so use the entire VoE seems redundant and inoperable to me. There is no such thing as a perfect road, aside from VoE, I found the module next to VoE - APM (Audio Preprocessing Module) - a pure audio preprocessing unit that integrates all the previous modules.

Step 1 - Download the Google WebRTC source code

The development progress of Google WebRTC is still considerable. This article will take the latest trunk revision 5125 of WebRTC as an example to explain. Please use SVN to synchronize the following directories by yourself (as for the synchronization method, please google it yourself):

http://webrtc.googlecode.com/svn/trunk/

Step 2 - Extract the resources needed to compile APM

The overall compilation of APM requires the following resources in the WebRTC source directory:

1) common_audio entire directory

2) modules directory (without video part)

3) system_wrappers entire directory

4) The three header files common_types.h | common.h | typedefs.h located in the root directory of the WebRTC source code.

5) The android-webrtc.mk file located in the WebRTC home directory.

Step 3 - Compile APM basic steps and some key points in Eclipse

This section is only explained according to its own jni directory organization structure, and readers can adjust it according to their own needs.

The jni organization structure in Eclipse is as follows:

All folders and header files in Step-2 are located in the webrtc subdirectory. android-webrtc.mk is located in the jni root directory.

Let's break it down step by step:

step 3.1

First, we need to describe and set the entire android project, open the Application.mk file in the jni root directory, and edit it as follows:


# Use of this source code is governed by a BSD-style license
# that can be found in the LICENSE file in the root of the source
# tree. An additional intellectual property rights grant can be found
# in the file PATENTS.  All contributing project authors may
# be found in the AUTHORS file in the root of the source tree.
APP_STL := gnustl_static
APP_CPPFLAGS := -frtti -fexceptions
APP_ABI := armeabi armeabi-v7a
APP_PLATFORM := android-9

The official description of APP_STL is as follows:

APP_STL
    By default, the NDK build system provides C++ headers for the minimal
    C++ runtime library (/system/lib/libstdc++.so) provided by the Android
    system.
    However, the NDK comes with alternative C++ implementations that you can
    use or link to in your own applications. Define APP_STL to select one of
    them. Examples are:
       APP_STL := stlport_static    --> static STLport library
       APP_STL := stlport_shared    --> shared STLport library
       APP_STL := system            --> default C++ runtime library
    For more information on the subject, please read docs/CPLUSPLUS-SUPPORT.html

Because NDK uses the minimum C++ runtime library by default to compile the project, it is impossible to compile the source code of STL container such as std::map in WebRTC. Therefore, we need to set our own C++ runtime library gnustl_static suitable for this project.

The benefits of this article, C++ audio and video learning kits, technical videos , including (audio and video development, interview questions, FFmpeg , webRTC , rtmp , hls , rtsp , ffplay , srs ) ↓↓↓↓↓↓ See below↓↓Click at the bottom of the article Receive↓↓

step 3.2

Open and edit the Android.mk file in the jni root directory as follows, this file only needs to tell NDK to call the Android.mk files in all subdirectories:


# Use of this source code is governed by a BSD-style license
# that can be found in the LICENSE file in the root of the source
# tree. An additional intellectual property rights grant can be found
# in the file PATENTS.  All contributing project authors may
# be found in the AUTHORS file in the root of the source tree.
include $(call all-subdir-makefiles)

step 3.3

Now that the preparations are ready, you can start compiling the entire APM unit. First, open the jni/webrtc directory and create a new Android.mk file as follows:


# Use of this source code is governed by a BSD-style license
# that can be found in the LICENSE file in the root of the source
# tree. An additional intellectual property rights grant can be found
# in the file PATENTS.  All contributing project authors may
# be found in the AUTHORS file in the root of the source tree.
#
MY_WEBRTC_ROOT_PATH := $(call my-dir)
#
# voice
include $(MY_WEBRTC_ROOT_PATH)/common_audio/signal_processing/Android.mk
include $(MY_WEBRTC_ROOT_PATH)/common_audio/vad/Android.mk
include $(MY_WEBRTC_ROOT_PATH)/modules/audio_processing/aec/Android.mk
include $(MY_WEBRTC_ROOT_PATH)/modules/audio_processing/aecm/Android.mk
include $(MY_WEBRTC_ROOT_PATH)/modules/audio_processing/agc/Android.mk
include $(MY_WEBRTC_ROOT_PATH)/modules/audio_processing/Android.mk
include $(MY_WEBRTC_ROOT_PATH)/modules/audio_processing/ns/Android.mk
include $(MY_WEBRTC_ROOT_PATH)/modules/audio_processing/utility/Android.mk
include $(MY_WEBRTC_ROOT_PATH)/modules/utility/source/Android.mk
include $(MY_WEBRTC_ROOT_PATH)/modules/audio_device/Android.mk
include $(MY_WEBRTC_ROOT_PATH)/system_wrappers/source/Android.mk
#
# build .so
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_ARM_MODE := arm
LOCAL_MODULE := liblu_audio_preprocessing
LOCAL_MODULE_TAGS := optional
LOCAL_WHOLE_STATIC_LIBRARIES := \
    libwebrtc_spl \
    libwebrtc_apm \
    libwebrtc_apm_utility \
    libwebrtc_vad \
    libwebrtc_ns \
    libwebrtc_agc \
    libwebrtc_aec \
    libwebrtc_aecm \
    libwebrtc_system_wrappers \
    libwebrtc_audio_device \
    libwebrtc_utility
#
# Add Neon libraries.
ifeq ($(WEBRTC_BUILD_NEON_LIBS),true)
LOCAL_WHOLE_STATIC_LIBRARIES += \
    libwebrtc_aecm_neon \
    libwebrtc_ns_neon \
    libwebrtc_spl_neon
endif
LOCAL_STATIC_LIBRARIES := \
    libprotobuf-cpp-2.3.0-lite
LOCAL_SHARED_LIBRARIES := \
    libcutils \
    libdl \
    libstlport
LOCAL_PRELINK_MODULE := false
#
#TODO(billhoo) find a properway to do this.
LOCAL_LDLIBS += $(NDK_ROOT)/sources/cxx-stl/gnu-libstdc++/4.6/libs/armeabi/libgnustl_static.a
LOCAL_LDLIBS += -lOpenSLES
ifndef NDK_ROOT
include external/stlport/libstlport.mk
endif
include $(BUILD_SHARED_LIBRARY)

A few points to note:

1) When compiling, if it prompts that the ../../../Android.mk file cannot be found, please check and correct your relative path.

2) The link path of the gnu static library on line 60 is for the NDK version r8d. If the reader version does not match, please find the path of the libgnustl_static.a static library and replace it.

3) This example does not intend to compile the WebRTC test project, please use the Eclipse search file function to find -DWEBRTC_AUDIOPROC_DEBUG_DUMP in the Android.mk file and comment it out.

step 3.4

Everything is ready, we can start compiling APM, but there will definitely be many small problems during the compilation process (such as incorrect relative path, unable to find the symbol of a certain function, etc.), these problems are left to the readers themselves Google, SO solved

Step 4 - Notes on using APM in android apps

After the above steps, readers can get the dynamic link library libwebrtc_audio_preprocessing.so. All we need to do is to write our own jni wrapper function to provide the APM interface to the android application layer. It should be noted here that if the reader intends to reference the compiled APM library in his own dynamic library, the order in which the two libraries are loaded by the android class is sensitive.

Assuming that the reader encapsulates his own JNI interface into a separate library libmy_jni_wrapper.so, and the library references libwebrtc_audio_preprocessing.so, then the following sequence should be followed when loading these two libraries:

static {
    // Ordering of loading these shared libraries is significant.
    System.loadLibrary("webrtc_audio_preprocessing");
    System.loadLibrary("my_jni_wrapper");
}

If the order is reversed, you will get an exception at runtime that the symbol in the webrtc_audio_preprocessing library cannot be found.

Summarize

The entire compilation work seems very simple now, but requires a lot of patience and searching, but the results are quite satisfactory. The effect of APM is much better than using each audio module alone before. However, for the influence of factors such as jitter, APM is powerless.