Audio and video editing and processing through FFmpeg commands in Android

The following article comes from DevYk, author DevYK

Audio and video editor

Preface

Sometimes we want to process audio and video, such as video editing, adding subtitles, cropping and other functional processing. Although some good projects are open source on Github, if we want to carry out secondary development on this project, for example, I want to The dynamic library of this project is based on OpenH264 to encode YUV. At this time, it is possible that the dynamic library does not integrate the OpenH64 library, so for extensibility, I made a set of universal libraries, which basically contains all commonly used libraries. Audio and video processing library, you do not need to compile.

Compiled header files and dynamic libraries can be obtained by themselves in the core/cpp directory of the project ( "fully open source" ).

Introduction

``Video and Audio Editor''  mainly transplants FFmpeg v4.4-dev + libx264 + freetype + fontconfig + fribidi + openh264 +libfdk-aac + gnutls + speex + libwebp + lame +opus + opencore-amr + https and other libraries. The fast processing framework for audio and video editing and video editing on the Android platform currently has built-in functions such as audio and video editing, editing, multiple video file merging, subtitles, watermarks, and reverse playback. You can also pass in the  "FFmpeg"  command for processing.

``Let's first look at the effect of the built-in functions:''

Audio and video clips:

Audio and video synthesis:

Audio and video editing:

how to use

1. Add dependencies

implementation 'com.devyk.ffmpeglib:AVFFmpegCore:1.0.1'

2. Function API introduction

"Callback processing"

public interface ExecuteCallback {
  /**
  *开始处理
  */
    void onStart(Long executionId);
    /**
     * 如果外部传递了当前操作视频的时长，那么返回的是百分比进度，反之返回的是操作视频对应的微妙时长
     *
     * @param v
     */
    void onProgress(float v);
  /**
  *处理成功
  */
    void onSuccess(long executionId);
  /**
  *处理失败
  */
    void onFailure(long executionId, String error);
  /**
  *取消处理
  */
    void onCancel(long executionId);
  /**
  * ffmpeg 执行的 log
  */
    void onFFmpegExecutionMessage(LogMessage logMessage);
}

「AVEditor」

• Clip:

AVVideo: //start: start time unit second
//duration: how many seconds to edit
fun clip(start: Float, duration: Float)
AVEditor:
fun exec(
epVideo: AVVideo, 
outputOption: OutputOption, 
executeCallback: ExecuteCallback)

• Rotate:

AVVideo: //rotation: rotation angle (only support 90, 180, 270 degree rotation)
//isFlip: whether to mirror
fun rotation(rotation: Int, isFlip: Boolean)
AVEditor:
fun exec(
epVideo: AVVideo, 
outputOption: OutputOption, 
executeCallback: ExecuteCallback)

• Cut:

AVVideo: //width: width of cropping
// height: height of cropping
// x: starting from x point
// y: starting from y point
fun crop(
width: Float, 
height: Float, 
x: Float, y: Float )
AVEditor:
fun exec(
epVideo: AVVideo, 
outputOption: OutputOption, 
executeCallback: ExecuteCallback)

• Add text watermark:

AVVideo: fun addText(avText: AVText)
AVEditor:
fun exec(
epVideo: AVVideo, 
outputOption: OutputOption, 
executeCallback: ExecuteCallback)

• Add image watermark:

AVVideo: fun addDraw(epDraw: AVDraw)
AVEditor:
fun exec(
epVideo: AVVideo, 
outputOption: OutputOption, 
executeCallback: ExecuteCallback)

• Video merge:

AVEditor: fun merge(
epVideos: List, 
outputOption: OutputOption, 
executeCallback: ExecuteCallback)

• Add background music:

AVEditor:
music(
videoin: String,
audioin: String,
output: String,
videoVolume: Float,
audioVolume: Float,
executeCallback: ExecuteCallback
)

• Audio and video separation:

AVEditor: fun demuxer(
inSource: String, outSource: String, 
format: Format, 
executeCallback: ExecuteCallback)

• Video playback:

AVEditor: fun reverse(
videoin: String, out: String, 
vr: Boolean,//whether the video is reversed
ar: Boolean, //whether the audio is reversed
executeCallback: ExecuteCallback)

• Video to picture:

AVEditor: fun video2pic(
videoin: String, //Video input file
out: String, //Picture output path-directory
w: Int, h: Int, //Width and height of the output picture
rate: Float, //Video generation per second Number of pictures
executeCallback: ExecuteCallback)

• Convert video to Gif:

AVEditor: fun video2Gif(
videoin: String,
gifOut: String,
startDuration: Int,
stopDuration: Int,
executeCallback: ExecuteCallback
)

• Custom command:

AVEditor: //cmd: FFmpeg command
//duration: duration of video processing, you can get
fun execCmd(cmd: String, duration: Long, executeCallback: ExecuteCallback) through VideoUitls.getDuration(videoPath )

FFmpeg compilation tips

Sometimes we find some good projects based on FFmpeg on Github, such as ijkplayer, RxFFmpeg, etc. We want to do secondary development based on it, because we don’t know how to compile, and we don’t know which nodes need to be opened to compile FFmpeg, this At that time, I wanted to take the compilation scripts of certain projects and use them to perform secondary compilation. Generally speaking, some projects will not open source to compile FFmpeg scripts. At this time, we can get the static or dynamic library of the open source project. Here I will take RxFFmpeg as an example. You can see if I get its compilation script.

1, destination clone RxFFmpeg

git clone https://github.com/microshow/RxFFmpeg.git

2. Associate librxffmpeg-core.so

Through the so, we know that it should be the dynamic library compiled by FFmpeg, and now we associate to the so through cmake

cmake_minimum_required(VERSION 3.4.1)
#JNI 路径
set(FFMpeg_include_PATH ${CMAKE_SOURCE_DIR})
include_directories(${FFMpeg_include_PATH}/include/)
add_library(RxFFmpeg SHARED IMPORTED)
set_target_properties(RxFFmpeg PROPERTIES IMPORTED_LOCATION ${PROJECT_SOURCE_DIR}/../../../libs/${CMAKE_ANDROID_ARCH_ABI}/librxffmpeg-core.so)
find_library(
        log-lib
        log)
FILE(GLOB JNI_ALL_C ${JNI_PATH}/*.cpp)
add_library(
        ffmpeg-tools
        SHARED
        ${JNI_ALL_C}
)
target_link_libraries(
        ffmpeg-tools
        RxFFmpeg
${log-lib}
)

3. Write JNI function and get the compiled script

//
// Created by DevYK on 2020-10-02.
//
#include <android/log.h>
extern "C"
{
#include "libavutil/avutil.h"
}
#include <jni.h>
#define  AV_TAG   "AVLOG"
#define LOGE(format, ...)  __android_log_print(ANDROID_LOG_ERROR, AV_TAG, format, ##__VA_ARGS__)
int JNI_OnLoad(JavaVM *javaVM, void *pVoid) {
    const char *config = avutil_configuration();
    LOGE("FFMPEG VERSION%s \n", av_version_info());
    LOGE("FFMPEG configuration %s \n", avutil_configuration());
    return JNI_VERSION_1_6;
}

View the information in the memory that the config pointer points to through debug as follows:

Well, after getting its compilation information, we can improve the compilation of our project based on it. We can compile it with more functions than it is. Just like the introduction at the beginning, I added some commonly used C++ libraries on the market. Basically reached omnipotence.

to sum up

Project address: AVFFmpegLib

I won’t introduce how to compile here. If you’re interested, you can see the mobile-ffmpeg project. I am also doing secondary packaging development based on it here.

reference

• EpMedia
• FFmpeg common commands
• mobile-ffmpeg
• Android audio and video editing experience summary and open source project sharing

Friends who need audio and video related and more Android learning materials can [ Private Message ] or click to get