Experience sharing of video editing of short video technology on android platform

Experience sharing of video editing in short video technology on the android platform.


Tip 1: Dear officials, what is shared here is video editing, that is, editing operations on videos such as cutting/splicing/separating/merging/graffiti/marking/overlaying/filtering .Not streaming media network playback and other functions, please pay attention.

Tip 2: 90% of these words are to popularize knowledge, 10% to promote our SDK, because sharing others, benefiting ourselves, is a virtuous circle, so that we can continue to share , after all, only sharing, not beneficial and difficult to continue. You have to support your family ^_^.

Tip 3: We are a professional team of android video editing, the text shared below is a summary of actual experience, you can find every knowledge point in Verify in our SDK or open source project.
Our SDK: https://github.com/LanSoSdk/LanSoEditor_common (charged, but wrote 30 detailed notes, suitable for beginners to learn, can be used to practice)
open source project : https://github.com/WritingMinds/ffmpeg-android (free, you need to compile it yourself, you need to know linux and ndk)
you can test both projects, after learning, and then decide which way to use for your Project is most beneficial.

Text:
1. Do you have to start editing ffmpeg when doing android video editing?
Not at all. Demand is the most important. Android has provided the MediaCodec class since 4.1. If you want to extract pictures from video, you can use mediacodec or mediaextractor; if you want to crop video, you can also use MediaCodec; if you want to zoom in, use MediaCodec + Opengl is the best Good option; not necessarily doing it with ffmepg at all.
2, then why is a lot done with ffmpeg?
Because ffmpeg encapsulates many formats, it is more flexible, simple and compatible to use. Its command line can save you a lot of time, such as video cropping ffmpeg -ss 10 -t 20 -i INPUT -acodec copy -vcodec copy OUTPUT; this is done, and you no longer need to write a lot of code yourself.
3. Can ffmpeg do most of the video editing functions for me?
Unoptimized ffmpeg is functional but not performance. Due to the performance limitation of the mobile phone CPU, some commands are slow to process, such as overlay command, colorchannelmixer command eq command, libx264 encoding function, so although your function is realized, it will take 2 minutes to process a 10-second video, so function, it is estimated that it is difficult to use in the APP. For such a situation, our SDK has made hard decoders and hard encoders to completely accelerate the execution of ffmpeg, which is also the core of our SDK.
4. How do I start doing ffmpeg?
    If you are a beginner, it is recommended not to compile ffmpeg right away, because there are too many compiled projects on github, and your purpose is to learn to use it in your project. Besides, if you want to use ffmpeg to implement some complex operations, even if you spend a week compiling it, it turns out that the function can be satisfied, but the processing is too slow to be used at all, and it is meaningless to do more with less. It is recommended to use the two links recommended above, and learn it first. Even if it is a paid version, the command code of ffmpeg is public, and the operation is unchanged. There are only time and other restrictions. There is no obstacle to learning ffmepg. It is recommended to learn it first, after all, you can use it flexibly after you are familiar with it.
5. Which functions of ffmpeg perform faster and can be implemented in free projects?
         Like ordinary audio and video cutting, separation, synthesis, splicing, packaging, format conversion, these are possible. Because the video data decoding and encoding operations are not used, the execution is very fast; for example, for the function of music in the second beat, the commands he uses are: audio crop, separate the audio in the video, and then combine the new audio with the video. ; To adjust the volume, you can also use the amixer of ffmpeg to achieve; another example is the video interception in the second shot, which is actually realized by the interception command of ffmpeg; for example, you directly save the H264 bare code stream encoded by MediaCodec as If you want to encapsulate the file into mp4 format so that other players can also play it, you can use the encapsulation command of ffmpeg to achieve it, and you do not need to encapsulate it yourself. If you only use these in your project, the free version will suffice, and there is no need for a paid SDK at all.
6. Which functions of ordinary ffmpeg are slow and not suitable for use?
         In some occasions where it is necessary to manipulate the pixels of the video screen, such as overlaying with overlay, colorchannelmixer or eq as a filter, scaling with scale, and occasions where video decoding --> processing --> re-encoding is used.
         The approximate workflow of these functions is: the
         first step of decoding, if you use hardware decoding, it will be much faster, if you use soft decoding, even after NEON batch optimization, the processing speed is still not ideal;
         the second step of processing, if you use The command is colorchannelmixer. Its working principle is to put all 255 pixel values ​​into a table in advance according to your settings. When processing a pixel, use the look-up table method to get the processed value, although it is faster, but Compared with opengl, it is still very slow. After all, this is done pixel by pixel, while opengl uses vector parallel processing, which is completely incomparable in speed;
         the third step: video encoding, ffmpeg itself does not have H264 video encoder, Need to use external, such as libx264, libopenh264 these, but also soft coding.
         Each frame of your video needs to go through these three steps: decoding, processing, and encoding. In this way, if the video of 20 seconds is 25 frames per second, it is 500 frames. If it is not done with hardware acceleration, it is estimated that it is difficult to complete the speed.
         Our SDK has done hard decoding and hard coding to speed up the execution of ffmpeg and make it run faster. This is also the core of our SDK.
7. Is there a good way, and want to realize the function, but also want to use the free version?
      have. Do it in the form of MediaCodec + OpenGL, and use MediaCodec + OpenGL to implement some slow operations of ffmpeg, such as filters, such as overlay, such as zooming, etc. Most of the operations in our advanced version are also done with opengl, We have done a lot of work to make it as easy to use as you would an ArrayList to add a media and delete a media.
     
8. If we do it ourselves, what knowledge do we need to know:
     First of all, you have to despise it strategically ^_^, after all, it is not for you to engage in scientific research, to engage in high-tech, to win the Nobel Prize. This is a technology, a very mature technology, similar to learning android UI and java programming. It requires accumulation of knowledge and time to do things. If you are a beginner, it takes a long time to learn, to fumble things, and then to master the process.
     Furthermore, you have to pay attention to it tactically ^_^. The knowledge points involved are:
      8.1 The basic operation of linux, it is strongly not recommended to use cygwin under windows.
      8.2 Knowledge of ndk, jni, and gcc, because what you need to do will eventually be encapsulated into a class and method of java, which can be called by engineers who do UI, and you need to know some java programming, so that you can debug it faster.
      8.3 Knowledge of video. This is essential, and needless to say, after all, it is video editing, such as video format, audio, pcm, basic knowledge of h264, video bit rate, resolution, frame rate, common commands of ffmpeg, etc.
      8.4 After understanding video knowledge, it is strongly recommended to be fully proficient in MediaCodec and video playback principle, because this can help you save a lot of time to implement a function.

Summary: Don’t compile as soon as you come up, it is important to apply what you have learned. Using the free version consumes labor costs and time costs. Using our SDK can speed up the development of your project. Besides, the cost of our SDK is far less than your labor cost. And the cost of time cost ^_^.