Development of audio and video learning - turn almost know the answer to
copyright reserved by the authors.
Commercial reprint please contact the author authorized, non-commercial reprint please indicate the source.
Author: Yao Dong
Link: http://www.zhihu.com/question/31156766/answer/54645514
Source: know almost
Let's assume that someone is in the zero-based audio and video, never learned any knowledge of digital signal processing, the basic high school math standards, but are familiar with C / C ++ development, at least familiar with the compiler IDE debugger under one platform.
It focuses on two open source projects ffmpeg and webRTC, mainly to see the video portion of the audio portion of ffmpeg and webRTC.
First of all, start from the video decoding, the most simple, basic code that calls ffmpeg video decoding only about 100 lines, specifically refer to @ Lei Xiao Hua's blog, there are detailed examples and explanation. As long as they get to know h.264 video decoding like 80% + of the scenes are the encoding format, and understand basic sps pps NAL and other H.264 file format concept, understand the principles of YUV format image, understand YUV to RGB conversion so that you can do image rendering can be achieved with a simple video player. (The first stage in line)
Then, learning video coding, it is recommended to start from x264, a simple call x264 encoding code is not to 100 lines. Coding complexity than that, the decoding is no need to configure any parameters, and a large number of encoding parameters to be configured, as long as the initial understand a few basic concepts like, such as a frame rate, common flow control ABR CRF, GOP, I / B / P Frame are what that means. (X264 familiar with coding again)
Audio and video codecs are roughly the same, only need to know the initial two kinds of encoders and EAAC + Silk, understand the concept of channels, sample rate, to know Wave file format. You can do a simple audio player. (Make an audio player)
Audio codec is not complicated place, but the sound, in the look WebRTC Audio Processing module, be understood that the following concepts, denoising NS, the AEC echo cancellation, silence the VAD detection, automatic gain control AGC, webRTC built these Although the algorithm is not the best, but you can solve the problem of + 90% of the worth under study. (Learn webrtc audio processing)
Learned here need to think about a problem, they are still want to go the route algorithm engineering course, if the algorithm route, you must go to studying mathematics, digital signal processing is behind a large number of theoretical mathematics. If the project route, then going to learn cross-platform development, audio and video learning acquisition of multiple platforms, playback and processing, learning assembly language to optimize multiple platforms. (Platform dependent)
Audio and video is a deep water field, learn a lifetime to learn not finish, but this area has the advantage of knowledge is updated more slowly, things are not easy to learn to be eliminated, the accumulation of experience is very important.