Original address (updated from time to time): https://github.com/xhunmon/VABlog
I. Introduction
As the saying goes, from open source, give back to open source! The following is a recommendation for learning audio and video routes. The rich audio and video materials are turned to the end.
2. Learning skills
| Skill | Importance | effect | study advice |
|---|---|---|---|
| c/c++ | ★★★★☆ | Audio and video open source libraries are basically written in c/c++, for example: FFmpeg library is written in C language, and the bottom layer of Webrtc is written in c++. | 1. Watch Brainstorm or NetEase c/c++ videos; 2. Read books: c++ primer 5th edition; 3. Watch FFmpeg source code; 4. Most importantly, do it yourself. |
| cmake | ★★★☆☆ | An important language for cross-platform bootstrap compilation. It is reflected in the CMakeList.txt file. | 1. Watch the video in Brainstorming CMake; 2. Getting started with CMake |
| shell | ★★☆☆☆ | Many open source libraries are compiled through shell scripts. Such as configure in ffmpeg and x264. | Shell script |
| Android NDK | ★★☆☆☆ | To use it on the android platform, you need to master some knowledge of NDK. Such as: cross-compilation, JNI access. | Watch the JNI and cross-compilation video in Brain NDK; |
| IOS | ★★☆☆☆ | (slightly) | (slightly) |
3. Learn the theoretical knowledge of audio and video
- Book: Advanced Guide to Audio and Video Development: Practice Based on Android and iOS Platforms (Jingdong) : Chapter 1 Basic Concepts of Audio and Video ; Turn the e-book to the end.
- Book: Android Audio and Video Development_He Junlin (Jingdong) : Chapter 1 Basic knowledge of audio and video ; turn to the end of the e-book.
- You can also search here 1 or here 2 .
Fourth, learn the basic principles of audio and video
1) Introduction to video and audio data processing
[Summary] A series of articles on the zero-based learning method of video and audio codec technology introduces the general principle and process of video and audio codec technology, which is easy to understand. Includes the following articles:
- Introduction to video and audio data processing: RGB, YUV pixel data processing : video is composed of them.
- Introduction to video and audio data processing: PCM audio sample data processing : audio is composed of them.
- Introduction to video and audio data processing: H.264 video stream analysis : a kind of video coding technology (must be mastered in modern audio and video development).
- Introduction to video and audio data processing: AAC audio stream analysis : a kind of audio coding technology (must be mastered in modern audio and video development).
- Introduction to video and audio data processing: FLV package format analysis : one of the audio and video package formats. To be more specific: tear up the FLV protocol by hand .
- Introduction to video and audio data processing: UDP-RTP protocol analysis : a kind of audio and video protocol.
2) Complete RTMP push small items
- Chapter 1: The whole process of live streaming: YUV of video data source (1)
- Chapter 2: The whole process of live streaming: PCM of audio data source (2)
- Chapter 3: The whole process of live streaming: H.264 of video coding (3)
- Chapter 4: The whole process of live streaming: AAC of audio coding (4)
- Chapter 5: The whole process of live streaming: RTMP for live streaming encoding (5)
- Benefits: The blogger bought a one-year server and set up an rtmp receiving server. The address is in the project, and everyone can use it for testing.
5. Learning FFmpeg
Audio and video development cannot be done without FFmpeg, because it is a "master", which already contains or can integrate almost all modern audio and video technologies (libraries).
1) Learning Path
- Read the official documentation
- Learn official examples (in source code
doc/examples/xxx) - [Summary] FFMPEG video and audio codec zero-based learning method
- Books (e-books to the last page)
2) Learning route
a) Source code compilation
Compile ffmpeg4.2.2 Through this article, we can basically compile the FFmpeg library we want
b) Source code reading
- Source code import: import FFmpeg to Clion (MacOS) , use Clion to read FFmpeg source code (support jump)
- Reading reference: FFMPEG/FFPLAY source code analysis (CSDN download) , I also put it in the bottom information, friends who have no conditions can download it from here.
c) Study official examples
- The important structure of FFmpeg (transferred from Raytheon) , because in learning FFmpeg, you must know the meaning of the important parameters in the structure, otherwise it will be difficult.
- FFmpeg Demuxing (decapsulation) corresponds to the decapsulation part
doc/examples/demuxing_decoding.cin . - FFmpeg Muxing (encapsulation) correspondence
doc/examples/muxing.c. - FFmpeg Remuxing (repackaging) corresponding
doc/examples/remuxing.c. - FFmpeg Decode (decoding) corresponds
doc/examples/decode_audio.cto anddoc/examples/decode_video.c. - FFmpeg Encode (encoding) corresponds
doc/examples/decode_audio.cto anddoc/examples/decode_video.c. - FFmpeg simply implements transcoding and summarizes decapsulation, decoding, encoding, and encapsulation together for easy understanding.
- FFmpeg Filter corresponds to SDL (Video)
doc/examples/filtering_video.c. - FFmpeg Filter corresponds to SDL (Audio)
doc/examples/filtering_video.c. - FFmpeg Transcode (transcoding) correspondence
doc/examples/transcoding.c.
Sixth, to be updated...
Audio and video materials
- Password: lqi9
- Brainstorming video
- NetEase Video
- Advanced C and C++ Compiling.pdf
- Android audio and video development_Junlin He.pdf
- C Primer Chinese Edition Fifth Edition.pdf
- C++ Primer Plus (6th edition) Chinese version.azw3
- C++ Primer (5th Edition) Chinese Edition.pdf
- Analysis of FFMPEG_FFPLAY source code.7z
- H.264-AVC-ISO_IEC_14496-10.pdf
- H.264-AVC-ISO_IEC_14496-15.pdf
- H.264_MPEG-4-Part-10-White-Paper.pdf
- H.264 official Chinese version.pdf
- ISO_IEC-14496-3-2009.pdf
- ISO_IEC_14496-14_2003-11-15.pdf
- SDL2-API Manual.doc
- aac-iso-13818-7.pdf
- STL source code analysis Simplified Chinese full version (clear scan with catalog).pdf
- amf0_spec_121207.pdf
- amf3_spec_121207.pdf
- hls-m3u8-draft-pantos-http-live-streaming-12.txt
- hls-mpeg-ts-VB_WhitePaper_TransportStreamVSProgramStream_rd2.pdf
- hls-mpeg-ts-iso13818-1.pdf
- rtmp.part1.Chunk-Stream.pdf
- rtmp.part2.Message-Formats.pdf
- rtmp.part3.Commands-Messages.pdf
- rtmp specification translation 1.0.docx
- rtmp_specification_1.0.pdf
- video_file_format_spec_v10_1.pdf
- "FFmpeg from entry to mastery".pdf
- Digital Signal Processing Tutorial (Fourth Edition).pdf
- A New Generation of Video Compression Coding Standard-H.264_AVC (Second Edition).pdf
- Advanced Guide to Audio and Video Development: Practice Based on Android and iOS Platforms.pdf
The author has something to say
If it helps, please star it, your encouragement is my motivation for open source!
In addition: Welcome to my blog && this navigation page is also very rich in content
Everything in this article comes from the Internet. If there is any infringement, please contact me (email: [email protected]) to deal with it.