RTMP (Real Time Messaging Protocol) is a streaming media protocol specially used to transmit audio and video data. It was originally created by Macromedia, and later owned by Adobe. It is a private protocol, mainly used to contact Flash Player and RtmpServer, such as FMS , Red5 , crtmpserver, etc. The RTMP protocol can be used to implement live broadcast and on-demand applications, and push audio and video data to RtmpServer through FMLE (Flash Media Live Encoder) , which can realize real-time live broadcast of the camera. However, after all, the application range of FMLE is limited. If you want to embed it in your own program, you still have to implement the push of RTMP protocol yourself. I have implemented a RTMPLiveEncoder , which captures camera video and microphone audio, encodes them in H.264 and AAC, and then sends them to FMS and crtmpserver to realize real-time live broadcast, which can be watched normally through flash player. The effect is good at present, and the delay time is 2 seconds or so. This article introduces the main ideas and key points of RTMPLiveEncoder, hoping to help friends who need this technology.
technical analysis
To implement RTMPLiveEncoder , the following four key technologies are required:
Capture camera video and microphone audio
H264 encoding and AAC encoding
Video and audio data are encapsulated into playable streams that can be recognized by streaming media servers
RTMP protocol realizes message sending
Encapsulating audio and video data into a playable stream is a difficult point. If you study carefully, you will find that the audio and video data stream encapsulated in RTMP Packet is actually the same as the way FLV encapsulates audio and video data. Therefore, we only need to follow the way FLV encapsulates H264 and AAC to generate the available data streams. Play the stream.
Let's look at the RTMP protocol again. Adobe once published a document "RTMP Specification", but wikipedia pointed out that this document hides many details, and RTMP cannot be implemented correctly based on it alone. However, it is still useful. In fact, before Adobe released, the RTMP protocol has been almost cracked, and now there are relatively complete implementations, such as: RTMPDump , which provides a C language interface, which means that it can be easily called in other languages.
program framework
DirectShow technology is used to realize audio and video acquisition, audio encoding and video encoding, which are cyclically carried out in their respective threads (AudioEncoderThread and VideoEncoderThread), and another thread (RtmpThread) is started for RTMP push. After the two encoding threads encode the audio and video data in real time, the data is handed over to the Rtmp thread, which is cyclically encapsulated into an Rtmp Packet by the Rtmp thread, and then sent out.
The data exchange between threads is realized through a queue DataBufferQueue. After AudioEncoderThread and VideoEncoderThread post the data pointer to DataBufferQueue, they return immediately, so as to avoid affecting the normal execution time of the encoding thread due to sending Rtmp messages.
The main job of RtmpThread is to send the decoding information header of the audio data stream and the decoding information header of the video data stream, and continuously fetch data from the DataBufferQueue, encapsulate it as RTMP Packet, and send it out. The process is shown in the following code: (process_buf_queue_, which is the DataBufferQueue in the above figure)
librtmp
一、编译librtmp
下载rtmpdump的代码，你会发现，它是一个地道的linux项目，除了一个简单的Makefile，其他什么都没有。好像librtmp不依赖于系统，我们可以不用费太多功夫，把它在windows上编译。不过，librtmp依赖于openssl和zlib，我们需要首先编译好它们。
1. 编译openssl1.0.0e
a) 下载并安装ActivePerl
b) 下载并安装nasm(http://nasm.sourceforge.net/)
c) 解压openssl压缩包
d) 运行cmd命令行，切到openssl目录，分别执行以下命令
>perl Configure VC-WIN32 --prefix=c:\some\dir
>ms\do_nasm
e) 运行Visual Studio Command Prompt(2010)，切到openssl目录，分别执行以下命令。
>nmake -f ms\nt.mak
>nmake -f ms\nt.mak install
f) 编译完毕后，即可在第一个命令所指定的目录下发现编译好的sdk。
2. 编译zlib
a) 解压zlib压缩包
b) 运行Visual Studio Command Prompt(2010)，切到openssl目录，分别执行以下命令
>cd contrib\masmx86
>bld_ml32.bat
c) 回到zlib目录，进入contrib\vstudio\vc10目录，打开vs2010解决方案文件，在zlibstat工程属性中，去掉预编译宏 ZLIB_WINAPI
d) 选择debug或release编译即可
3. 编译librtmp
a) 首先打开visual studio 2010，新建一个win32 console工程，指定为静态链接库
b) 将librtmp的代码导入工程，把openssl、zlib的头文件和librtmp放在一起，把编译好的openssl和zlib的静态库放在一起
c) 在工程设置中，添加之前编译好的openssl和zlib的库，编译即可。
二、librtmp的使用
首先初始化RTMP结构
开始之后，就要向RTMP Server发起握手连接报文
连接成功，就可以开始循环发送报文了，这里需要指定时戳和数据类型（Audio、Video、Metadata）。这里有一点需要注意的是，在调用Send之前，buf中的数据，必须是已经封装好的H264或AAC数据流。
关闭
最后是释放
H264和AAC数据流
本文提到过，RTMP推送的音视频流的封装形式和FLV格式相似，由此可知，向FMS推送H264和AAC直播流，需要首先发送"AVC sequence header"和"AAC sequence header"，这两项数据包含的是重要的编码信息，没有它们，解码器将无法解码。
AVC sequence header就是AVCDecoderConfigurationRecord结构，该结构在标准文档“ISO-14496-15 AVC file format”中有详细说明。
AAC sequence header存放的是AudioSpecificConfig结构，该结构则在“ISO-14496-3 Audio”中描述。AudioSpecificConfig结构的描述非常复杂，这里我做一下简化，事先设定要将要编码的音频格式，其中，选择"AAC-LC"为音频编码，音频采样率为44100，于是AudioSpecificConfig简化为下表：
这样，AVC sequence header和AAC sequence header的内容可以基本确定了，更详细的信息，大家可以去翻阅相关文档。
FFmpeg/WebRTC/RTMP/NDK/Android音视频流媒体高级开发 学习
★文末名片可以免费领取音视频开发学习资料，内容包括（FFmpeg ，webRTC ，rtmp ，hls ，rtsp ，ffplay ，srs）以及音视频学习路线图等等。
见下方!↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓