RTMP (Real Time Messaging Protocol) is a streaming media protocol specially used to transmit audio and video data. It was originally created by Macromedia and later owned by Adobe. It is a proprietary protocol mainly used to contact Flash Player and RtmpServer, such as FMS , Red5 , crtmpserver , etc. RTMP protocol can be used to implement live broadcast and VOD applications, and push audio and video data to RtmpServer through FMLE (Flash Media Live Encoder) , which can realize real-time live broadcast of the camera. However, after all, the application scope of FMLE is limited. If you want to embed it into your own program, you still have to implement the push of the RTMP protocol yourself. I have implemented a RTMPLiveEncoder , which collects camera video and microphone audio, encodes H.264 and AAC, and then sends it to FMS and crtmpserver to achieve real-time live broadcast, which can be watched normally through flash player. The current effect is good, and the delay time is 2 seconds or so. This article introduces the main ideas and key points of RTMPLiveEncoder, in order to help friends who need this technology.
technical analysis
To implement RTMPLiveEncoder , the following four key technologies are required:
- Capture camera video and microphone audio
- H264 encoding and AAC encoding
- Video and audio data are encapsulated into playable streams that can be recognized by the streaming server
- RTMP protocol realizes message sending
Among them, the first two technologies have been introduced in my previous article " Capture audio and camera video and real-time H264 encoding and AAC encoding ", so I won't repeat them here.
It is difficult to encapsulate audio and video data into playable streams. If you study it carefully, you will find that the audio and video data stream encapsulated in RTMP Packet is actually the same as the way FLV encapsulates audio and video data. Therefore, we only need to encapsulate H264 and AAC according to the way of FLV to generate Play the stream.
Let's look at the RTMP protocol again. Adobe once published a document "RTMP Specification", but wikipedia pointed out that this document hides a lot of details, and it is impossible to correctly implement RTMP based on it alone. However, it is still useful. In fact, before Adobe released, the RTMP protocol has been almost cracked, and now there are relatively complete implementations, such as: RTMPDump , which provides an interface in C language, which means that it can be easily called in other languages.
Program framework
It is the same as the article I wrote before " Capturing audio and camera video and real-time H264 encoding and AAC encoding ", using DirectShow technology to achieve audio and video capture, audio encoding and video encoding, looping in their respective threads (AudioEncoderThread and VideoEncoderThread) , the push of RTMP starts another thread (RtmpThread). After the two encoding threads encode the audio and video data in real time, the data is handed over to the Rtmp thread, which is cyclically packaged into an Rtmp Packet by the Rtmp thread, and then sent out.
Data exchange between threads is implemented through a queue DataBufferQueue. AudioEncoderThread and VideoEncoderThread return the data pointer immediately after posting the data pointer to the DataBufferQueue, so as to avoid the normal execution time of the encoding thread being affected by sending Rtmp messages.
The main job of RtmpThread is to send the decoding information header of the audio data stream and the decoding information header of the video data stream, and continuously take out the data from the DataBufferQueue, encapsulate it as RTMP Packet, and send it out. The process is shown in the following code: (process_buf_queue_, which is the DataBufferQueue in the above figure)
librtmp
1. Compile librtmp
Download the code of rtmpdump and you will find that it is an authentic linux project with nothing but a simple Makefile. It seems that librtmp is not system dependent, we can compile it on windows without much effort. However, librtmp depends on openssl and zlib, which we need to compile first.
1. Compile openssl1.0.0e
a) Download and install ActivePerl
b) Download and install nasm (http://nasm.sourceforge.net/)
c) Unzip the openssl archive
d) Run the cmd command line, switch to the openssl directory, and execute the following commands respectively
>perl Configure VC-WIN32 --prefix=c:\some\dir >ms\do_nasm
e) Run the Visual Studio Command Prompt (2010), switch to the openssl directory, and execute the following commands respectively.
>nmake -f ms\nt.mak >nmake -f ms\nt.mak install
f) After compiling, you can find the compiled sdk in the directory specified by the first command.
2. Compile zlib
a) Unzip the zlib archive
b) Run the Visual Studio Command Prompt (2010), switch to the openssl directory, and execute the following commands respectively
>cd contrib\masmx86 >bld_ml32.bat
c) Go back to the zlib directory, enter the contrib\vstudio\vc10 directory, open the vs2010 solution file,
在zlibstat工程属性中，去掉预编译宏 ZLIB_WINAPI
d) 选择debug或release编译即可
3. 编译librtmp
a) 首先打开visual studio 2010，新建一个win32 console工程，指定为静态链接库
b) 将librtmp的代码导入工程，把openssl、zlib的头文件和librtmp放在一起，把编译好的openssl和zlib的静态库放在一起
c) 在工程设置中，添加之前编译好的openssl和zlib的库，编译即可。
二、librtmp的使用
首先初始化RTMP结构
开始之后，就要向RTMP Server发起握手连接报文
连接成功，就可以开始循环发送报文了，这里需要指定时戳和数据类型（Audio、Video、Metadata）。这里有一点需要注意的是，在调用Send之前，buf中的数据，必须是已经封装好的H264或AAC数据流。
关闭
最后是释放
H264和AAC数据流
本文提到过，RTMP推送的音视频流的封装形式和FLV格式相似，由此可知，向FMS推送H264和AAC直播流，需要首先发送"AVC sequence header"和"AAC sequence header"，这两项数据包含的是重要的编码信息，没有它们，解码器将无法解码。
AVC sequence header就是AVCDecoderConfigurationRecord结构，该结构在标准文档“ISO-14496-15 AVC file format”中有详细说明。
AAC sequence header存放的是AudioSpecificConfig结构，该结构则在“ISO-14496-3 Audio”中描述。AudioSpecificConfig结构的描述非常复杂，这里我做一下简化，事先设定要将要编码的音频格式，其中，选择"AAC-LC"为音频编码，音频采样率为44100，于是AudioSpecificConfig简化为下表：
这样，AVC sequence header和AAC sequence header的内容可以基本确定了，更详细的信息，大家可以去翻阅相关文档。
运行效果
RtmpLiveEncoder开始运行
用FMS自带的一个flash播放器播放
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
转自：http://www.cnblogs.com/haibindev/archive/2011/12/29/2305712.html