Implementing RtspServer and high-definition, high-resolution and high-bitrate video transmission optimization based on Live555
Recently I have done some RTSP server projects for PC and embedded platforms. Most of the requirements are simple but comprehensive functions, and the performance is even stronger. After comprehensive consideration, development is basically based on live555. After optimizing Live555 itself and optimizing the video data transmission within the program, not only the requirements were met, but the performance exceeded expectations, achieving 1080p with a high code rate of 10Mbps . Smooth live streaming of HD video at above resolutions . Here are some optimization points to share:
Why develop based on Live555
In fact, I have developed an RTSP Server program before and wrote an article to introduce it " Design and Implementation of an RtspServer and Introduction to RTSP2.0 ". However, the purpose of development at that time was not only to realize RTSP live broadcast, but also to simplify it. The code is designed to facilitate customization, so it does not fully implement all the interaction details in the RTSP protocol. To fully expand it based on it may delay the progress of the project. Based on project considerations, I chose Live555, an RTSP open source project that I know well and think is excellent, as the basis to develop the RTSP Server program.
Live555 is a cross-platform streaming media solution that uses C++ as the development language. It implements the entire RTSP structure including server-client, and supports H.264, H.265, MPEG, AAC and other video and audio encodings. It is A very well-known open source project. As an RTSP Server, the source code only contains video sources for local files, but it is highly scalable and can develop service programs suitable for your own project needs based on some base classes provided by Live555.
Live555 architecture and RTSP data flow
Core module of Live555
The interaction process between RTSP server and client
Live555 streaming media module and server processing flow
The streaming media module of Live555 is basically divided into two parts: Source and Sink. Of course, they also have a common base class Medium. For the server, Source is the data source and Sink is the data output. The video data is passed to MediaSink through MediaSource and finally transmitted to the client through the RTPInterface network. Here are the modules and inheritance relationships used by the server:
As shown in the figure above, by completing your own ServerMediaSubsession and MediaSource, you can pass the H.264 encoded data that needs to be broadcast to live555 to achieve RTSP live broadcast.
Optimization points for high bit rate video data transmission
For high-definition and high-bitrate video images, the video data of each frame will be relatively large, and this value will often exceed the default internal memory processing size of live555, because the optimization of live555 mainly focuses on the expansion of the memory buffer size, and Avoid memory data copying. The following are effective optimization points summarized based on actual development and testing:
- The extended frame parsing buffer size, that is, BANK_SIZE, has a default value of 150k and is set to at least 300k according to the size of the transmitted H264 data frame. Otherwise, if it exceeds the size, it may be abandoned by Live555.
- Increase the OutPacketBuffer::maxSize size, also to accommodate oversized frame data, otherwise data loss may occur.
- In RTPInterface, increase the socket send buffer size, that is, the parameter value of the increaseSendBufferTo function
- For MultiFramedRTPSink::sendPacketIfNecessary, you can directly call sendNext to try to form an RTP message to send a data packet. The advantage of this modification is that the read data will be sent out as soon as possible, but it will also take up more thread time.
- When an application transfers data from its own thread to Live555, it should minimize memory copies, preferably through a memory pool, to avoid copying memory from blocking the Live555 event loop.
After the above modifications and the optimization of the internal code of the application, in actual applications, smooth live broadcast of high-resolution HD videos above 1080p with a high bit rate of 10Mbps has been achieved.
Please contact QQ for cooperation. (Please indicate the author and source when reprinting~)