More domestic application of open source stream media server nginx-rtmp-module there has been less functional, large clusters of difficulty and other issues. In LiveVideoStack online to share in, PingOS open source project team development engineers, UCloud RTC R & D engineer Zhu Jianping introduced in detail based on nginx-rtmp-module of PingOS streaming media server at http-flv, http-ts, hls +, multi-process, retweet, back to the source and clustered deployment of technical implementation details.
Wen / Zhu Jianping
Finishing / LiveVideoStack
Live Playbackhttps://www2.tutormeetplus.com/v2/render/playback?mode=playback&token=006643cdea15499d96f19ab676924e88
1. Nginx streaming expansion: http-flv, http-ts, hls +
The most original nginx-rtmp-module associated with most models, including streaming media server SRS, including virtually identical (one producer, n number of consumers). Nginx keep a problem: it merely made a consumption model of RTMP, if you want to expand http-flv form or http-ts will be more difficult. Since rtmp-session only use RTMP protocol, if you want to expand http-flv, first of all we need to understand its underlying distribution model (shown above): All producers and consumers will be mounted to the same stream, the production who is responsible for receiving data from the network, it sends the consumer to obtain data from the buffer.
If flv transmission data, the original can be retained rtmp-session, when the server receives a HTTP request, to create a rtmp-session, this session is not associated with the network, only a logical session. The session then injected into the stream which, if based on the consumer's role is injected into the stream of them, you can achieve access to the data and distribute out. If at this time the server receives a request for http-flv, you can create a session on a logical, and inject it into the stream, then theoretically we can get is rtmp data. But what we need is data flv, flv data due to the similarity with rtmp data, we can be very easy to restore data to rtmp flv data through tag-header manner. According to the above ideas, in producer and consumer model, consumers can reuse the previous distribution processes and fulfilled the http-flv agreement by creating a form of http-fake-session. We extend them to create a http-fake-session as producers, and let the http-fake-session with a http client association, after the association http client is responsible for passing data from a remote server to download to the producers, the producers will these data can be distributed through the distribution model following rtmp-session. This also indirectly a function return a http source. We can achieve http-flv player and pull flow quickly through the above ideas. Similarly, we can continue to expand the agreement in accordance with the above ideas. If we after receiving a http request to create a similar rtmp-fake-session (session on logic, not related to the network), we put it in the role of the consumer is inserted into the stream of them. This can be obtained from the stream of data which needs to be distributed downwards. Note that: stream rtmp is initially stored data rather than the data ts, ts can not get the data directly.
1.1 http-flv implemented in the Nginx
Note the following details based on Nginx achieve http-flv: First, the realization of multiplexing the distribution model and Nginx http modules. (Nginx support for http protocol stack more perfect, including http1.0, http1.1 protocol) in some lines of business, the customer may need to add a suffix when you download http-flv, in accordance with past practice, we will be in the code which logic filter suffix. If you meet more complex requirements such as whether the changes need to open http chunked encoding, we can only modify the code. And if it is to achieve Nginx http-flv reuse existing modules by http-based, we will be able to implement these operations by nginx-http-rewrite function. Therefore the use of native features nginx-http to develop http-flv can bring more benefits, such as significantly reduce the amount of code.
Here I have seen one case: that reuse the http module, but no distribution process of multiplexing rtmp. This will cause the process we need to distribute the http-flv to re-do it again, control of the business will become very complicated.
For example, if it was requested to be played at this time, it is necessary to notify message to the service server. At this point, if rtmp and http-flv achieve two protocols are separate, it means that if both are triggered, they are required to report to the business server. So we need to pay double the code logic maintenance work, which will undoubtedly significantly increase development and maintenance costs. Therefore, the simplest solution is to implement flv no treatment-related business, format conversion only when the next issue, equivalent to send data only when rtmp rtmp format distribution, while rtmp flv only need to distribute data marked flv the tag-header, and then issued, thus eliminating the need for the development of the business layer.
http-flv player realization
The figure shows that the cache for rtmp rtmp and http-flv support both protocols. Both http-flv and rtmp share a cache, in fact, flv rtmp data transmission itself, but is the tag-header to throw away.
http-flv is issued and issued only difference points rtmp is that different send function: http-flv call is send function http and called when issued rtmp is native send function, before the next made to add the respective protocol header. Shared memory can achieve both a saving memory effect, and to achieve uniform traffic, reduce development costs.
http-flv back to the source of realization
The figure shows that the http-flv back to the source in the realization of nginx. Http-flv thinking back to the source to achieve the realization of the http-flv player similar idea: create a http client that is in need back to the source, things http client did was to download data to a local http. Before downloading data to a local http client need to create a rtmp fake session and as producers injected into the stream of them. Http client then begins to download data from the Internet and downloaded to the fIv rtmp data split into data. Why split into rtmp data? This is because the cached data rtmp plug-flow type rtmp coming, so downloading data from the Internet to the need to do a split flv, rtmp split into data and then into the cache. The requirements of the final actual data into or rtmp flv format. Thus, according to http-flv player in logic rtmp fake session, and will be able to quickly realize the operation back to the source of http-flv.
1.2 http-ts is implemented in Nginx
The figure shows that the realization http-ts in the Nginx. Its implementation and realization of ideas http-flv basically the same, just different in operation, except that the http-ts require a separate buffer cache. Since the difference http-ts and http-flv large data format for data to rtmp flv, the data only need to split into individual pieces, and adds a header in front. Even the most one or a block of data is missing flv do not filled.
However, different data ts, more stringent requirements that each block must be 188 bytes, including a header and a payload section ts. And if the database size is less than 188 bytes, you need filled. Rtmp while not strictly fixed data block size required length thereof. Ts for data, the order data into the flv ts data, this process is the need to consume some amount of calculation. Since flv ts data and data formats that much difference, so here we will ts of the buffer and the buffer is completely independent rtmp open. But this is not enabled by default, you need to be configured in the server.
After the open configuration, the buffer will be rtmp generate a data mirroring ts, ts this part of the data will only be for http-ts and hls two protocol. Server also relates to a service hls native, where we did not make any changes, but added hls + service to use this buffer. Whether ts or hls +, they have registered their fake session, the purpose of doing so is to unify the business. For example, when there is a request to enter the play, we need to let the server know the current business there are requests.
This network is similar to the notification, event notification interfaces, in the process of development we all want to just write a business data, rather than saying do hls agreement to write a notification for hls play, but also for the agreements do ts ts write a notice, so the code will become a huge business, leading to service almost hard to maintain.
Thus fakesession effect is very large, which service layer and the network layer will completely separate. Even if the server itself issued a protocol is not rtmp, create and mount a rtmp-session to the service server can be. Overall, http-ts The only difference with the http-flv to achieve is to get the different positions of the buffer. http-flv need to get from rtmp buffer, http-ts is obtained from ts buffer in. If you can understand http-flv agreement process, then it is not hard to understand http-ts implementation process.
1.3 hls + Nginx is implemented in
It is shown in FIG hls + nginx is implemented. hls hls + with different traditions, traditional hls no status on the server side, the server contains a large number of pieces of data, the client continues to perform the download, but hls + each client's status is logged. For how to record each client's status, before I try to connect to hls + to create a virtual connection used to record status. But we found that business would be more complicated, and the latter will have many problems, including the amount of code, bug and maintenance costs. Then replace the other way, or using fake session of ways. As consumers into using fake session, in accordance with each incoming http, connecting, and then bind the session ID.
Since the transmission request when the client hls 1st sessionID is not known to the server if a connection is not to acquire the session ID, the client is considered to enter the first time. The client will receive a reply of 302, 302 replies will tell the client to a new address, which contains a session ID. After the client to get session ID, request m3u8 again, will join the session ID, the server can obtain the corresponding session ID and client identity to distinguish. This session ID can be recorded by the play status of each client. Why record this state? This is mainly because the server is not hard but the data is written directly into memory, it needs to know each user, each client's download progress and positioning ts data from memory depending on the schedule.
and hls + http-ts ts they share a buffer, and real-time positioning hls + ts content from the buffer. So for hls +, the ts and no real data generation, but a record for each file offset of the code in memory. Therefore hls + there is no problem reading and writing, doing hls service, you may encounter a problem before - the hard disk read and write bottlenecks. Mechanical hard disk read and write speed is relatively slow, the general idea is to solve mount a virtual hard disk, memory mapped to the directory to read and write.
If a hls + program, reducing the need to mount operations, but also for memory and not much consumption. And if there is demand and hls + http-ts at the same time, this time for memory utilization is very high.
2. Static push-pull flow
Static sliding flow primarily to meet the needs of the cluster. If a single server is insufficient to support high-concurrency services, then we need to consider the scalability of the server. In addition, if the user scattered throughout the country, also we need to open up between the servers. But if the business is not as complicated as you may choose to use a static push-pull flow.
Static push-pull configuration streaming service shown above, first of all look at the static pull streams: the presence of a target source station first, if you are using a static back to the source, the destination address is configured in the configuration file which target the source station can be altered.
The figure shows a simple static pull flow model: if the data is pushed from the anchor flows to the source station A, then we need to ensure that the address server A does not change.
In addition, if you want to build a complete streaming media system, you need to include static and static push pull stream flow. If the audience is played to the server C request, the server will pull the stream C to the server A, the server regardless of whether there is video streams A, C will pull server. Therefore, this model applies only to simple business scenarios.
3. Dynamic Control: The dynamic back to the source, dynamic push switch, authentication
As opposed to "no brain" push and pull sliding flow static flow, more suitable for most people's needs is a push-pull dynamic flow.
Nginx's RTMP service for each of the functions are done in different stages of the trigger. To oclp_play example, when someone starts playing triggers play a message, play the message carries a start parameter.
During playback, play the message will still be triggered, but this time will carry update parameters. At the end of play will trigger a play message, carried by the parameter is down. With these parameters, we can achieve specific business server requests notification to the stage to play and play.
3.1 Dynamic back to the source
There is a similar plug-flow process operations, publish plug flow, the same is divided into three stages, play in the main application, and publish the authentication operation. If the start phase, the server returns the results of a business or non-404 200, the server will interrupt the current play request, publish true. In addition, pull and push pull is mainly used in dynamic flow stage. When the server receives a request to play, and found that there is no objective flow of current server, that publish the stream does not exist, it will trigger the start of stage pull. After transmitting the service start request if the server returns a result 302, the location and has written a new address or http-flv rtmp address, this server will mark the target server that pulling rtmp stream or fIv stream, this process is called dynamic pulled stream.
3.2 Dynamic Retweet
Pull dynamic stream corresponding dynamic plug flow, which is appreciated that the dynamic pull manner substantially the same flow. If you push the stream to the server, the server sends a request to start the configured target address. If you add a new address in the returned results rtmp among the media server will push rtmp stream to a new address, which is the operating dynamic plug flow. This is the premise of all is to return 302 results, and if do not want to launch the stream, then back to the server 400, or other non-200, the flow is interrupted.
Oclp_stream with a relatively small, with only the creation is triggered when the road disappeared in the stream. Either play or publish, or publish play only if there is, you will think this way the flow of the life cycle is not over, and only when the two disappeared before the road was identified flow life cycle has ended.
Similarly, if the stream is not released all the way through but only the first time someone requests at this time will be considered a trigger start and the flow path is created, but it is not the producer. This application scenario is relatively small, only on the business requirements are relatively high system might use this message.
The figure shows a configuration examples, including IP query server, query server operating'd like to see the stage play support.
Cluster deployment dependent business (scheduling) server, if there is demand back to the source so that the edge server B to query the server business in oclp_hold stage, when business will tell the server edge server B a 302 address, which contains the source address. Edge Server B will come out from the tag table (media server A) to pull the stream, in order to achieve dynamic back to the source.
Dynamic Retweet mainly to the local flow pushed out. In the CDN services in different clusters responsible for different functions. For example, some cluster responsible for the recording, some are only responsible for transcoding, this time we hope to be able to put these core machine needs to be transcoded or want to record streaming on demand forwarded to the appropriate cluster. Dynamic Retweet very important, if these different types of services included, you need to add configuration oclp_push to achieve dynamic retweet.
3.3 Authentication
Authentication operation, we will only publish or play for authentication.
If the play when the feedback is allowed to play 200, 403 if the feedback is not allowed to play, publish, too, by a customer service server controls whether a service request can be allowed.
When the front-end play or is publish, how the authentication token brought over?
All custom parameters args = k = v & pargs = $ pargs while sending play query outwardly, if added args = k = v & pargs = $ pargs, will bring these parameters retransmission request so that it can be rtmp of: primarily by variable passed over.
4. Multi-process: inter-process back to the source
There are many multi-process problems bug in nginx rtmp native, the practice is now on the list of each process stream down through shared memory record. If the play does not flow process, the query stream list, by unix socket and pull back to the source to the target flow process. In addition, inter-process back to the source does not trigger ocl_playoclp_publish oclp_pull messages.
The more instructions
Drops:https://github.com/im-pingo/pingos
[Trailer] large-scale online sharing and interactive live online video user experience optimization practice
Accompanied by sudden epidemic, online education, video conferencing, interactive live online video needs brought to the outlet, how in large-scale, highly concurrent video needs to provide a smoother, more clear, with a low latency live video viewing experience for the user? How to give Whiteboard function for online learning education to optimize the user experience and teaching effectiveness? How to stimulate user interest in online video, interactive video to enhance the user experience?
At 19:00 on March 29 , LiveVideoStack hand in hand UCloud RTC chief architect Wang Lifei, learning and thinking Wang Xiao senior architect Zhao Wenjie, beep beep Mile Mile senior R & D engineer Tang Jun-line, to explore the underlying technology infrastructure and large-scale interactive live online video behind and share user experience optimization practices.
- "Optimizing practice URTC million people live even Jimmy" Wang Lifei UCloud RTC Chief Architect
- "Education at a whiteboard technology to optimize online / offline" Zhao Wenjie learning and thinking Wang Xiao senior architect
- "High Energy bars: The interactive video experience optimization," Tang Jun-line beep beep Mile Mile senior R & D engineer
Reservation inlet open class:
