background:
The first two chapters briefly introduces the RTSP protocol, FFmpeg simple interactive process of RTSP. Mentioned before, practical applications for different projects, FFmpeg need to be revamped to accommodate a variety of special circumstances.
Forefront of the case are as follows:
. 1, the SETUP protocol selection phase: carrier protocol is the TCP or UDP, RTP whether the bearer handover procedure attempts after a protocol not supported by the carrier
2, the redirection of sound: Normally, usually after SETUP stage, there will be no redirect, because this requires re-opened and re-connected, but some actual server will be redirected to PLAY stage
when using UDP carriers: 3, NAT penetration , in the network of the client how NAT holes to let the server can transmit data to the client
4, select media server: the RTP-Info to the server and DESCRIBE / SETUP to the server not the same, how RTP-Info is switched to the specified server.
5, and integration of MediaPlayer: PLAY when used in describing the absolute time, which is how to integrate with MediaPlayer
Next explained related changes.
Relevant amendments
1, protocol selection SETUP stage
This is relatively simple, it said the previous section, SETUP is carried out in stages connect, resolve playback parameters carried url address, according to the included "udp" / "tcp" / "http", set lower_transport_mask flag, according to the flag one will try.
As for the item, or the server has been determined to be configured according to the type of property or a server (server type may "Server:" carried in ff_rtsp_parse_line function parameter determination), designated in what carriers ff_rtsp_make_setup_request, to avoid excess SETUP try again.
2, redirected perfect
Native FFmpeg does not support redirection operation when the PLAY command sent by the server in general will not have such an operation, because if redirected required, connect phase (DESCRIBE-> SETUP) can redirect operation, and If we redirect pLAY stage, you need to disconnect and reconnect, will be extended from broadcast play speed.
Of course, the actual circumstances ZTE server will redirect PLAY operation, in accordance with the above said process, need to be disconnected and re-connected.
Modifications to such a case, the first modification is carried out inside rtsp_read_header function modification, to redirect 302 returns such a specific error code, skip to the beginning again disconnected after ff_rtsp_connect function, reconnecting with the new URL.
amend as below:
+++ rtspdec.c (working copy)
@@ -237,7 +237,16 @@
}
ff_rtsp_send_cmd(s, "PLAY", rt->control_uri, cmd, reply, NULL);
if (reply->status_code != RTSP_STATUS_OK) {
- return -1;
+ if(reply->status_code >=300 && reply->status_code < 400 && s->iformat){
+ av_strlcpy(s->filename, reply->location, sizeof(s->filename));
+ if(rt->playseekFlag == 1){
+ av_strlcatf(s->filename, sizeof(s->filename),"&playseek=%sZ", rt->playseekTime);
+ }
+ av_log(s, AV_LOG_ERROR, "Redirecting to %s\n", s->filename);
+ return -(reply->status_code);
+ }
+ else
+ return -1;
}
if (rt->transport == RTSP_TRANSPORT_RTP &&
reply->range_start != AV_NOPTS_VALUE) {
@@ -365,6 +374,8 @@
{
RTSPState *rt = s->priv_data;
int ret;
+
+redirect:
av_log(NULL, AV_LOG_INFO, "[%s:%d]\n", __FUNCTION__, __LINE__);
rt->send_keepalive=0;
ret = ff_rtsp_connect(s);
@@ -407,9 +418,12 @@
if (rt->initial_pause) {
/* do not start immediately */
} else {
- if (rtsp_read_play(s) < 0) {
+ ret = rtsp_read_play(s);
+ if (ret < 0) {
ff_rtsp_close_streams(s);
ff_rtsp_close_connections(s);
+ if(ret <=-300&& ret > -400)//add by wusc for redirect
+ goto redirect;
return AVERROR_INVALIDDATA;
}
}
The principle is so, but such a simple modification will lead to the collapse of many places, for example, has been closed connections, stream, etc., when there will be reconnected again free and other operations resulting in the collapse, some of which are already members of alloc alloc repeatedly caused the collapse, some relate to modify the underlying RTP / TCP layer, in order to avoid an impact on other modules, and finally did not take this way, but in the upper layer to be modified.
Previous section said rtsp_read_header entrance is in avformat_open_input, when the error code is the error code RTSP redirection, call the avformat_open_input again, because AVFormatContext had been free in RTSP demux in, so I can not redirect the URL to take home the top only with a relatively ugly way (global variable) to the URL to avformat_open_input. This amendment can be fully re-open the process file FFmpeg go again, the benefits are less modified place, more reliable, simpler, bad place is the code a bit confusing.
amend as below:
avformat_open_input：
if (!(s->flags&AVFMT_FLAG_PRIV_OPT) && s->iformat->read_header)
if ((ret = s->iformat->read_header(s)) < 0)
goto fail;
Index: rtsp.c
===================================================================
--- rtsp.c (revision 151774)
+++ rtsp.c (revision 151775)
@@ -538,8 +538,10 @@
avformat_free_context(rtpctx);
} else if (rt->transport == RTSP_TRANSPORT_RDT && CONFIG_RTPDEC)
ff_rdt_parse_close(rtsp_st->transport_priv);
- else if (CONFIG_RTPDEC)
+ else if (CONFIG_RTPDEC){
+ if(!rtsp_st->transport_priv)
rtp_parse_close(rtsp_st->transport_priv);
+ }
}
rtsp_st->transport_priv = NULL;
if (rtsp_st->rtp_handle)
Index: utils.c
===================================================================
--- utils.c (revision 151774)
+++ utils.c (revision 151775)
@@ -1018,6 +1018,7 @@
return av_probe_input_buffer(s->pb, &s->iformat, filename, s, 0, 0);
}
+extern char rtsp_play_redirect_url[1024] = {0};
int avformat_open_input_header(AVFormatContext **ps, const char *filename, AVInputFormat *fmt, AVDictionary **options, const char *headers)
{
@@ -1035,6 +1036,10 @@
if ((ret = av_opt_set_dict(s, &tmp)) < 0)
goto fail;
+ if(rtsp_play_redirect_url[0] ){
+ filename = av_strdup(rtsp_play_redirect_url);
+ av_log(NULL,NULL,"ugly code for RTSP PLAY 302 -%s---\n",filename);
+ memset(rtsp_play_redirect_url,0,sizeof(rtsp_play_redirect_url));}
av_log(NULL,NULL,"-%s---\n",__FUNCTION__);
if ((ret = init_input(s, filename, headers, &tmp)) < 0)
goto fail;
Index: avio.c
===================================================================
--- avio.c (revision 151774)
+++ avio.c (revision 151775)
@@ -530,8 +530,12 @@
int ret = 0;
if (!h) return 0; /* can happen when ffurl_open fails */
+ if(!h->prot)
+ return 0;
if (h->is_connected && h->prot->url_close)
+ {
ret = h->prot->url_close(h);
+ }
#if CONFIG_NETWORK
ff_network_close();
#endif
Index: rtspdec.c
===================================================================
--- rtspdec.c (revision 151774)
+++ rtspdec.c (revision 151775)
@@ -45,6 +45,7 @@
static int _rtsp_read_packet(AVFormatContext *s, AVPacket *pkt) ;
static int rtsp_read_flush(AVFormatContext *s);
void rtsp_invoke(void *para);
+char rtsp_play_redirect_url[1024] ;
static void *recv_buffer_task( void *_AVFormatContext)
{
@@ -386,7 +387,6 @@
RTSPState *rt = s->priv_data;
int ret;
av_log(NULL, AV_LOG_INFO, "[%s:%d]\n", __FUNCTION__, __LINE__);
rt->send_keepalive=0;
ret = ff_rtsp_connect(s);
@@ -431,26 +431,20 @@
} else {
ret = rtsp_read_play(s);
if (ret < 0) {
ff_rtsp_close_streams(s);
ff_rtsp_close_connections(s);
if(ret <=-300 && ret > -400){//add by wusc for redirect
+ av_log(NULL, AV_LOG_INFO, "play redirect to :%s",s->filename);
+ av_strlcpy(rtsp_play_redirect_url, s->filename, 1024);
+ rtsp_play_redirect_url[1023] = '\0';
+ return ret;
}
return AVERROR_INVALIDDATA;
}
}
av_log(s, AV_LOG_INFO, "[%s]transport=%d timeout=%d div=%d\n",__FUNCTION__,rt->transport,rt->timeout,rt->keeplive_div);
return 0;
}
3, NAT penetration
When using UDP carrier, according to the situation of the network, in the network may require the client's NAT holes, to allow the server to the client, data can be transmitted.
FFmpeg itself in such a processing mechanism, different versions of the function name, ff_rtp_send_punch_packets / rtp_send_punch_packets. Native code, after holes are after SETUP, before PLAY, but for ZTE server, you need to put it in PLAY, and burrows and heartbeat mechanism requires the same room periodically sends once. How transmission, and 4:00 modifications described together.
4. Select the media server
According to the RTSP protocol, media server DESCRIBE / SETUP will give us the address of the server,
the RTP-Info is the server replies to our media server information, but to the server and DESCRIBE / SETUP to the server is not the same as the actual situation RTP-Info, Further according to standard procedures received RTSP DESCRIBE / SETUP specified address is no flow, we have to consider the use of RTP-Info server address, the same, if the carrier is UDP, NAT needs to send the packet.
Parsing a media server addresses and RTP-Info field of the handover by the following:
Index: rtsp.c
===================================================================
--- rtsp.c (revision 151103)
+++ rtsp.c (revision 151104)
@@ -781,8 +781,17 @@
p++;
get_word_sep(value, sizeof(value), ";, ", &p);
read++;
- if (!strcmp(key, "url"))
+ if (!strcmp(key, "url")){
av_strlcpy(url, value, sizeof(url));
+
+ //wusc add for zte server rtpinfo
+ if(NULL != strstr(rt->server, "ZXUS") || NULL != strstr(rt->server, "ZMSS")){
+ memset(rt->rtpinfo_url,0,sizeof(rt->rtpinfo_url));
+ av_strlcpy(rt->rtpinfo_url, value, sizeof(rt->rtpinfo_url));
+ av_log(NULL, AV_LOG_ERROR, "ZXUS/ZMSS server should ignore transports source\n");
+ }
+ }
+
else if (!strcmp(key, "seq"))
seq = strtoul(value, NULL, 10);
else if (!strcmp(key, "rtptime"))
@@ -1173,7 +1182,7 @@
/* we will use two ports per rtp stream (rtp and rtcp) */
j += 2;
if (ffurl_open(&rtsp_st->rtp_handle, buf, AVIO_FLAG_READ_WRITE) == 0){
- av_log(NULL, AV_LOG_INFO, "[%s]ffurl_open,handle=%d\n", __FUNCTION__, rtsp_st->rtp_handle);
+ av_log(NULL, AV_LOG_INFO, "[%s]ffurl_open,handle=%d\n", __FUNCTION__, rtsp_st->rtp_handle);
goto rtp_opened;
}
}
@@ -1307,6 +1316,7 @@
ff_url_join(url, sizeof(url), "rtp", NULL, host,
reply->transports[0].server_port_min, "%s", options);
}
if (!(rt->server_type == RTSP_SERVER_WMS && i > 1) &&
rtp_set_remote_url(rtsp_st->rtp_handle, url) < 0) {
err = AVERROR_INVALIDDATA;
@@ -1318,8 +1328,12 @@
*/
if (!(rt->server_type == RTSP_SERVER_WMS && i > 1) && s->iformat &&
CONFIG_RTPDEC){
- rtp_send_zte_punch_packets(rtsp_st->rtp_handle,rt->rtsp_hd);//ZTE NAT
- rtp_send_punch_packets(rtsp_st->rtp_handle);
+ if(NULL != strstr(rt->server, "ZXUS") || NULL != strstr(rt->server, "ZMSS")){
+ av_log(s, AV_LOG_ERROR, "ZXUS/ZMSS server send NAT after PLAY\n");
+ }else{
+ rtp_send_zte_punch_packets(rtsp_st->rtp_handle,rt->rtsp_hd);//ZTE NAT
+ rtp_send_punch_packets(rtsp_st->rtp_handle);
+ }
}
break;
}
Index: rtsp.h
===================================================================
--- rtsp.h (revision 151103)
+++ rtsp.h (revision 151104)
@@ -373,6 +373,7 @@
//add by wusc for UT Apk eg:playseek=20180423T115645Z
int playseekFlag;
char playseekTime[128];
+ char rtpinfo_url[1024];//for zte server
} RTSPState;
/**
Index: rtspdec.c
===================================================================
--- rtspdec.c (revision 151103)
+++ rtspdec.c (revision 151104)
@@ -553,6 +568,27 @@
/* send dummy request to keep TCP connection alive */
if ((av_gettime() - rt->last_cmd_time) / 1000000 >= (rt->timeout / keeplive_div) || rt->send_keepalive == 0) {
+ //add by wusc for zte server rtpinfo
+ if(rt->lower_transport == RTSP_LOWER_TRANSPORT_UDP && rt->rtpinfo_url[0] != 0
+ && (NULL != strstr(rt->server, "ZXUS") || NULL != strstr(rt->server, "ZMSS"))){
+ char url[1024],host[20]="";
+ int port=0;
+ av_url_split(NULL, 0, NULL, 0, host, sizeof(host), &port,NULL, 0,rt->rtpinfo_url);
+ ff_url_join(url, sizeof(url), "rtp", NULL, host,port , "%s", "");
+ av_log(NULL, AV_LOG_ERROR, "rt->rtpinfo_url %s \nset rtp url[%s] \n",rt->rtpinfo_url,url);
+ for (int i = 0; i < rt->nb_rtsp_streams; i++){
+ RTSPStream *rtsp_st = rt->rtsp_streams[i];
+ RTPDemuxContext *rtpctx = rtsp_st->transport_priv;
+ if (!rtpctx || rtsp_st->stream_index < 0)
+ continue;
+
+ int ret = rtp_set_remote_url(rtsp_st->rtp_handle, url) ;
+ av_log(NULL, AV_LOG_ERROR, "ZTE NAT rtp_set_remote_url ret = %d\n",ret);
+ rtp_send_zte_punch_packets(rtsp_st->rtp_handle,rt->rtsp_hd);//ZTE NAT
+ rtp_send_punch_packets(rtsp_st->rtp_handle);
+ }
+ }
+
if (rt->server_type == RTSP_SERVER_WMS ||
(rt->server_type != RTSP_SERVER_REAL &&
rt->get_parameter_supported)) {
Logic is simple, that is parse out the RTP-Info carries the URL after the PLAY command is successful, correctly returns PLAY command to reset the RTP handle the URL (receiving media data URL) is a new URL, call rtp_send_punch_packets new network address to send NAT penetrate the package.
Note that, in the case of audio and video separately, it is nb_rtsp_streams greater than 1, RTP-Info is also carried by the URL is greater than 1, you need to re-set separately, but this in itself does not meet the standard process flow RTSP protocol, so that modification did not consider at this situation.
5, real-time streaming playback, use absolute time to send PLAY command description
Android MediaPlayer in, RTSP play is generally used for on-demand, are relative to basically use, and for some time live streaming on the move, is to send PLAY command using absolute time, APK will carry playseek field in the URL, the client parsing the fields PLAY operation.
amend as below:
Index: rtsp.c
===================================================================
--- rtsp.c (revision 149739)
+++ rtsp.c (revision 149740)
@@ -1374,7 +1387,9 @@
socklen_t peer_len = sizeof(peer);
int support_options =1;
int failed_to_retry =0 ;
+ rt->playseekFlag = 0;
+ memset(rt->playseekTime,0x00,sizeof(rt->playseekTime));
+
if (!ff_network_init())
return AVERROR(EIO);
redirect:
@@ -1413,7 +1428,22 @@
rt->control_transport = RTSP_MODE_TUNNEL;
} else if (!strcmp(option, "filter_src")) {
rt->filter_source = 1;
- } else {
+ } else if (strstr(option, "playseek=")) {//add by wusc for UT Apk eg:playseek=20180423T115645Z
+ lower_transport_mask |= (1<< RTSP_LOWER_TRANSPORT_UDP);
+ rt->playseekFlag = 1;
+ av_log(NULL, AV_LOG_INFO, "[%s:%d]playseek option:%s", __FUNCTION__,__LINE__, option);
+ char *start = strstr(option, "playseek=") + 9;
+ int len = 0;
+ if(strchr(start, '&')){
+ len = strchr(start, '&') - start;
+ }else{
+ len = strlen(start);
+ }
+ av_strlcpy(rt->playseekTime, start, len);
+ rt->playseekTime[len] = '\0';
+ av_log(NULL, AV_LOG_INFO, "[%s:%d]len is %d,set playseek:%s", __FUNCTION__,__LINE__,len, rt->playseekTime);
+ }
+ else {
/* Write options back into the buffer, using memmove instead
* of strcpy since the strings may overlap. */
int len = strlen(option);
Index: rtsp.h
===================================================================
--- rtsp.h (revision 149739)
+++ rtsp.h (revision 149740)
@@ -368,6 +368,9 @@
int use_protocol_mode;
void *priv_data;
int send_keepalive;
+ //add by wusc for eg:playseek=20180423T115645Z
+ int playseekFlag;
+ char playseekTime[128];
} RTSPState;
/**
Index: rtspdec.c
===================================================================
--- rtspdec.c (revision 149739)
+++ rtspdec.c (revision 149740)
@@ -186,6 +186,7 @@
for (i = 0; i < rt->nb_rtsp_streams; i++) {
RTSPStream *rtsp_st = rt->rtsp_streams[i];
RTPDemuxContext *rtpctx = rtsp_st->transport_priv;
if (!rtpctx)
continue;
ff_rtp_reset_packet_queue(rtpctx);
@@ -198,7 +199,7 @@
if (rt->state == RTSP_STATE_PAUSED) {
cmd[0] = 0;
} else {
@@ -208,20 +209,24 @@
// seek, fast-forward/fast-rewind triggered seek
if (rt->playback_rate_permille != rt->playback_rate_permille_next)
rt->playback_rate_permille = rt->playback_rate_permille_next;
-
- if (rt->playback_rate_permille >= 0) {
- // forward
- snprintf(cmd, sizeof(cmd),
- "Range: npt=%"PRId64".%03"PRId64"-\r\n",
- rt->seek_timestamp / AV_TIME_BASE,
- rt->seek_timestamp / (AV_TIME_BASE / 1000) % 1000);
- } else {
- // backward
- snprintf(cmd, sizeof(cmd),
- "Range: npt=%"PRId64".%03"PRId64"-0\r\n",
- rt->seek_timestamp / AV_TIME_BASE,
- rt->seek_timestamp / (AV_TIME_BASE / 1000) % 1000);
- }
+ if(rt->playseekFlag == 1){
+ snprintf(cmd, sizeof(cmd),"Range: clock=%s.00Z-\r\n",rt->playseekTime);
+ }else{
+ if (rt->playback_rate_permille >= 0) {
+ // forward
+ snprintf(cmd, sizeof(cmd),
+ "Range: npt=%"PRId64".%03"PRId64"-\r\n",
+ rt->seek_timestamp / AV_TIME_BASE,
+ rt->seek_timestamp / (AV_TIME_BASE / 1000) % 1000);
+ } else {
+ // backward
+ snprintf(cmd, sizeof(cmd),
+ "Range: npt=%"PRId64".%03"PRId64"-0\r\n",
+ rt->seek_timestamp / AV_TIME_BASE,
+ rt->seek_timestamp / (AV_TIME_BASE / 1000) % 1000);
+ }
+ }
+
length = strlen (cmd);
snprintf(&cmd [length], sizeof(cmd) - length,
"Scale: %d.%d\r\n",
Modification is relatively simple, when resolving URL options in ff_rtsp_connect function, if carrying playseek = parameter, it is saved, when sent to PLAY stage Range: clock =% xxxxxxxxx.00Z- form of send.
Note that, if the return redirection PLAY, redirected address may not carry playseek field, if there is playseek field before the redirect, the new URL may also need to carry on.
to sum up
Several more targeted at some specific modifications to integrate the front end, in fact, actually there are some small point to note, for example, some server settings carriers need to send "MP2T / RTP" prefix, some servers can not be so quick to send NAT traversal package some server requests the URL must carry certain parameters and so on.
On the basis of the actual situation needs to be familiar with RTSP protocol, one for each case will improve step by step, this article provides some modifications reference, three articles on the entry and implementation of FFmpeg in RTSP RTSP protocol can have a preliminary understanding.