Use FFmpeg to read video stream and save

I recently came into contact with FFmpeg, and need to implement a program that reads and displays the code stream of the rtsp protocol. While moving the code online, I also wrote some understanding of tools such as FFmpeg and Qt.

Prepare

First define the macro, its function is to avoid the error of 'UINT64_C' was not declared in this  scope .

#ifndef INT64_C 
#define INT64_C(c) (c ## LL) 
#define UINT64_C(c) (c ## ULL) 
#endif

Add FFmpeg and C++ header files

extern "C" {
	/*Include ffmpeg header file*/
#include <libavformat/avformat.h> 
#include <libavcodec/avcodec.h> 
#include <libswscale/swscale.h> 

#include <libavutil/imgutils.h>  
#include <libavutil/opt.h>     
#include <libavutil/mathematics.h>   
#include <libavutil/samplefmt.h>
}

#include <iostream>
using namespace std;

main function

First, define the input and output AVFormatContext structure, which stores audio and video data, which is an abstraction and encapsulation of audio and video files. Note that FFmpeg developers can only use pointers. Then define the input and output file names, and the input is the address of the rtsp protocol. Here I use my own Haikang camera address. The output is saved as a flv file. The avformat_network_init function, as the name implies, initializes the network.

int main(void)
{
	AVFormatContext* ifmt_ctx = NULL, * ofmt_ctx = NULL;
	const char* in_filename, * out_filename;
	in_filename = "rtsp://admin:WY@[email protected]/h264/ch1/main/av_stream";
	out_filename = "output.flv";
	
    avformat_network_init();

Set a configuration dictionary, in FFmpeg we use the AVDictionary structure configuration.

	AVDictionary* avdic = NULL;
	char option_key[] = "rtsp_transport";
	char option_value[] = "tcp";
	av_dict_set(&avdic, option_key, option_value, 0);
	char option_key2[] = "max_delay";
	char option_value2[] = "5000000";
	av_dict_set(&avdic, option_key2, option_value2, 0);

The next step is to open the input stream. AV needs to be used. Let's take a look at its declaration:、

int avformat_open_input(AVFormatContext **ps, const char *url, ff_const59 AVInputFormat *fmt, AVDictionary **options);

The function has four parameters, first a pointer to the AVFormatContext pointer, followed by a pointer to the url. AVInputFormat is the encapsulation format for the specified input. Generally, NULL is passed, and FFmpeg detects it by itself. AVDictionary **options other parameter settings, it is a dictionary for parameter passing.

Before opening the input video stream, we define a few more parameters:

	AVPacket pkt;
	AVOutputFormat* ofmt = NULL;
	int video_index = -1;
	int frame_index = 0;
	
    int i;

The AVPacket class saves the data before decoding after demultiplexing. As for what is demultiplexing, we all know that signals are time-division multiplexed, frequency-division multiplexed, etc. Video and audio are often multiplexed in audio and video signals, and they have to be separated independently at the receiving end, that is, Source-> Demux->Stream changes. See: AVPacket Analysis

Next we open the input stream:

	//打开输入流
	int ret;
	if ((ret = avformat_open_input(&ifmt_ctx, in_filename, 0, &avdic)) < 0)
	{
		cout<<"Could not open input file."<<endl;
		goto end;
	}
	if ((ret = avformat_find_stream_info(ifmt_ctx, 0)) < 0)
	{
		cout<<"Failed to retrieve input stream information"<<endl;
		goto end;
	}

The end here represents the end of the program, and some finishing work such as closing the input stream is done in the end. If avformat_open_input returns a negative value, output an error and end. The role of avformat_open_input is to open the input stream and read the file header, but not to open the decoder. And avformat_find_stream_info reads the package in the media file to get the streaming information.

Continue to look at the code:

	//nb_streams代表有几路流

	for (i = 0; i < ifmt_ctx->nb_streams; i++) 
	{
		if (ifmt_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO)
		{
			//视频流
			video_index = i;
			cout << "get videostream." << endl;
			break;
		}
	}
	av_dump_format(ifmt_ctx, 0, in_filename, 0);

We use a for loop. If the codec_type in the codecpar parameter of the i-th stream is AVMEDIA_TYPE_VIDEO, we know that the video stream has been obtained. Then we assign the i of the video stream to video_index, and print the "get videostream." information on the console. The av_dump_format function prints information about some other streams. As shown below:

Next, we open the output stream:

    avformat_alloc_output_context2(&ofmt_ctx, NULL, NULL, out_filename);

	if (!ofmt_ctx)
	{
		printf("Could not create output context\n");
		ret = AVERROR_UNKNOWN;
		goto end;
	}

The function of this function is to find the appropriate AVFormatContext management structure according to the file name. Next write the file header to the output file:

    //写文件头到输出文件
    ret = avformat_write_header(ofmt_ctx, NULL);
    if (ret < 0)
    {
    	printf("Error occured when opening output URL\n");
    	goto end;
    }

The next step is to save the data to the video file. We use a while loop:

	while (1)
	{
		AVStream* in_stream, * out_stream;
		//从输入流获取一个数据包
		ret = av_read_frame(ifmt_ctx, &pkt);//读一帧并放到pkt中去
		if (ret < 0)
			break;//读取失败

		in_stream = ifmt_ctx->streams[pkt.stream_index];
		out_stream = ofmt_ctx->streams[pkt.stream_index];
		//copy packet
		//转换 PTS/DTS 时序
		pkt.pts = av_rescale_q_rnd(pkt.pts, in_stream->time_base, out_stream->time_base, (enum AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
		pkt.dts = av_rescale_q_rnd(pkt.dts, in_stream->time_base, out_stream->time_base, (enum AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
		//printf("pts %d dts %d base %d\n",pkt.pts,pkt.dts, in_stream->time_base);
		pkt.duration = av_rescale_q(pkt.duration, in_stream->time_base, out_stream->time_base);
		pkt.pos = -1;

		//此while循环中并非所有packet都是视频帧，当收到视频帧时记录一下，仅此而已
		if (pkt.stream_index == video_index)
		{
			printf("Receive %8d video frames from input URL\n", frame_index);
			frame_index++;
		}

		//将包数据写入到文件。
		ret = av_interleaved_write_frame(ofmt_ctx, &pkt);
		if (ret < 0)
		{
			if (ret == -22) {
				continue;
			}
			else {
				printf("Error muxing packet.error code %d\n", ret);
				break;
			}
		}

The following steps are required to play network video files: protocol solution, decapsulation, video and audio decoding, and video and audio synchronization. Among them, the output after receiving the protocol is the AVStream class. We first read a frame of data with av_read_frame. Next, we use the pointer in_stream to point to the data of a certain stream in the packet, and out_stream to point to the data of another stream. After obtaining the stream pointer, perform pts/dts conversion. pts and dts are video playback and decoding timestamps respectively. The following article describes in more detail:

The concept of pts, dts - Author: SamirChen

If a video frame is received, it is printed to the console:

		if (pkt.stream_index == video_index)
		{
			printf("Receive %8d video frames from input URL\n", frame_index);
			frame_index++;
		}

Finally, call av_interleaved_write_frame to write data. Compared with av_write_frame, av_interleaved_write_frame function provides the functions of packet caching and pts inspection.

		//将包数据写入到文件。
		ret = av_interleaved_write_frame(ofmt_ctx, &pkt);
		if (ret < 0)
		{
			if (ret == -22) {
				continue;
			}
			else {
				printf("Error muxing packet.error code %d\n", ret);
				break;
			}
		}
				av_packet_unref(&pkt);
	}

Finally, we write the end of the file and follow up on previous errors:

	//写文件尾
	av_write_trailer(ofmt_ctx);

end:
	av_dict_free(&avdic);
	avformat_close_input(&ifmt_ctx);
	//Close input
	if (ofmt_ctx && !(ofmt->flags & AVFMT_NOFILE))
		avio_close(ofmt_ctx->pb);
	avformat_free_context(ofmt_ctx);
	if (ret < 0 && ret != AVERROR_EOF)
	{
		cout<<"Error occured."<<endl;
		return -1;
	}

	return 0;
}

When running with visual studio, you need to turn off SDL checking, otherwise an error will be reported due to version reasons

Summarize

It is the first time to use FFmpeg for multimedia development. Although the code is ported, the process is still very interesting. I will continue to learn this piece when I have time.
Original link:
FFmpeg grabs stream from rtsp
Complete project code:

#ifndef INT64_C 
#define INT64_C(c) (c ## LL) 
#define UINT64_C(c) (c ## ULL) 
#endif 

extern "C" {
	/*Include ffmpeg header file*/
#include <libavformat/avformat.h> 
#include <libavcodec/avcodec.h> 
#include <libswscale/swscale.h> 

#include <libavutil/imgutils.h>  
#include <libavutil/opt.h>     
#include <libavutil/mathematics.h>   
#include <libavutil/samplefmt.h>
}

#include <iostream>
using namespace std;

int main(void)
{
	AVFormatContext* ifmt_ctx = NULL, * ofmt_ctx = NULL;
	const char* in_filename, * out_filename;
	in_filename = "rtsp://admin:WY@[email protected]/h264/ch1/main/av_stream";
	out_filename = "output.flv";

	avformat_network_init();

	AVDictionary* avdic = NULL;
	char option_key[] = "rtsp_transport";
	char option_value[] = "tcp";
	av_dict_set(&avdic, option_key, option_value, 0);
	char option_key2[] = "max_delay";
	char option_value2[] = "5000000";
	av_dict_set(&avdic, option_key2, option_value2, 0);

	AVPacket pkt;
	AVOutputFormat* ofmt = NULL;
	int video_index = -1;
	int frame_index = 0;

	int i;

	//打开输入流
	int ret;
	if ((ret = avformat_open_input(&ifmt_ctx, in_filename, 0, &avdic)) < 0)
	{
		cout<<"Could not open input file."<<endl;
		goto end;
	}
	if ((ret = avformat_find_stream_info(ifmt_ctx, 0)) < 0)
	{
		cout<<"Failed to retrieve input stream information"<<endl;
		goto end;
	}


	//nb_streams代表有几路流

	for (i = 0; i < ifmt_ctx->nb_streams; i++) 
	{
		if (ifmt_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO)
		{
			//视频流
			video_index = i;
			cout << "get videostream." << endl;
			break;
		}
	}

	av_dump_format(ifmt_ctx, 0, in_filename, 0);

	//打开输出流
	avformat_alloc_output_context2(&ofmt_ctx, NULL, NULL, out_filename);

	if (!ofmt_ctx)
	{
		printf("Could not create output context\n");
		ret = AVERROR_UNKNOWN;
		goto end;
	}
	ofmt = ofmt_ctx->oformat;

	for (i = 0; i < ifmt_ctx->nb_streams; i++)
	{
		AVStream* in_stream = ifmt_ctx->streams[i];
		AVStream* out_stream = avformat_new_stream(ofmt_ctx, in_stream->codec->codec);

		if (!out_stream)
		{
			printf("Failed allocating output stream.\n");
			ret = AVERROR_UNKNOWN;
			goto end;
		}

		//将输出流的编码信息复制到输入流
		ret = avcodec_copy_context(out_stream->codec, in_stream->codec);
		if (ret < 0)
		{
			printf("Failed to copy context from input to output stream codec context\n");
			goto end;
		}

		out_stream->codec->codec_tag = 0;

		if (ofmt_ctx->oformat->flags & AVFMT_GLOBALHEADER)
			out_stream->codec->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;

	}

	av_dump_format(ofmt_ctx, 0, out_filename, 1);

	if (!(ofmt->flags & AVFMT_NOFILE))
	{
		ret = avio_open(&ofmt_ctx->pb, out_filename, AVIO_FLAG_WRITE);
		if (ret < 0)
		{
			printf("Could not open output URL '%s'", out_filename);
			goto end;
		}
	}

	//写文件头到输出文件
	ret = avformat_write_header(ofmt_ctx, NULL);
	if (ret < 0)
	{
		printf("Error occured when opening output URL\n");
		goto end;
	}


	//while循环中持续获取数据包，不管音频视频都存入文件
	while (1)
	{
		AVStream* in_stream, * out_stream;
		//从输入流获取一个数据包
		ret = av_read_frame(ifmt_ctx, &pkt);
		if (ret < 0)
			break;

		in_stream = ifmt_ctx->streams[pkt.stream_index];
		out_stream = ofmt_ctx->streams[pkt.stream_index];
		//copy packet
		//转换 PTS/DTS 时序
		pkt.pts = av_rescale_q_rnd(pkt.pts, in_stream->time_base, out_stream->time_base, (enum AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
		pkt.dts = av_rescale_q_rnd(pkt.dts, in_stream->time_base, out_stream->time_base, (enum AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
		//printf("pts %d dts %d base %d\n",pkt.pts,pkt.dts, in_stream->time_base);
		pkt.duration = av_rescale_q(pkt.duration, in_stream->time_base, out_stream->time_base);
		pkt.pos = -1;

		//此while循环中并非所有packet都是视频帧，当收到视频帧时记录一下
		if (pkt.stream_index == video_index)
		{
			printf("Receive %8d video frames from input URL\n", frame_index);
			frame_index++;
		}

		//将包数据写入到文件。
		ret = av_interleaved_write_frame(ofmt_ctx, &pkt);
		if (ret < 0)
		{
			if (ret == -22) {
				continue;
			}
			else {
				printf("Error muxing packet.error code %d\n", ret);
				break;
			}

		}

		//av_free_packet(&pkt); //此句在新版本中已deprecated 由av_packet_unref代替
		av_packet_unref(&pkt);
	}


	//写文件尾
	av_write_trailer(ofmt_ctx);

end:
	av_dict_free(&avdic);
	avformat_close_input(&ifmt_ctx);
	//Close input
	if (ofmt_ctx && !(ofmt->flags & AVFMT_NOFILE))
		avio_close(ofmt_ctx->pb);
	avformat_free_context(ofmt_ctx);
	if (ret < 0 && ret != AVERROR_EOF)
	{
		cout<<"Error occured."<<endl;
		return -1;
	}

	return 0;
}

The original text  uses FFmpeg to read the video stream and save it_ffmpeg saves the video stream_Leopard Fakemi's Blog-CSDN Blog

★The business card at the end of the article can receive audio and video development learning materials for free, including (FFmpeg, webRTC, rtmp, hls, rtsp, ffplay, srs) and audio and video learning roadmaps, etc.

see below!