(2) -FFmpeg FFmpeg video decoding based video player with SDL

(2) -FFmpeg FFmpeg video decoding based video player with SDL

Video decoding knowledge

• Decoding h.264 format type of the code stream of pure

  Specifically speaking is compressed video data into encoded video screen pixel data, such as decoding H.264, H.264 stream is decoded into the YUV data format.

• The decoded video files packaged

  We viewed the video streams are stored in a certain most encapsulation formats (e.g. MP4, AVI, etc.). Encapsulation format typically contains contents of the audio stream and the like. For packaging video format, the video stream needs to be extracted start encapsulation format, and then decoded. For example decoding avi video file format is "avi-> H.264 streams -> YUV".

QT-based development environment to build FFmpeg

1. Get FFmpeg

   FFmpeg get address

According to their need to select the appropriate version, which by definition is a static library which Static version, Shared is a dynamic library versions, Dev Lib file is available to developers to use. Here we are Shared and download Dev.

2. In reference to FFmpeg project

The project creation process is not described here, only explain part of the introduction of FFmpeg, unzip it and just downloaded Shared Dev archive, which Shared compressed .dll files in the package only when a project runs needed, Dev Unzip the package we just needs which include and lib folders, copy directly to our project directory

and then is introduced in the .pro file

INCLUDEPATH += $$PWD/lib/ffmpeg/include

LIBS += $$PWD/lib/ffmpeg/lib/avcodec.lib\
        $$PWD/lib/ffmpeg/lib/avdevice.lib\
        $$PWD/lib/ffmpeg/lib/avfilter.lib\
        $$PWD/lib/ffmpeg/lib/avformat.lib\
        $$PWD/lib/ffmpeg/lib/avutil.lib\
        $$PWD/lib/ffmpeg/lib/postproc.lib\
        $$PWD/lib/ffmpeg/lib/swresample.lib\
        $$PWD/lib/ffmpeg/lib/swscale.lib

As we build a C ++ project, compiler used when compiling C ++, and C is the FFMPEG library, so there need to add extern "C".

extern "C" {
#include <libavcodec\avcodec.h>
#include <libavformat\avformat.h>
#include <libswscale\swscale.h>
#include <libswresample\swresample.h>
}

Decoding related functions and data structures described

function

1. ** av_register_all (): ** register all file formats and codec library

2. ** avformat_open_input (): ** Open the file or the URL, and based on the underlying byte stream input module is initialized; parsing header information of the multimedia file or multimedia stream, create and fill the structure wherein AVFormatContext key fields, each followed by original flow setup AVStream structure.

3. ** avformat_find_stream_info (): ** acquiring video file information.

4. ** avcodec_find_decoder (): ** Find decoder.

5. * Int av_seek_frame (the AVFormatContext S, int stream_index, an int64_t timestamp, the flags int):
   action: to achieve random access to the media file by changing the pointers to read and write the media file, the media player mostly from fast-forward, rewind etc. .
   parameter:

   s: AVFormatContext pointer;
   avformat_open_input returns obtained.
   stream_index: Specifies the media stream.
   timestamp: Time label.
   flags: targeting

6. ** avcodec_open2 (): ** Open the decoder.

7. ** av_read_frame (): ** a compressed data read from the input file.

8. ** avcodec_decode_video2 (): ** a decoded compressed data.

9. ** avcodec_close (): ** Close the decoder.

10. ** avformat_close_input (): ** close the input video files.

data structure

1. AVFormatContext: encapsulation format context structure, the structure is overall command, the saved video file format information package

• iformat: AVInputFormat input video
• The number of input video AVStream: nb_streams
• streams: the AVStream input video [] array
• duration: an input video duration (in microseconds)
• bit_rate: input video bit rate

2. ** AVInputFormat: ** Each package format (e.g., FLV, MKV, MP4, AVI) that corresponds to a structure

• name: the name of the encapsulation format
• long_name：封装格式的长名称
• extensions：封装格式的扩展名
• id：封装格式ID
• 一些封装格式处理的接口函数

3. **AVStream ：**视频文件中每个视频（音频）流对应一个该结构体。

• id：序号
• codec：该流对应的AVCodecContext
• time_base：该流的时基
• r_frame_rate：该流的帧率

4. **AVCodecContext ：**编码器上下文结构体，保存了视频（音频）编解码相关信息。

• codec：编解码器的AVCodec
• width, height：图像的宽高（只针对视频）
• pix_fmt：像素格式（只针对视频）
• sample_rate：采样率（只针对音频）
• channels：声道数（只针对音频）
• sample_fmt：采样格式（只针对音频）

5. **AVCodec ：**每种视频（音频）编解码器(例如H.264解码器)对应一个该结构体。

• name：编解码器名称
• long_name：编解码器长名称
• type：编解码器类型
• id：编解码器ID
• 一些编解码的接口函数

6. **AVPacket ：**存储一帧压缩编码数据。

• pts：显示时间戳
• dts ：解码时间戳
• data ：压缩编码数据
• size ：压缩编码数据大小
• stream_index ：所属的AVStream

7. **AVFrame ：**存储一帧解码后像素（采样）数据。

• data：解码后的图像像素数据（音频采样数据）。
• linesize：对视频来说是图像中一行像素的大小；对音频来说是整个音 频帧的大小。
• width, height：图像的宽高（只针对视频）。
• key_frame：是否为关键帧（只针对视频） 。
• pict_type：帧类型（只针对视频） 。例如I，P，B。

FFmpeg解码的流程

解码视频并保存50张图片的完整代码

#include <iostream>
using namespace std;

#define __STDC_CONSTANT_MACROS

extern "C"{
    #include "libavcodec/avcodec.h"
    #include "libavformat/avformat.h"
    #include "libavutil/pixfmt.h"
    #include "libswscale/swscale.h"
}
///由于我们建立的是C++的工程
///编译的时候使用的C++的编译器编译
///而FFMPEG是C的库
///因此这里需要加上extern "C"
///否则会提示各种未定义
#include <stdio.h>

///现在我们需要做的是让SaveFrame函数能把RGB信息定稿到一个PPM格式的文件中。
///我们将生成一个简单的PPM格式文件，请相信，它是可以工作的。
void SaveFrame(AVFrame *pFrame, int width, int height,int index)
{

FILE *pFile;
  char szFilename[32];
  int  y;

  // Open file
  sprintf(szFilename, "d:/frame%d.ppm", index);
  pFile=fopen(szFilename, "wb");
  cout<<szFilename;
  if(pFile==NULL)
    return;

  // Write header
  fprintf(pFile, "P6 %d %d 255", width, height);

  // Write pixel data
  for(y=0; y<height; y++)
  {
    fwrite(pFrame->data[0]+y*pFrame->linesize[0], 1, width*3, pFile);
  }
  // Close file
  fclose(pFile);

}


int main(int argc, char *argv[])
{
    char *file_path = "E:1.avi";

    AVFormatContext *pFormatCtx;
    AVCodecContext *pCodecCtx;
    AVCodec *pCodec;
    AVFrame *pFrame, *pFrameRGB;
    AVPacket *packet;
    uint8_t *out_buffer;

    static struct SwsContext *img_convert_ctx;

    int videoStream, i, numBytes;
    int ret, got_picture;

    av_register_all(); //初始化FFMPEG  调用了这个才能正常适用编码器和解码器

    //Allocate an AVFormatContext.
    pFormatCtx = avformat_alloc_context();

    if (avformat_open_input(&pFormatCtx, file_path, NULL, NULL) != 0) {
        printf("can't open the file.");
        return -1;
    }

    if (avformat_find_stream_info(pFormatCtx, NULL) < 0) {
        printf("Could't find stream infomation.");
        return -1;
    }

    videoStream = -1;

    ///循环查找视频中包含的流信息，直到找到视频类型的流
    ///便将其记录下来 保存到videoStream变量中
    ///这里我们现在只处理视频流  音频流先不管他
    for (i = 0; i < pFormatCtx->nb_streams; i++) {
        if (pFormatCtx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO) {
            videoStream = i;
        }
    }

    ///如果videoStream为-1 说明没有找到视频流
    if (videoStream == -1) {
        printf("Didn't find a video stream.");
        return -1;
    }

    ///查找解码器
    pCodecCtx = pFormatCtx->streams[videoStream]->codec;
    pCodec = avcodec_find_decoder(pCodecCtx->codec_id);

    if (pCodec == NULL) {
        printf("Codec not found.");
        return -1;
    }

    ///打开解码器
    if (avcodec_open2(pCodecCtx, pCodec, NULL) < 0) {
        printf("Could not open codec.");
        return -1;
    }

    pFrame = av_frame_alloc();
    pFrameRGB = av_frame_alloc();

    img_convert_ctx = sws_getContext(pCodecCtx->width, pCodecCtx->height,
            pCodecCtx->pix_fmt, pCodecCtx->width, pCodecCtx->height,
            AV_PIX_FMT_RGB32, SWS_BICUBIC, NULL, NULL, NULL);

    numBytes = avpicture_get_size(AV_PIX_FMT_RGB32, pCodecCtx->width,pCodecCtx->height);

    out_buffer = (uint8_t *) av_malloc(numBytes * sizeof(uint8_t));
    avpicture_fill((AVPicture *) pFrameRGB, out_buffer, AV_PIX_FMT_RGB32,
            pCodecCtx->width, pCodecCtx->height);

    int y_size = pCodecCtx->width * pCodecCtx->height;

    packet = (AVPacket *) malloc(sizeof(AVPacket)); //分配一个packet
    av_new_packet(packet, y_size); //分配packet的数据

    av_dump_format(pFormatCtx, 0, file_path, 0); //输出视频信息

    int index = 0;

    
    //循环
    while (1)
    {
        if (av_read_frame(pFormatCtx, packet) < 0)
        {
            break; //这里认为视频读取完了
        }

        if (packet->stream_index == videoStream) {
            //这里就是将读取的每帧数据进行解码
            ret = avcodec_decode_video2(pCodecCtx, pFrame, &got_picture,packet);

            if (ret < 0) {
                printf("decode error.");
                return -1;
            }

            if (got_picture) {
                //通过该函数将解码的YUV格式文件转化为RGB格式文件
                sws_scale(img_convert_ctx,
                        (uint8_t const * const *) pFrame->data,
                        pFrame->linesize, 0, pCodecCtx->height, pFrameRGB->data,
                        pFrameRGB->linesize);

                SaveFrame(pFrameRGB, pCodecCtx->width,pCodecCtx->height,index++); //保存图片
                if (index > 50) return 0; //这里我们就保存50张图片
            }
        }
        av_free_packet(packet);
    }
    av_free(out_buffer);
    av_free(pFrameRGB);
    avcodec_close(pCodecCtx);
    avformat_close_input(&pFormatCtx);

    return 0;
}