Problem Description
Recently, colleagues in the same group encountered a problem: when FFmpeg splices ts files to generate mp4 (demux -> mux, no codec), it can be played on the Android platform, but it cannot be played on mac (QuickTime Player) and iOS. Among them, the FFmpeg version is 3.3, the audio track contained in the ts stream is lc aac, and the video track is hevc, as shown below:
// 视频流
Stream #0:0[0x101]: Video: hevc (Main) ([36][0][0][0] / 0x0024), yuv420p(tv), 720x1280, 250 tbr, 90k tbn, 90k tbc
// 音频流
Stream #0:1[0x102]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 100 kb/sThe basic information of the spliced mp4 is as follows. When playing it with QuickTime Player on a mac, it will prompt that the format is not compatible, but it can be played with QQ Video and VLC. In addition, it can also be played through MediaPlayer and self-developed players on Android.
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'leon.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2mp41
encoder : Lavf57.71.100
Duration: 00:02:04.42, start: 99.935011, bitrate: 173 kb/s
Stream #0:0(und): Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv), 720x1280, 159 kb/s, SAR 1:1 DAR 9:16, 1.93 fps, 250 tbr, 90k tbn, 90k tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 12 kb/s (default)
Metadata:
handler_name : SoundHandlerPreliminary solution to the problem
Where is the problem? I found the clue Encode H265 to hvc1 codec on the world's largest same-sex dating website . The general idea is: hev1 or hvc1 are two codec tags, indicating different packaging methods of the hevc stream in the mp4 container. Quicktime Player and iOS no longer support hev1 tagged mp4. Hev1 or hvc1 are just two different packaging methods of hevc in mp4. The encoding format itself is hevc, so you can switch between these two tags without re-encoding.
So try to convert the codec tag of mp4 by the following command:
ffmpeg -i leon.mp4 -c:v copy -tag:v hvc1 -c:a copy leon-hvc1.mp4It was found that FFmpeg3.3 and previous versions made a direct error:
<mux.c init_muxer 376> Tag hvc1 incompatible with output codec id '174' ([35][0][0][0])In 3.4 and later versions, it can run normally.
leon.mp4 is in hev1 tag mode, which -tag:v hvc1can be converted to hvc1 tag mode, and then played with Quicktime Player.
Similarly, in FFmpeg3.4 and later versions, you can use the following commands to convert ts into mp4, and control the -tag:vcodec tag mode of mp4.
// 把ts（aac和hevc）转格式为MP4，默认情况下leon.mp4的codec tag是hev1，Quicktime Player无法播放
ffmpeg -i leon.ts -c:v copy -c:a copy leon.mp4
// 把ts（aac和hevc）转格式为MP4，强制指定hvc1，所以leon-hvc1.mp4的codec tag是hvc1，Quicktime Player可以播放
ffmpeg -i leon.ts -c:v copy -tag:v hvc1 -c:a copy leon-hvc1.mp4
Then why can't FFmpeg3.3 and previous versions be used -tag:v hvc1? After testing, it is found -tag:v hev1that it can't work, and it is also a direct error:
<mux.c init_muxer 376> Tag hev1 incompatible with output codec id '174' ([35][0][0][0])That is to say, FFmpeg3.3 and previous versions do not support -tag:v hvc1and -tag:v hev1, but FFmpeg3.4 and later versions are possible.
root of the problem
The difference between FFmpeg versions can only be solved in the source code. Fortunately, the file name, function name and line number have been specified in the error message: libavformat/mux.cfile, init_muxer function, line 376.
The call stack is: avformat_write_header -> avformat_init_output -> init_muxer, which is triggered when the header information of the wrapper is written.
The core code segment is as follows:
// par是hevc流的编码参数：AVStream->codecpar
// st是hevc流：AVStream
// s是AVFormatContext
// of表示封装格式，这里是mp4：AVFormatContext->oformat;
if (par->codec_tag) { // 如果AVStream->codecpar->codec_tag不为0
// 检查AVStream->codecpar->codec_tag是否在封装容器支持的codec_tag列表中，即：AVOutputFormat->codec_tag列表
if (!validate_codec_tag(s, st)) {
// 从AVOutputFormat->codec_tag列表中找出编码ID对应的codec_tag
const uint32_t otag = av_codec_get_tag(s->oformat->codec_tag, par->codec_id);
// 现在设置的codec_tag校验失败，与codec_id对应的codec_tag是otag
av_log(s, AV_LOG_ERROR, "Tag %s incompatible with output codec id '%d' (%s)\n", av_fourcc2str(par->codec_tag), par->codec_id, av_fourcc2str(otag));
ret = AVERROR_INVALIDDATA;
goto fail;
}
} else {
// 若AVStream->codecpar->codec_tag为0（默认情况下为0），则FFmpeg会根据编码ID（AVCodecID）从封装格式的codec_tag列表中，找到一个codec_tag。
par->codec_tag = av_codec_get_tag(of->codec_tag, par->codec_id);
}avformat_write_headerWhen writing the header information of the encapsulation container, the codec_tag will be checked: if AVStream->codecpar->codec_tag has a value, it will be checked whether AVStream->codecpar->codec_tag is in the codec_tag list supported by the encapsulation format, if not
If AVStream->codecpar->codec_tag is 0, it will find a matching codec_tag from the codec_tag list in the encapsulation format according to AVCodecID.
So far, the logic of FFmpeg3.3 and later versions is consistent. The reason for the above hvc1 tag difference may be that the codec_tag list supported by mp4 in different FFmpeg versions is different.
Let's look at the error message of FFmpeg3.3:
// 表示当前为hevc流设置的hvc1 codec_tag不受支持，174是hevc的AVCodecID，表示与hevc对应的codec_tag应该是10进制35（16进制0x23）
<mux.c init_muxer 376> Tag hvc1 incompatible with output codec id '174' ([35][0][0][0])OK, the cause of the error is clear: In FFmpeg3.3, the hvc1 tag we set for hevc is not supported by the mp4 encapsulation format. Let's look at the list of codec_tags supported by mp4.
libavformat/movenc.cIt is the Muxer class of mp4 and mov, where AVOutputFormat->codec_tag specifies the codec_tag list supported by a certain package format, and the codec_tag list supported by mp4 is in the libavformat/isom.cff_mp4_obj_type structure. The code is too long, so post a few of our commonly used ones:
{ AV_CODEC_ID_H264 , 0x21 },
{ AV_CODEC_ID_HEVC , 0x23 },
{ AV_CODEC_ID_AAC , 0x40 },The codec_tag corresponding to hevc is 0x23, which is 35 in decimal, which is exactly the same as the above error message. In other words, the only supported codec_tag of the hevc encoding format in mp4 is 0x23, which is hev1, so we will fail if we explicitly specify hvc1.
Let's look at the latest trunk version of FFmpeg, the list of codec_tags supported by mp4, and only a few commonly used ones:
{ AV_CODEC_ID_H264 , MKTAG('a', 'v', 'c', '1') },
{ AV_CODEC_ID_H264 , MKTAG('a', 'v', 'c', '3') },
{ AV_CODEC_ID_HEVC , MKTAG('h', 'e', 'v', '1') },
{ AV_CODEC_ID_HEVC , MKTAG('h', 'v', 'c', '1') },
{ AV_CODEC_ID_AAC , MKTAG('m', 'p', '4', 'a') },The codec_tag corresponding to hevc is hev1 and hvc1, and because hev1 is in front of the list, hev1 is used by default, that is to say: if we do not actively specify -tag:v, then the codec_tag corresponding to hevc is hev1, which cannot be played with Quicktime Player play, so pass the
tag:v hvc1It is mandatory to specify codec_tag to be compatible with all platforms.
Some people may have doubts, since FFmpeg3.3 supports hev1 tag, why tag:v hev1is the specification wrong? That's because the way FFmpeg3.3 and the latest version represent codec_tag has changed (the same is hev1 tag, FFmpeg3.3 is ox23, and the latest version is MKTAG('h', 'e', 'v', '1')), Although they are all hev1, but the value has changed, the two versions are not compatible, if specified -tag:v 35, there will be no error in FFmpeg3.3.
So far, all doubts have been resolved, in summary:
- In FFmpeg3.3 and earlier versions, the mp4 container only supports the hev1 tag of hevc, and the value is 0x23.
- For FFmpeg3.4 and later versions, the mp4 container supports both the hev1 and hvc1 tags of hevc, and uses the hev1 tag by default.
So if you want to use the hvc1 tag of hevc in FFmpeg3.3 and earlier versions, what should you do? You can use the mov container instead. After analyzing the code, it is found that the mov package format supports both the hev1 and hvc1 tags of hevc in different versions of FFmpeg, and the hev1 tag is used by default.
The above analysis is based on the FFmpeg command line, so how to set the codec_tag in the code is relatively simple. Before avformat_write_header, it is enough to set the codec_tag for the hevc stream:
AVStream->codecpar->codec_tag = MKTAG('h', 'v', 'c', '1');What is the difference between the two different tags in Mp4? Different tags correspond to different Box types.
hev1 tag：stsd -> hev1 -> hvcC
hvc1 tag：stsd -> hvc1 -> hvcC
Original FFmpeg hevc codec_tag compatibility issue
★The business card at the end of the article can receive audio and video development learning materials for free, including (FFmpeg, webRTC, rtmp, hls, rtsp, ffplay, srs) and audio and video learning roadmaps, etc.
see below!