h.264判断一帧的结束

最近上了一个项目，有些地方要抠一些细节，达到效果最优，其中有一处就是要从es流中整合出完整的帧。搜了一些资料，然后参照live555的代码做一下总结，留着以后备用。

PS: 从事流媒体开发时间不长，有哪些地方理解有误，还希望能帮我指正出来，共同进步^_^

首先讲一下es流的结构，看完标准发现h.264没有帧的概念，是以片（slice也有翻译成条带的）为单元的。

如下图：

参考文章:http://zhongcong386.blog.163.com/blog/static/1347278042012112762445528/

NAL_header(1Byte)+(slice_header)+(slice_data)+......+NAL_header(1Byte)+(slice_header)+(slice_data)

slice_data= rbsp_byte=宏块（实实在在的视频数据），当然上面nal_header之后的rbsp数据格式不同类型的NAL也不同。

比如：nal_unit_type ==5 的idr slice 格式是slice_header( )+slice_data( )+rbsp_slice_trailing_bits( )。具体的各个nal slice的格式参看标准的7.3.2.*

（nal_unit_type <= 5 && nal_unit_type > 0）的nal单元都负载的VCL数据（视频流），每个vcl类型的nal单元存储的是一个片（slice）。

接下来就是分理出完整一帧数据。假设我们已经分离出了连续的nal_unit_type =1~5的NAL。

第一种情况没有下一个nal则表示当前的nal是当前帧的最后一部分数据。

第二种情况 nal_unit_type ！=next_nal_unit_type 则表示上一帧已经结束。

第三种情况 nal_unit_type == next_nal_unit_type的时候有可能是同一帧的数据，也有可能是不同帧的数据。（如：两个PP帧在一块就有可能出现这种情况）

出现第三种情况的时候我们需要分析slice_header里面的frame_num这一个值。如果frame_num != next_frame_num 则表示当前nal已经是当前帧的最后一部分数据。

slice_header()的句法如下： PS 太多啦，有需求直接去看标准；

slice_header( ) { 
 first_mb_in_slice 2 ue(v)
 slice_type 2 ue(v)
 pic_parameter_set_id 2 ue(v)
 frame_num 2 u(v)
 if( !frame_mbs_only_flag ) {
  field_pic_flag 2 u(1)
 if( field_pic_flag )
 bottom_field_flag 2 u(1)
 }
 if( nal_unit_type = = 5 )
 idr_pic_id 2 ue(v)
 if( pic_order_cnt_type = = 0 ) {
 pic_order_cnt_lsb 2 u(v)
 if( pic_order_present_flag && !field_pic_flag )
 delta_pic_order_cnt_bottom 2 se(v)
 }
 ..................................
 ..................................
 if( num_slice_groups_minus1 > 0 && slice_group_map_type >= 3 && slice_group_map_type <= 5)
 slice_group_change_cycle 2 u(v)
}

具体live555分析的代码我摘录下来，贴在下面
fHaveSeenFirstByteOfNALUnit =true表示当前帧结束
unsigned H264VideoStreamParser::parse() {
..........................
..........................
   // If this NAL unit is a VCL NAL unit, we also scan the start of the next NAL unit, to determine whether this NAL unit
    // ends the current 'access unit'.  We need this information to figure out when to increment "fPresentationTime".
    // (RTP streamers also need to know this in order to figure out whether or not to set the "M" bit.)
    Boolean thisNALUnitEndsAccessUnit = False; // until we learn otherwise 
    if (haveSeenEOF()) {//最后一个nal肯定是帧的最后一部分了
      thisNALUnitEndsAccessUnit = True;
    } else {
      Boolean const isVCL = nal_unit_type <= 5 && nal_unit_type > 0; //Would need to include type 20 for SVC and MVC #####
      if (isVCL) {//
		  u_int8_t firstByteOfNextNALUnit;
		  testBytes(&firstByteOfNextNALUnit, 1);
		  u_int8_t next_nal_ref_idc = (firstByteOfNextNALUnit&0x60)>>5;
		  u_int8_t next_nal_unit_type = firstByteOfNextNALUnit&0x1F;
	if (next_nal_unit_type >= 6) {//nal_unit_type ！=next_nal_unit_type的情况
 	// The next NAL unit is not a VCL; therefore, we assume that this NAL unit ends the current 'access unit':
	  thisNALUnitEndsAccessUnit = True;
	} else {
	  // The next NAL unit is also a VCL.  We need to examine it a little to figure out if it's a different 'access unit'.
	  // (We use many of the criteria described in section 7.4.1.2.4 of the H.264 specification.)
	  Boolean IdrPicFlag = nal_unit_type == 5;
	  Boolean next_IdrPicFlag = next_nal_unit_type == 5;
	  if (next_IdrPicFlag != IdrPicFlag) {
	    // IdrPicFlag differs in value
	    thisNALUnitEndsAccessUnit = True;
	  } else if (next_nal_ref_idc != nal_ref_idc && next_nal_ref_idc*nal_ref_idc == 0) {
	    //nal_ref_idc differs in value with one of the nal_ref_idc values being equal to 0
	    thisNALUnitEndsAccessUnit = True;
	  } else if ((nal_unit_type == 1 || nal_unit_type == 2 || nal_unit_type == 5)
		     && (next_nal_unit_type == 1 || next_nal_unit_type == 2 || next_nal_unit_type == 5)) {//这是第三种情况主要的功能在
//analyze_slice_header中
	    // Both this and the next NAL units begin with a "slice_header".
	    // Parse this (for each), to get parameters that we can compare:
	    // Current NAL unit's "slice_header":
	    unsigned frame_num, pic_parameter_set_id, idr_pic_id;
	    Boolean field_pic_flag, bottom_field_flag;
		analyze_slice_header(fStartOfFrame + fOutputStartCodeSize, fTo, nal_unit_type,
			frame_num, pic_parameter_set_id, idr_pic_id, field_pic_flag, bottom_field_flag);  
	    // Next NAL unit's "slice_header":
	    u_int8_t next_slice_header[NUM_NEXT_SLICE_HEADER_BYTES_TO_ANALYZE];
	    testBytes(next_slice_header, sizeof next_slice_header);
	    unsigned next_frame_num, next_pic_parameter_set_id, next_idr_pic_id;
	    Boolean next_field_pic_flag, next_bottom_field_flag;
		analyze_slice_header(next_slice_header, &next_slice_header[sizeof next_slice_header], next_nal_unit_type,
			next_frame_num, next_pic_parameter_set_id, next_idr_pic_id, next_field_pic_flag, next_bottom_field_flag);
	    if (next_frame_num != frame_num) {
	      // frame_num differs in value
	      thisNALUnitEndsAccessUnit = True;
	    } else if (next_pic_parameter_set_id != pic_parameter_set_id) {
	      // pic_parameter_set_id differs in value
	      thisNALUnitEndsAccessUnit = True;
	    } else if (next_field_pic_flag != field_pic_flag) {
	      // field_pic_flag differs in value
	      thisNALUnitEndsAccessUnit = True;
	    } else if (next_bottom_field_flag != bottom_field_flag) {
	      // bottom_field_flag differs in value
	      thisNALUnitEndsAccessUnit = True;
	    } else if (next_IdrPicFlag == 1 && next_idr_pic_id != idr_pic_id) {
	      // IdrPicFlag is equal to 1 for both and idr_pic_id differs in value
	      // Note: We already know that IdrPicFlag is the same for both.
	      thisNALUnitEndsAccessUnit = True;
	    }
	  }
..........................
}

先就这样，下次再补充。

这一篇博客有更好的方法和大家一块分享http://www.360doc.com/content/13/0913/15/13084517_314201133.shtml

h.264判断一帧的结束

猜你喜欢