本文包含以下内容
1、x264的码率控制原理与对应源码解析
基本概念
对于此前了解过H.264的码率控制原理的读者（如果不了解，也可以去看我的文章H.264码率控制算法研究及JM相应代码分析），首先需要说明的是x264采用的码率控制算法并没有采用拉格朗日代价函数来控制编码，而是使用一种更简单的方法，即利用半精度帧的SATD作为量化等级参数选择的依据。SATD即将残差经哈德曼变换的4×4块的预测残差绝对值总和，可以将其看作简单的时频变换，其值在一定程度上可以反映生成码流的大小。
X264显式支持的一趟码率控制方法有：ABR, CQP, CRF. 缺省方法是CRF。
详见x264_param_default()方法中
param->rc.i_rc_method = X264_RC_CRF;
param->rc.i_bitrate = 0;
param->rc.f_rate_tolerance = 1.0;
param->rc.i_vbv_max_bitrate = 0;
param->rc.i_vbv_buffer_size = 0;
param->rc.f_vbv_buffer_init = 0.9;
param->rc.i_qp_constant = 23 + QP_BD_OFFSET;
param->rc.f_rf_constant = 23;对于bitrate和QP，一旦设置他们就表示要按照相应的码率控制方法进行编码，CRF有缺省值23，没有任何关于编码控制的设置时就按照CRF缺省值23来编码。
详见x264_param_parse()方法中
OPT("bitrate")
{
p->rc.i_bitrate = atoi(value);
p->rc.i_rc_method = X264_RC_ABR;
}
OPT2("qp", "qp_constant")
{
p->rc.i_qp_constant = atoi(value);
p->rc.i_rc_method = X264_RC_CQP;
}
OPT("crf")
{
p->rc.f_rf_constant = atof(value);
p->rc.i_rc_method = X264_RC_CRF;
} 源码分析
x264中和码率控制相关的内容都在ratecontrol.h ratecontrol.c这两个文件中，基本的调用流程如下，图中最左边的几个函数代表码率控制的内容是在哪里被调用的，对这几个函数不清楚的读者可以参见我的上一篇文章
本文福利， 免费领取C++音视频学习资料包、技术视频，内容包括（音视频开发，面试题，FFmpeg ，webRTC ，rtmp ，hls ，rtsp ，ffplay ，srs）↓↓↓↓↓↓见下面↓↓文章底部点击免费领取↓↓
首先需要知道，在整个码率控制流程中的关键结构体就是x264_ratecontrol_t了，其定义如下
struct x264_ratecontrol_t
{
/* constants */
int b_abr;
/* 2pass 是变码率(VBR)压缩的一项关键技术，意思是通过两次分析来压缩一个文件。
* 第一次分析影片各部分的动作快慢及分配相应的码率并作以相应记录，
* 第二次再根据第一次的记录进行压缩，这样就能让码率得到最佳分配，
* 从而在指定文件大小或规定码流内最大限度的提高视频质量。*/
int b_2pass;
int b_vbv; //视频缓冲检验器
int b_vbv_min_rate;
double fps;
double bitrate;
double rate_tolerance;
double qcompress;
int nmb; /* number of macroblocks in a frame */
int qp_constant[3];
/* current frame */
ratecontrol_entry_t *rce; //当前帧码率控制的相关内容
int qp; /* qp for current frame */
float qpm; /* qp for current macroblock: precise float for AQ */
float qpa_rc; /* average of macroblocks' qp before aq */
float qpa_rc_prev;
int qpa_aq; /* average of macroblocks' qp after aq */
int qpa_aq_prev;
float qp_novbv; /* QP for the current frame if 1-pass VBV was disabled. */
/* VBV stuff */
double buffer_size;
int64_t buffer_fill_final; /* 上一个完成的帧实际使用的缓冲区大小 */
int64_t buffer_fill_final_min;
double buffer_fill; /* planned buffer, if all in-progress frames hit their bit budget */
/* 计划的缓冲区大小, 就是处理的所有帧都算上可能出现的最大缓冲区使用量 */
double buffer_rate; /* # of bits added to buffer_fill after each frame */
/* 每帧过后要加在buffer_fill上的比特数 */
double vbv_max_rate; /* # of bits added to buffer_fill per second */
predictor_t *pred; /* predict frame size from satd */
int single_frame_vbv;
float rate_factor_max_increment; /* Don't allow RF above (CRF + this value). */
/* ABR stuff */
int last_satd;
double last_rceq;//last rc estimated qscale
double cplxr_sum; /* sum of bits*qscale/rceq */
double expected_bits_sum; /* sum of qscale2bits after rceq, ratefactor, and overflow, only includes finished frames */
int64_t filler_bits_sum; /* sum in bits of finished frames' filler data */
double wanted_bits_window; /* target bitrate * window */
double cbr_decay;//还不清楚是做什么的
double short_term_cplxsum;
double short_term_cplxcount;
double rate_factor_constant;
double ip_offset;
double pb_offset;
/* 2pass stuff */
FILE *p_stat_file_out;
char *psz_stat_file_tmpname;
FILE *p_mbtree_stat_file_out;
char *psz_mbtree_stat_file_tmpname;
char *psz_mbtree_stat_file_name;
FILE *p_mbtree_stat_file_in;
int num_entries; /* number of ratecontrol_entry_ts */
ratecontrol_entry_t *entry; /* FIXME: copy needed data and free this once init is done */
double last_qscale;
double last_qscale_for[3]; /* last qscale for a specific pict type, used for max_diff & ipb factor stuff */
int last_non_b_pict_type;
double accum_p_qp; /* for determining I-frame quant;accumulation of p frame qp*/
/* 用于决定I帧的量化系数 */
double accum_p_norm;
double last_accum_p_norm;
double lmin[3]; /* min qscale by frame type */
double lmax[3];
double lstep; /* max change (multiply) in qscale per frame */
struct
{
uint16_t *qp_buffer[2]; /* Global buffers for converting MB-tree quantizer data. */
int qpbuf_pos; /* In order to handle pyramid reordering, QP buffer acts as a stack.
* This value is the current position (0 or 1). */
int src_mb_count;
/* For rescaling */
int rescale_enabled;
float *scale_buffer[2]; /* Intermediate buffers */
int filtersize[2]; /* filter size (H/V) */
float *coeffs[2];
int *pos[2];
int srcdim[2]; /* Source dimensions (W/H) */
} mbtree;
/* MBRC stuff */
float frame_size_estimated; /* Access to this variable must be atomic: double is
* not atomic on all arches we care about */
double frame_size_maximum; /* Maximum frame size due to MinCR */
double frame_size_planned;
double slice_size_planned;
predictor_t *row_pred;//用satd和qscale预测framesize的一组系数
predictor_t row_preds[3][2];//3代表IPB帧，但是2代表什么还不清楚
predictor_t *pred_b_from_p; /* predict B-frame size from P-frame satd */
int bframes; /* # consecutive B-frames before this P-frame */
int bframe_bits; /* total cost of those frames */
int i_zones;
x264_zone_t *zones;
x264_zone_t *prev_zone;
/* hrd stuff */
int initial_cpb_removal_delay;
int initial_cpb_removal_delay_offset;
double nrt_first_access_unit; /* nominal removal time */
double previous_cpb_final_arrival_time;
uint64_t hrd_multiply_denom;
};根据我的学习经验，看完这个结构体之后肯定会好奇里面一些奇怪的变量是干什么的，这时候就有必要先大致了解一下x264码率控制的理论方法了。这方面建议大家搜索一些论文来学习，给出一个我认为讲的比较好的。
步骤中有很多经验公式，这方面应该也有一些论述文章，以后有时间再做补充。
下面就依照前面的流程图和理论步骤来分析代码。依照顺序分析各个关键函数，各个函数的作用，代码详细分析的内容都写在注释中。希望三者结合能有助于大家理解x264的码率控制。如果有任何问题或者错误，欢迎大家指出，代码分析中还有一些我自己不理解的地方，也希望了解的朋友多多指点。
x264_ratecontrol_new
//call by x264_encoder_open
//用于各种赋值和初始化，管理与2pass码率控制相关的输入输出文件
int x264_ratecontrol_new( x264_t *h )
{
x264_ratecontrol_t *rc;
x264_emms();
CHECKED_MALLOCZERO( h->rc, h->param.i_threads * sizeof(x264_ratecontrol_t) );
rc = h->rc;
//判断码率控制方法
rc->b_abr = h->param.rc.i_rc_method != X264_RC_CQP && !h->param.rc.b_stat_read;
rc->b_2pass = h->param.rc.i_rc_method == X264_RC_ABR && h->param.rc.b_stat_read;
/* FIXME: use integers */
if( h->param.i_fps_num > 0 && h->param.i_fps_den > 0 )
rc->fps = (float) h->param.i_fps_num / h->param.i_fps_den;
else
rc->fps = 25.0; //default fps
// macroblock tree：决定该MB使用何种大小的qp值进行量化。对每个MB处理，
// 向前预测一定数量的帧，记录该MB被参考的情况，qp的大小与被参考次数成反比。
// 与mb_tree相关的参数：rc-lookahead 决定mb_tree向前预测的帧数
if( h->param.rc.b_mb_tree )
{
h->param.rc.f_pb_factor = 1;
// Qcomp (Quantizer Curve Compression) : 控制整个视频的码率波动大小，
// 它会影响mb_tree的强度 (Higher qcomp = weaker mbtree)
// 0% 时相当于CBR模式，100%时相当于CQP模式
rc->qcompress = 1;
}
else
rc->qcompress = h->param.rc.f_qcompress; // 0.0 => cbr, 1.0 => constant qp
//码率单位一般是1000
rc->bitrate = h->param.rc.i_bitrate * (h->param.i_avcintra_class ? 1024. : 1000.);
rc->rate_tolerance = h->param.rc.f_rate_tolerance;
rc->nmb = h->mb.i_mb_count; //number of mbs in a frame
rc->last_non_b_pict_type = -1;
rc->cbr_decay = 1.0;
if( h->param.rc.i_rc_method == X264_RC_CRF && h->param.rc.b_stat_read )
{
x264_log( h, X264_LOG_ERROR, "constant rate-factor is incompatible with 2pass.\n" );
return -1;
}
x264_ratecontrol_init_reconfigurable( h, 1 );
if( h->param.i_nal_hrd )
{
uint64_t denom = (uint64_t)h->sps->vui.hrd.i_bit_rate_unscaled * h->sps->vui.i_time_scale;
uint64_t num = 90000;
x264_reduce_fraction64( &num, &denom );
rc->hrd_multiply_denom = 90000 / num;
//计算需要的比特数
double bits_required = log2( 90000 / rc->hrd_multiply_denom )
+ log2( h->sps->vui.i_time_scale )
+ log2( h->sps->vui.hrd.i_cpb_size_unscaled );
if( bits_required >= 63 )
{
x264_log( h, X264_LOG_ERROR, "HRD with very large timescale and bufsize not supported\n" );
return -1;
}
}
if( rc->rate_tolerance < 0.01 )
{
x264_log( h, X264_LOG_WARNING, "bitrate tolerance too small, using .01\n" );
rc->rate_tolerance = 0.01;
}
/* whether qp is allowed to vary per macroblock */
h->mb.b_variable_qp = rc->b_vbv || h->param.rc.i_aq_mode;
if( rc->b_abr )
{
/* FIXME ABR_INIT_QP is actually used only in CRF */
#define ABR_INIT_QP (( h->param.rc.i_rc_method == X264_RC_CRF ? h->param.rc.f_rf_constant : 24 ) + QP_BD_OFFSET)
rc->accum_p_norm = .01;
rc->accum_p_qp = ABR_INIT_QP * rc->accum_p_norm;
/* estimated ratio that produces a reasonable QP for the first I-frame */
// 估计初始复杂度 = 0.01 * (700000 ^ qcomp) * (宏块总数目^ 0.5)
rc->cplxr_sum = .01 * pow( 7.0e5, rc->qcompress ) * pow( h->mb.i_mb_count, 0.5 );
//每个帧的比特数
rc->wanted_bits_window = 1.0 * rc->bitrate / rc->fps;
rc->last_non_b_pict_type = SLICE_TYPE_I;
}
rc->ip_offset = 6.0 * log2f( h->param.rc.f_ip_factor );
rc->pb_offset = 6.0 * log2f( h->param.rc.f_pb_factor );
//get qp for p frame at first, then use offset to calculate values for i framne and b frame
rc->qp_constant[SLICE_TYPE_P] = h->param.rc.i_qp_constant;
rc->qp_constant[SLICE_TYPE_I] = x264_clip3( h->param.rc.i_qp_constant - rc->ip_offset + 0.5, 0, QP_MAX );
rc->qp_constant[SLICE_TYPE_B] = x264_clip3( h->param.rc.i_qp_constant + rc->pb_offset + 0.5, 0, QP_MAX );
h->mb.ip_offset = rc->ip_offset + 0.5;
//每帧qscale的最大改变倍数=qp_step/6
rc->lstep = pow( 2, h->param.rc.i_qp_step / 6.0 );
rc->last_qscale = qp2qscale( 26 );
int num_preds = h->param.b_sliced_threads * h->param.i_threads + 1;
//预测误差都用satd来计算
CHECKED_MALLOC( rc->pred, 5 * sizeof(predictor_t) * num_preds );
CHECKED_MALLOC( rc->pred_b_from_p, sizeof(predictor_t) );
for( int i = 0; i < 3; i++ )
{//对应P B I帧
rc->last_qscale_for[i] = qp2qscale( ABR_INIT_QP );
rc->lmin[i] = qp2qscale( h->param.rc.i_qp_min );
rc->lmax[i] = qp2qscale( h->param.rc.i_qp_max );
for( int j = 0; j < num_preds; j++ )
{
rc->pred[i+j*5].coeff_min = 2.0 / 4;
rc->pred[i+j*5].coeff = 2.0;
rc->pred[i+j*5].count = 1.0;
rc->pred[i+j*5].decay = 0.5;
rc->pred[i+j*5].offset = 0.0;
}
for( int j = 0; j < 2; j++ )
{
rc->row_preds[i][j].coeff_min = .25 / 4;
rc->row_preds[i][j].coeff = .25;
rc->row_preds[i][j].count = 1.0;
rc->row_preds[i][j].decay = 0.5;
rc->row_preds[i][j].offset = 0.0;
}
}
*rc->pred_b_from_p = rc->pred[0];
// 用于手动分配低或高码率给视频的某个特定部分
if( parse_zones( h ) < 0 )
{
x264_log( h, X264_LOG_ERROR, "failed to parse zones\n" );
return -1;
}……2pass相关的内容省略不看本文福利， 免费领取C++音视频学习资料包、技术视频，内容包括（音视频开发，面试题，FFmpeg ，webRTC ，rtmp ，hls ，rtsp ，ffplay ，srs）↓↓↓↓↓↓见下面↓↓文章底部点击免费领取↓↓
其中调用的x264_ratecontrol_init_reconfigurable主要是对vbv做一些初始化的设置。
x264_ratecontrol_start
/* Before encoding a frame, choose a QP for it */
//called by x264_encode_encode
//计算一帧的QP值，帧层码率控制,到这一步，一帧中所有宏块还是统一qp的
void x264_ratecontrol_start( x264_t *h, int i_force_qp, int overhead )
{
x264_ratecontrol_t *rc = h->rc;
ratecontrol_entry_t *rce = NULL; //used for 2pass
x264_zone_t *zone = get_zone( h, h->fenc->i_frame );
float q;
x264_emms();
if( zone && (!rc->prev_zone || zone->param != rc->prev_zone->param) )
x264_encoder_reconfig_apply( h, zone->param );
rc->prev_zone = zone;
if( h->param.rc.b_stat_read )
{
int frame = h->fenc->i_frame;
assert( frame >= 0 && frame < rc->num_entries );
rce = h->rc->rce = &h->rc->entry[frame];
if( h->sh.i_type == SLICE_TYPE_B
&& h->param.analyse.i_direct_mv_pred == X264_DIRECT_PRED_AUTO )
{
h->sh.b_direct_spatial_mv_pred = ( rce->direct_mode == 's' );
h->mb.b_direct_auto_read = ( rce->direct_mode == 's' || rce->direct_mode == 't' );
}
}
if( rc->b_vbv )
{
// 初始化重构帧的的几个参数，主要用于自适应B帧的确定
memset( h->fdec->i_row_bits, 0, h->mb.i_mb_height * sizeof(int) );
memset( h->fdec->f_row_qp, 0, h->mb.i_mb_height * sizeof(float) );
memset( h->fdec->f_row_qscale, 0, h->mb.i_mb_height * sizeof(float) );
rc->row_pred = rc->row_preds[h->sh.i_type];
rc->buffer_rate = h->fenc->i_cpb_duration * rc->vbv_max_rate * h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale;
// 根据当前所有帧预计的大小临时更新VBV，也即更新h->rc->buffer_fill，就是更新缓冲区
update_vbv_plan( h, overhead );
//h264级别
const x264_level_t *l = x264_levels;
while( l->level_idc != 0 && l->level_idc != h->param.i_level_idc )
l++;
// 获取该h264级别的最小压缩比，例如：级别3.1，mincr值为4
int mincr = l->mincr;
// 蓝光碟压缩，最小压缩比为4
if( h->param.b_bluray_compat )
mincr = 4;
/* Profiles above High don't require minCR, so just set the maximum to a large value. */
// profile高于high的档次不需要mincr，因此将帧的最大值设为一个非常大的值
// profile的各种取值定义在文件common/set.h中
if( h->sps->i_profile_idc > PROFILE_HIGH )
rc->frame_size_maximum = 1e9;
else
{
/* The spec has a bizarre special case for the first frame. */
if( h->i_frame == 0 )
{
//384 * ( Max( PicSizeInMbs, fR * MaxMBPS ) + MaxMBPS * ( tr( 0 ) - tr,n( 0 ) ) ) / MinCR
double fr = 1. / 172;
int pic_size_in_mbs = h->mb.i_mb_width * h->mb.i_mb_height;
// Maximum frame size due to MinCR，计算每帧最大尺寸
rc->frame_size_maximum = 384 * BIT_DEPTH * X264_MAX( pic_size_in_mbs, fr*l->mbps ) / mincr;
}
else
{
//384 * MaxMBPS * ( tr( n ) - tr( n - 1 ) ) / MinCR
rc->frame_size_maximum = 384 * BIT_DEPTH * ((double)h->fenc->i_cpb_duration * h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale) * l->mbps / mincr;
}
}
}//if(rc->vbv)
if( h->sh.i_type != SLICE_TYPE_B )
rc->bframes = h->fenc->i_bframes;
// 根据不同码率控制方式计算qp值
if( rc->b_abr )
{
// update qscale for 1 frame based on actual bits used so far
q = qscale2qp( rate_estimate_qscale( h ) );
}
else if( rc->b_2pass )
{
rce->new_qscale = rate_estimate_qscale( h );
q = qscale2qp( rce->new_qscale );
}
else /* CQP */
{
if( h->sh.i_type == SLICE_TYPE_B && h->fdec->b_kept_as_ref )
q = ( rc->qp_constant[ SLICE_TYPE_B ] + rc->qp_constant[ SLICE_TYPE_P ] ) / 2;
else
q = rc->qp_constant[ h->sh.i_type ];
if( zone )
{
if( zone->b_force_qp )
q += zone->i_qp - rc->qp_constant[SLICE_TYPE_P];
else
q -= 6*log2f( zone->f_bitrate_factor );
}
}
if( i_force_qp != X264_QP_AUTO )
q = i_force_qp - 1;
q = x264_clip3f( q, h->param.rc.i_qp_min, h->param.rc.i_qp_max );
// 设置当前帧的qp，以及aq之前和之后的宏块平均qp
rc->qpa_rc = rc->qpa_rc_prev =
rc->qpa_aq = rc->qpa_aq_prev = 0;
rc->qp = x264_clip3( q + 0.5f, 0, QP_MAX );
h->fdec->f_qp_avg_rc =
h->fdec->f_qp_avg_aq =
rc->qpm = q;// 当前宏块的qp值
if( rce )
rce->new_qp = rc->qp;
//更新h->rc->accum_p_qp的值
accum_p_qp_update( h, rc->qpm );
// 记录最后一个非B帧的类型
if( h->sh.i_type != SLICE_TYPE_B )
rc->last_non_b_pict_type = h->sh.i_type;
}rate_estimate_qscale
// update qscale for 1 frame based on actual bits used so far
//得到一帧的QP和satd值
/*码率控制部分的核心之一，另一个是get_scale
* 0、计算SATD和图像的模糊复杂度
* 1、在get_scale中，按复杂度采用指数模型得到qscale
* rcc->last_qscale = pow( rce->blurred_complexity, 1 - rcc->qcompress )/rate_factor
* 2、在get_scale中根据复杂度和目标比特数调整qp
* q /= rate_factor; //rcc->wanted_bits_window / rcc->cplxr_sum
* 3、根据已编码帧的实际比特数和目标比特数的偏差再次调整qp
* overflow = x264_clip3f( 1.0 + (total_bits - wanted_bits) / abr_buffer, .5, 2 );
q *= overflow;
*/
static float rate_estimate_qscale( x264_t *h )
{
float q;
x264_ratecontrol_t *rcc = h->rc;
ratecontrol_entry_t rce = {0};
int pict_type = h->sh.i_type;
//zhanghui:will there be total_bits if not 2pass，no
int64_t total_bits = 8*(h->stat.i_frame_size[SLICE_TYPE_I]
+ h->stat.i_frame_size[SLICE_TYPE_P]
+ h->stat.i_frame_size[SLICE_TYPE_B])
- rcc->filler_bits_sum;
if( rcc->b_2pass )
{
rce = *rcc->rce;
if( pict_type != rce.pict_type )
{
x264_log( h, X264_LOG_ERROR, "slice=%c but 2pass stats say %c\n",
slice_type_to_char[pict_type], slice_type_to_char[rce.pict_type] );
}
}
if( pict_type == SLICE_TYPE_B )
{
/* B-frames don't have independent ratecontrol, but rather get the
* average QP of the two adjacent P-frames + an offset */
//B帧的QP由两个相邻的参考帧的QP值计算而来，求平均
int i0 = IS_X264_TYPE_I(h->fref_nearest[0]->i_type);
int i1 = IS_X264_TYPE_I(h->fref_nearest[1]->i_type);
int dt0 = abs(h->fenc->i_poc - h->fref_nearest[0]->i_poc);
int dt1 = abs(h->fenc->i_poc - h->fref_nearest[1]->i_poc);
float q0 = h->fref_nearest[0]->f_qp_avg_rc;
float q1 = h->fref_nearest[1]->f_qp_avg_rc;
if( h->fref_nearest[0]->i_type == X264_TYPE_BREF )
q0 -= rcc->pb_offset/2;
if( h->fref_nearest[1]->i_type == X264_TYPE_BREF )
q1 -= rcc->pb_offset/2;
if( i0 && i1 )
q = (q0 + q1) / 2 + rcc->ip_offset;
else if( i0 ) //取不是I帧的那个
q = q1;
else if( i1 )
q = q0;
else //如果都不是I帧
q = (q0*dt1 + q1*dt0) / (dt0 + dt1);
if( h->fenc->b_kept_as_ref )
q += rcc->pb_offset/2;
else
q += rcc->pb_offset;
//估算frame_size
if( rcc->b_2pass && rcc->b_vbv )
rcc->frame_size_planned = qscale2bits( &rce, qp2qscale( q ) );
else
rcc->frame_size_planned = predict_size( rcc->pred_b_from_p, qp2qscale( q ), h->fref[1][h->i_ref[1]-1]->i_satd );
/* Limit planned size by MinCR */
if( rcc->b_vbv )
rcc->frame_size_planned = X264_MIN( rcc->frame_size_planned, rcc->frame_size_maximum );
h->rc->frame_size_estimated = rcc->frame_size_planned;
/* For row SATDs */
if( rcc->b_vbv )
//从slice decide那里得到计算好的satd值
rcc->last_satd = x264_rc_analyse_slice( h );
rcc->qp_novbv = q;
return qp2qscale( q );
}//if( pict_type == SLICE_TYPE_B )
else
{
//缓冲区初始值
double abr_buffer = 2 * rcc->rate_tolerance * rcc->bitrate;
if( rcc->b_2pass )
{
double lmin = rcc->lmin[pict_type];
double lmax = rcc->lmax[pict_type];
int64_t diff;
int64_t predicted_bits = total_bits;
if( rcc->b_vbv )
{
if( h->i_thread_frames > 1 )
{
int j = h->rc - h->thread[0]->rc;
for( int i = 1; i < h->i_thread_frames; i++ )
{
x264_t *t = h->thread[ (j+i)%h->i_thread_frames ];
double bits = t->rc->frame_size_planned;
if( !t->b_thread_active )
continue;
bits = X264_MAX(bits, t->rc->frame_size_estimated);
predicted_bits += (int64_t)bits;
}
}
}
else
{
if( h->i_frame < h->i_thread_frames )
predicted_bits += (int64_t)h->i_frame * rcc->bitrate / rcc->fps;
else
predicted_bits += (int64_t)(h->i_thread_frames - 1) * rcc->bitrate / rcc->fps;
}
/* Adjust ABR buffer based on distance to the end of the video. */
if( rcc->num_entries > h->i_frame )
{
double final_bits = rcc->entry[rcc->num_entries-1].expected_bits;
double video_pos = rce.expected_bits / final_bits;
double scale_factor = sqrt( (1 - video_pos) * rcc->num_entries );
abr_buffer *= 0.5 * X264_MAX( scale_factor, 0.5 );
}
diff = predicted_bits - (int64_t)rce.expected_bits;
q = rce.new_qscale;
q /= x264_clip3f((double)(abr_buffer - diff) / abr_buffer, .5, 2);
if( ((h->i_frame + 1 - h->i_thread_frames) >= rcc->fps) &&
(rcc->expected_bits_sum > 0))
{
/* Adjust quant based on the difference between
* achieved and expected bitrate so far */
double cur_time = (double)h->i_frame / rcc->num_entries;
double w = x264_clip3f( cur_time*100, 0.0, 1.0 );
q *= pow( (double)total_bits / rcc->expected_bits_sum, w );
}
rcc->qp_novbv = qscale2qp( q );
if( rcc->b_vbv )
{
/* Do not overflow vbv */
double expected_size = qscale2bits( &rce, q );
double expected_vbv = rcc->buffer_fill + rcc->buffer_rate - expected_size;
double expected_fullness = rce.expected_vbv / rcc->buffer_size;
double qmax = q*(2 - expected_fullness);
double size_constraint = 1 + expected_fullness;
qmax = X264_MAX( qmax, rce.new_qscale );
if( expected_fullness < .05 )
qmax = lmax;
qmax = X264_MIN(qmax, lmax);
while( ((expected_vbv < rce.expected_vbv/size_constraint) && (q < qmax)) ||
((expected_vbv < 0) && (q < lmax)))
{
q *= 1.05;
expected_size = qscale2bits(&rce, q);
expected_vbv = rcc->buffer_fill + rcc->buffer_rate - expected_size;
}
rcc->last_satd = x264_rc_analyse_slice( h );
}
q = x264_clip3f( q, lmin, lmax );
}//if( rcc->b_2pass )
else /* 1pass ABR */
{
/* Calculate the quantizer which would have produced the desired
* average bitrate if it had been applied to all frames so far.
* Then modulate that quant based on the current frame's complexity
* relative to the average complexity so far (using the 2pass RCEQ).
* Then bias the quant up or down if total size so far was far from
* the target.
* 1、计算出一个Qp值，如果将该值应用到当前所有的帧，则可以获得目标平均吗率
* 2、根据当前帧的复杂度和平均复杂度的差距，调整QP
* 3、如果total size和目标相差太多，再调整QP
* Result: Depending on the value of rate_tolerance, there is a
* tradeoff between quality and bitrate precision. But at large
* tolerances, the bit distribution approaches that of 2pass. */
double wanted_bits, overflow = 1;
//首先计算当前帧的SATD
rcc->last_satd = x264_rc_analyse_slice( h );
//计算cplxsum (累计复杂度) 和 cplxcount（加权累计帧数）
rcc->short_term_cplxsum *= 0.5;
rcc->short_term_cplxcount *= 0.5;
rcc->short_term_cplxsum += rcc->last_satd / (CLIP_DURATION(h->fenc->f_duration) / BASE_FRAME_DURATION);
rcc->short_term_cplxcount ++;
rce.tex_bits = rcc->last_satd;
//计算图像的模糊复杂度
rce.blurred_complexity = rcc->short_term_cplxsum / rcc->short_term_cplxcount;
rce.mv_bits = 0;
rce.p_count = rcc->nmb;
rce.i_count = 0;
rce.s_count = 0;
rce.qscale = 1;
rce.pict_type = pict_type;
rce.i_duration = h->fenc->i_duration;
if( h->param.rc.i_rc_method == X264_RC_CRF )
{
q = get_qscale( h, &rce, rcc->rate_factor_constant, h->fenc->i_frame );
}
else
{
q = get_qscale( h, &rce, rcc->wanted_bits_window / rcc->cplxr_sum, h->fenc->i_frame );
/* ABR code can potentially be counterproductive in CBR, so just don't bother.
* Don't run it if the frame complexity is zero either. */
if( !rcc->b_vbv_min_rate && rcc->last_satd )
{
// FIXME is it simpler to keep track of wanted_bits in ratecontrol_end?
int i_frame_done = h->i_frame + 1 - h->i_thread_frames;
double time_done = i_frame_done / rcc->fps;
if( h->param.b_vfr_input && i_frame_done > 0 )
time_done = ((double)(h->fenc->i_reordered_pts - h->i_reordered_pts_delay)) * h->param.i_timebase_num / h->param.i_timebase_den;
wanted_bits = time_done * rcc->bitrate;
if( wanted_bits > 0 )
{
//缓存区增长函数
abr_buffer *= X264_MAX( 1, sqrt( time_done ) );
overflow = x264_clip3f( 1.0 + (total_bits - wanted_bits) / abr_buffer, .5, 2 );
q *= overflow;
}
}
}
//如果不是第一个I帧
if( pict_type == SLICE_TYPE_I && h->param.i_keyint_max > 1
/* should test _next_ pict type, but that isn't decided yet */
&& rcc->last_non_b_pict_type != SLICE_TYPE_I )
{
q = qp2qscale( rcc->accum_p_qp / rcc->accum_p_norm );
q /= fabs( h->param.rc.f_ip_factor );
}
//否则如果不是第一帧
else if( h->i_frame > 0 )
{
if( h->param.rc.i_rc_method != X264_RC_CRF )
{
/* Asymmetric clipping, because symmetric would prevent
* overflow control in areas of rapidly oscillating complexity */
double lmin = rcc->last_qscale_for[pict_type] / rcc->lstep;
double lmax = rcc->last_qscale_for[pict_type] * rcc->lstep;
if( overflow > 1.1 && h->i_frame > 3 )
lmax *= rcc->lstep;
else if( overflow < 0.9 )
lmin /= rcc->lstep;
q = x264_clip3f(q, lmin, lmax);
}
}
else if( h->param.rc.i_rc_method == X264_RC_CRF && rcc->qcompress != 1 )
{
q = qp2qscale( ABR_INIT_QP ) / fabs( h->param.rc.f_ip_factor );
}
rcc->qp_novbv = qscale2qp( q );
//FIXME use get_diff_limited_q() ?
q = clip_qscale( h, pict_type, q );
}
rcc->last_qscale_for[pict_type] =
rcc->last_qscale = q;
//根据最后的qscale预测frame size
if( !(rcc->b_2pass && !rcc->b_vbv) && h->fenc->i_frame == 0 )
rcc->last_qscale_for[SLICE_TYPE_P] = q * fabs( h->param.rc.f_ip_factor );
if( rcc->b_2pass && rcc->b_vbv )
rcc->frame_size_planned = qscale2bits(&rce, q);
else
rcc->frame_size_planned = predict_size( &rcc->pred[h->sh.i_type], q, rcc->last_satd );
/* Always use up the whole VBV in this case. */
if( rcc->single_frame_vbv )
rcc->frame_size_planned = rcc->buffer_rate;
/* Limit planned size by MinCR */
if( rcc->b_vbv )
rcc->frame_size_planned = X264_MIN( rcc->frame_size_planned, rcc->frame_size_maximum );
h->rc->frame_size_estimated = rcc->frame_size_planned;
return q;
}
}get_qscale
/**
* modify the bitrate curve from pass1 for one frame
* 码率控制部分的核心之一，另一个是rate_estimate_qscale
* 1、按复杂度采用指数模型得到qscale
* rcc->last_qscale = pow( rce->blurred_complexity, 1 - rcc->qcompress )/rate_factor
* 2、根据复杂度和目标比特数调整qp
* q /= rate_factor; //rcc->wanted_bits_window / rcc->cplxr_sum
* 3、在rate_estimate_qscale中，根据已编码帧的实际比特数和目标比特数的偏差再次调整qp
*/
static double get_qscale(x264_t *h, ratecontrol_entry_t *rce, double rate_factor, int frame_num)
{
x264_ratecontrol_t *rcc= h->rc;
x264_zone_t *zone = get_zone( h, frame_num );
double q;
if( h->param.rc.b_mb_tree )
{
double timescale = (double)h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale;
q = pow( BASE_FRAME_DURATION / CLIP_DURATION(rce->i_duration * timescale), 1 - h->param.rc.f_qcompress );
}
else
q = pow( rce->blurred_complexity, 1 - rcc->qcompress );
// avoid NaN's in the rc_eq
if( !isfinite(q) || rce->tex_bits + rce->mv_bits == 0 )
q = rcc->last_qscale_for[rce->pict_type];
else
{
rcc->last_rceq = q;
q /= rate_factor; //rcc->wanted_bits_window / rcc->cplxr_sum
rcc->last_qscale = q;
}
if( zone )
{
if( zone->b_force_qp )
q = qp2qscale( zone->i_qp );
else
q /= zone->f_bitrate_factor;
}
return q;
}clip_qscale
// apply VBV constraints and clip qscale to between lmin and lmax
//按一阶模型根据satd估计分配的bits，结合vbv限制，修正qscale
static double clip_qscale( x264_t *h, int pict_type, double q )
{
x264_ratecontrol_t *rcc = h->rc;
double lmin = rcc->lmin[pict_type];
double lmax = rcc->lmax[pict_type];
if( rcc->rate_factor_max_increment )
lmax = X264_MIN( lmax, qp2qscale( rcc->qp_novbv + rcc->rate_factor_max_increment ) );
double q0 = q;
/* B-frames are not directly subject to VBV,
* since they are controlled by the P-frames' QPs. */
if( rcc->b_vbv && rcc->last_satd > 0 )
{
double fenc_cpb_duration = (double)h->fenc->i_cpb_duration *
h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale;
/* Lookahead VBV: raise the quantizer as necessary such that no frames in
* the lookahead overflow and such that the buffer is in a reasonable state
* by the end of the lookahead. */
if( h->param.rc.i_lookahead )
{
int terminate = 0;
/* Avoid an infinite loop. */
for( int iterations = 0; iterations < 1000 && terminate != 3; iterations++ )
{
double frame_q[3];
double cur_bits = predict_size( &rcc->pred[h->sh.i_type], q, rcc->last_satd );
double buffer_fill_cur = rcc->buffer_fill - cur_bits;
double target_fill;
double total_duration = 0;
double last_duration = fenc_cpb_duration;
frame_q[0] = h->sh.i_type == SLICE_TYPE_I ? q * h->param.rc.f_ip_factor : q;
frame_q[1] = frame_q[0] * h->param.rc.f_pb_factor;
frame_q[2] = frame_q[0] / h->param.rc.f_ip_factor;
/* Loop over the planned future frames. */
for( int j = 0; buffer_fill_cur >= 0 && buffer_fill_cur <= rcc->buffer_size; j++ )
{//如果未来几帧都按照这个qscale设置来编码的话，全都编码完会用掉多少buffer
total_duration += last_duration;
buffer_fill_cur += rcc->vbv_max_rate * last_duration;
int i_type = h->fenc->i_planned_type[j];
int i_satd = h->fenc->i_planned_satd[j];
if( i_type == X264_TYPE_AUTO )
break;
i_type = IS_X264_TYPE_I( i_type ) ? SLICE_TYPE_I : IS_X264_TYPE_B( i_type ) ? SLICE_TYPE_B : SLICE_TYPE_P;
cur_bits = predict_size( &rcc->pred[i_type], frame_q[i_type], i_satd );
buffer_fill_cur -= cur_bits;
last_duration = h->fenc->f_planned_cpb_duration[j];
}
/* Try to get to get the buffer at least 50% filled, but don't set an impossible goal. */
target_fill = X264_MIN( rcc->buffer_fill + total_duration * rcc->vbv_max_rate * 0.5, rcc->buffer_size * 0.5 );
if( buffer_fill_cur < target_fill )
{//zhanghui:如果达不到预期，就加大qscale？？为什么，这样难道不会更小吗？
q *= 1.01;
terminate |= 1;
continue;
}
/* Try to get the buffer no more than 80% filled, but don't set an impossible goal. */
target_fill = x264_clip3f( rcc->buffer_fill - total_duration * rcc->vbv_max_rate * 0.5, rcc->buffer_size * 0.8, rcc->buffer_size );
if( rcc->b_vbv_min_rate && buffer_fill_cur > target_fill )
{//zhanghui:如果超过预期，就减小qscale？？为什么？这样难道不会更大吗？
q /= 1.01;
terminate |= 2;
continue;
}
break;
}
}
/* Fallback to old purely-reactive algorithm: no lookahead. */
//if( h->param.rc.i_lookahead )
else
{
if( ( pict_type == SLICE_TYPE_P ||
( pict_type == SLICE_TYPE_I && rcc->last_non_b_pict_type == SLICE_TYPE_I ) ) &&
rcc->buffer_fill/rcc->buffer_size < 0.5 )
{
q /= x264_clip3f( 2.0*rcc->buffer_fill/rcc->buffer_size, 0.5, 1.0 );
}
/* Now a hard threshold to make sure the frame fits in VBV.
* This one is mostly for I-frames. */
double bits = predict_size( &rcc->pred[h->sh.i_type], q, rcc->last_satd );
/* For small VBVs, allow the frame to use up the entire VBV. */
double max_fill_factor = h->param.rc.i_vbv_buffer_size >= 5*h->param.rc.i_vbv_max_bitrate / rcc->fps ? 2 : 1;
/* For single-frame VBVs, request that the frame use up the entire VBV. */
double min_fill_factor = rcc->single_frame_vbv ? 1 : 2;
if( bits > rcc->buffer_fill/max_fill_factor )
{
double qf = x264_clip3f( rcc->buffer_fill/(max_fill_factor*bits), 0.2, 1.0 );
q /= qf;
bits *= qf;
}
if( bits < rcc->buffer_rate/min_fill_factor )
{
double qf = x264_clip3f( bits*min_fill_factor/rcc->buffer_rate, 0.001, 1.0 );
q *= qf;
}
q = X264_MAX( q0, q );
}
/* Check B-frame complexity, and use up any bits that would
* overflow before the next P-frame. */
if( h->sh.i_type == SLICE_TYPE_P && !rcc->single_frame_vbv )
{
int nb = rcc->bframes;
double bits = predict_size( &rcc->pred[h->sh.i_type], q, rcc->last_satd );
double pbbits = bits;
double bbits = predict_size( rcc->pred_b_from_p, q * h->param.rc.f_pb_factor, rcc->last_satd );
double space;
double bframe_cpb_duration = 0;
double minigop_cpb_duration;
for( int i = 0; i < nb; i++ )
bframe_cpb_duration += h->fenc->f_planned_cpb_duration[i];
if( bbits * nb > bframe_cpb_duration * rcc->vbv_max_rate )
nb = 0;
pbbits += nb * bbits;
minigop_cpb_duration = bframe_cpb_duration + fenc_cpb_duration;
space = rcc->buffer_fill + minigop_cpb_duration*rcc->vbv_max_rate - rcc->buffer_size;
if( pbbits < space )
{
q *= X264_MAX( pbbits / space, bits / (0.5 * rcc->buffer_size) );
}
q = X264_MAX( q0/2, q );
}
/* Apply MinCR and buffer fill restrictions */
double bits = predict_size( &rcc->pred[h->sh.i_type], q, rcc->last_satd );
double frame_size_maximum = X264_MIN( rcc->frame_size_maximum, X264_MAX( rcc->buffer_fill, 0.001 ) );
if( bits > frame_size_maximum )
q *= bits / frame_size_maximum;
if( !rcc->b_vbv_min_rate )
q = X264_MAX( q0, q );
}
if( lmin==lmax )
return lmin;
else if( rcc->b_2pass )
{
double min2 = log( lmin );
double max2 = log( lmax );
q = (log(q) - min2)/(max2-min2) - 0.5;
q = 1.0/(1.0 + exp( -4*q ));
q = q*(max2-min2) + min2;
return exp( q );
}
else
return x264_clip3f( q, lmin, lmax );
}本文福利， 免费领取C++音视频学习资料包、技术视频，内容包括（音视频开发，面试题，FFmpeg ，webRTC ，rtmp ，hls ，rtsp ，ffplay ，srs）↓↓↓↓↓↓见下面↓↓文章底部点击免费领取↓↓
accum_p_qp_update
//更新h->rc->accum_p_qp的值
static void accum_p_qp_update( x264_t *h, float qp )
{
x264_ratecontrol_t *rc = h->rc;
rc->accum_p_qp *= .95;
rc->accum_p_norm *= .95;
rc->accum_p_norm += 1;
if( h->sh.i_type == SLICE_TYPE_I )
rc->accum_p_qp += qp + rc->ip_offset;
else
rc->accum_p_qp += qp;
}x264_ratecontrol_qp
//确定QP在规定范围内
int x264_ratecontrol_qp( x264_t *h )
{
x264_emms();
return x264_clip3( h->rc->qpm + 0.5f, h->param.rc.i_qp_min, h->param.rc.i_qp_max );
}x264_ratecontrol_mb_qp
//自适应量化的作用在此体现
int x264_ratecontrol_mb_qp( x264_t *h )
{
x264_emms();
float qp = h->rc->qpm;
if( h->param.rc.i_aq_mode )
{
/* MB-tree currently doesn't adjust quantizers in unreferenced frames. */
float qp_offset = h->fdec->b_kept_as_ref ? h->fenc->f_qp_offset[h->mb.i_mb_xy] : h->fenc->f_qp_offset_aq[h->mb.i_mb_xy];
/* Scale AQ's effect towards zero in emergency mode. */
if( qp > QP_MAX_SPEC )
qp_offset *= (QP_MAX - qp) / (QP_MAX - QP_MAX_SPEC);
qp += qp_offset;
}
return x264_clip3( qp + 0.5f, h->param.rc.i_qp_min, h->param.rc.i_qp_max );
}x264_ratecontrol_mb
/* TODO:
* eliminate all use of qp in row ratecontrol: make it entirely qscale-based.
* make this function stop being needlessly O(N^2)
* update more often than once per row? */
//宏块级的码率控制？感觉更多的是根据缓冲区和framesize做一些增减，是以一行宏块为单位的
//为什么要在熵编码的最后部分，应该因为是要给下一行编码做指导，相当于模型的更新
int x264_ratecontrol_mb( x264_t *h, int bits )
{
x264_ratecontrol_t *rc = h->rc;
const int y = h->mb.i_mb_y;
//这一行的qp
h->fdec->i_row_bits[y] += bits;
//加上这个qp，计算平均qp用，这个qp是被aq过的
rc->qpa_aq += h->mb.i_qp;
//没到这一行的结尾，返回
if( h->mb.i_mb_x != h->mb.i_mb_width - 1 )
return 0;
x264_emms();
//aq之前，所有qp都一样，所以这样计算，用于算平均qp
rc->qpa_rc += rc->qpm * h->mb.i_mb_width;
//没有vbv，返回
if( !rc->b_vbv )
return 0;
float qscale = qp2qscale( rc->qpm );
//重建帧，这一行的qp都一样，qscale也一样
h->fdec->f_row_qp[y] = rc->qpm;
h->fdec->f_row_qscale[y] = qscale;
//根据这一行的satd，比特数，qscale更新predictor
update_predictor( &rc->row_pred[0], qscale, h->fdec->i_row_satd[y], h->fdec->i_row_bits[y] );
if( h->sh.i_type == SLICE_TYPE_P && rc->qpm < h->fref[0][0]->f_row_qp[y] )
update_predictor( &rc->row_pred[1], qscale, h->fdec->i_row_satds[0][0][y], h->fdec->i_row_bits[y] );
/* update ratecontrol per-mbpair in MBAFF */
if( SLICE_MBAFF && !(y&1) )
return 0;
/* FIXME: We don't currently support the case where there's a slice
* boundary in between.
* slice一定是整行，其实一般也就是整个帧*/
int can_reencode_row = h->sh.i_first_mb <= ((h->mb.i_mb_y - SLICE_MBAFF) * h->mb.i_mb_stride);
/* tweak quality based on difference from predicted size */
//根据与预测大小的差距调整quality
float prev_row_qp = h->fdec->f_row_qp[y];
float qp_absolute_max = h->param.rc.i_qp_max;
if( rc->rate_factor_max_increment )
qp_absolute_max = X264_MIN( qp_absolute_max, rc->qp_novbv + rc->rate_factor_max_increment );
float qp_max = X264_MIN( prev_row_qp + h->param.rc.i_qp_step, qp_absolute_max );
float qp_min = X264_MAX( prev_row_qp - h->param.rc.i_qp_step, h->param.rc.i_qp_min );
float step_size = 0.5f;
float buffer_left_planned = rc->buffer_fill - rc->frame_size_planned;
float slice_size_planned = h->param.b_sliced_threads ? rc->slice_size_planned : rc->frame_size_planned;
float max_frame_error = X264_MAX( 0.05f, 1.0f / h->mb.i_mb_height );
float size_of_other_slices = 0;
if( h->param.b_sliced_threads )
{
float size_of_other_slices_planned = 0;
for( int i = 0; i < h->param.i_threads; i++ )
if( h != h->thread[i] )
{
size_of_other_slices += h->thread[i]->rc->frame_size_estimated;
size_of_other_slices_planned += h->thread[i]->rc->slice_size_planned;
}
float weight = rc->slice_size_planned / rc->frame_size_planned;
size_of_other_slices = (size_of_other_slices - size_of_other_slices_planned) * weight + size_of_other_slices_planned;
}
if( y < h->i_threadslice_end-1 )
{//没到最后一行
/* B-frames shouldn't use lower QP than their reference frames. */
if( h->sh.i_type == SLICE_TYPE_B )
{
qp_min = X264_MAX( qp_min, X264_MAX( h->fref[0][0]->f_row_qp[y+1], h->fref[1][0]->f_row_qp[y+1] ) );
rc->qpm = X264_MAX( rc->qpm, qp_min );
}
/* More threads means we have to be more cautious in letting ratecontrol use up extra bits. */
float rc_tol = buffer_left_planned / h->param.i_threads * rc->rate_tolerance;
float b1 = predict_row_size_sum( h, y, rc->qpm ) + size_of_other_slices;//预测以当前qp编码下去的framesize
/* Don't increase the row QPs until a sufficent amount of the bits of the frame have been processed, in case a flat */
/* area at the top of the frame was measured inaccurately. */
if( row_bits_so_far( h, y ) < 0.05f * slice_size_planned )
qp_max = qp_absolute_max = prev_row_qp;
//不是I slice
if( h->sh.i_type != SLICE_TYPE_I )
rc_tol *= 0.5f;
if( !rc->b_vbv_min_rate )
qp_min = X264_MAX( qp_min, rc->qp_novbv );
while( rc->qpm < qp_max //不超过最大qp
&& ((b1 > rc->frame_size_planned + rc_tol) || //预计framesize超标
(rc->buffer_fill - b1 < buffer_left_planned * 0.5f) || //预计缓冲区不足
(b1 > rc->frame_size_planned && rc->qpm < rc->qp_novbv)) ) //预计framesize超标&&qp不超过qp_novbv
{
rc->qpm += step_size;
b1 = predict_row_size_sum( h, y, rc->qpm ) + size_of_other_slices;
}
while( rc->qpm > qp_min //不低于最小qp
&& (rc->qpm > h->fdec->f_row_qp[0] || rc->single_frame_vbv)
&& ((b1 < rc->frame_size_planned * 0.8f && rc->qpm <= prev_row_qp) //framesize比预期小太多，且当前行qp比前一行的qp小
|| b1 < (rc->buffer_fill - rc->buffer_size + rc->buffer_rate) * 1.1f) )
{
rc->qpm -= step_size;
b1 = predict_row_size_sum( h, y, rc->qpm ) + size_of_other_slices;
}
/* avoid VBV underflow or MinCR violation */
while( (rc->qpm < qp_absolute_max)
&& ((rc->buffer_fill - b1 < rc->buffer_rate * max_frame_error) ||
(rc->frame_size_maximum - b1 < rc->frame_size_maximum * max_frame_error)))
{
rc->qpm += step_size;
b1 = predict_row_size_sum( h, y, rc->qpm ) + size_of_other_slices;
}
//经过反复调整后的预期framesize
h->rc->frame_size_estimated = b1 - size_of_other_slices;
/* If the current row was large enough to cause a large QP jump, try re-encoding it. */
//经过各种调整的qp值过大，尝试重新编码
if( rc->qpm > qp_max && prev_row_qp < qp_max && can_reencode_row )
{
/* Bump QP to halfway in between... close enough. */
rc->qpm = x264_clip3f( (prev_row_qp + rc->qpm)*0.5f, prev_row_qp + 1.0f, qp_max );
rc->qpa_rc = rc->qpa_rc_prev;
rc->qpa_aq = rc->qpa_aq_prev;
h->fdec->i_row_bits[y] = 0;
h->fdec->i_row_bits[y-SLICE_MBAFF] = 0;
return -1;
}
}
else
{//到了最后一行
h->rc->frame_size_estimated = predict_row_size_sum( h, y, rc->qpm );
/* Last-ditch attempt: if the last row of the frame underflowed the VBV,
* try again.
* 如果最后一行导致VBV 下溢，尝试重新编码*/
if( (h->rc->frame_size_estimated + size_of_other_slices) > (rc->buffer_fill - rc->buffer_rate * max_frame_error) &&
rc->qpm < qp_max && can_reencode_row )
{
rc->qpm = qp_max;
rc->qpa_rc = rc->qpa_rc_prev;
rc->qpa_aq = rc->qpa_aq_prev;
h->fdec->i_row_bits[y] = 0;
h->fdec->i_row_bits[y-SLICE_MBAFF] = 0;
return -1;
}
}
rc->qpa_rc_prev = rc->qpa_rc;
rc->qpa_aq_prev = rc->qpa_aq;
return 0;
}update_predictor
//更新frame size预测模型的系数
static void update_predictor( predictor_t *p, float q, float var, float bits )
{
float range = 1.5;
if( var < 10 )
return;
float old_coeff = p->coeff / p->count;
float new_coeff = X264_MAX( bits*q / var, p->coeff_min );
float new_coeff_clipped = x264_clip3f( new_coeff, old_coeff/range, old_coeff*range );
float new_offset = bits*q - new_coeff_clipped * var;
if( new_offset >= 0 )
new_coeff = new_coeff_clipped;
else
new_offset = 0;
p->count *= p->decay;
p->coeff *= p->decay;
p->offset *= p->decay;
p->count ++;
p->coeff += new_coeff;
p->offset += new_offset;
}x264_ratecontrol_end
/* After encoding one frame, save stats and update ratecontrol state */
//编码一帧之后，保存数据，更新rc状态
int x264_ratecontrol_end( x264_t *h, int bits, int *filler )
{
x264_ratecontrol_t *rc = h->rc;
const int *mbs = h->stat.frame.i_mb_count;
x264_emms();
//各个类型的帧计数
h->stat.frame.i_mb_count_skip = mbs[P_SKIP] + mbs[B_SKIP];
h->stat.frame.i_mb_count_i = mbs[I_16x16] + mbs[I_8x8] + mbs[I_4x4];
h->stat.frame.i_mb_count_p = mbs[P_L0] + mbs[P_8x8];
for( int i = B_DIRECT; i < B_8x8; i++ )
h->stat.frame.i_mb_count_p += mbs[i];
//qp均值，crf均值
h->fdec->f_qp_avg_rc = rc->qpa_rc /= h->mb.i_mb_count;
h->fdec->f_qp_avg_aq = (float)rc->qpa_aq / h->mb.i_mb_count;
h->fdec->f_crf_avg = h->param.rc.f_rf_constant + h->fdec->f_qp_avg_rc - rc->qp_novbv;
//输出stat
if( h->param.rc.b_stat_write )
{
char c_type = h->sh.i_type==SLICE_TYPE_I ? (h->fenc->i_poc==0 ? 'I' : 'i')
: h->sh.i_type==SLICE_TYPE_P ? 'P'
: h->fenc->b_kept_as_ref ? 'B' : 'b';
int dir_frame = h->stat.frame.i_direct_score[1] - h->stat.frame.i_direct_score[0];
int dir_avg = h->stat.i_direct_score[1] - h->stat.i_direct_score[0];
char c_direct = h->mb.b_direct_auto_write ?
( dir_frame>0 ? 's' : dir_frame<0 ? 't' :
dir_avg>0 ? 's' : dir_avg<0 ? 't' : '-' )
: '-';
if( fprintf( rc->p_stat_file_out,
"in:%d out:%d type:%c dur:%"PRId64" cpbdur:%"PRId64" q:%.2f aq:%.2f tex:%d mv:%d misc:%d imb:%d pmb:%d smb:%d d:%c ref:",
h->fenc->i_frame, h->i_frame,
c_type, h->fenc->i_duration,
h->fenc->i_cpb_duration,
rc->qpa_rc, h->fdec->f_qp_avg_aq,
h->stat.frame.i_tex_bits,
h->stat.frame.i_mv_bits,
h->stat.frame.i_misc_bits,
h->stat.frame.i_mb_count_i,
h->stat.frame.i_mb_count_p,
h->stat.frame.i_mb_count_skip,
c_direct) < 0 )
goto fail;
/* Only write information for reference reordering once. */
int use_old_stats = h->param.rc.b_stat_read && rc->rce->refs > 1;
for( int i = 0; i < (use_old_stats ? rc->rce->refs : h->i_ref[0]); i++ )
{
int refcount = use_old_stats ? rc->rce->refcount[i]
: PARAM_INTERLACED ? h->stat.frame.i_mb_count_ref[0][i*2]
+ h->stat.frame.i_mb_count_ref[0][i*2+1]
: h->stat.frame.i_mb_count_ref[0][i];
if( fprintf( rc->p_stat_file_out, "%d ", refcount ) < 0 )
goto fail;
}
if( h->param.analyse.i_weighted_pred >= X264_WEIGHTP_SIMPLE && h->sh.weight[0][0].weightfn )
{
if( fprintf( rc->p_stat_file_out, "w:%d,%d,%d",
h->sh.weight[0][0].i_denom, h->sh.weight[0][0].i_scale, h->sh.weight[0][0].i_offset ) < 0 )
goto fail;
if( h->sh.weight[0][1].weightfn || h->sh.weight[0][2].weightfn )
{
if( fprintf( rc->p_stat_file_out, ",%d,%d,%d,%d,%d ",
h->sh.weight[0][1].i_denom, h->sh.weight[0][1].i_scale, h->sh.weight[0][1].i_offset,
h->sh.weight[0][2].i_scale, h->sh.weight[0][2].i_offset ) < 0 )
goto fail;
}
else if( fprintf( rc->p_stat_file_out, " " ) < 0 )
goto fail;
}
if( fprintf( rc->p_stat_file_out, ";\n") < 0 )
goto fail;
/* Don't re-write the data in multi-pass mode. */
if( h->param.rc.b_mb_tree && h->fenc->b_kept_as_ref && !h->param.rc.b_stat_read )
{
uint8_t i_type = h->sh.i_type;
/* Values are stored as big-endian FIX8.8 */
for( int i = 0; i < h->mb.i_mb_count; i++ )
rc->mbtree.qp_buffer[0][i] = endian_fix16( h->fenc->f_qp_offset[i]*256.0 );
if( fwrite( &i_type, 1, 1, rc->p_mbtree_stat_file_out ) < 1 )
goto fail;
if( fwrite( rc->mbtree.qp_buffer[0], sizeof(uint16_t), h->mb.i_mb_count, rc->p_mbtree_stat_file_out ) < h->mb.i_mb_count )
goto fail;
}
}
if( rc->b_abr )
{
//更新cplxr_sum和wanted_bits_window
if( h->sh.i_type != SLICE_TYPE_B )
rc->cplxr_sum += bits * qp2qscale( rc->qpa_rc ) / rc->last_rceq;
else
{
/* Depends on the fact that B-frame's QP is an offset from the following P-frame's.
* Not perfectly accurate with B-refs, but good enough. */
rc->cplxr_sum += bits * qp2qscale( rc->qpa_rc ) / (rc->last_rceq * fabs( h->param.rc.f_pb_factor ));
}
rc->cplxr_sum *= rc->cbr_decay;
rc->wanted_bits_window += h->fenc->f_duration * rc->bitrate;
rc->wanted_bits_window *= rc->cbr_decay;
}
if( rc->b_2pass )
rc->expected_bits_sum += qscale2bits( rc->rce, qp2qscale( rc->rce->new_qp ) );
if( h->mb.b_variable_qp )
{
if( h->sh.i_type == SLICE_TYPE_B )
{
rc->bframe_bits += bits;
if( h->fenc->b_last_minigop_bframe )
{
update_predictor( rc->pred_b_from_p, qp2qscale( rc->qpa_rc ),
h->fref[1][h->i_ref[1]-1]->i_satd, rc->bframe_bits / rc->bframes );
rc->bframe_bits = 0;
}
}
}
//更新vbv
*filler = update_vbv( h, bits );
rc->filler_bits_sum += *filler * 8;
if( h->sps->vui.b_nal_hrd_parameters_present )
{
if( h->fenc->i_frame == 0 )
{
// access unit initialises the HRD
h->fenc->hrd_timing.cpb_initial_arrival_time = 0;
rc->initial_cpb_removal_delay = h->initial_cpb_removal_delay;
rc->initial_cpb_removal_delay_offset = h->initial_cpb_removal_delay_offset;
h->fenc->hrd_timing.cpb_removal_time = rc->nrt_first_access_unit = (double)rc->initial_cpb_removal_delay / 90000;
}
else
{
h->fenc->hrd_timing.cpb_removal_time = rc->nrt_first_access_unit + (double)(h->fenc->i_cpb_delay - h->i_cpb_delay_pir_offset) *
h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale;
if( h->fenc->b_keyframe )
{
rc->nrt_first_access_unit = h->fenc->hrd_timing.cpb_removal_time;
rc->initial_cpb_removal_delay = h->initial_cpb_removal_delay;
rc->initial_cpb_removal_delay_offset = h->initial_cpb_removal_delay_offset;
}
double cpb_earliest_arrival_time = h->fenc->hrd_timing.cpb_removal_time - (double)rc->initial_cpb_removal_delay / 90000;
if( !h->fenc->b_keyframe )
cpb_earliest_arrival_time -= (double)rc->initial_cpb_removal_delay_offset / 90000;
if( h->sps->vui.hrd.b_cbr_hrd )
h->fenc->hrd_timing.cpb_initial_arrival_time = rc->previous_cpb_final_arrival_time;
else
h->fenc->hrd_timing.cpb_initial_arrival_time = X264_MAX( rc->previous_cpb_final_arrival_time, cpb_earliest_arrival_time );
}
int filler_bits = *filler ? X264_MAX( (FILLER_OVERHEAD - h->param.b_annexb), *filler )*8 : 0;
// Equation C-6
h->fenc->hrd_timing.cpb_final_arrival_time = rc->previous_cpb_final_arrival_time = h->fenc->hrd_timing.cpb_initial_arrival_time +
(double)(bits + filler_bits) / h->sps->vui.hrd.i_bit_rate_unscaled;
h->fenc->hrd_timing.dpb_output_time = (double)h->fenc->i_dpb_output_delay * h->sps->vui.i_num_units_in_tick / h->sps->vui.i_time_scale +
h->fenc->hrd_timing.cpb_removal_time;
}
return 0;
fail:
x264_log( h, X264_LOG_ERROR, "ratecontrol_end: stats file could not be written to\n" );
return -1;
}update_vbv
// update VBV after encoding a frame
static int update_vbv( x264_t *h, int bits )
{
int filler = 0;
int bitrate = h->sps->vui.hrd.i_bit_rate_unscaled;
x264_ratecontrol_t *rcc = h->rc;
x264_ratecontrol_t *rct = h->thread[0]->rc;
int64_t buffer_size = (int64_t)h->sps->vui.hrd.i_cpb_size_unscaled * h->sps->vui.i_time_scale;
if( rcc->last_satd >= h->mb.i_mb_count ) //为什么要把这两个比较
update_predictor( &rct->pred[h->sh.i_type], qp2qscale( rcc->qpa_rc ), rcc->last_satd, bits );
if( !rcc->b_vbv )
return filler;
uint64_t buffer_diff = (uint64_t)bits * h->sps->vui.i_time_scale;
rct->buffer_fill_final -= buffer_diff;
rct->buffer_fill_final_min -= buffer_diff;
if( rct->buffer_fill_final_min < 0 )
{//underflow happens
double underflow = (double)rct->buffer_fill_final_min / h->sps->vui.i_time_scale;
if( rcc->rate_factor_max_increment && rcc->qpm >= rcc->qp_novbv + rcc->rate_factor_max_increment )
x264_log( h, X264_LOG_DEBUG, "VBV underflow due to CRF-max (frame %d, %.0f bits)\n", h->i_frame, underflow );
else
x264_log( h, X264_LOG_WARNING, "VBV underflow (frame %d, %.0f bits)\n", h->i_frame, underflow );
rct->buffer_fill_final =
rct->buffer_fill_final_min = 0;
}
if( h->param.i_avcintra_class )
buffer_diff = buffer_size;
else
buffer_diff = (uint64_t)bitrate * h->sps->vui.i_num_units_in_tick * h->fenc->i_cpb_duration;
rct->buffer_fill_final += buffer_diff;
rct->buffer_fill_final_min += buffer_diff;
if( rct->buffer_fill_final > buffer_size )
{
if( h->param.rc.b_filler )
{
int64_t scale = (int64_t)h->sps->vui.i_time_scale * 8;
filler = (rct->buffer_fill_final - buffer_size + scale - 1) / scale;
bits = h->param.i_avcintra_class ? filler * 8 : X264_MAX( (FILLER_OVERHEAD - h->param.b_annexb), filler ) * 8;
buffer_diff = (uint64_t)bits * h->sps->vui.i_time_scale;
rct->buffer_fill_final -= buffer_diff;
rct->buffer_fill_final_min -= buffer_diff;
}
else
{
rct->buffer_fill_final = X264_MIN( rct->buffer_fill_final, buffer_size );
rct->buffer_fill_final_min = X264_MIN( rct->buffer_fill_final_min, buffer_size );
}
}
return filler;
}最后的x264_ratecontrol_summary和x264_ratecontrol_delete略。
一些补充内容
1、什么是Macroblock Tree
Macroblock Tree是一个基于macroblock的qp控制方法。MB Tree的工作原理类似于古典的qp compression，只不过qcomp处理的对象是整张frame而MB Tree针对的是每个MB进行处理。工作过程简单来说，是对于每个MB，向前预测一定数量的帧（该数量由rc-lookahead和keyint的较小值决定）中该MB被参考的情况，根据引用次数的多寡，决定对该MB使用何种大小的qp进行quantization。而qp的大小与被参考次数成反比，也就是说，对于被参考次数多的MB，264的解码器认为此对应于缓慢变化的场景，因此给与比较高的质量（比较低的qp数值）。至于视频的变化率与人眼感知能力的关系，这是一个基于主观测试的经验结果：视频变化率越大 人眼的敏感度越低，也就是说，人眼可以容忍快速变化场景的某些缺陷，但相对而言某些平滑场景的缺陷，人眼则相当敏感。注意此处说的平滑，指的是沿时间维度上场景的变化频率，而非普通意义上的像素域中的场景。
2、MBTree File
这是一个临时文件，记录了每个P帧中每个MB被参考的情况。
3、MB Tree的处理对象
据说目前mbtree只处理p frames的mb，同时也不支持bpyramid。还没有细看。
4、与Mbtree相关的参数
--qcomp qcomp有削弱mbtree强度的倾向，具体来说，qcomp的值越趋近于1（Constant Quantizer），mbtree的效力越差。
--rc-lookahead 决定mbtree向前预测的帧数。
5、x264中可以调整每个mb的qp的方式有2种
（1） 启动aq---这种方式会略微扰乱码率控制；
（2） 启用vbv---vbv和abr都是接近cbr的模式，vbv会逐行调整qp，码率控制会更准确一些；
6、rate_tolerance的单位是啥？没有单位，就是一个简单的相乘系数，double abr_buffer = 2 * rcc->rate_tolerance * rcc->bitrate;
7、rate_estimate_qscale中的total_bits事一直都有吗？不是，只有在2pass的时候才能通过stat文件读取到
本文福利， 免费领取C++音视频学习资料包、技术视频，内容包括（音视频开发，面试题，FFmpeg ，webRTC ，rtmp ，hls ，rtsp ，ffplay ，srs）↓↓↓↓↓↓见下面↓↓文章底部点击免费领取↓↓