JitterBuffer(1) of WebRTC series

In the process of audio and video network transmission, due to network jitter, the video at the receiving end is not received in time, causing playback freezes. In order to eliminate inter-frame jitter, a solution is JitterBuffer. JitterBuffer includes sorting of RTP packets, intra-GOP frame sorting and inter-GOP sorting. (Explanation of terms is noted at the end of the article).

1. RTP packet sorting: PacketBuffer

1.1 Insert RTP packet (PacketBuffer::InsertPacket)

This function first judges whether it is the first package, if it is, record it, and the next package starts to sort backwards, if not, it calls the package serial number comparison function AheadOf. If the position of the package calculated using the index in the cache is occupied and the serial number is the same, it is a duplicate package and discarded. If it is occupied but the serial number is different, it means that the cache is full and needs to be expanded, and the index value of the package should be recalculated. If it is still full after expansion, it needs to be cached

PacketBuffer::InsertResult PacketBuffer::InsertPacket(
    std::unique_ptr<PacketBuffer::Packet> packet) {
  PacketBuffer::InsertResult result;
  //当前包序号
  uint16_t seq_num = packet->seq_num;
  //当前包在缓存中的索引
  size_t index = seq_num % buffer_.size();
  if (!first_packet_received_) {
    //保存第一个包
    first_seq_num_ = seq_num;
    //第一个包的序号
    first_packet_received_ = true;
    //收到了第一个包
  } else if (AheadOf(first_seq_num_, seq_num)) {
    // If we have explicitly cleared past this packet then it's old,
    // don't insert it, just silently ignore it.
    // 如果当前包比之前记录的第一个包first_seq_num_还老
    // 并且之前已经清理过第一个包序列号,说明已经至少成功解码过一帧,RtpVideoStreamReceiver::FrameDecoded
    // 会调用PacketBuffer::ClearTo(seq_num),清理first_seq_num_之前的所有缓存,这个时候还来一个比first_seq_num_还
    // 老的包,就没有必要再留着了。
    if (is_cleared_to_first_seq_num_) {
      return result;
    }
    // 相反如果没有被清理过,则是有必要保留成第一个包的,比如发生了乱序。
    first_seq_num_ = seq_num;
  }
  //如果缓存的槽被占了,而且序号一样,说明是重复包,丢掉
  if (buffer_[index] != nullptr) {
    // Duplicate packet, just delete the payload.
    if (buffer_[index]->seq_num == packet->seq_num) {
      return result;
    }
    // The packet buffer is full, try to expand the buffer.
    // 如果槽被占,但是输入包和对应槽的包序列号不等,说明缓存满了,需要扩容。
    // ExpandBufferSize() 会更新缓存在新的队列的位置,并不会引起位置错误
    while (ExpandBufferSize() && buffer_[seq_num % buffer_.size()] != nullptr) {
    }
    // 重新计算输入包索引.
    index = seq_num % buffer_.size();
    // Packet buffer is still full since we were unable to expand the buffer.
    // 如果对应的槽还是被占用了,还是满,已经不行了,致命错误.
    if (buffer_[index] != nullptr) {
      // Clear the buffer, delete payload, and return false to signal that a
      // new keyframe is needed.
      RTC_LOG(LS_WARNING) << "Clear PacketBuffer and request key frame.";
      ClearInternal();
      result.buffer_cleared = true;
      return result;
    }
  }
  //之前的包是否连续,这里初始为false,在FindFrames中置位
  packet->continuous = false;
  //此处的move移动语义提升了效率
  buffer_[index] = std::move(packet);
  // 更新丢包信息,检查收到当前包后是否有丢包导致的空洞,也就是不连续.
  UpdateMissingPackets(seq_num);
  result.packets = FindFrames(seq_num);
  return result;
}

1.2 Insert padding packet (PacketBuffer::InsertPadding)

The padding packet here is similar to padding, mainly due to the padding performed by the sender to meet the output bit rate.

PacketBuffer::InsertResult PacketBuffer::InsertPadding(uint16_t seq_num) {
  PacketBuffer::InsertResult result;
  // 更新丢包信息,检查收到当前包后是否有丢包导致的空洞,也就是不连续.
  UpdateMissingPackets(seq_num);
  // 分析排序缓存,检查是否能够组装出完整的帧并返回.
  result.packets = FindFrames(static_cast<uint16_t>(seq_num + 1));
  return result;
}

1.3 Packet loss detection (PacketBuffer::UpdateMissingPackets)

This function mainly completes whether the packets are continuous, mainly relying on packet loss cache missing_packets_ to maintain the packet sequence number.

void PacketBuffer::UpdateMissingPackets(uint16_t seq_num) {
  // 如果最新插入的包序列号还未设置过,这里直接设置一次.
  if (!newest_inserted_seq_num_)
    newest_inserted_seq_num_ = seq_num;
  const int kMaxPaddingAge = 1000;
  // 如果当前包的序列号新于之前的最新包序列号,没有发生乱序
  if (AheadOf(seq_num, *newest_inserted_seq_num_)) {
    // 丢包缓存missing_packets_最大保存1000个包,这里得到当前包1000个包以前的序列号,
    // 也就差不多是丢包缓存里应该保存的最老的包.
    uint16_t old_seq_num = seq_num - kMaxPaddingAge;
    // 第一个>= old_seq_num的包的位置
    auto erase_to = missing_packets_.lower_bound(old_seq_num);
    // 删除丢包缓存里所有1000个包之前的所有包(如果有的话)
    missing_packets_.erase(missing_packets_.begin(), erase_to);
    // Guard against inserting a large amount of missing packets if there is a
    // jump in the sequence number.
    // 如果最老的包的序列号都比当前最新包序列号新,那么更新一下当前最新包序列号
    if (AheadOf(old_seq_num, *newest_inserted_seq_num_))
      *newest_inserted_seq_num_ = old_seq_num;
    // 因为seq_num >newest_inserted_seq_num_,这里开始统计(newest_inserted_seq_num_, sum)之间的空洞.
    ++*newest_inserted_seq_num_;
    // 从newest_inserted_seq_num_开始,每个小于当前seq_num的包都进入丢包缓存,直到newest_inserted_seq_num_ ==
    // seq_num,也就是最新包的序列号变成了当前seq_num.
    while (AheadOf(seq_num, *newest_inserted_seq_num_)) {
      missing_packets_.insert(*newest_inserted_seq_num_);
      ++*newest_inserted_seq_num_;
    }
  } else {
    // 如果当前收到的包的序列号小于当前收到的最新包序列号,则从丢包缓存中删除(之前应该已经进入丢包缓存)
    missing_packets_.erase(seq_num);
  }
}

1.4 Continuous packet detection (PacketBuffer::PotentialNewFrame)

The main function is to detect whether the front of the current packet is continuous. If it is continuous, the complete frame detection will be carried out.

bool PacketBuffer::PotentialNewFrame(uint16_t seq_num) const {
  // 通过序列号获取缓存索引
  size_t index = seq_num % buffer_.size();
  // 上个包的索引
  int prev_index = index > 0 ? index - 1 : buffer_.size() - 1;
  const auto& entry = buffer_[index];
  const auto& prev_entry = buffer_[prev_index];
  // 如果当前包的槽位没有被占用,那么该包之前没有处理过,不连续
  if (entry == nullptr)
    return false;
  // 如果当前包的槽位的序列号和当前包序列号不一致,不连续.
  if (entry->seq_num != seq_num)
    return false;
  // 如果当前包的帧开始标识frame_begin为true,那么该包是帧第一个包,连续.
  if (entry->is_first_packet_in_frame())
    return true;
  // 如果上个包的槽位没有被占用,那么上个包之前没有处理过,不连续.  
  if (prev_entry == nullptr)
    return false;
  // 如果上个包和当前包的序列号不连续,不连续.
  if (prev_entry->seq_num != static_cast<uint16_t>(entry->seq_num - 1))
    return false;
  // 如果上个包的时间戳和当前包的时间戳不相等,不连续.
  if (prev_entry->timestamp != entry->timestamp)
    return false;
  // 排除掉以上所有错误后,如果上个包连续,则可以认为当前包连续.
  if (prev_entry->continuous)
    return true;
  // 如果上个包不连续或者有其他错误,就返回不连续.
  return false;
}

1.5 Integrity detection of frames (PacketBuffer::FindFrames)

The PacketBuffer::FindFrames function will traverse the consecutive packets in the sorting buffer and check the boundary of a frame, but here the processing of VPX and H264 is distinguished:

For VPX, this function considers the frame_begin of the packet to be credible, so that a complete frame of VPX is completely dependent on the detection of the two packets of frame_begin and frame_end;

[Learning address]: FFmpeg/WebRTC/RTMP/NDK/Android audio and video streaming media advanced development

[Article Benefits]: Receive more audio and video learning packages, Dachang interview questions, technical videos and learning roadmaps for free. The materials include (C/C++, Linux, FFmpeg webRTC rtmp hls rtsp ffplay srs, etc.) Click 1079654574 to join the group to receive it~

For H264, this function considers the frame_begin of the packet to be untrustworthy, and does not rely on frame_begin to determine the start of the frame, but frame_end is still credible. Specifically, the start identifier of H264 is traced forward from the last packet of a frame identified by frame_end , until a fault with a different time stamp is found, it is considered that a complete H264 frame has been found.

In addition, some special processing is done for the P frame of H264 here. Although the P frame may be complete, if there is still a packet loss hole in front of the P frame, it will not be passed back immediately, and will wait until all the holes are filled, because P A frame must have a reference frame in order to be decoded correctly.

std::vector<std::unique_ptr<PacketBuffer::Packet>> PacketBuffer::FindFrames(
    uint16_t seq_num) {
  std::vector<std::unique_ptr<PacketBuffer::Packet>> found_frames;
  // 基本算法:遍历所有连续包,先找到带有frame_end标识的帧最后一个包,然后向前回溯,
  // 找到帧的第一个包(VPX是frame_begin, H264是时间戳不连续),组成完整一帧,
  // PotentialNewFrame(seq_num)检测seq_num之前的所有包是否连续.
  for (size_t i = 0; i < buffer_.size() && PotentialNewFrame(seq_num); ++i) {
    // 当前包的缓存索引
    size_t index = seq_num % buffer_.size();
    // 如果seq_num之前所有包连续,那么seq_num自己也连续.
    buffer_[index]->continuous = true;
    // If all packets of the frame is continuous, find the first packet of the
    // frame and add all packets of the frame to the returned packets.
    // 找到了帧的最后一个包.
    if (buffer_[index]->is_last_packet_in_frame()) {
      // 帧开始序列号,从帧尾部开始.
      uint16_t start_seq_num = seq_num;
      // Find the start index by searching backward until the packet with
      // the |frame_begin| flag is set.
      // 开始向前回溯,找帧的第一个包.
      // 帧开始的索引,从帧尾部开始.
      int start_index = index;
      // 已经测试的包数
      size_t tested_packets = 0;
      // 当前包的时间戳. 也就是帧的时间戳
      int64_t frame_timestamp = buffer_[start_index]->timestamp;
      // Identify H.264 keyframes by means of SPS, PPS, and IDR.
      bool is_h264 = buffer_[start_index]->codec() == kVideoCodecH264;
      bool has_h264_sps = false;
      bool has_h264_pps = false;
      bool has_h264_idr = false;
      bool is_h264_keyframe = false;
      int idr_width = -1;
      int idr_height = -1;
      // 从帧尾部的包开始回溯.
      while (true) {
        // 测试包数++
        ++tested_packets;
        // 如果是VPX,并且找到了frame_begin标识的第一个包,一帧完整,回溯结束.
        if (!is_h264 && buffer_[start_index]->is_first_packet_in_frame())
          break;
        // h264 判断方式
        if (is_h264) {
          //获取h264 相关信息,
          const auto* h264_header = absl::get_if<RTPVideoHeaderH264>(
              &buffer_[start_index]->video_header.video_type_header);
          if (!h264_header || h264_header->nalus_length >= kMaxNalusPerPacket)
            return found_frames;
          // 遍历所有NALU,注意WebRTC所有IDR帧前面都会带SPS、PPS.
          for (size_t j = 0; j < h264_header->nalus_length; ++j) {
            if (h264_header->nalus[j].type == H264::NaluType::kSps) {
              has_h264_sps = true;
            } else if (h264_header->nalus[j].type == H264::NaluType::kPps) {
              has_h264_pps = true;
            } else if (h264_header->nalus[j].type == H264::NaluType::kIdr) {
              has_h264_idr = true;
            }
          }
          // 默认sps_pps_idr_is_h264_keyframe_为false,也就是说只需要有IDR帧就认为是关键帧,
          // 而不需要等待SPS、PPS完整.
          if ((sps_pps_idr_is_h264_keyframe_ && has_h264_idr && has_h264_sps &&
               has_h264_pps) ||
              (!sps_pps_idr_is_h264_keyframe_ && has_h264_idr)) {
            is_h264_keyframe = true;
            // Store the resolution of key frame which is the packet with
            // smallest index and valid resolution; typically its IDR or SPS
            // packet; there may be packet preceeding this packet, IDR's
            // resolution will be applied to them.
            if (buffer_[start_index]->width() > 0 &&
                buffer_[start_index]->height() > 0) {
              idr_width = buffer_[start_index]->width();
              idr_height = buffer_[start_index]->height();
            }
          }
        }
        // 如果检测包数已经达到缓存容量,中止.
        if (tested_packets == buffer_.size())
          break;
        start_index = start_index > 0 ? start_index - 1 : buffer_.size() - 1;
        // In the case of H264 we don't have a frame_begin bit (yes,
        // |frame_begin| might be set to true but that is a lie). So instead
        // we traverese backwards as long as we have a previous packet and
        // the timestamp of that packet is the same as this one. This may cause
        // the PacketBuffer to hand out incomplete frames.
        // See: https://bugs.chromium.org/p/webrtc/issues/detail?id=7106
        // 这里保留了注释,可以看看H264不使用frame_begin的原因,
        //已timestamp发生变化,认为是一帧结束
        if (is_h264 && (buffer_[start_index] == nullptr ||
                        buffer_[start_index]->timestamp != frame_timestamp)) {
          break;
        }
        // 如果仍然在一帧内,开始包序列号--.
        --start_seq_num;
      }
      //如果没有sps或者pps 异常警告
      if (is_h264) {
        // Warn if this is an unsafe frame.
        if (has_h264_idr && (!has_h264_sps || !has_h264_pps)) {
          RTC_LOG(LS_WARNING)
              << "Received H.264-IDR frame "
                 "(SPS: "
              << has_h264_sps << ", PPS: " << has_h264_pps << "). Treating as "
              << (sps_pps_idr_is_h264_keyframe_ ? "delta" : "key")
              << " frame since WebRTC-SpsPpsIdrIsH264Keyframe is "
              << (sps_pps_idr_is_h264_keyframe_ ? "enabled." : "disabled");
        }
        // Now that we have decided whether to treat this frame as a key frame
        // or delta frame in the frame buffer, we update the field that
        // determines if the RtpFrameObject is a key frame or delta frame.
        //帧的起始位置
        const size_t first_packet_index = start_seq_num % buffer_.size();
        // 设置数据缓存中的关键帧标识.
        if (is_h264_keyframe) {
          buffer_[first_packet_index]->video_header.frame_type =
              VideoFrameType::kVideoFrameKey;
          if (idr_width > 0 && idr_height > 0) {
            // IDR frame was finalized and we have the correct resolution for
            // IDR; update first packet to have same resolution as IDR.
            buffer_[first_packet_index]->video_header.width = idr_width;
            buffer_[first_packet_index]->video_header.height = idr_height;
          }
        } else {
          buffer_[first_packet_index]->video_header.frame_type =
              VideoFrameType::kVideoFrameDelta;
        }
        // If this is not a keyframe, make sure there are no gaps in the packet
        // sequence numbers up until this point.
        // 这个条件是说在丢包的列表里搜索>start_seq_num(帧开始序列号)的第一个位置,
        // 发现其不等于丢包列表的开头, 有些丢的包序列号小于start_seq_num,
        // 也就是说P帧前面有丢包空洞, 举例1: missing_packets_ = { 3, 4, 6},
        // start_seq_num = 5, missing_packets_.upper_bound(start_seq_num)==6
        // 作为一帧开始位置的序列号5,前面还有3、4这两个包还未收到,那么对P帧来说,虽然完整,但是向后传递也可能是没有意义的,
        // 所以这里又清除了frame_created状态,先继续缓存,等待丢包的空洞填满.
        // 举例2:
        // missing_packets_ = { 10, 16, 17}, start_seq_num = 3,
        // missing_packets_.upper_bound(start_seq_num)==10
        // 作为一帧开始位置的序列号3,前面并没有丢包,并且帧完整,那么可以向后传递.
        if (!is_h264_keyframe && missing_packets_.upper_bound(start_seq_num) !=
                                     missing_packets_.begin()) {
          return found_frames;
        }
      }
      const uint16_t end_seq_num = seq_num + 1;
      // Use uint16_t type to handle sequence number wrap around case.
      uint16_t num_packets = end_seq_num - start_seq_num;
      found_frames.reserve(found_frames.size() + num_packets);
      for (uint16_t i = start_seq_num; i != end_seq_num; ++i) {
        std::unique_ptr<Packet>& packet = buffer_[i % buffer_.size()];
        RTC_DCHECK(packet);
        RTC_DCHECK_EQ(i, packet->seq_num);
        // Ensure frame boundary flags are properly set.
        packet->video_header.is_first_packet_in_frame = (i == start_seq_num);
        packet->video_header.is_last_packet_in_frame = (i == seq_num);
        found_frames.push_back(std::move(packet));
      }
      // 马上要组帧了,清除丢包列表中到帧开始位置之前的丢包.
      // 对H264 P帧来说,如果P帧前面有空洞不会运行到这里,在上面已经解释.
      // 对I帧来说,可以丢弃前面的丢包信息(?).
      missing_packets_.erase(missing_packets_.begin(),
                             missing_packets_.upper_bound(seq_num));
    }
    // 向后扩大搜索的范围,假设丢包、乱序,当前包的seq_num刚好填补了之前的一个空洞,
    // 该包并不能检测出一个完整帧,需要这里向后移动指针到frame_end再进行回溯,直到检测出完整帧,
    // 这里会继续检测之前缓存的因为前面有空洞而没有向后传递的P帧。
    ++seq_num;
  }
  return found_frames;
}

2. Sorting of frames

A P frame in a GOP depends on the previous P frame and I key frame. , RtpFrameReferenceFinder is to find the reference frame of each frame. The I frame is the self-reference of the GOP start frame, and each frame in the subsequent GOP must refer to the previous frame. RtpFrameReferenceFinder maintains the latest GOP table. After receiving the P frame, RtpFrameReferenceFinder finds the GOP to which the P frame belongs, sets the reference frame of the P frame as the previous frame of the frame in the GOP, and then passes it to FrameBuffer.

2.1 Set the reference frame (RtpSeqNumOnlyRefFinder::ManageFrameInternal)

This function mainly deals with the continuity of frames within the GOP and sets the reference frame.

RtpSeqNumOnlyRefFinder::FrameDecision
RtpSeqNumOnlyRefFinder::ManageFrameInternal(RtpFrameObject* frame) {
  // 如果是关键帧,插入GOP表,key是last_seq_num,初始value是{last_seq_num,last_seq_num}
  if (frame->frame_type() == VideoFrameType::kVideoFrameKey) {
    last_seq_num_gop_.insert(std::make_pair(
        frame->last_seq_num(),
        std::make_pair(frame->last_seq_num(), frame->last_seq_num())));
  }
  // We have received a frame but not yet a keyframe, stash this frame.
  // 如果GOP表空,那么就不可能找到参考帧,先缓存.
  if (last_seq_num_gop_.empty())
    return kStash;
  // Clean up info for old keyframes but make sure to keep info
  // for the last keyframe.
  // 删除较老的关键帧(PID小于last_seq_num - 100), 但是至少保留一个。
  auto clean_to = last_seq_num_gop_.lower_bound(frame->last_seq_num() - 100);
  for (auto it = last_seq_num_gop_.begin();
       it != clean_to && last_seq_num_gop_.size() > 1;) {
    it = last_seq_num_gop_.erase(it);
  }
  // Find the last sequence number of the last frame for the keyframe
  // that this frame indirectly references.
  // 在GOP表中搜索第一个比当前帧新的关键帧。
  auto seq_num_it = last_seq_num_gop_.upper_bound(frame->last_seq_num());
  // 如果搜索到的关键帧是最老的,说明当前帧比最老的关键帧还老,无法设置参考帧,丢弃.
  if (seq_num_it == last_seq_num_gop_.begin()) {
    RTC_LOG(LS_WARNING) << "Generic frame with packet range ["
                        << frame->first_seq_num() << ", "
                        << frame->last_seq_num()
                        << "] has no GoP, dropping frame.";
    return kDrop;
  }
  // 如果搜索到的关键帧不是最老的,那么搜索到的关键帧的上一个关键帧所在的GOP里应该可以找到参考帧,
  // 如果找不到关键帧,seq_num_it为end(), seq_num_it--则为最后一个关键帧.
  seq_num_it--;
  // Make sure the packet sequence numbers are continuous, otherwise stash
  // this frame.
  // 保证帧的连续,不连续则先缓存.
  // 当前GOP的最新一个帧的最后一个包的序列号.
  uint16_t last_picture_id_gop = seq_num_it->second.first;
  // 当前GOP的最新包的序列号,可能是last_picture_id_gop, 也可能是填充包.
  uint16_t last_picture_id_with_padding_gop = seq_num_it->second.second;
  // P帧的连续性检查.
  if (frame->frame_type() == VideoFrameType::kVideoFrameDelta) {
    // 获得P帧第一个包的上个包的序列号.
    uint16_t prev_seq_num = frame->first_seq_num() - 1;
    // 如果P帧第一个包的上个包的序列号与当前GOP的最新包的序列号不等,说明不连续,先缓存.
    if (prev_seq_num != last_picture_id_with_padding_gop)
      return kStash;
  }
  // 现在这个帧是连续的了
  RTC_DCHECK(AheadOrAt(frame->last_seq_num(), seq_num_it->first));
  // Since keyframes can cause reordering we can't simply assign the
  // picture id according to some incrementing counter.
  // 获得当前帧的最后一个包的序列号,设置为初始PID,后面还会设置一次Unwrap.
  frame->SetId(frame->last_seq_num());
  // 设置帧的参考帧数,P帧才需要1个参考帧.
  frame->num_references =
      frame->frame_type() == VideoFrameType::kVideoFrameDelta;
  // 设置参考帧为当前GOP的最新一个帧的最后一个包的序列号,
  // 既然该帧是连续的,那么其参考帧自然也就是上个帧.
  frame->references[0] = rtp_seq_num_unwrapper_.Unwrap(last_picture_id_gop);
  // 如果当前帧比当前GOP的最新一个帧的最后一个包还新,则更新GOP的最新一个帧的最后一个包(first)
  // 以及GOP的最新包(second).
  if (AheadOf<uint16_t>(frame->Id(), last_picture_id_gop)) {
    seq_num_it->second.first = frame->Id();// 更新GOP的最新一个帧的最后一个包
    seq_num_it->second.second = frame->Id();// 更新GOP的最新包,可能被填充包更新.
  }
  // 更新填充包状态.
  UpdateLastPictureIdWithPadding(frame->Id());
  frame->SetSpatialIndex(0);
  // 设置当前帧的PID为Unwrap形式.
  frame->SetId(rtp_seq_num_unwrapper_.Unwrap(frame->Id()));
  return kHandOff;

2.2 Processing Padding (RtpSeqNumOnlyRefFinder::PaddingReceived)

This function updates the padding packet. If the padding packet fills the sequence number hole in the GOP, then P can be continuous, and try to process the P frame.

RtpFrameReferenceFinder::ReturnVector RtpSeqNumOnlyRefFinder::PaddingReceived(
    uint16_t seq_num) {
  // 只保留最近100个填充包.
  auto clean_padding_to =
      stashed_padding_.lower_bound(seq_num - kMaxPaddingAge);
  stashed_padding_.erase(stashed_padding_.begin(), clean_padding_to);
  // 缓存填充包.
  stashed_padding_.insert(seq_num);
  // 更新填充包状态.
  UpdateLastPictureIdWithPadding(seq_num);
  RtpFrameReferenceFinder::ReturnVector res;
  // 尝试处理一次缓存的P帧,有可能序列号连续了.
  RetryStashedFrames(res);
  return res;
}

3 Processing cached packets (RtpSeqNumOnlyRefFinder::RetryStashedFrames)

The most common is to find consecutive frames with reference frames. If the above-mentioned Padding packet sequence number is just satisfied, it will also try to process it.

void RtpSeqNumOnlyRefFinder::RetryStashedFrames(
    RtpFrameReferenceFinder::ReturnVector& res) {
  bool complete_frame = false;
  // 遍历缓存的帧
  do {
    complete_frame = false;
    for (auto frame_it = stashed_frames_.begin();
         frame_it != stashed_frames_.end();) {
      // 调用ManageFramePidOrSeqNum来处理一个缓存帧,检查是否可以输出带参考帧的连续的帧.
      FrameDecision decision = ManageFrameInternal(frame_it->get());
      switch (decision) {
        case kStash:// 仍然不连续,或者没有参考帧.
          ++frame_it;// 检查下一个缓存帧.
          break;
        case kHandOff:// 找到了一个带参考帧的连续的帧.
          complete_frame = true;
          res.push_back(std::move(*frame_it));
          ABSL_FALLTHROUGH_INTENDED;
        case kDrop:// 无论kHandOff、kDrop都可以从缓存中删除了.
          frame_it = stashed_frames_.erase(frame_it);// 删除并检查下一个缓存帧.
      }
    }
  } while (complete_frame);// 如果能持续找到带参考帧的连续的帧则继续.
}

Write so much today, the specific reference link is a blog written by a great god, the specific link is attached:

http://t.csdn.cn/Sf0Dl

There will be time to fill in the follow-up content.

Original link: JitterBuffer(1) of WebRTC series

Guess you like

Origin blog.csdn.net/irainsa/article/details/130117036