webrtc code walk five (audio neteq code walk)

I. Overview

• The functions implemented by the NetEqImpl class are:

1. Audio redundant message analysis
2. nack message request
3. Audio jitterbuffer
4. Audio speed change algorithm function

• The key external interfaces of this class are:

1. NetEqImpl::InsertPacket: input audio RTP packet
2. NetEqImpl::GetAudio: The audio rendering module regularly requests audio packets
3. std::vector<uint16_t> GetNackList(int64_t round_trip_time_ms): output the serial number of the message that needs nack retransmission

2. Detailed code explanation 

• NetEqImpl::InsertPacket

     The core function call relationship is as follows:

NetEqImpl::InsertPacket
->NetEqImpl::InsertPacketInternal
->PacketBuffer::InsertPacketList
->PacketBuffer::InsertPacket
->buffer_.insert(it, std::move(packet));  // Insert the packet at that position.

The main purpose of this function is to store the packet in buffer_ of the PacketBuffer object.

Note the std::unique_ptr<AudioDecoder::EncodedAudioFrame> frame parameter of the buffer_ structure:

struct Packet {
  struct Priority {
    Priority() : codec_level(0), red_level(0) {}
    Priority(int codec_level, int red_level)
        : codec_level(codec_level), red_level(red_level) {
      CheckInvariant();
    }
  //...
  }

  uint32_t timestamp;
  uint16_t sequence_number;
  uint8_t payload_type;
  // Datagram excluding RTP header and header extension.
  rtc::Buffer payload;
  Priority priority;
  RtpPacketInfo packet_info;
  std::unique_ptr<TickTimer::Stopwatch> waiting_time;
  std::unique_ptr<AudioDecoder::EncodedAudioFrame> frame;

  //......
};

 The handle of the decoder is hung in the std::unique_ptr<AudioDecoder::EncodedAudioFrame> frame, which means that each message specifies its own decoder handle.

In the function of receiving packets into the queue, it will call:

std::unique_ptr<NackTracker> nack_: Update the received packet information to facilitate the determination of packets that need to be retransmitted by nack.

std::unique_ptr<StatisticsCalculator> stats_: The time interval between update messages, used to determine the jitterbuffer cache time.

• NetEqImpl::GetAudio

Description of key function processing: 

NetEqImpl::ExtractPackets: Get RTP packets, to be decoded

 packet_list->front().frame->Decode for audio decoding

 

Encapsulate output audio PCM frame data.