webrtc视频处理（待续）

一、远端视频数据处理

1）创建RTP接收线程

RTP接收线程是收到http承载的sdp命令中的m行，根据m行的信息创建的。

Received session description :{
   "sdp" : "v=0\r\no=- 3138862463696765007 2 IN IP4 127.0.0.1\r\n
            s=-\r\nt=0 0\r\n
			a=group:BUNDLE audio video\r\n
			a=msid-semantic: WMS stream_label\r\n
			m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126\r\n
			c=IN IP4 0.0.0.0\r\n
			a=rtcp:9 IN IP4 0.0.0.0\r\n
			a=ice-ufrag:xxx\r\n
			a=ice-pwd:xxxx\r\n
			a=ice-options:trickle\r\n
			a=fingerprint:sha-256 xxxx\r\n
			a=setup:active\r\n
			a=mid:audio\r\n
			a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\n
			a=sendrecv\r\n
			a=rtcp-mux\r\n
			a=rtpmap:111 opus/48000/2\r\n
			a=rtcp-fb:111 transport-cc\r\n
			a=fmtp:111 minptime=10;useinbandfec=1\r\n
			a=rtpmap:103 ISAC/16000\r\n
			a=rtpmap:104 ISAC/32000\r\n
			a=rtpmap:9 G722/8000\r\n
			a=rtpmap:102 ILBC/8000\r\n
			a=rtpmap:0 PCMU/8000\r\n
			a=rtpmap:8 PCMA/8000\r\n
			a=rtpmap:106 CN/32000\r\n
			a=rtpmap:105 CN/16000\r\n
			a=rtpmap:13 CN/8000\r\n
			a=rtpmap:110 telephone-event/48000\r\n
			a=rtpmap:112 telephone-event/32000\r\n
			a=rtpmap:113 telephone-event/16000\r\n
			a=rtpmap:126 telephone-event/8000\r\n
			a=ssrc:1737881718 cname:xxxx\r\n
			a=ssrc:1737881718 msid:stream_label audio_label\r\n
			a=ssrc:1737881718 mslabel:stream_label\r\n
			a=ssrc:1737881718 label:audio_label\r\n
			m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 124 125\r\n
			c=IN IP4 0.0.0.0\r\n
			a=rtcp:9 IN IP4 0.0.0.0\r\n
			a=ice-ufrag:xxxx\r\n
			a=ice-pwd:+xxxx\r\n
			a=ice-options:trickle\r\n
			a=fingerprint:sha-256 xxxx\r\n
			a=setup:active\r\n
			a=mid:video\r\n
			a=extmap:2 urn:ietf:params:rtp-hdrext:toffset\r\n
			a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\n
			a=extmap:4 urn:3gpp:video-orientation\r\n
			a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\n
			a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\n
			a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\n
			a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\n
			a=sendrecv\r\n
			a=rtcp-mux\r\n
			a=rtcp-rsize\r\n
			a=rtpmap:96 H264/90000\r\n
			a=rtcp-fb:96 ccm fir\r\n
			a=rtcp-fb:96 nack\r\n
			a=rtcp-fb:96 nack pli\r\n
			a=rtcp-fb:96 goog-remb\r\n
			a=rtcp-fb:96 transport-cc\r\n
			a=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\n
			a=rtpmap:97 rtx/90000\r\n
			a=fmtp:97 apt=96\r\n
			a=rtpmap:98 VP8/90000\r\n
			a=rtcp-fb:98 ccm fir\r\n
			a=rtcp-fb:98 nack\r\n
			a=rtcp-fb:98 nack pli\r\n
			a=rtcp-fb:98 goog-remb\r\n
			a=rtcp-fb:98 transport-cc\r\n
			a=rtpmap:99 rtx/90000\r\n
			a=fmtp:99 apt=98\r\n
			a=rtpmap:100 VP9/90000\r\n
			a=rtcp-fb:100 ccm fir\r\n
			a=rtcp-fb:100 nack\r\n
			a=rtcp-fb:100 nack pli\r\n
			a=rtcp-fb:100 goog-remb\r\n
			a=rtcp-fb:100 transport-cc\r\n
			a=rtpmap:101 rtx/90000\r\n
			a=fmtp:101 apt=100\r\n
			a=rtpmap:127 red/90000\r\n
			a=rtpmap:124 rtx/90000\r\n
			a=fmtp:124 apt=127\r\n
			a=rtpmap:125 ulpfec/90000\r\n
			a=ssrc-group:FID 489266959 3672397520\r\n
			a=ssrc:489266959 cname:xxxx\r\n
			a=ssrc:489266959 msid:stream_label Ultrasonic\r\n
			a=ssrc:489266959 mslabel:stream_label\r\n
			a=ssrc:489266959 label:Ultrasonic\r\n
			a=ssrc:3672397520 cname:xxxx\r\n
			a=ssrc:3672397520 msid:stream_label Ultrasonic\r\n
			a=ssrc:3672397520 mslabel:stream_label\r\n
			a=ssrc:3672397520 label:Ultrasonic\r\n
			a=ssrc-group:FID 3677845131 3184560832\r\n
			a=ssrc:3677845131 cname:xxxx\r\n
			a=ssrc:3677845131 msid:stream_label camera_0\r\n
			a=ssrc:3677845131 mslabel:stream_label\r\n
			a=ssrc:3677845131 label:camera_0\r\n
			a=ssrc:3184560832 cname:xxxx\r\n
			a=ssrc:3184560832 msid:stream_label camera_0\r\n
			a=ssrc:3184560832 mslabel:stream_label\r\n
			a=ssrc:3184560832 label:camera_0\r\n",
   "type" : "answer"
}

->RtpVideoStreamReceiver :: RtpVideoStreamReceiver
->VideoReceiveStream :: VideoReceiveStream 
->Call:: CreateVideoReceiveStream 
->WebRtcVideoChannel :: WebRtcVideoReceiveStream :: RecreateWebRtcVideoStream
->WebRtcVideoChannel :: WebRtcVideoReceiveStream :: WebRtcVideoReceiveStream
->WebRtcVideoChannel :: AddRecvStream
->WebRtcVideoChannel :: AddRecvStream
->BaseChannel :: AddRecvStream_w   //创建接收流资源。
->BaseChannel :: UpdateRemoteStreams_w 
VideoChannel :: SetRemoteContent_w  //根据SDP协议，创建接收的音视频、数据报文资源

2）创建解码线程

解码器是当有rtp报文输入的时候，动态创建的。

->H264DecoderImpl::InitDecode
->VCMGenericDecoder::InitDecode          //初始化解码器
->VCMCodecDataBase::CreateAndInitDecoder //根据PT查找map表，确定解码格式，创建对应解码器
->VCMCodecDataBase::GetDecoder           //每来一包，判断一下视频解码器是否创建，已经创建，用原有的。没有创建，创建新解码器
->VideoReceiver::Decode
->VideoReceiver::Decode
->VideoReceiveStream::Decode             //解码线程读取RTP数据
VideoReceiveStream::DecodeThreadFunction

3）创建渲染线程

二，本端视频数据处理

1）创建视频capture线程

VideoCapturer::OnFrame
WebRtcVideoCapturer::OnFrame
videocapturemodule::VideoCaptureImpl::DeliverCapturedFrame
videocapturemodule::VideoCaptureImpl::IncomingFrame
videocapturemodule::CaptureSinkFilter::ProcessCapturedFrame
videocapturemodule::CaptureInputPin::Receive

2）创建编码线程

3）创建发送线程

webrtc::RTPSenderVideo::SendVideo
webrtc::RTPSender::SendOutgoingData
webrtc::ModuleRtpRtcpImpl::SendOutgoingData
webrtc::PayloadRouter::OnEncodedImage
webrtc::internal::VideoSendStreamImpl::OnEncodedImage
webrtc::VideoStreamEncoder::OnEncodedImage
webrtc::VCMEncodedFrameCallback::OnEncodedImage
webrtc::H264EncoderImpl::Encode
webrtc::VCMGenericEncoder::Encode
webrtc::vcm::VideoSender::AddVideoFrame
webrtc::VideoStreamEncoder::EncodeVideoFrame
webrtc::VideoStreamEncoder::EncodeTask::Run
rtc::TaskQueue::Impl::RunPendingTasks
rtc::TaskQueue::Impl::ThreadState::RunThreadMain
rtc::TaskQueue::Impl::ThreadMain
rtc::PlatformThread::Run
rtc::PlatformThread::StartThread

webrtc视频处理（待续）

猜你喜欢