Beginners may not know the transmission process of the audio and video project code stream, here is this project as an example to take you to understand
background
This project is based on Junzheng's ptz ipc
PTZ: In the security monitoring application, it is the abbreviation of Pan/Tilt/Zoom, which represents the omni-directional (left-right/up-down) movement of the pan/tilt and the lens zoom and zoom control.
The backend server uses Tuya's solution.
flow chart
The video stream is collected by the sensor and transmitted to the ISP, and the video stream format collected at this time is the IVS format
After ISP tuning, the IVS format video stream is transmitted to FrameSource
There are two in FrameSource, the main stream and the secondary stream, the main stream is 1920 x 1080p, and the secondary stream is 640 x 360p.
There is also a picture capture process in FrameSource, grab a YUV image and scan the QR code in the app to configure the network, and get the ssid and psk of the mobile phone connected to the wifi
The main stream is transmitted in channel0, and the YUV video stream is encoded into H.264 format video stream after ENCODE encoding.
What is transmitted in channel1 is the secondary code stream, and the IVS image correlation algorithm is carried out in this channel, and then coded. In chnnel1, jpeg images are also captured from the encoded video stream as a camera function.
p2p is made by Tuya Smart, using MQTT to transfer to the server, and then using MQTT to transfer to the mobile app
There is a module dedicated to video screen decoding in the mobile phone, we don’t need to do it
ISP
Image tuning
The main function in this area is to adjust the image. Mainly some parameter settings, image rotation and so on.
And the configuration of the sensor driver
FrameSource
Video source module
The video source is the image data source of the IMP system. You can set the image resolution, crop, zoom and other attributes, as well
as the back-end noise reduction function.
FrameSource is a data stream related concept, which can set the image resolution, format, etc., and
provide the original image to the back end
OSD
Time watermark function
IVS
Image-related algorithm modules
Human-shaped clock chasing, human-shaped detection and alarm algorithms are used in this project
p2p
The peer to peer
is done by Tuya Smart, using MQTT transmission
The device and the mobile phone are directly encoded on the same network segment and transmitted to the mobile phone without going through the server