I. Introduction
The local camera streaming is similar to the local desktop streaming, except that the source of the collection device is replaced by a local camera device instead of a desktop, and the other codes are exactly the same. When collecting real-time video from a local camera, it should be noted that if you set the resolution and frame rate, it must be supported by the device itself. If it does not support it, stop. For example, the maximum resolution of the device itself is 1280x720, and you actively set the resolution to capture 1080x720 The image, which cannot be collected, will fail. If it is not set by default, it will generally use 640x480 resolution and 25 frame rate to collect. The command line to capture the local camera device is ffmpeg -f dshow -i video="USB Video Device":audio="Microphone (USB Audio Device)" -vcodec libx264 -preset:v ultrafast -tune:v zerolatency -f rtsp rtsp:/ /192.168.0.110:6907/stream , you can specify to bring a microphone, so that audio and video are available.
General collection steps:
- Find the format av_find_input_format, parameter dshow/v4l2/avfoundation
- Open the desktop avformat_open_input, parameter video=USB Video Device
- Find video stream av_find_best_stream
- Find decoder avcodec_find_decoder
- Open the decoder avcodec_open2
- Loop read av_read_frame
- Decode video avcodec_send_packet/avcodec_receive_frame
- close release avcodec_free_context/avformat_close_input
The general steps of push flow:
- create output avformat_alloc_output_context2
- Create video stream avformat_new_stream
- Open the output avio_open, and fill in the complete address of the push stream as a parameter
- Write start character avformat_write_header
- Write frame data av_interleaved_write_frame
- close release avio_close/avformat_free_context
2. Rendering
3. Experience address
- Domestic site: https://gitee.com/feiyangqingyun
- International site: https://github.com/feiyangqingyun
- Personal works: https://blog.csdn.net/feiyangqingyun/article/details/97565652
- Experience address: https://pan.baidu.com/s/1d7TH_GEYl5nOecuNlWJJ7g Extraction code: 01jf File name: bin_video_push.
4. Related codes
void FFmpegThread::initInputFormat()
{
//本地摄像头/桌面录屏
if (videoType == VideoType_Camera) {
#if defined(Q_OS_WIN)
//ifmt = av_find_input_format("vfwcap");
ifmt = av_find_input_format("dshow");
#elif defined(Q_OS_LINUX)
//可以打开cheese程序查看本地摄像头(如果是在虚拟机中需要设置usb选项3.1)
//ifmt = av_find_input_format("v4l2");
ifmt = av_find_input_format("video4linux2");
#elif defined(Q_OS_MAC)
ifmt = av_find_input_format("avfoundation");
#endif
} else if (videoType == VideoType_Desktop) {
#if defined(Q_OS_WIN)
ifmt = av_find_input_format("gdigrab");
#elif defined(Q_OS_LINUX)
ifmt = av_find_input_format("x11grab");
#elif defined(Q_OS_MAC)
ifmt = av_find_input_format("avfoundation");
#endif
}
}
bool FFmpegThread::initInput()
{
//实例化格式处理上下文
formatCtx = avformat_alloc_context();
//设置超时回调(有些不存在的地址或者网络不好的情况下要卡很久)
formatCtx->interrupt_callback.callback = FFmpegHelper::avinterruptCallBackFun;
formatCtx->interrupt_callback.opaque = this;
//打开输入(通过标志位控制回调那边做超时判断)
//其他地方调用 formatCtx->url formatCtx->filename 可以拿到设置的地址(两个变量值一样)
tryOpen = true;
QByteArray urlData = VideoHelper::getRightUrl(videoType, videoUrl).toUtf8();
int result = avformat_open_input(&formatCtx, urlData.data(), ifmt, &options);
tryOpen = false;
if (result < 0) {
debug("打开出错", "错误: " + FFmpegHelper::getError(result));
return false;
}
//根据自己项目需要开启下面部分代码加快视频流打开速度
//开启后由于值太小可能会出现部分视频流获取不到分辨率
if (decodeType == DecodeType_Fastest && videoType == VideoType_Rtsp) {
//接口内部读取的最大数据量(从源文件中读取的最大字节数)
//默认值5000000导致这里卡很久最耗时(可以调小来加快打开速度)
formatCtx->probesize = 50000;
//从文件中读取的最大时长(单位为 AV_TIME_BASE units)
formatCtx->max_analyze_duration = 5 * AV_TIME_BASE;
//内部读取的数据包不放入缓冲区
//formatCtx->flags |= AVFMT_FLAG_NOBUFFER;
//设置解码错误验证过滤花屏
//formatCtx->error_recognition |= AV_EF_EXPLODE;
}
//获取流信息
result = avformat_find_stream_info(formatCtx, NULL);
if (result < 0) {
debug("找流失败", "错误: " + FFmpegHelper::getError(result));
return false;
}
//解码格式
formatName = formatCtx->iformat->name;
//某些格式比如视频流不做音视频同步(响应速度快)
if (formatName == "rtsp" || videoUrl.endsWith(".sdp")) {
useSync = false;
}
//设置了最快速度则不启用音视频同步
if (decodeType == DecodeType_Fastest) {
useSync = false;
}
//有些格式不支持硬解码
if (formatName.contains("rm") || formatName.contains("avi") || formatName.contains("webm")) {
hardware = "none";
}
//本地摄像头设备解码出来的直接就是yuv显示不需要硬解码
if (videoType == VideoType_Camera || videoType == VideoType_Desktop) {
useSync = false;
hardware = "none";
}
//过低版本不支持硬解码
#if (FFMPEG_VERSION_MAJOR < 3)
hardware = "none";
#endif
//获取文件时长(这里获取到的是秒)
double length = (double)formatCtx->duration / AV_TIME_BASE;
duration = length * 1000;
this->checkVideoType();
//有时候网络地址也可能是纯音频
if (videoType == VideoType_FileHttp) {
onlyAudio = VideoHelper::getOnlyAudio(videoUrl, formatName);
}
if (getIsFile()) {
//文件必须要音视频同步
useSync = true;
//发送文件时长信号
emit receiveDuration(duration > 0 ? duration : 0);
}
QString msg = QString("格式: %1 时长: %2 秒 加速: %3").arg(formatName).arg(duration / 1000).arg(hardware);
debug("媒体信息", msg);
return true;
}
5. Features
5.1 File streaming
- Specify the network card and listening port, and receive network requests to push various files such as audio and video.
- Real-time statistics display the number of visits corresponding to each file, the total number of visits, and the number of visits from different IP addresses.
- Multiple modes can be specified, 0-direct play, 1-download play.
- Real-time printing displays various sending and receiving request and response data.
- Each file corresponds to an MD5-encrypted unique identifier, which is used to request the address suffix to distinguish which file to access.
- Support various browsers (Google chromium/Microsoft edge/Firefox firefox, etc.), various players (vlc/mpv/ffplay/potplayer/mpchc, etc.) to open requests.
- During the playback process, the playback progress can be switched arbitrarily, and double-speed playback is supported.
- The name history of the file that needs to be streamed is automatically stored and opened to load the application.
- Switch files to get the access address, and automatically copy the address to the clipboard for direct pasting and testing.
- Extremely low CPU usage, 128-channel 1080P simultaneous streaming is less than 1% CPU usage, asynchronous data sending mechanism.
- Pure QTcpSocket communication, does not depend on the streaming media service program, the core source code is less than 500 lines, with detailed annotations and complete functions.
- Support any version of Qt4/Qt5/Qt6, support any system (windows/linux/macos/android/embedded linux, etc.).
5.2 Network streaming
- Support various local video files and network video files.
- Support various network video streams, webcams, protocols including rtsp, rtmp, http.
- It supports streaming of local camera devices, and the resolution and frame rate can be specified.
- Supports streaming of local desktops, specifying screen area and frame rate, etc.
- Automatically start the streaming media service program, the default mediamtx (formerly rtsp-simple-server), you can choose srs, EasyDarwin, LiveQing, ZLMediaKit, etc.
- You can switch and preview video files in real time.
- The clarity and quality of streaming can be adjusted.
- Files, directories and addresses can be added dynamically.
- Video files are automatically streamed in a loop. If the video source is a video stream, it will automatically reconnect after being disconnected.
- The network video stream is automatically reconnected, and the reconnection is successful and the streaming will continue automatically.
- The real-time performance of network video streaming is extremely high, and the delay is extremely low. The delay time is about 100ms.
- After pushing the stream, in addition to using the rtmp address to access, it also supports direct hls/webrtc access, and you can directly open the browser to view the real-time screen.
- Support any version of Qt4/Qt5/Qt6, support any system (windows/linux/macos/android/embedded linux, etc.).