I. Introduction
Setting the rotation angle is a relatively niche requirement. If the video itself has a rotation angle, it will be rotated to the corresponding angle when decoding and playing. For example, videos shot on mobile phones are generally rotated 90 degrees. Yes, if the video file is opened on a computer, some early players may play it while lying down, because when the early players were designed, mobile phones were not yet popular, and this demand was not considered. Only with the development of society has this new demand gradually emerged.
In addition to correcting the rotation angle of source video recognition, you can also actively specify the rotation angle to play the video. This is also the need of some users. For example, some users have large-screen machines that are vertical and hope that the video can be played by setting the rotation angle. After checking the multimedia classes in Qt, almost none of them provide an interface for rotation angles. The approach in Qt is to put the playback in a container, such as QGraphicsItem or qml, and their containers provide a good rotation interface. If it is related to the ffmpeg kernel, rotation is basically achieved through filters, and vlc/mpv itself provides a rotation interface. By comparing the rotation interfaces of various kernels, we found that the general approach is to rotate on the drawing side, rather than on the source data side. The ffmpeg filter rotates on the source data side. Source rotation is relatively resource intensive. , and the drawing side is basically handled through opengl.
2. Effect drawing
3. Experience address
- Domestic site: https://gitee.com/feiyangqingyun
- International site: https://github.com/feiyangqingyun
- Personal work: https://blog.csdn.net/feiyangqingyun/article/details/97565652
- Experience address: https://pan.baidu.com/s/1d7TH_GEYl5nOecuNlWJJ7g Extraction code: 01jf File name: bin_video_demo.
- Video homepage: https://space.bilibili.com/687803542
4. Functional features
4.1. Basic functions
- Supports various audio and video file formats, such as mp3, wav, mp4, asf, rm, rmvb, mkv, etc.
- Supports local camera devices and local desktop collection, and supports multiple devices and multiple screens.
- Supports various video streaming formats, such as rtp, rtsp, rtmp, http, udp, etc.
- Local audio and video files and network audio and video files can automatically identify file length, playback progress, volume, mute status, etc.
- Files can specify the playback position, adjust the volume, set the mute status, etc.
- Supports double-speed playback of files, with optional speeds of 0.5x, 1.0x, 2.5x, 5.0x, etc., which is equivalent to slow playback and fast playback.
- Supports starting, stopping, pausing, and continuing playback.
- Supports capturing screenshots, you can specify the file path, and you can choose whether to automatically display a preview after the capture is completed.
- Support video storage, manually start recording, stop recording, some kernels support continuing recording after pausing recording, skipping the parts that are not needed for recording.
- Supports mechanisms such as senseless switching loop playback and automatic reconnection.
- Provides signals such as successful playback, playback completion, decoded picture received, captured picture received, video size change, recording status change, etc.
- Multi-thread processing, one decoding thread, no stuck on the main interface.
4.2. Features
- Supports multiple decoding kernels at the same time, including qmedia kernel (Qt4/Qt5/Qt6), ffmpeg kernel (ffmpeg2/ffmpeg3/ffmpeg4/ffmpeg5/ffmpeg6), vlc kernel (vlc2/vlc3), mpv kernel (mpv1/mp2), and mdk kernel , Hikvision sdk, easyplayer kernel, etc.
- With a very complete multiple base class design, adding a new decoding kernel only requires a very small amount of code to apply the entire mechanism, making it easy to expand.
- Supports multiple screen display strategies at the same time, automatic adjustment (the original resolution is smaller than the size of the display control, it will be displayed according to the original resolution, otherwise it will be proportionally scaled), proportional scaling (always proportional scaling), stretch filling (always stretched and filled) ). Three picture display strategies are supported in all cores and in all video display modes.
- Supports multiple video display modes at the same time, handle mode (the incoming control handle is handed over to the other party for drawing control), drawing mode (the callback gets the data and then converts it to QImage for drawing with QPainter), GPU mode (the callback gets the data and then converts it to yuv) QOpenglWidget drawing).
- Supports multiple hardware acceleration types, ffmpeg can choose dxva2, d3d11va, etc., vlc can choose any, dxva2, d3d11va, mpv can choose auto, dxva2, d3d11va, mdk can choose dxva2, d3d11va, cuda, mft, etc. Different system environments have different types to choose from. For example, Linux systems have vaapi and vdpau, and macos systems have videotoolbox.
- The decoding thread and the display form are separated, and any decoding core can be specified to be mounted to any display form and switched dynamically.
- Supports shared decoding thread, which is enabled by default and processed automatically. When the same video address is recognized, a decoding thread is shared, which can greatly save network traffic and streaming pressure on the other device in a network video environment. The top domestic video manufacturers all adopt this strategy. In this way, as long as one video stream is pulled, it can be shared to dozens or hundreds of channels for display.
- Automatically identify the video rotation angle and draw it. For example, videos shot on mobile phones are generally rotated 90 degrees. They must be automatically rotated during playback, otherwise they will be upside down by default.
- Automatically identify changes in resolution during video stream playback and automatically adjust the size on the video controls. For example, the camera can dynamically configure the resolution during use, and the corresponding video controls must also respond synchronously when the resolution is changed.
- Audio and video files automatically switch and play in a loop without any perception, and there will be no visible switching traces such as a black screen during switching.
- The video control also supports any decoding core, any picture display strategy, and any video display mode.
- The video control floating bar supports three modes: handle, drawing, and GPU at the same time, and non-absolute coordinates can be moved around.
- The local camera device supports specifying the device name, resolution, and frame rate for playback.
- Local desktop collection supports setting the collection area, offset value, specified desktop index, frame rate, simultaneous collection of multiple desktops, etc. It also supports specifying the window title to collect fixed windows.
- Recording files also support open video files, local cameras, local desktops, network video streams, etc.
- Instant response to opening and closing, whether it is opening a non-existent video or network stream, detecting the presence of a device, waiting for a timeout during reading, and immediately interrupting the previous operation and responding after receiving a shutdown command.
- Supports opening various picture files, and supports drag-and-drop playback of local audio and video files.
- The video streaming communication method can be tcp/udp. Some devices may only provide a certain protocol communication such as tcp. You need to specify the protocol method to open.
- You can set the connection timeout (timeout for video stream detection) and read timeout (timeout during collection).
- Supports frame-by-frame playback, provides previous frame/next frame function interface, and can view the collected images frame by frame.
- Audio files automatically extract album information such as title, artist, album, and album cover, and automatically display the album cover.
- The video response has extremely low latency of about 0.2s, and the extremely fast response to opening the video stream is about 0.5s, which has been specially optimized.
- Supports H264/H265 encoding (more and more surveillance cameras now use H265 video stream format) to generate video files, and automatically recognizes and switches the encoding format internally.
- Supports playback of video streams containing special characters in user information (for example, characters such as +#@ in user information), with built-in parsing and escaping processing.
- Supports filters, various watermarks and graphic effects, supports multiple watermarks and images, and can write OSD tag information and various graphic information to MP4 files.
- Supports various audio formats in video streams, including AAC, PCM, G.726, G.711A, G.711Mu, G.711ulaw, G.711alaw, MP2L2, etc. It is recommended to choose AAC for the best cross-platform compatibility.
- The kernel ffmpeg uses pure qt+ffmpeg decoding and does not rely on third-party drawing and playback such as SDL. The gpu drawing uses qopenglwidget and the audio playback uses qaudiooutput.
- Kernel ffmpeg and kernel mdk support Android, among which mdk supports Android hard decoding, and the performance is very brutal.
- You can switch audio and video tracks, that is, program channels. Maybe the ts file contains multiple audio and video program streams. You can set which one to play respectively. You can set it before playing and set it dynamically during playback.
- The video rotation angle can be set before playback and dynamically changed during playback.
- The video control floating bar comes with functions such as starting and stopping recording, muting the sound, taking screenshots, and closing the video.
- The audio component supports sound waveform value data analysis. Waveform curves and columnar sound bars can be drawn based on this value. Sound amplitude signals are provided by default.
- Labels and graphic information support three drawing methods: drawing to mask layer, drawing to picture, and source drawing (corresponding information can be stored in a file).
- By passing in a url address, the address can bring communication protocol, resolution, frame rate and other information without any other settings.
- Three strategies are supported for saving videos to files: automatic processing, file only, and all transcoding. The transcoding strategy supports automatic identification, conversion to 264, and conversion to 265. Encoding saving supports specified resolution scaling or equal scaling. For example, if you have requirements on the size of the saved file, you can specify scaling before saving.
- Supports encrypted saving files and decrypted playback files, and you can specify the secret key text.
- The provided monitoring layout class supports 64-channel simultaneous display, and also supports various special-shaped layouts, such as 13-channel, 6-row and 2-column layout on mobile phones. Various layouts can be freely defined.
- It supports electronic magnification. Switch to the electronic magnification mode on the floating bar, select the area that needs to be enlarged on the screen, and it will automatically enlarge after the selection. It can be reset by switching the magnification mode again.
- Extremely detailed printing information prompts in each component, especially error message prompts, and a unified printing format for packaging. It is extremely convenient and useful to test the complex equipment environment on site, which is equivalent to pinpointing the specific channel and step that went wrong.
- At the same time, simple examples, video players, multi-screen video monitoring, monitoring playback, frame-by-frame playback, multi-screen rendering and other separate form examples are provided to specifically demonstrate how to use the corresponding functions.
- Monitoring playback can select different manufacturer types, playback time periods, user information, and designated channels. Support switching playback progress.
- You can select a sound card to play sound from the sound card device drop-down box, and provide a corresponding function interface for switching sound cards.
- It supports compilation to mobile app and provides a special mobile app layout interface, which can be used as video surveillance on mobile phones.
- The code framework and structure are optimized to the best, with powerful performance, detailed annotations, and continuous iterative updates and upgrades.
- The source code supports windows, linux, mac, android, etc., and supports various domestic Linux systems, including but not limited to Tongxin UOS/Winning Kirin/Galaxy Kirin, etc. Embedded linux is also supported.
- The source code supports Qt4, Qt5, and Qt6 and is compatible with all versions.
4.3. Video controls
- Any number of osd label information can be added dynamically. The label information includes name, visible or not, font size, text text, text color, background color, label image, label coordinates, label format (text, date, time, date time, picture) , label position (upper left corner, lower left corner, upper right corner, lower right corner, center, custom coordinates).
- Any amount of graphic information can be added dynamically, which is very useful. For example, the graphic area information analyzed by the artificial intelligence algorithm can be directly sent to the video control. Graphic information supports arbitrary shapes and is drawn directly on the original image using absolute coordinates.
- Graphic information includes name, border size, border color, background color, rectangular area, path set, point coordinate set, etc.
- Each graphic information can specify one or more of the three areas, and the specified areas will be drawn.
- Built-in floating bar control, the floating bar position supports top, bottom, left and right.
- The parameters of the floating bar control include margins, spacing, background transparency, background color, text color, pressed color, position, button icon code set, button name identification set, and button prompt information set.
- The row of tool buttons in the floating bar control can be customized. Through the structure parameter settings, the icon can choose graphic fonts or custom pictures.
- The floating bar button internally implements functions such as video switching, capturing screenshots, mute switching, and closing videos. You can also add your own corresponding functions in the source code.
- The floating bar button corresponds to the button that has implemented the function, and has corresponding icon switching processing. For example, after pressing the video button, it will switch to the icon that is recording, and after the sound button is switched, it will become the mute icon, and it will be restored by switching again.
- When the floating bar button is clicked, it is sent out with a unique name as a signal, and can be associated with response processing by itself.
- Prompt information can be displayed in the blank area of the floating bar. The current video resolution is displayed by default. Information such as frame rate and bit stream size can be added.
- Video control parameters include border size, border color, focus color, background color (default transparent), text color (default global text color), fill color (the blank space outside the video is filled with black), background text, background image (if set Pictures are taken first), whether to copy pictures, zoom display mode (automatic adjustment, proportional scaling, stretch filling), video display mode (handle, drawing, GPU), enable floating bar, floating bar size (horizontal is height, vertical is is the width), the position of the floating bar (top, bottom, left, right).
5. Related codes
QString FilterHelper::getFilter(int rotate)
{
//0=逆时针旋转90度并垂直翻转
//1=顺时针旋转90度
//2=逆时针旋转90度
//3=顺时针旋转90度并水平翻转
//hflip=水平翻转
//vflip=垂直翻转
//180度=先水平翻转再垂直翻转(查阅ffplay代码获知)
QString filter;
if (rotate == 90) {
filter = "transpose=1";
} else if (rotate == 180) {
filter = "hflip,vflip";
} else if (rotate == 270) {
filter = "transpose=2";
} else if (rotate > 0) {
filter = QString("rotate=%1*PI/180").arg(rotate);
}
return filter;
}
void FFmpegThread::setRotate(int rotate)
{
if (rotate < 0) {
return;
}
//只有变化了才需要触发
bool changed = (this->rotate != rotate);
this->rotate = rotate;
if (isOk && rotate >= 0 && changed) {
//旋转通过滤镜实现效果
this->resetFilter();
QMetaObject::invokeMethod(this, "receiveSizeChanged");
}
}
void MdkThread::setRotate(int rotate)
{
//限定90度的倍数才能旋转
if (this->rotate != rotate && rotate >= 0 && rotate % 90 == 0) {
this->rotate = rotate;
mdkPlayer->rotate(rotate, mdkWidget);
QMetaObject::invokeMethod(this, "receiveSizeChanged");
}
}
void MpvThread::setRotate(int rotate)
{
if (this->rotate != rotate && rotate >= 0) {
this->rotate = rotate;
setOption("video-rotate", rotate);
QMetaObject::invokeMethod(this, "receiveSizeChanged");
}
}
void QtavThread::setRotate(int rotate)
{
//限定90度的倍数才能旋转
if (this->rotate != rotate && rotate >= 0 && rotate % 90 == 0) {
this->rotate = rotate;
videoOutput->setOrientation(rotate);
QMetaObject::invokeMethod(this, "receiveSizeChanged");
}
}