1. It is not recommended to use QPainter to draw video

Display video on Qidget, usually through paintEvent, because QPainter is not used to render video, it is used to draw GUI by itself, it uses CPU rendering instead of GPU, lacks graphics card acceleration, and its performance cannot meet the requirements of drawing video. In addition, video drawing should not use event-driven, because using event-driven will make the drawing operation run in the main thread, which may block the UI event response, which is obviously unreasonable. In addition, there are the following defects:

YUV to RGB conversion consumes a lot of CPU;
QPainter's large-area drawing is not efficient. It has been tested. If it is only a small preview window, it is okay to draw. If it is a large window, the larger the window, the more CPU will be consumed. As for the full screen, it will be stuck.

2. It is recommended to use QOpenGLWidget to draw video

Nowadays, embedded boards with better performance generally have GPUs. If they support OpenGL, it is most recommended to use the official QOpenGLWidget that comes with Qt for drawing.

Specifically, it is the class that displays the video, let it inherit QOpenGLWidget, so that GPU rendering is used, and then the paintEvent function is reused, so that the CPU is not consumed much.

3. It is recommended to use third-party libraries such as SDL and OpenCV for CPU drawing

Get the widget's window handle (it seems to be the return value of the winid() function) or the window position, and use other tools (opengl, directx, etc., I recommend trying SDL in linux, the interface is more friendly) to draw by yourself. If it is an embedded board, libraries such as SDL need to be cross-compiled.

The benefits of this article, free to receive Qt development learning materials package, technical video, including (C++ language foundation, introduction to Qt programming, QT signal and slot mechanism, QT interface development-image drawing, QT network, QT database programming, QT project combat, QSS, OpenCV, Quick module, interview questions, etc.) ↓↓↓↓↓↓See below↓↓Click on the bottom of the article to receive the fee↓↓

4. If only the optimization method under QPainter can be used

If the embedded board does not have a GPU, does not support OpenGL, and can only draw with the CPU, and does not want to use a third-party library due to technical reasons or personal preferences, etc., it can only use QPainter. There are several optimization methods as follows, especially double buffering Mechanics and multi-threaded rendering.

4.1 Drawing with double buffering mechanism

The method is to first draw the content to be drawn in a picture, and then draw the picture to the control at one time, instead of rendering in the drawing event. I won’t go into details here, please see my previous blog for details: Qt Drawing Advancement II: Double Buffering Mechanism and Examples

4.2 Update drawing after multi-threaded rendering

You can use multi-threads to draw in the image memory, then send QPixmap through the signal, and then draw in the UI main thread.

4.3 Split drawing data into hash table, trade space for time

I won’t go into details here, please see the blog for details: QT 2d drawing optimization (1)

4.4 Optimization using QPainterPath

Batch all paint calls of the same type, one of which uses the same brush and pen. To do this, use the class QPainterPath and add the required objects using functions like addRect. Make sure to do this outside of the paint function!

4.5 Define drawing area

We only need to draw a small part of the area without drawing the data of the entire icon, so we need to define a viewport.

4.6 Optimize drawing instructions

Clipping: Clip data that does not need to be drawn before rendering
Partial update: do not render where there is no change
Split cache: layer or divide into multiple objects, in short, separate variable elements from less variable elements, and cache less variable elements

4.7 Other Notes

For the class that displays the video, set the parent class to nullptr, so that the CPU usage will decrease;
QPainter should be set to smooth mode;
It is better to choose QWidget's drawing event drawing instead of using QLabel drawing, QLabel is only suitable for displaying a single picture;
If you draw multiple pictures, it is better to use Qidget's drawing event drawing instead of QLabel's setPixmap;
The conversion from QImage to QPixmap consumes more resources. Displaying QImage on QLabel is easy to get stuck;
If the decoded data is placed in the cache through memcpy, it will consume more CPU.

V. Summary

If Qt needs to draw video at high frequency, choose a solution according to whether the embedded board has a GPU:

If the embedded board has a GPU, use the EGLFS platform plug-in, and it is most recommended to use the official QOpenGLWidget that comes with Qt for drawing;
If there is no GPU, generally only the LinuxFB platform plug-in can be used, and third-party libraries such as SDL and OpenCV are recommended;
If you don't want to use a third-party library, you can only use QPainter to achieve drawing, and you must pay attention to optimization, otherwise the CPU usage is easy to be high.