翻译自What about Qt 3D in Qt 6?
原文作者：Sean Harmer
In Qt 6, we hope to Qt 3D can be improved in many aspects, this article will focus on several main directions: to support the work of the renderer caches and modern graphics API, such as the Vulkan, Metal and DirectX 12.
Renderer cache
Qt 3D operation is based on two prior data structures:
Scene Graph- description of the scene;
Frame Graph- described rendering Scene Graph approach.
Each rendering a picture, we have to do a lot of work in order to describe abstract scene graph and frame graph of conversion and transfer calls to the GPU to draw the bottom. Briefly, the following steps:
- 1. identifying the respective traverse frame graph and the rendering phase. Each stage consists of rendering target (screen or FBO); which camera to use; which window to use; what parts should draw the scene graph; GPU set a specific state (for example, disable the depth test or write, or enable mask testing) .
- 2. Step 1 for each rendering stage requires screening entity we are concerned in the scene graph.
- 3. Each entity and select the appropriate phase current rendering shader. Entities may be used at different stages of different shaders, e.g., using a simple early Z fragment shader performs padding or shadow map generation, full illumination used to achieve the final effect on the shader screen.
- 4. The combined uniform variables (for custom shader variables).
- 5. All of this information is bound to RenderCommands.
- 6. Once all phases are completed, we will submit RenderCommands to OpenGL through a separate thread because OpenGL long history, its thread is very picky.
- 7.OpenGL submit each thread iteration stage and rendering commands contained therein, describe them translated into OpenGL format from our middle, and assigned to the original OpenGL function calls.
All this makes Qt 3D becomes very flexible, but at the expense of runtime performance. Usual method can significantly improve performance by caching nothing more than to avoid unnecessary rendering overhead. Theoretically, we can get upgrade by caching some intermediate results. In fact, many things to consider, such as how to combine dynamic rendering mode, rendering cache is really difficult to do.
Of which there are too many things that can affect the appearance of the rendered scene need to track, but also to figure out if what the minimum set of tasks between different screens after some attribute update must be re-drawn Yes. We've added some tracking in Qt 5 version, but to do so requires more complete reconstruction.
Before describing in detail the work we have done in this regard, I would like to discuss another issue:
Modern graphics API
So far, Qt Quick (basically) has been completely built on top of OpenGL (or OpenGL ES), Qt 3D is probably the case. Although OpenGL has long provided a good service for the graphic engineer, but it is a very old API, there are some root-rooted structural problems that can not be solved without introducing new API. In addition, OpenGL extension and after years of "reform," trying to keep up with the actual work of modern GPU, and the amount of data processing required by artists and growing. Although OpenGL has done a lot to promote this impressive improvement, but it is still limited, especially its multi-threading model and drive implementation of heuristic models that try to predict the driving behavior of the application developer mode.
As the previous section, in the interior of the driver, OpenGL and Qt 3D mode of operation is very similar. When you send out a bunch of OpenGL function calls, these calls will be converted into a command stored in the command buffer and then submitted for processing to the hardware at some point in time (the best estimate is determined by the driver).
Once the hardware buffer commands are processed out of order, we must send the next frame OpenGL function call again. The same process will be a pick a place, it can be very wasteful.
In the drive, create operational command is a very resource-intensive, but also in OpenGL, all confined within a thread. So, to clear the command buffer bit of a waste. Write-driven GPU manufacturers add a variety of heuristics, trying to predict the actual intention of libraries and application developers, to take this as far as possible to cache data and optimize operations. This allows the drive to become larger and more complex, harder to maintain, and led to a huge difference in performance between the GPU manufacturers in some cases.
OpenGL is essentially the threading model is single-threaded. Yes, it can support multi-threaded through a number of ways such as sharing context, but within the driver calls will still be serialized. Considering the OpenGL than 20 years of history, it is not surprising.
OpenGL standard obsolete is another question. Apple has announced deprecated OpenGL, will only focus on the Metal as its graphics API. At some point in the future, we may find that the OpenGL disappear from the MacOS and iOS. Even before that, the OpenGL libraries on the platform will not see any new function of (in fact, they have not been updated for many years).
What can we do for these problems? Well, in the past few years, the emergence of modern graphics API is used to solve these and other problems. Vulkan, Metal and DirectX 12 are very popular API, compared with OpenGL, they provide a more direct interface to control the GPU.
You might say that's great, but in fact there is at compromise. Most OpenGL drivers are now responsible for the work done by the library or application developer. It may sound scary, but true to some extent. But, after all, we can use their understanding of the application macro mode of operation to extract performance from the GPU. On the other hand we can choose to do similar work in a shorter period of time, so that the CPU / GPU into sleep or power-saving mode, and ultimately improve endurance performance. This is a huge improvement for mobile devices and desktop computers in terms.
OpenGL drivers will discard command buffer, and it creates consumed in each frame are high, but when we use Vulkan or similar tool as an application developer, we can know when to keep these command buffer and the next a resubmitted they are safe.
You may want to know what benefits. Submit the same command buffer can only let us see exactly the same with the previous frame content on the screen, is not it? If so, then what is it doing?
This is a good question. In fact, even if we once again submit the same to the GPU command buffer, they refer to the resources they may contain different data. Not only is the vertex buffer and texture, may further include a generally uniform buffer object for saving material properties and the camera transformation matrix. If we can keep track of which things change in the scene, you can determine whether you can re-submit the same command to the GPU, thus saving a lot of work, which is great.
There is a case of icing on the cake! Vulkan uses the concept of primary and secondary command buffer of the command buffer. Our main command buffer is submitted to the GPU content, it may contain calls to auxiliary command buffer. A common use is recorded in advance certain entities drawing command, and stores the secondary command buffer.
When we want to draw the whole scene, our renderer can create a master command buffer, invoke the command buffer those visible entity. When visibility change (for example, if the camera is moving or certain entities move), we can re-record the main command buffer. It is also very good.
More icing on the cake! Use Vulkan, we can also read and write commands in different threads buffers! We have to be responsible for submitting to the GPU command buffer queue, and (graphics / computing / transmission, etc.) synchronization tasks between CPU and GPU and GPU in a different queue.
As you can see, we can gain more control over operations and the hardware involved, but we have to do more work. Overall, this is a great opportunity to enhance performance.
Continue to talk in Qt 6 Qt 3D
Qt 6 node to development time, we are actively studying both the general direction. As can be seen from the above description, both of these tasks involve a lot of work on how to modify the tracking user status on the scene graph and frame graph and a subsequent Qt 3D remaining work to be done. This includes how we ultimately cache command other intermediate state between the frame buffer and to avoid unnecessary duplication of work.
As you may already know, Qt Quick and Qt Quick 3D re-built QRhi layer basis, providing support for Vulkan, Metal, DirectX 11 and OpenGL QRhi layer. We are still studying whether it can reasonably extend this functionality to meet the needs of Qt 3D functionality and multithreading, or whether you need to use other means integrated graphics API, so Qt 3D can still work well with Qt Quick and Qt Widgets module use.
That there is still much work to do, but the initial results look very promising. We tested the scene contains about 1000 entities, in a mid-range desktop platform fooled when we try to maximize the use of GPU, can achieve 600 per second (no screen tearing) rendering speed, or when we are limited to 60fps, you can achieve 1% of the CPU load! It also now uses only a single core! In order to further improve multi-threaded architecture, beyond the limits of Qt 5 series, we are verifying some ideas.
There is a by-product of this work, we have developed a version of the next iteration of frame graph, it's very natural smooth update, therefore easier to understand and easier to Qt 3D user modifications.
to sum up
As you can see, the Qt 5.x cycle and a later time, we will do a lot of work behind the scenes to improve the Qt 3D. We will also look for ways to improve public API, but we do not expect much change in this area, but some of the less desirable functions and attribute names some cleanup.
All of these improvements will also contribute to the Kuesa based on Qt 3D and any other 3D applications using Qt 3D development. These changes will help build a solid foundation, so that we can add more exciting expansion in the Qt 6.
