Inscription
-Holding the sword in the world, starting from your accumulation of bits and pieces, you must strive for perfection wherever you go, that is, toss every day.
| ** | You may need |
|---|---|
| CSDN | NetEase Cloud Classroom Tutorial |
| Nuggets | EDU Academy Tutorial |
| Know almost | Flutter series articles |
This article will describe the cooperation principle of CPU, GPU and monitor display image, Vsync mechanism, Flutter Vsync process
1 Graphic drawing principle
The display (screen) is composed of physical display units (pixels), and each pixel can emit multiple colors. The principle of display phase is to display different colors on different physical pixels, and finally constitute a complete image.
Graphics calculation and drawing are all done by corresponding hardware. The operating system generally encapsulates these underlying hardware operation instructions and provides some encapsulated APIs for the application layer on the operating system to call.
Then encapsulate the native API of the operating system in a programming framework and model, and then define a simple development rule to develop GUI applications, and this level of abstraction is the so-called UI system.
For example, the Android SDK encapsulates the Android operating system API and provides a UI system that uses the UI description file XML+Java to operate the DOM, while iOS provides the UIKit series of abstract operations on View.
1.1 The cooperation principle of CPU, GPU and display
The full name of CPU central processing unit is the computing and control core of the computer system, and the final execution unit of information processing and program operation.
The full name of GPU is Graphics Processing Unit. It is a graphics processor, a microprocessor that specializes in image and graphics-related operations on personal computers, workstations, game consoles and some mobile devices (such as tablets, smart phones, etc.)
The graphics processor generally consists of three parts:
1. Display the core of the main chip graphics card, commonly known as GPU, its main task is to construct and render the image information input by the system.
2. The display buffer memory is used to store the graphic information to be displayed and to save the intermediate data of the graphic operation.
3. The RAMD/A converter converts the binary number into an analog signal suitable for the display.
In terms of the original computer system, the CPU, GPU, and display cooperate in a specific way: the CPU submits the calculated display content to the GPU, and the GPU puts it into the frame buffer after rendering. They are image producers. The frame buffer (BufferQueue) is constantly filled with data. The display can be understood as a consumer, and then the frame data (BufferQueue) is fetched from the frame buffer at a fixed frequency, and then the rendered content is presented on the screen, such as a screen refresh The frequency is 60 Hz, which means that data will be fetched 60 times within 1 second.
The display is refreshed at a fixed frequency (to fetch data from the GPU), through a vertical synchronization signal (such as VSync), a 60Hz screen will send out such a signal 60 times within one second, this signal is used to synchronize the CPU, GPU and display The signal that prompts the CPU and GPU to work on the next frame.
In the later baptismal world of mobile devices, each mobile phone was similar to a small computer system. The Android system issued a VSYNC signal every 16.6ms to notify the interface to perform input, animation, and drawing actions.
1.2 Vsync mechanism
The current graphics card can usually render the CS frame rate above 120, that is, 120FPS. FPS represents the number of frames that the GPU draws in one second. 120FPS represents 120 frames in one second. The usually used display can only reach 60HZ. Refresh rate, like this situation, the graphics card changes the picture 120 times in 1 second, but the monitor only has the ability to display 60 frames, which obviously loses half of the frames. This phenomenon is that the image production capacity exceeds the display capacity , Oversupply.
On the contrary, the graphics card changes the picture 60 times within 1 second, but the monitor has the ability to display 120 frames. This is in short supply, and when the supply exceeds supply, it is easy to cause lag.
The Vsync mechanism can well coordinate the above two situations of oversupply and short supply. The Vsync mechanism can be understood as a communication bridge between the graphics card and the display. The graphics card waits for the vertical synchronization signal before rendering each frame. Only when the display completes a refresh , Send the vertical synchronization signal, the graphics card will render the next frame, to ensure that the refresh rate and frame rate are kept in sync, in order to achieve the effect of supply and demand balance, to prevent stuck phenomenon.
2 React Native for cross-platform development
The following figure shows the technical architecture diagram of React Native. ReactJS itself does not draw the UI directly, but calls native components to perform page rendering operations. Bridges is a bridge that is used to draw instructions for native components to draw.
3 Flutter for cross-platform development
Unlike React Native, which uses native components to render the interface, Flutter does not need to use native components to render the interface, but uses its own rendering engine (Engine layer) to draw page components, as shown in the following figure. Flutter's technical architecture sketch.
The engine part of the Flutter engine is developed in C++, and the Skia part is an open source two-dimensional graphics library that provides a common API for a variety of software and hardware platforms. Skia acts as a rendering/GPU.
The Engine layer of Flutter exposes the Canvas, PictureRecorder and other interfaces to the Dart layer. Using these interfaces, you can draw the image you want.
Drawing images by directly calling APIs is more like an instructional operation.
In Flutter, through the Widget component provided by Flutter Sdk, you can build exquisite image layouts. These widgets are not components that are finally displayed on the interface, just like ReactNative's virtual DOM.
Each Widget is built to the final image displayed on the monitor through three stages: Widget --> Element --> RenderObject
The drawing process of Widget, Element, RenderObjec will be updated in the next section of my big lead career.
4 Flutter Vsync process
The Vsync communication mechanism in flutter is shown in the following figure:
complete
