1. The origin of VSync
The display process of a frame on the display screen is a process of scanning pixels from top to bottom line by line. If the screen starts to scan the pixels of the next frame before the scanning of the previous frame is over, then a The situation of tearing is shown in the figure below.
This problem was first paid attention to and solved on the PC. GPU manufacturers developed a technical solution to prevent the screen from being torn, the full name is Vertical Synchronization (Chinese name vertical synchronization, referred to as VSync). The basic idea is to provide a signal to the outside before the screen is refreshed, and the host side selects an appropriate strategy to complete the screen refresh according to this signal, so as to avoid the mismatch (tearing) between data refresh and screen scanning. So VSync signal is also called TE signal or VBlank signal.
The figure below shows the different display conditions of the screen when Vsync is turned on and off. Here we need to explain the traditional display architecture. It mainly consists of three parts. The first part is responsible for rendering, including CPU, GPU and some system modules; the second part is called frame buffer, which is essentially a block of memory. The rendered data will be saved. In this memory; the third part is the screen, which is used to draw the data on the frame buffer. Generally speaking, there are two frame buffers, one is called backbuffer, which is used to write rendering data, and the other is called frontbuffer, which is used to send rendering data to the screen. The state of these two buffers is constantly changing, that is to say, when the backbuffer is finished writing data and waiting to be displayed, it becomes the frontbuffer, and when the data of the frontbuffer is displayed, it becomes the backbuffer.
VSync off：
VSync on：
Specifically, VSync in the hardware perspective is actually a level signal. There is a separate pin on the Panel, and a separate GPIO needs to be connected to it on the host side to obtain its signal changes; VSync in the software perspective is actually a GPIO interrupts are generally rising edge interrupts, and the software completes the corresponding display logic based on this interrupt.
2. Vsync in Android
2.1 Background
Android's display system has always used double buffering and VSync to prevent screen tearing, which is also the normal operation of other systems. The difference between Android is that VSync is applied to the rendering system as part of the Butter Project (Project Butter) to improve the fluency of the system.
Butter project is introduced from android4.1, mainly has three parts: VSync, Choreographer and Triple Buffer
VSync:
The role of VSync in Android is to unify the rhythm of system drawing and display (Apps and SurfaceFlinger). Everyone performs their duties and ensures that they work when VSync comes, so that the system is theoretically smooth.
As shown in the figure below, in the absence of VSync, the rhythm of the system rendering is inconsistent with the rhythm of the screen refresh. If the system renders a certain frame later, the screen will display the same content twice, that is, Jank (dropped frames).
With VSync, the system will draw when VSync arrives, consistent with the refresh rhythm of the screen, which greatly reduces the probability of jank.
So the question is, how to make Apps draw according to the rhythm of VSync? App relies on the system's drawing system, so the drawing system must listen to commands, which is why Choreographer appears.
Choreographer：
The role of Choreographer (choreographer) has been clearly written in the source code comments. It is a module used to receive timing pulse signals to control drawing. In other words, with Choreographer, apps can perform periodic drawing work according to the Vsync signal.
The combination of the above two has basically completed the transformation of VSync in Android, but there is another reason for Jank that cannot be ignored, which is the Jank risk brought by the double buffer mechanism.
Triple Buffer：
As shown in the figure below, there is no problem with double buffers under ideal conditions. This ideal state means that the drawing work (which can be roughly understood as the total time consumed by a frame of CPU and GPU execution) is completed within one VSync cycle. In this case, Jank will not happen.
Unfortunately, things don't always work out the way we expect, and if the draw takes longer than one VSync period, then jank is bound to happen. As shown in the figure below, if the drawing of two frames exceeds one VSync cycle, then Jank will occur twice.
The three-buffer mechanism is actually based on the above-mentioned backbuffer and frontbuffer, adding another buffer for rotation. In this case, it is also assumed that the drawing of two frames is greater than one VSync cycle, then only one Jank will be caused.
Although the three-buffer mechanism can reduce the probability of jank, it will also bring negative effects of slow Touch response and high memory consumption, but these negative effects are ignored compared to the effects it brings.
This article focuses on VSync. Let's take a look at how VSync is generated and used in the Android system.
2.2 Virtualization of VSync
From the above introduction, we can know that VSync actually originated from the display screen, but if every App and SurfaceFlinger directly monitor VSync from the hardware driver, it would be a bit too complicated, and the coupling is too high. then what should we do?
Therefore, it is best to have a module to communicate with the driver, and then it will broadcast the VSync signal to everyone, just like a hub. However, the VSync frequency is so high, and the consumption from the kernel to the userspace is also a lot each time, and VSync is periodic, so it is easy to guess, so there is no need to monitor from the kernel all the time, but the system always needs VSync to control the rendering and synthesis, so It is necessary to engage in a virtual VSync to simulate the hardware VSync. The general structure is as follows:
Among them, DisplayVSync in SurfaceFlinger (renamed VsyncController after Android S) is a virtual VSync source, which requires two parameters to ensure synchronization with hardware VSync, the first is the reference point, and the second is the cycle. These can be solved by turning on hardware VSync synchronization.
2.3 Synchronization of VSync
The essence of VSync virtualization is to simulate hardware VSync at the software level. Since it is a software simulation, there will be errors. If the error is relatively large, then it is necessary to enable hardware VSync synchronization for calibration. Then there are two problems. How do you find that your error is relatively large? And how to synchronize?
The first is how to find that the error is relatively large? The answer is through the fence mechanism. When SurfaceFlinger hands over each frame to HWC, it will also get the PresentFence of this frame from HWC, which is the signal when this frame starts to refresh to the screen. When does the driver start to refresh a frame to the screen? The answer is when the screen VSync comes. So now it can be connected. According to the signal time of PresentFence, the real VSync time can be known, and then things will be simple.
Get PresentFence in HWComposer::presentAndGetReleaseFences,
After the fence is obtained, it will be aligned and monitored.
Once it is inaccurate, turn on the hardware VSync for calibration. Usually, the calibration can be completed after receiving six hardware VSyncs.
Performance in Systrace
SurfaceFlinger：
Turn on hardware VSync:
Receive hardware VSync via hwbinder:
HWC：
2.4 Distribution of VSync
App and SurfaceFlinger are different processes. Passing VSync between them involves inter-process communication, and the frequency of VSync is very high, and there are many Apps, so the distribution efficiency of VSync must be very high. There are only a few ways to communicate between Linux processes. Android chose Domain Socket because it is efficient, simple, and orderly, and encapsulated it into an easier-to-use BitTube.
VSync-app/sf
All aspects of Android drawing and display are driven by VSync. Specifically, the drawing of each frame of App starts from receiving the VSync signal (VSync-app), and SurfaceFlinger synthesizes the current layer from receiving the VSync signal (VSync -sf) started. In order to avoid waste, the distribution of VSync is on-demand, that is, DisplayVSync will send VSync to it only when the user needs (requestNextVsync).
Introduction to Vsync related classes
First, let’s introduce some vsync-related classes. Basically, all vsync-related methods are implemented in these three classes (the following codes are source codes of Anrdoid T version).
VsyncTracker: It actually creates a VSyncPredictor object, which is used to predict the future VSync timestamp based on the previous VSync signal timestamp. That is, the Vsync model is trained based on HWVsync. In this way, the future VSync time can still be predicted when HWVsync is turned off.
VsyncDispatcher: As the name suggests, this class is used to distribute Vsync signals. In fact, a VSyncDispatchTimerQueue object is finally created, which is responsible for distributing vsync callback events. Modules that need to receive Vsync events can register callbacks to it through registerCallback. When a Vsync event occurs, it will traverse the registered callbacks to distribute Vsync.
VSyncController: The final method is implemented in a VSyncReactor object. From the code point of view, the main function of this object is to transmit HWVsync and presentFence signals.
sf apply for vsync
When sf needs to request refresh, it will call the scheduleFrame function in MessageQueue
Then directly call the schedule function in VSyncCallbackRegistration, and then go to the schedule function in VSyncDispatchTimerQueue.
Among them, rearmTimerSkippingUpdateFor is a key function. This function will get the timestamp of the next vsync trigger, and set this timestamp to the timer through the setTimer function. When the timer is woken up, trigger the callback to send vsync.
Let's see how the callback is triggered layer by layer.
When the timer arrives, the first callback is the timerCallback function in VSyncDispatchTimerQueue
It holds a VSyncDispatchTimerQueueEntry object in the structure Invocation. If you follow it further, you can know that the mCallback is finally transferred to the VsyncCallback function in the MessageQueue.
The last part of the red box is where we usually see the vsync-sf jump in the trace!
app application vsync
Compared with the application of sf, the application of app is more complicated. The app usually applies for vsync by calling the binder interface requestNextVsync.
This interface will call the requestNextVsync function in eventthread, which will send the broadcast through mCondition.
When threadMain listens to the broadcast, it will continue to execute the loop.
What will eventThread execute? The key function is the setVSyncEnabled function in dispSyncSource. When the incoming parameter is true, it will call the start function in CallbackRepeater.
Continue to look down, it will call the Schedule function in VSyncCallbackRegistration, and then go to the schedule function in VSyncDispatchTimerQueue.
The following process is basically the same as the sf application for vsync, and the place where it calls back is here
Call the callback in CallbackRepeater;
Finally, the onVsyncCallback in DispSyncSource is called, which is where the vsync-app jump we saw in the trace.
Compared with vsync-sf, vsync-app has one more process of sending vsync to the applicant. Continue to look down and call onVSyncEvent in EventThread, which will save VsyncEvent to mPendingEvents.
So where are these events distributed? The answer is still in threadMain, this dispatchEvent function is used to distribute vsync to each consumer.
Speaking of this, everyone will definitely have a question, why are vsync-app and vsync-sf triggered by the same timer, but the final callback positions are different?
The answer is that the positions of the two vsync register callbacks are different.
vsync-sf is registered in messagequeue
And vsync-app is registered in callbackRepeater.
This is also a change made by Google on Android T. The reason is that Google believes that the internal transfer process of vsync-sf should be simplified, and it is only used by sf itself anyway.
2.5 VSync-offset/duration
Another advantage of virtualized VSync is that you can perform some customized operations on VSync, and offset is one of them.
Next is the definition of offset, offset is divided into two categories, namely phase-app and phase-sf:
phase-app: the phase difference between VSync-app and hw_vsync;
phase-sf: the phase difference between VSync-sf and hw_vsync;
Still taking the trace as an example, it can be seen that each vsync-app is 1.2ms later than the corresponding TE signal, so the app-offset in this trace is +1200000 (in ns)
Similarly, each vsync-sf is 3.6ms earlier than the corresponding TE, so sf-offset is -3600000.
To sum up, offset represents the phase difference between vsync-app and vsync-sf and hw_vsync, and this value can be obtained by dump sf.
One of the more difficult points of Offset is how to set the Offset time. Its advantages and disadvantages are dynamic and have a lot to do with the performance and usage scenarios of the model.
If the Offset configuration is too short, it is possible that the rendering has not been completed after the App receives the Vsync-App, and SurfaceFlinger receives the Vsync-SF and starts to synthesize. If there is no previously accumulated Buffer in the App's BufferQueue, then SurfaceFlinger will synthesize this time. There will be no App content in it, you need to wait until the next Vsync-SF to synthesize the content of this App, the time is equivalent to Vsync cycle + Offset, not the Offset we expected.
If the Offset configuration is too long, there is no way to play its original role.
In addition, it is beneficial to slightly stagger the VSync of app and sf, because after the staggering, the tasks that occupy the CPU at the same time in the whole system will be reduced, and theoretically it will be a bit optimized. Generally, Android has different offset default configurations for different frame rates.
In Android S and later versions, Google introduced the concept of duration, which partially replaced offset.
The definition of duration is relatively clear
app duration: the duration from app drawing a buffer to sf consuming this buffer (interval between vsync-app and corresponding vsync-sf);
sf duration: the duration from sf consuming a buffer to the screen on this buffer (the interval from vsync-sf to TE);
That is to say, the sum of app duration and sf duration is the total duration of a certain frame from drawing to refreshing on the screen.
This representation is relatively intuitive. Therefore, in the S version and later, Google uses the configuration of duration to determine the vsync phase difference by default, but this does not mean that offset is abandoned. These two values affect each other. Modify one of the values. Another value will also change. The specific conversion relationship is
app phase=n * vsync period - (app duration + sf duration)
sf phase = m * vsync period - sf duration
From the concept of duration, it can be seen that the shorter the duration, the more limited the time for app drawing and sf composition, so the CPU and GPU frequency required for drawing threads and sf threads will be higher, and the power consumption will be higher.
The length of the duration will affect the life cycle of a buffer from drawing to on-screen, and then affect the chirality. The shorter the duration, the stronger the chirality. Secondly, the configuration of the duration will affect the number of buffer blocks preloaded in the bufferqueue, and then affect the memory size occupied by sf. The longer the duration, the more buffer blocks, and the larger the memory occupied by sf. In addition, the manufacturer's frequency adjustment strategy for CPU and GPU will also be affected by the duration, and changing the duration rashly may lead to an abnormal increase in the frequency of specific scenarios.
Having said so much, to sum up the architecture of the VSync module in Andriod is as follows:
HW_VSync is a pulse signal generated by the screen to control the refresh of the screen.
VSync-app and VSync-sf are collectively referred to as software VSync. They are VSync semaphores generated by SurfaceFlinger by simulating hardware VSync, and then distributed to app and sf to control their synthesis rhythm.
3. Summary
This article first introduces the source of VSync, and then focuses on the VSync mechanism of the Android system, including VSync virtualization, VSync synchronization, and VSync distribution. I hope that after reading this article, readers have a certain understanding of the architecture of VSync in Android and how the system uses VSync to control the synthesis rhythm. VSync is one of the very important components of the Android display system. This article only gives a rough introduction to the overall architecture. Subsequent readers can conduct in-depth research on each module of VSync by referring to the AOSP source code.
4 References and links
【1】https://source.android.com/docs/core/graphics/implement-vsync
【2】https://android.googlesource.com/
【3】https://www.digitaltrends.com/computing/what-is-vsync/
【4】https://www.hp.com/us-en/shop/tech-takes/vsync-should-i-turn-it-on-or-off
Past
Expect
push
recommend
An article to understand the bandwidth and synchronization of Vulkan in mobile rendering
Best Sound Quality Practices of RTC in Different Business Scenarios
Android VNDK/VSDK Snapshot compilation framework
Long press to follow Kernel Craftsman WeChat
Linux Kernel Black Technology | Technical Articles | Featured Tutorials