Browser rendering principle and barrage 【Reprint】

background

As the number of bullet chats increases, and we will continue to add more and more animations to videos, how to smoothly display various bullet chats to our users has become a problem that we must consider. This requires us to understand the underlying rendering principles of the browser in order to achieve our various barrage effects with the lowest performance consumption, and to know which performance consumption can be avoided by our front-end.

Target

Through this introduction, we can understand: how the animation we implemented is displayed on the screen on the browser, and where consumption can be reduced to achieve smoother animation.

text

(1) Chrome multi-process model

1.1 Main process

The specific tasks of each process are as follows:

• Browser is responsible for the "Chrome" part of the browser, including the navigation bar, bookmarks, forward and back buttons. At the same time, this process will also control those parts that we cannot see, including the sending of network requests and the reading and writing of files.
• Renderer By default, each Tab page will start a rendering process, which is mainly responsible for our html parsing and js execution
• Plugin is mainly responsible for the operation of plug-ins, because each of us can write plug-ins. In order to prevent the crash of plug-ins from affecting the entire browser, chrome starts an independent process for each plug-in. GPU Chrome was first released when it
  was There is no GPU process, and the original intention of using the GPU is to achieve the effect of 3D CSS, but then the web page and the UI interface of Chrome all choose to use the GPU to draw, which makes the GPU a common requirement for browsers

In addition to the processes listed above, Chrome has many other processes at work, such as the extension process (Extension Process) and tool process (utility process).

We can see the cpu and memory used by each process in the upper right corner of the browser > more tools > task manager:

1.2 Advantages and disadvantages of Chrome multi-process architecture

benefit:

One: Multi-process can make the browser have good fault tolerance.

If you have three tabs, you will have three separate rendering processes. When one of the tabs crashes, you can close this tab at any time and the other tabs will not be affected. But if all the tabs run in the same process, they will be related, and one hangs and all hangs.

Two: It can provide security and sandboxing.

Because the operating system can provide a way for you to limit the capabilities of each process, the browser can make certain processes not have certain functions. For example, Chrome restricts the ability of the tab rendering processes to read and write system files randomly, since they may process random input from the user.

Disadvantages:

Since each process has its own independent memory space, it will take up a lot of memory.

Improve:

One: In order to save memory, Chrome will limit the number of processes that are started. When the number of processes reaches a certain limit, Chrome will run tabs that visit the same website in one process.

Two: When Chrome runs on some hardware with better performance, services related to the browser process will be put into different processes to improve system stability. On the contrary, if the hardware performance is not good, these services will be executed in the same process to reduce memory usage.

(2) What happened from entering the URL to displaying the page?

2.1 What tasks are performed in the browser process

Step 1: Processing Input

The browser's UI thread will perform a series of analysis to determine whether the input string is a search keyword or a url address.

If it is not a url address, the default search engine of the browser will be called to perform a search operation.

Step Two: Start Navigating

If it is a url address: the UI thread will call the network thread to initiate a network request to obtain the content of the site, that is, dns addressing, tcp three-way handshake, arp addressing and similar operations.

At this time, a loading icon indicating that the resource is being loaded will be displayed on the tab.

The ui thread will also notify the browser process to start a rendering process to prepare for rendering the page.

Step 3: Read the response

After getting the response: If the network thread receives a 301 redirect from the server, it will tell the UI thread to redirect, and then it will initiate a new network request again.

If it is data content, it will first detect the specific media type of the response data.

The media type of the response body can generally be determined by the Content-Type of the HTTP header, but the Content-Type is sometimes missing or wrong. In this case, the browser needs to perform MIME type sniffing to determine the response type.

If the received response data is a compressed file, the response data will be handed over to the download manager for downloading.

Step 4: Find a renderer process

If the body of the response is an HTML file, the network thread does some necessary security checks on the content and looks for a renderer process.

Step 5: Submit Navigation

The data and rendering process are ready, and the browser process will tell the rendering process to submit this navigation through IPC.

In addition, the browser process will also pass the response data stream just received to the corresponding rendering process so that it can continue to receive incoming HTML data.

Once the browser process receives a reply from the rendering thread that the navigation has been submitted, the navigation process is over and the document loading phase begins.

At this point, the navigation bar will be updated and the session history for the current tab will be generated.

Step 6: The rendering process continues to receive data and parse it

After the navigation submission is completed, the rendering process will continue to receive html data, and parse and load page-related resources. Once all resources are onloaded, the rendering process will notify the browser process that all resources have been loaded. At this time, the UI thread starts Will stop the loading icon on the navigation bar.

(3) Tasks performed by the rendering process and the gpu process

3.1 The role of the rendering process

The main task of the rendering process is to convert HTML, CSS, and JS into web content that we can interact with.

The main thread in the rendering process:

• A main thread is responsible for html, css parsing and js execution
• A compositing thread is responsible for segmentation, generating frame data, and receiving user events
• A raster thread pool converts the divided tiles into bitmaps
• Several worker threads such as web worker and service worker threads

3.2 Main thread task

After the rendering process receives the html data, it will first execute on the main thread: convert it into a character stream, and generate a token sequence after lexical analysis (at this time, the browser will do an optimization, and will give the identified resource address to Network thread to load data), then syntax analysis, parsing html, generating DOM tree, parsing css to generate css rule tree.

3.2.1 After the DOM tree is generated, four tree structures will be generated successively. The first is the layout tree generated by combining the DOM tree and the css rule tree

The main thread will traverse the DOM tree, combined with the css rule tree, to determine the calculation style of each DOM node.

Then it will traverse the DOM tree again, and calculate a layout tree according to the calculation style of the DOM node. Each node on the layout tree will have specific information about its x, y coordinates on the page and the size of the box. The layout tree looks similar to the previously built DOM tree, the difference is that this tree only has the information of those visible nodes.

For example, if a node is set to display:none, the node will not appear on the layout tree but the visibility:hidden node will appear on the layout tree. Likewise, if a pseudo-element node is visible, it will appear on the layout, but not on the DOM tree.

3.2.2 In the painting stage, the painting tree is generated from the layout tree

The main thread will traverse the previously obtained layout tree to generate a series of drawing records. The drawing record is a comment on the drawing process, such as "first draw the background, then the text", similar to the operation record of drawing from one point to another when we use canvas to draw graphics.

As shown in the figure, we can see that the drawing tree generated by the layout tree will have fewer nodes:

Here, we first look at a phenomenon:

As shown in the figure, a node and b node are in a parallel relationship in the document structure, and then we set the a and b nodes to overlap through a negative margin, and then this happens: the background of b will cover the background of a , but it does not cover the text of a. This is because a and b do not meet the requirements listed below and do not form their own painting layer. They share a painting layer, and the order in which the painting records are generated is to paint the background first. Draw text again, so this will happen.

To have an independent painting layer, the following conditions need to be met:

1. root object of the page

2. have an explicit CSS position property (relative, absolute or transformed)

3. is transparent

4. Has CSS filter

5. A canvas element with a 3D (WebGL) context or an accelerated 2D context

6. video element

3.2.3 In the synthesis stage, the graphics tree is generated from the drawing tree, and each graphics layer is what we usually call the synthesis layer

It is similar to the process from the layout tree to the painting tree described above. Only the painting layer that meets the special conditions can form its own graphics layer, otherwise, the graphics layer of its first ancestor element will be used.

In order to have an independent compositing layer, the following conditions need to be met:

1. Layers have 3D or perspective transform CSS properties

2. Layers used by video elements using accelerated video decoding

3. Layers are used by canvas elements with a 3D context or an accelerated 2D context

4. Layers for compositing plugins

5. Layers use CSS animations for their opacity, or use animated webkit transforms

6. Layers use Accelerated CSS Filters

7. Layer's children are compositing layers

8. The layer has a sibling with a lower z-index that has a compositing layer (in other words, the layer that overlaps the compositing layer and should be rendered on top of it)

As we usually use:

We can view all graphics layers of the current page in the chrome console.

When css3 animation is executed, the browser actually just moves the corresponding graphics layer.

3.3 Composite thread tasks

• Divide the layer into 256x256 or 512x512 tiles
• Rasterization refers to converting the drawing records on the tiles into bitmaps, and the bitmaps are stored in the memory of the gpu
• The drawing quad contains information such as the location of the tile in memory and the location of the tile on the page after the layer is composited
• Composite Frames A collection of drawing quads representing the content of a frame of the page

The compositing thread divides each layer into tiles, and then sends the tile data to a series of raster threads. The compositing thread also assigns different priorities to different raster threads so that those in or near the viewport The tiles can be rasterized first. The rasterizer thread rasterizes each tile and stores them in GPU memory. When we use css3 animation and upgrade the composite layer, when each composite layer is animating, it directly operates on the rasterized layer instead of rasterizing it every time.

By the way: Rasterization is divided into software rasterization and hardware rasterization. The new browsers basically use GPU hardware rasterization, also known as fast rasterization. When the tiles on the layer are rasterized, the compositing thread collects information called drawing quadrilaterals on the tiles to build a compositing frame, and then the compositing thread generates a series of instruction calls. Due to the limitation of the sandbox, the renderer process cannot directly call the 3D API provided by the operating system. Therefore, a separate process is required to access, which is the GPU process mentioned above.

3.4 gpu process

A series of instruction calls generated by the compositing thread are serialized and stored in a shared memory, and then the gpu process fetches the serialized instructions from the shared memory, parses them and executes the appropriate graphics call.

After the GPU rendering is completed, the rendering result will be stored in the frame buffer, and the video controller will read the data of the frame buffer frame by frame according to the VSync signal, and finally display it on the display after data conversion.

(4) Realization principle of barrage

First of all, we will have a barrage list to maintain all the barrage of the current video, and then we will get the barrage that will be displayed in the next frame frame by frame. There is a judgment here, whether there is a cached dom node, if No, we will create a dom node and fill it with barrage. Then, get the width and height of the bullet chat to perform collision detection with the bullet chat that has already appeared on the screen, determine which track the bullet chat should be displayed on, and then start the displacement animation. When the bullet chatting has completely moved out of the display area, we will cache the dom node of this bullet chatting, and then reuse it for the subsequent bullet chatting (because creating a dom node is also a very performance-consuming operation).

Reprint please specify:

Reprinted from the article of the snowman boss, only for learning and use (no commercial use), transfer and delete, you can privately message the blogger.