write in front
Hello everyone, I'm Ranmo , this is a topic I gave at the Global Open Source Technology Summit GOTC - "Flutter-based Web Rendering Engine "Beihai Kraken". I mainly share some implementation principles and key technical features of Kraken from a technical point of view, and now I will share it with you in a text version.
Kraken Github：https://github.com/openkraken/kraken
Kraken official website: https://openkraken.com/
The technical background of the North Sea
When it comes to the technical background of Beihai, we have to mention the evolution of cross-end technology. Many students should be familiar with the process of cross-end technology, so I will briefly talk about it.
We know that the browser is the most mature natural cross-platform solution. As early as the PC era, the browser has become the entrance to the Internet, and everyone will habitually browse web pages through the browser to absorb various information. At that time, we called this way of surfing the Internet "surfing". However, in the mobile era, browsers do not have an eye-catching performance on mobile devices. On the contrary, due to problems such as large memory, long white screen in weak network environment, and lack of sensor capabilities (slow standard follow-up), various doubts are endless.
In order to make up for some deficiencies of the above browsers on the mobile side, the Hybrid technology has emerged, which implements some non-standard supersets on the Web through the capabilities of containers, and also improves the loading performance of the first screen through various technologies such as prefetch and offline packages. .
Since then, an RN-like solution (typically representing React Native) has emerged. Its principle is to bridge the Native control and the front-end ecosystem through the JS engine, improve efficiency through Web development business logic, and improve performance through Native control rendering. and experience. However, the disadvantage of this type of solution is that it cannot completely smooth out the differences between the two ends, does not solve the problem of consistency, and ultimately exposes the complexity to developers.
As the new darling of the cross-end industry, Flutter has also gained more and more attention in the past two years. Let's introduce Flutter.
The advantages of Flutter are good performance and high cross-end consistency due to its self-drawing rendering, but it also has its own shortcomings, such as its own ecology, neither front-end nor Android/iOS.
This leads to a series of questions.
-
First of all, the front-end (JavaScript) or client-side (Swift / JAVA) transformation has a certain cost, but because the GUI system on the end side is similar. From the perspective of a front-end developer, the cost of language learning is not particularly high. Students with experience in front-end frameworks such as React or Vue can quickly get started with simple learning. For some small entrepreneurial teams, it is true that they can learn and develop quickly in small steps, but when the organization is large to a certain extent, the cost of this conversion will increase exponentially.
-
Secondly, the ecosystem is waiting to be rebuilt. Some Flutter developer friends may feel that there are already quite a few pubs in Flutter development that can be used directly. But in fact, the ecosystem is not limited to Flutter pub, there are also various existing basic links, such as construction-related CI/CD, and construction and so on. This series of ecology needs to be rebuilt, and the cost is very large.
-
Thirdly, many of the existing businesses are front-end projects developed through JavaScript + front-end framework. If we want to migrate them to Dart + Widget, the cost is undoubtedly very huge.
While facing so many problems and the high cost of switching, we also expect to bring more technical possibilities to our business through Flutter, while improving some performance and experience problems of web containers on the side. Well, the first step in introducing a new technology is to solve the cost of introducing this new technology , so we are actively exploring a solution that combines the front-end ecosystem with Flutter.
So the protagonist of this topic - Beihai Kraken was born.
Kraken is a high-performance web standard self-drawing rendering engine, featuring high performance, easy expansion, Flutter-based and web standards compliance.
Below I list some of Beihai's application scenarios in Ali. On C-end APPs or IoT devices, Beihai has related applications.
Technical principles of the North Sea
Before introducing the technical principle of Kraken, I will demonstrate how to develop a Kraken application. Because Kraken is a web rendering engine developed based on the W3C standard, the upper layer is independent of the framework. Whether developers use Vue, React or Rax, they can develop an application on Kraken.
Taking Vue.js development as an example, the following is a project I started with vue-cli provided by Vue. See the official example for the specific code .
It can be seen that the leftmost is the relevant code of Vue, and the right is the result of the application running on Chrome (left) and the result of running on Kraken (right). You can see that the results are exactly the same.
Knowing how to develop a Kraken application, let's understand the technical principle of Kraken. For a better understanding, let's first compare the rendering process of Flutter and Webview.
I believe everyone is very familiar with the rendering process of WebView. A very classic topic in the interview is how a URL input is finally rendered on the screen. In general, it parses HTML, JS and CSS files, executes the corresponding JS to call the DOM API, and finally generates the DOM Tree and CSSOM Tree, and then calculates the final Render Tree, generates a series of Layers through the Layout and Paint processes, and finally passes the Compositing and rasterizing rendering to the screen.
Looking at Flutter, there are three classic Flutter trees - Widget Tree, Element Tree and RenderObject Tree. Widget Tree corresponds to the front-end layer similar to the front-end framework, and Element corresponds to DOM Tree, RenderObject Tree and Render Tree respectively. Finally, Layer will be generated through a series of calculations of Layout and Paint, and then rendered to the screen through synthesis and rasterization.
Then, let's add the front-end framework to our entire process for a more intuitive comparison. Here we take Vue.js as an example.
Vue.js will generate a series of Vdom at runtime to generate Vdom Tree, and then call the API of the specific platform through the abstraction of platform.
Then we will find that we only need to exchange the part of the process I circled with the red frame to achieve the effect we finally want to achieve (upper-level web development, lower-level rendering based on Flutter).
Based on the above assumptions, the rendering process of Beihai came out.
At present, the mainstream front-end frameworks will group the product into a JS Bundle, and operate the specific view through the standard DOM API, while there is generally only one root node in HTML. Under the Web, the page will first request the HTML file, and then parse the Script tag to load the corresponding JS file. The entrance of Kraken is designed as a JS file, which can reduce one request and speed up the rendering of the first screen.
The JS file will be executed in JS Engine. Kraken's runtime provides a series of Web standard API interfaces through JS Engine Binding. Calling the corresponding API will execute the relevant logic and create a series of instructions that need to be sent to the Dart layer for processing. Stored in a struct. C++ sends the underlying address of the corresponding instruction to Dart through FFI, and Dart processes the relevant instruction and generates a Dom Tree. Similarly, CSS will generate the corresponding CSSOM Tree through Parser, which will eventually generate Flutter's RenderObject Tree. After a series of calculations of Layout and Paint, a corresponding series of Layers will be generated, and then finally displayed on the screen through synthetic rasterization.
Similarly, in the latest implementation, we considered the SSR application scenario, so we added the Beihai application development method with HTML as the entry point, and the corresponding HTML file can be parsed through HTML Parser, and the subsequent process is the same. The support of SSR also makes the second opening rate of the first screen to a higher level.
So after understanding the entire rendering process of Kraken, how do we complete the development of a Web standard rendering engine based on Flutter?
So to do this based on Flutter, you must first understand the architecture of Flutter.
The top layer of Flutter is the Framework implemented by Dart, which includes the responsive framework, the official website component library, and the parts that implement the layout and drawing protocol. In the middle is the Flutter Engine implemented by C++, which is the second half of the rendering process, providing some basic capabilities, as well as layer synthesis and rasterization output. The bottom layer of Embedder is responsible for some implementations of specific platforms to achieve cross-platform.
It is not difficult to find that the most Dart Framework Widget is an abstraction of UI and implements a set of responsive frameworks, corresponding to front-end frameworks such as Vue / React. The layout protocol below can correspond to the W3C standard to implement a set of layout and rendering protocols based on front-end standards.
Then we can draw the architectural design of Beihai.
Look at the left side first, the left side is still the overall architecture of Flutter introduced above. Flutter's Widget capabilities can be registered in Kraken through plug-ins and become a front-end standard Tag. JS can dynamically call and control rendering. The Flutter architecture on the left supports the upper-level Flutter ecosystem, so that the Flutter ecosystem can also be integrated into the entire Kraken rendering system in the form of plug-ins.
On the right is the architecture implementation of Kraken, which does not intrude the implementation into the Flutter Engine. In the Dart layer, by implementing a series of layout and rendering capabilities of the W3C standard, a series of standardized capabilities are provided for the upper layer, such as Element, CSS, and modules of various Web standards. In the upper layer, Kraken's runtime provides a series of Web standard API interfaces through JS Engine Binding. Calling the corresponding API will execute the relevant logic and create a series of instructions that need to be sent to the Dart layer for processing. The instructions are stored in a struct. C++ sends the address of the bottom layer of the corresponding instruction to Dart through FFI, and finally Dart invokes the standardization capability mentioned above according to the instruction to complete the docking. Through this implementation, it provides support for the upper front-end ecosystem. With the rich front-end ecosystem, developers can enjoy the efficient development experience brought by the front-end ecosystem.
Key technical features
The loading performance of the first screen is a key indicator of a C-side scenario, and a long white screen will greatly affect the user experience.
Kraken needs to create a large number of nodes when the first screen is initialized, and a lot of time is spent on communication, so it is urgent to optimize the performance of the first screen.
In the technical principle part above, we know that Kraken needs to complete the communication between C++ (JS Engine) and Dart through Bridge to achieve the purpose of passing instructions to the Dart layer. The architecture of Bridge has also evolved in three versions.
In the original first-generation solution, we hacked into the Flutter Engine, passed the data from the JS Engine to the Flutter Engine, and finally sent the data to the Dart layer through native bingding. The obvious disadvantage of this generation of solutions is that it invades the Flutter Engine, and it takes a lot of time to compile the Flutter Engine during development. At the same time, hacking into the Flutter Engine is not a reasonable design for Kraken's architecture.
Later, Dart FFI appeared, which can realize efficient communication between C++ and Dart, so the second-generation solution was born. The second-generation Bridge solution serializes the JSON data and transmits the data to the Dart layer through Dart FFI, and the Dart layer then deserializes JSON to get the final data. Compared with the previous generation, this generation of solutions can solve the shortcomings of invading Flutter Engine, but it introduces the problem of string copying and JSON serialization and deserialization.
In order to solve the above problems, the third generation Bridge solution was produced. The third-generation Bridge solution defines a standard 40 Bytes Struct to store instructions by sharing memory, and only the address of the instruction is passed through Dart FFI. Both C++ and Dart rely on the address to access related data. This solves the problem of JSON serialization and deserialization, saves time, and saves one data copy. At the same time, since the memory is 40 Bytes aligned, the memory access efficiency can be improved.
Here are some above-the-fold benefits from actual online pages.
The long list of infinite scrolling is a historical problem that has plagued front-end developers for a long time. A large number of layouts cause the page to freeze, and the long Paint time during scrolling causes the scrolling frame to drop. The page experience is very bad. The community also has a lot of front-end solutions to deal with this problem, and on Kraken, we also expect to solve this problem at the container layer.
There are also RecyclerView and TableView on Android and iOS respectively to solve this problem. Their principle is to define a buffer area outside the viewport of the visible area, dynamically release when the node exceeds the area, dynamically create when entering the area, and A series of nodes are used for attribute replacement to ensure that the number of nodes does not explode. A similar implementation of Sliver is also provided in Flutter, so can we use Sliver to empower the front end to solve this problem?
Kraken defines a new displayattribute sliver. By setting the node's displayattribute to sliver, you can directly use Flutter's Sliver capability to achieve a dynamic recycling capability after the node exceeds the visible and cached area. It can be seen that we use a demo of 1000 cards for testing, sliverand blockthere are .
At the same time, this standard has also been discussed in the W3C Chinese Interest Group . It is hoped that after everyone has fully discussed and reached a consensus, they will try to submit this proposal to W3C and feed back to the front-end community.
A large front-end team often has both a client and a front-end, and will accumulate a series of end-to-end capabilities. Different needs will have different technical selections. For example, a player is often developed through Native technology. We expect to integrate the capabilities on the end (including Flutter Widget, Web, Native, and third-party SDKs, etc.) into a large front-end development system, so how do we integrate this series of capabilities on the end in Kraken? At the same time, we also expect to introduce on-demand, which can optimize the package size. In different business domains, we expect to quickly carry out customized development and quickly form a set of domain capabilities for vertical business domains.
Kraken provides a set of extension capabilities to solve the above problems. Through the rendering capability extension interface, developers can quickly integrate the developed standard-compliant Flutter Widget and Native rendering capabilities into the Kraken system, and finally provide a dynamic through JavaScript. ability to call. Similarly, through MethodChannel, developers can call some Native or Dart API capabilities through this channel, such as some two-party or three-party SDK capabilities.
Developers can customize the capabilities required by the business domain by extending capabilities, and plug and unplug as needed to achieve the purpose of optimizing the package size. Likewise, plugins registered to kraken can be controlled via JavaScript code, providing dynamism.
Below is a series of demos that extend the Flutter Widget, Native API, and Native player inside Kraken.
The following is to improve interactivity. Before introducing Kraken's interactivity, let's take a look at some interactive problems under the Web.
When developing applications with rich interactive capabilities under the Web, front-end developers often need to introduce an additional lib to provide enhanced gesture capabilities (for example, a gesture library such as Hammer.js). Then, when the front-end developer introduces lib, it will cause an additional request for the corresponding JS library after index.html is loaded, which causes an extra request overhead and prolongs the interactive time of the first screen.
When the user performs an operation on the screen, it may be the user's hand or a device such as the Apple Pencil or a mouse due to the way the user operates. Therefore, in the W3C standard, the touch points of user-operated interactive applications are abstracted into a pointer , and these pointers will form a gesture according to the operation, which are three processes of down, move, and up, of which move can be omitted (for example, click).
In the Web, this series of pointers need to be dispatched to the element tree, and these pointers are frequently sent to the JS layer through bubbling, and then JS completes the identification of the interaction by encapsulating the Touch API. This brings several problems. First, frequently passing pointers from C++ to JS brings unnecessary overhead. In addition, the ability to encapsulate standards will also cause additional development costs, and the ease of use is not outstanding. At this time, if some solutions of the community are used, it will also lead to non-standardization and misalignment of standards, resulting in inconsistent interactive experience on different pages in the same application.
In order to solve the above problems, we expect to provide a set of standardized interaction capabilities from the aspects of standardization, ease of use, and standardization. Different gesture capabilities are obtained by encapsulating the underlying pointer, so that developers can quickly develop rich interactive applications.
Below is a flowchart of the enhanced interactivity in Kraken. When the user performs some interactive operations, the pointer of each touch point will be passed from Native to Kraken, and the Pointer will be distributed to GestureManager (gesture recognizer management class) and Scroll recognizer at the same time. GestureManager will recognize that developers register and distribute to the corresponding gesture recognizer through the Web standard listening behavior (EventTarget.addEventListener), and the Scroll recognizer will also distribute the pointer. These recognizers are added to the Flutter arena for gesture competition to ensure that only a specific action is triggered (interaction controllable). The Scroll recognizer will trigger the scrolling operation of the scroll area, the gesture recognizer will bubble and dispatch through the standard Web process, and finally the developer completes the custom behavior by listening to the event.
When developing applications, debugging capabilities are essential. The high front-end development efficiency is not only due to a prosperous ecosystem, but also a friendly development and debugging experience is an artifact to improve efficiency.
Kraken abstracts the Inspector to connect to Chrome DevTools through the Chrome DevTools Protocol, and provides a series of debugging experiences that are completely consistent with front-end development of web applications. Whether developers prefer to use Console.log or through JS Debugger, they can quickly get started.
In addition, Kraken also provides the ability of local hot update by supporting all standard Web APIs of HMR, so that the development of Kraken applications can be consistent with the local hot update debugging experience under the Web, which greatly improves the development and debugging experience of developers.
Finally, all of Kraken's code has been open sourced. Kraken provides an open TSC mechanism. It is expected that all developers can communicate and make decisions equally, so that Kraken can develop better, and more developers are welcome to build Kraken together.
Kraken Github：https://github.com/openkraken/kraken
Kraken official website: https://openkraken.com/