Author | Sun Jianbo (Tian Yuan)
Source | Alibaba Cloud Native Official Account
This article is modified and compiled from the KubeVela live broadcast content of the third issue of "GO Kaiyuan". The video content is longer, and the content of this article has been cut and reconstructed.
KubeVela's background
KubeVela is a cloud-native platform-level open source project based on the Go language, which was officially released in mid-November last year . Although it has been less than two months since its release, KubeVela, as the core dependency behind the "Alibaba Unified Cloud Native Application Platform Kernel", has actually been running behind Alibaba's multiple products for a relatively long period of time. Participate in the core construction of these products and projects.
KubeVela ：https://github.com/oam-dev/kubevela
This kernel system was born from the implementation of the Open Application Model ( OAM ) model jointly launched by Alibaba Cloud and Microsoft at the end of 2019 based on Kubernetes. In the continuous evolution and iteration, a large number of open source communities (especially Microsoft, ByteDance, The feedback and contributions of the fourth paradigm, Tencent and Manbang Group's community participants, finally met with the open source community under the name "KubeVela" at the KubeCon North America Summit 2020 . The KubeVela project has received continuous attention from the entire cloud native ecosystem after the official announcement, and reached the top of the Go language open source trend list on the fourth day after its release.
Figure 1 KubeVela's GitHub Star is growing rapidly
KubeVela's github address:https://github.com/oam-dev/kubevela/
What is KubeVela?
In a word, KubeVela is a simple, easy-to-use yet highly scalable cloud-native platform construction engine for platform builders .
Specifically, KubeVela's goal is to enable any platform team to quickly and efficiently create a cloud-native platform suitable for different business scenarios that can directly face users in a Kubernetes-native way. For example: building application PaaS, database PaaS, AI PaaS or continuous delivery system, etc.
Figure 2 KubeVela "Separation of Concerns" workflow
In terms of design, KubeVela exposed two core APIs to the platform team, including:
-
Capability template : "Capability" in KubeVela refers to the atomic functions that can form a complete application. For example, StatefulSet and Ingress belong to two different "capabilities". KubeVela allows the platform team to preset various capabilities in Kubernetes by defining various capability "templates".
- Deployment environment template : Similar to "ability", the application deployment environment is pre-defined and initialized in KubeVela through the "environment" template. For example, "test cluster" and "production cluster" belong to two "environments".
As platform users, such as business teams, they only need to "one-click" to initialize their expected deployment clusters through the environment template provided by the platform team, and then "assemble" the required capability templates into a complete application. Direct application delivery and operation and maintenance to any Kubernetes cluster.
Since these capabilities and environments are abstracted through "templates", for business teams, they do not need to learn the complete Kubernetes concepts and details. They only need to understand the parameters exposed by the above templates. Use Kubernetes seamlessly to do what you want to do. The specific configurable items exposed through the template, how the template behind it is rendered, and what Kubernetes objects are rendered into are completely under the control of the platform team and can be adjusted and modified at any time.
The above-mentioned "platform team providing capability template" combined with the "business team assembly template declaration application" workflow is also a concentrated expression of the "separation of concerns" idea advocated by the OAM project jointly released by Ali and Microsoft. In terms of specific template support, the first phase of KubeVela supports Google's open source CUELang template language, and the second phase will support the Helm Chart package directly as a capability template.
What can KubeVela do for you?
After understanding what KubeVela is, let’s answer the second question that everyone has always been concerned about: As a platform builder, what can KubeVela help you do?
1. Quickly build abstractions
Building "abstraction" is the most basic and inevitable function of any cloud native platform.
We know that what Kubernetes exposes is a set of declarative APIs, and the so-called abstraction is actually the operable and configurable items that a platform exposes to users on the basis of these declarative APIs . As the platform team, the ultimate goal of providing "abstraction" is to simplify the user's mind, so that the business team can only focus on what they care about, and avoid introducing a large number of platform-level details that are not related to the business to make users "daunt" . It can be said that providing "abstraction" is the first step for any platform team to implement system-level open source projects such as Kubernetes.
The most common abstraction method in the industry is to provide users with a graphical interface (such as Console or Dashboard) for operation. The common point of these graphical interfaces is that they only allow users to fill in certain specific field parameters, thereby simplifying the user's mind. , Such as the Console of an open source K8s PaaS project shown in the figure below:
Figure 3 Console of an open source K8s PaaS project
Some projects (such as Racher Rio) choose to provide users with a command line tool. In fact, its function is completely similar to the graphical interface, except that the parameters that are allowed to fill in become CLI parameters.
Of course, for some teams with higher technical levels, they will develop the upper-level CRD + Operator based on Kubernetes as an "abstraction". For example, Knative is actually an abstraction for serverless scenarios, while Pinterest has its own Application CRD abstraction.
So, as a platform team, how do we decide which configurable parameters to expose to users? This involves the three basic modes of "abstraction" (more complex situations are further combinations of these three modes):
-
Combination abstraction , this model is common when we combine two atomic capabilities into one capability provision. For example, when we actually develop the Console, we often "combine" K8s Deployment and Service, exposing the concept of a Web Service to allow users You can define both the container image and the exposed port in one form.
-
Split abstraction , this pattern is common when we want to divide the fields on an object into several forms for filling in steps in the use process, thereby decoupling the configuration during deployment and operation and maintenance. For example, for multiple containers in a Pod, I want users to fill in the business container in the first form, and let the operation and maintenance fill in the Sidecar container in the other form. Another example is the ArgoRollout object. I would like one form for users to fill in the container image, and another form for the operation and maintenance to fill in the grayscale strategy.
- Conversion abstraction . This mode is usually used to change the name or remove some unrelated concepts. For example, Knative Revision and Deployment are essentially one-to-one correspondence, but there are fields like LabelSelector that users don’t need to care about in Knative. Just removed it.
Figure 4 Common abstract patterns
The above-mentioned abstract patterns are reflected in many platform-level projects and products in the industry. But on the other hand, how to properly design abstraction and how to ensure that abstraction can meet the needs and habits of business users is actually a very challenging problem. The key point here is that whether it is a graphical interface or a CRD Operator, once these "abstracts" are online, it is very difficult to modify it. But on the other hand, the needs of business users are almost impossible to be static (the actual situation is even "the same day").
KubeVela's design and realization of "abstraction"
As the platform team of Alibaba, in the practice of large-scale cloud-native application infrastructure, we also encountered the problems and challenges of how to design "abstraction". After a lot of attempts and conclusions, we finally chose the technology combination of GitOps + IaC (Infrastructure as Code) together with the R&D efficiency team to solve this problem (for details, you can read this article: "Cloud Native Era, How will the application architecture evolve" ?" ).
Among them, GitOps is more an innovation in the delivery pipeline, and IaC exists to solve the problem of "abstraction". Specifically, the power of IaC is that its definition of "abstraction" is expressed through "templates." This means that behind an "abstraction", there is no need for complex server-side programming work such as CRD Operator. As a platform team, we only need to submit a template, and the user will "automatically" have the abstracted fields; and if we want to modify these abstractions For fields, we only need to update the corresponding template, and the abstraction on the user side will "automatically" change. The adjustment and update of this abstract mechanism does not require any recompilation and online links, so we call it "client abstraction".
Figure 5 The relationship between users, abstractions, templates and the original K8s API
In terms of specific implementation, Alibaba uses CUELang, a template language developed by Google, to define abstract templates. This is why KubeVela first open sourced the CUE-based abstract mechanism in the first phase. In terms of specific use, the platform team only needs to register (kubectl apply) the CUE template to the Kubernetes cluster in accordance with the OAM specifications (ie WorkloadDefinition and TraitDefinition objects), and business users can immediately use this abstraction (the specific usage will be detailed later. Description).
On the other hand, the reason why CUE is favored by Google and Ali lies in its relatively complete abstraction layer implementation capabilities. For example, we have summarized the three abstract modes above. Among them, "conversion abstraction" and "combination abstraction" are easy to do during template rendering. It is nothing more than changing the field name when template rendering, and the generated content becomes multiple objects. However, it is actually very difficult to split abstraction, which involves the independent operation of the split capabilities and the final patch-merge process of the two capabilities.
With KubeVela, this split is relatively simple. Taking the previously mentioned definition process of decoupling business container and Sidecar container as an example, we hope to separate "Define Business Container" and "Define Sidecar Container" into two different forms on the user side. In the specific implementation, the platform team only needs to register a WorkloadDefinition object (named worker), which carries the deployment template of the business container, and then register a TraitDefinition object (named sidecar), which only carries the template of the Sidecar container, then KubeVela Two sets of completely independent configurable items, worker and sidecar, will be exposed to the user side, so that the user can only fill in the business container parameters in the worker during deployment, while the operation and maintenance can independently fill in the configuration parameters of the sidecar in the subsequent operation and maintenance. , Without any modification to the user's worker part at all.
Figure 6 An example of KubeVela "splitting" the Kubernetes API
Of course, in addition to CUE, the above abstract mechanism can also be implemented through Helm, and seamlessly integrated with the GitOps pipeline. This feature will be released as the next important feature of KubeVela. At that time, we will share cases and best practices of building a continuous delivery system based on KubeVela.
2. Quickly build user interface
With the above-mentioned "abstractions", as the end users of the platform, the business team can use these abstractions in some way to deliver and manage applications. At this level, KubeVela does not make any constraints. On the contrary, its goal is to allow abstractions to be directly exposed on the user interface, so that when the platform team adjusts these abstractions, business users can immediately To use the latest abstraction, there is no need to update or upgrade the system.
In terms of specific implementation, KubeVela will automatically generate a JSON schema for the above abstraction. The content of this JSON schema is the list of parameters and types that the abstract allows users to fill in. So whether it is a graphical interface or other user interfaces, you can directly use this JSON schema to render user forms and even generate usage documents .
For example, in the previous example of decoupling the Sidecar container definition, KubeVela will expose two JSON schemas for users: one is used to define the parameter list of the business container, and the other is used to define the parameter list of the Sidecar container. The front end can be rendered into two independent Forms for users to fill out.
It is the above-mentioned IaC abstraction + automatic schema generation mechanism that makes it not only very simple to build user-oriented user interfaces based on KubeVela, but also highly extensible: as long as the templates behind these abstractions are modified by the platform administrator, they will be immediately reflected in On the user's graphical interface form, there is no need to upgrade the system and go online again.
In KubeVela, it has a built-in simplified graphical interface called Appfile, which actually displays the above abstract schema in YAML, allowing users to modify and configure. In the following example, we can visualize See how each “capability abstraction” (route, autoscaler, etc.) is embodied as a configurable item in Appfile.
Figure 7 Use KubeVela abstraction in Appfile
Figure 8 Use vela traits to view the registered capabilities
Figure 9 Use vela show to view the use document of the ability (automatically generated)
Currently, Appfile is the main user interface built in KubeVela that uses "abstraction". At the same time, the Dashboard and Restful API with the same mechanism are planned to be released in KubeVela in 2021 Q2, allowing users to define and use this abstract mechanism through a graphical interface and API.
It is worth mentioning that, as the native platform construction engine of Kubernetes, the above-mentioned abstraction mechanism of KubeVela and Appfile itself are all implemented in Kubernetes in a declarative API, where the CRD corresponding to Appfile is called the Application object .
Therefore, as a platform team, they register abstract templates through Definition CRD. As users of the platform, they actually use abstract templates through this Application CRD. The whole set of mechanisms is completely run as a Kubernetes plugin, providing the most native integrated And scalability.
3. Unified definition and management of cloud resources with Terraform
With the Definition + Application system (which is also the core content of the OAM specification), KubeVela can integrate more capability providers, such as Terraform , under a unified user experience and API .
Terraform is a well-known tool for creating cloud resources in the industry. Its rich ecology includes most of the cloud resources of all mainstream cloud vendors, and it is the best supplement to the lack of Kubernetes cloud resource management capabilities. Using Terraform in KubeVela to define and pull up cloud resources is very simple, as shown in the following figure:
Figure 10 KubeVela uses Terraform to pull up cloud resources
1) Platform team: Register cloud resource templates and abstractions
The work of the platform team is to define a WorkloadDefinition object named "aliyun-rds", and define a template for Terraform Alibaba Cloud RDS cloud resources in it. In the above example, we also wrote the Terraform configuration through CUE. This is because Terraform cloud resources support JSON format description, and CUE is a superset of JSON, so you can naturally use all the capabilities of Terraform.
Of course, on the other hand, we are also planning to support Terraform's HCL syntax as another template language of KubeVela. In the CUE template, we quote Alibaba Cloud's RDS definition and abstract it into a small number of user fields (parameters) such as user and password.
2) User: Define and use cloud resources
In this way, the user only needs to fill in a new Service in the Appfile, named sample-db and its type is the aliyun-rds defined above, and then define the user, password and other parameters provided in the template in this section.
In addition, users can also define data binding in the business application of express-server above, and fill in the configuration named sample-db and the name of the environment variable to which it maps.
Finally, the user only needs a vela up command, KubeVela will pull up the business container, and then automatically pass the link information returned by the Aliyun RDS created by Terraform to the business container. We can see in the last part that the application has been successfully started , And got the connection information of the database. Of course, the data transfer and orchestration functions in this process are also core capabilities built into KubeVela.
to sum up
As the first cloud-native platform construction engine on Kubernetes and the complete implementation of the OAM model, KubeVela provides platform builders with more capabilities than these, such as the upcoming open-source unified application grayscale framework, multi-cluster and multi-environment delivery components, CRD Controller's life cycle management, etc., are the core capabilities that KubeVela focuses on. Due to space limitations, we will not expand one by one. Everyone is very welcome to use and feedback in the community to learn more details.
Welcome to join the KubeVela community
As stated at the beginning, KubeVela is a community-built project initiated by the community, so in the early stage of the project, we have already harvested 38 contributors from dozens of different companies. This is a very open community. A large number of new features are being planned and implemented, and everyone's contributions, use and feedback are welcome.
If you want to better understand the KubeVela project, welcome to learn specific examples and manuals on its official website. More advanced, you can try to add plug-in capabilities from the open source community to KubeVela . In addition, if you have any wonderful ideas about extending KubeVela, such as developing your own cloud-native database scenario PaaS or AI scenario PaaS based on KubeVela, welcome to the OAM community to discuss through Issue.