As more and more enterprises and applications turn to cloud-native architecture, the requirements for Kubernetes clusters are becoming increasingly diverse and flexible. For fault isolation considerations, enterprise data centers often require multiple independent Kubernetes clusters to carry different services, rather than all being deployed on a shared Kubernetes cluster.
In addition, since business loads are generally characterized by dynamic changes, the scale of each Kubernetes cluster also needs to be dynamically changed to adapt to the increasing business load, and when the business load decreases, computing resources can be released in a timely manner to save overhead. These Kubernetes cluster operation and maintenance requirements can be collectively classified into Kubernetes as a service, that is, KaaS service.
On mainstream public clouds, KaaS is a very important and core component. Public cloud KaaS can quickly and easily create production-ready, high-availability Kubernetes clusters for its users, and provide subsequent operation and maintenance operations for cluster expansion and reduction. Generally speaking, the nodes of the Kubernetes cluster provided by public cloud KaaS are virtual machines to improve the utilization of physical resources.
In private enterprise data centers, KaaS requires the enterprise IT department to build it from scratch, including the procurement and integration of multiple systems such as virtualization platforms, Kubernetes management platforms, and user terminals, which is often time-consuming and expensive. In addition, traditional mainstream virtualization platforms often integrate too many functions because they need to support a wide range of virtualization scenarios. When they only support Kubernetes clusters, they appear cumbersome and inefficient.
To this end, SmartX has open sourced Virtink , the Kubernetes native lightweight virtualization platform based on Cloud Hypervisor , and further open sourced the knest command line tool based on Virtink to help users create and operate Kubernetes clusters on Virtink with one click to more efficiently Provide KaaS capability foundation for private data centers in a direct, simpler and more efficient way. This article will introduce the functions and uses of knest in detail to help users know and understand knest.
Prepare Kubernetes host cluster
As we mentioned above, Virtink is Kubernetes' native virtualization platform, which means that it needs to be deployed and run on a Kubernetes cluster. To avoid confusion, in the following content, our Kubernetes cluster is called the Kubernetes host cluster, and the Kubernetes cluster deployed on the Virtink virtual machine is called the Kubernetes virtual machine cluster.
It is worth noting that the nodes of the Kubernetes host cluster do not have to be physical machines, but can also be virtual machines that support nested virtualization. If you already have an existing Kubernetes cluster, you can choose to use it as the next host cluster, but you need to ensure that each node in the cluster supports KVM.
If you currently do not have an available Kubernetes cluster, you can choose to use minikube to create a temporary Kubernetes cluster as a host cluster locally, for example:
minikube start --memory 6g --cni calico.yaml |
Here we choose not to use minikube's default CNI but to use Calico because Calico in the Kubernetes virtual machine cluster to be created next requires the CNI of the Kubernetes host cluster to support the IPIP package in the pod. Minikube's default CNI does not support this feature because Calico is chosen. Moreover, Calico turns off this feature by default. We need to add an environment variable to turn on this feature:
curl -LO https://projectcalico.docs.tigera.io/manifests/calico.yaml sed -i '/^ # Use Kubernetes API as the backing datastore.$/i \ \ \ \ \ \ \ \ \ \ \ \ - name: FELIX_ALLOWIPIPPACKETSFROMWORKLOADS\n value: "true"' calico.yaml |
Use knest to create a Kubernetes virtual machine cluster
First, if you have not installed knest, please select the binary file of the corresponding platform in the latest release of the project and install it, for example:
curl -LO https://github.com/smartxworks/knest/releases/download/v0.6.0/knest-linux-amd64 sudo install knest-linux-amd64 /usr/local/bin/knest |
Next, use knest to create a cluster of Kubernetes virtual machines:
knest create demo --pod-network-cidr 10.245.0.0/16 --service-cidr 10.112.0.0/12 |
It is worth noting that the pod network segment and service network segment of the Kubernetes virtual machine cluster must be separated from the Kubernetes host cluster, and there must be no overlap between network segments.
If your Kubernetes host cluster is minikube, then the default pod network segment and service network segment of minikube are 10.244.0.0/16 and 10.96.0.0/12 respectively, so here you choose to go up a network segment as the Kubernetes virtual machine cluster pod network segment and service network segment.
Expanding the Kubernetes virtual machine cluster
The Kubernetes virtual machine cluster created above contains only one control plane node and no worker nodes. If your Kubernetes host cluster has sufficient computing resources, you can choose to use the knest expansion command to add more control plane nodes and worker nodes.
And if you are using minikube as the Kubernetes host cluster, and you have at least 10G of available memory locally, you can choose to expand the content of the minikube cluster to 10G, and then use knest to add a worker node to the Kubernetes virtual machine cluster:
knest scale demo --worker-machine-count 1 |
Create a persistent Kubernetes virtual machine cluster
The Kubernetes virtual machine cluster created above uses Virtink's ContainerRootfs type storage by default. ContainerRootfs storage can use the directory in the Docker image as the rootfs of the virtual machine, so it can fully utilize the Docker tool chain to build the virtual machine image, but it cannot be persisted.
If the virtual machine is shut down and then started again, its rootfs will be rebuilt from the Docker image, that is, files saved to the rootfs while the virtual machine is running cannot be persisted. If you want each node of your Kubernetes virtual machine cluster to be persistent, knest also supports creating persistent Kubernetes virtual machine clusters, for example:
knest create demo --pod-network-cidr 10.245.0.0/16 --service-cidr 10.112.0.0/12 --persistent --host-cluster-cni calico --machine-addresses 10.244.0.100-10.244.0.110 |
It is worth noting that considering that the nodes of the Kubernetes virtual machine cluster need fixed IPs, it is recommended that the CNI of the Kubernetes host cluster choose a CNI that supports fixed IPs, such as Calico and kube-ovn.
When using knest to create a persistent Kubernetes virtual machine cluster, you need to provide the CNI type of the Kubernetes host cluster through the --host-cluster-cni parameter, so that knest can mark the IP on the VM in an appropriate way, so that it can finally Correctly obtain fixed IP via CNI.
In addition, you also need to provide a certain number of fixed IPs through the --machine-addresses parameter to meet the fixed IP allocation needs of the Kubernetes virtual machine cluster.
Summarize
Building Kubernetes as a Service in an enterprise data center is not an easy task, which involves the integration of multiple platforms such as virtualization platform, Kubernetes management operation and maintenance platform, and user terminals. SmartX's open source knest tool, based on its other open source Kubernetes native lightweight virtualization engine Virtink project, provides one-click creation and operation of efficient, secure, and low-cost Kubernetes virtualization clusters, which is an ideal foundation for data center KaaS. member.
You are welcome to scan the QR code below, add the SmartX open source community administrator WeChat, join the Virtink community, and discuss Knest and Virtink with cloud native experts and engineers. Or you can also contact us via email ( [email protected] ) to communicate your ideas and questions.
