Using Java to develop their own Kubernetes controller, you want to try?

Author | Nicolas Fränkel

Translator | Inf, Zebian | Xu Veyron

Cover photo | CSDN download the visual China

In this article, we will begin to develop their own Kubernetes controller.

Technology stack can be Python, NodeJS or Ruby. Because this blog is named as "Java Geeks" and therefore choose Java is normal.

As a use case, we will achieve sidecar mode: Whenever a pod is scheduled, sidecar pod will also be scheduled. If you delete the former, the latter must also be deleted.

Select the right tools

In order to perform REST call in Java, you first need to generate the binding. There are several ways to do this. The most cumbersome this operation is performed manually: the need to carefully control all of the possible combinations JSON request and response, the development of the respective Java object serialization framework and JSON select HTTP client. The second best option is to use a proprietary code generators, e.g. Swagger (https://swagger.io/) or Apiary (https://apiary.io/). This requires API provider model to provide a possible format. The downside is the need to use tools. Sometimes, form more or less open, e.g. OpenAPI specification (https://swagger.io/specification/). In this case, it is possible to achieve the selected tool from the format. In the best case, it may have provided binding.

Kubernetes is the case: The project provides its own binding (https://kubernetes.io/docs/reference/using-api/client-libraries/) for a variety of languages. The problem is that the language REST API wrapper and very close to the author too familiar. For example, this is to list all the names of all the space pod ways:

ApiClient client = Config.defaultClient();
CoreV1Api core = new CoreV1Api(client);
V1PodList pods =
    core.listPodForAllNamespaces(null, null, null, null, null, null, null, null);

Note: All null parameters need to be passed.

This is the "wrapper code is very close to the REST API" means. Fortunately, there is another option. Fabric8 organization provides a smooth Java API on Github. The above equivalent code is the code:

KubernetesClient client = new DefaultKubernetesClient();
PodList pods = client.pods().inAnyNamespace().list();    

注意：无需传递无用的null参数。

Fabric8 organization provides a smooth Java API on Github:

https://github.com/fabric8io/kubernetes-client)

Fabric8 quick overview

In simple terms, using Fabric8 the API, all Kubernetes resources can be used on KubernetesClient instance, for example:

client.namespaces()
client.services()
client.nodes()

Depending on the nature of the resource, its scope can be a namespace, you can not:

client.pods().inAnyNamespace()
client.pods().inNamespace("ns")

At this point, you can call this action:

Pod lists all the names of all space:

client.pods().inAnyNamespace().list();

Delete all pod namespace ns in:

client.pods().delete(client.pods().inNamespace("ns").list().getItems());

创建一个名为ns的新名称空间：

client.namespaces()
  .createNew()
    .withApiVersion("v1")
    .withNewMetadata()
      .withName("ns")
    .endMetadata()
  .done();

To achieve control loop

Note that, only one control loop Kubernetes controller, which monitors the state of the cluster, and coordinate with the desired state. In order to be able to schedule / delete events, use the Observer pattern. The application will subscribe to such incidents when they occur, associated callback will be triggered.

Below is a very simple API map:

To truly realize a monitoring program, simply execute the following lines of code:

public class DummyWatcher implements Watcher<Pod> {

  @Override
  public void eventReceived(Action action, Pod pod) {
    switch (action) {
      case ADDED:    //注意1        
        break;
      case MODIFIED: //注意2       
        break;
      case DELETED:  //注意3
        break;
      case ERROR:    //注意4       
        break;
    }
  }

  @Override
  public void onClose(KubernetesClientException cause) {
           //注意5               
  }
}

client.pods()
  .inAnyNamespace()
  .watch(DummyWatcher());

注意：

When adding a new pod works
Function when modifying existing pod
When delete a pod works
It works when an error occurs
Clean up any resources. If the client properly closed, cause will be null

details

Now, we have everything to achieve the required sidecar mode. I will not show all the code, which can be found on GitHub: https: //github.com/nfrankel/jvm-controller, but it is important to emphasize some of the key content.

1) mark sidecar

Essentially, the viewer need to add a sidecar pod when you add a new pod, and remove it when you remove it. This basic approach will not work: If you schedule a sidecar pod, it will trigger an observer, and add new sidecar pod sidecar. And this situation will continue. Thus, a label sidecar pod is essential. Detects such pod, it should not trigger the creation logic.

There are several ways of marking sidecar pod:

After the name of the sidecar POD plus specific string, e.g. sidecar
Add the following specific tags:

client.pods()
  .inNamespace("ns")
  .createNew()
    .withNewMetadata()
      .addToLabels("sidecar", "true")
    .endMetadata()
  .done();

2) together with the pod together to remove sidecar

pod should be only a sidecar. As mentioned above, it should be created when you add Pod, and delete it should delete the latter.

Therefore, a reference to the main pod should be added to the sidecar. Thus, when the pod is removed, if it's not a sidecar, we should find sidecar allocation and delete it.

The first method is simple to explicitly delete sidecar when deleting the main pod. But this is a lot of work, it takes a lot of time. Kubernetes allows you to bind the life cycle to another pod Pod lifecycle. Then, delete the logical processing by the Kubernetes itself. This supports the concept of ownerReference.

The API makes it easy to implement:

client.pods()
  .inNamespace("ns")
  .createNew()
    .withNewMetadata()
      .addNewOwnerReference()
        .withApiVersion("v1")
        .withKind("Pod")
        .withName(podName)
        .withUid(pod.getMetadata().getUid())
      .endOwnerReference()
    .endMetadata()
  .done();

3) 始终保持一个sidecar

Adding a sidecar does not mean it will always stay that way. For example, you can delete pod belonging to deployment. Deployment goal is to re-create a pod, to achieve the desired number of copies.

Similarly, if you delete a sidecar while retaining the main pod, you should use the correct reference to generate a new sidecar.

in conclusion

In this article, we describe how to use the Java language Kubernetes controller on the JVM. With Fabric8 the API, the operation is very simple to achieve . The main problem from the scheduler / delete logic in extreme cases. Under article in this series (last one) article, we will finally see how to deploy and run the code.

The complete source code for this article can be found at Maven format on Github:

https://github.com/nfrankel/jvm-controller

Hopefully this article useful to you, and we welcome the comments section to discuss.

Original: https: //blog.frankel.ch/your-own-kubernetes-controller/2/

【End】