Many enterprises choose Kubernetes as their container orchestration system when deploying containers. It's a nod to Kubernetes' reliability, flexibility, and wide-ranging features. In this article, we'll take a deep dive into how Kubernetes handles a very common and necessary job - load balancing. Load balancing is a relatively simple task in many non-container environments (i.e. balancing between servers), but when containers are involved, some additional, special handling is required.
Manage containers
To understand Kubernetes load balancing, you first need to understand how Kubernetes organizes containers. Containers are typically used to execute a specific service or set of services, so they need to be viewed in terms of the service they provide, not just as a single instance of a service (ie, a single container). In fact, that's what Kubernetes does.
Put them in Pods
In Kubernetes, a pod is a basic unit of functionality. A pod is a set of containers and their shared volumes. Containers are often closely related in terms of functionality and services. Pods with the same feature set are abstracted into collections, called services. These services are accessed by clients of Kubernetes-based applications; the services in these individual pods, in turn, can manage access to the containers that make them up, keeping clients isolated from the containers themselves.
Manage Pods
Now let's look at some specifics. Pods are typically created and destroyed by Kubernetes and are not designed to be persistent entities. Each pod has its own IP address (based on the local address), UID, and port number; newly created pods, whether they are the current pod or a copy of a previous pod, are assigned a new UID and IP address. Communication between containers is possible within each pod, but it cannot communicate directly with containers in different pods.
Let Kubernetes handle transactions
Kubernetes uses its own built-in tools to manage communication with individual pods. This shows that in general, it is enough to rely on Kubernetes to monitor pods internally, without worrying about the creation, deletion or replication of pods. However, there are times when at least some internal elements of a Kubernetes-managed application are required to be visible to the underlying network. When this happens, the plan must consider what to do with the lack of a permanent IP address.
Pods and Nodes
In many ways, Kubernetes can be thought of as a pod management system, just like a container management system. Most infrastructure deals with containers at the pod level, not at the container level. From the perspective of Kubernetes internal management, the organizational level above a pod is equivalent to a node, which is a virtual machine that contains management and communication resources and is an environment for deploying pods. Nodes themselves can also be created, destroyed and replaced/redeployed internally. Whether at the node or pod level, their creation, destruction, redeployment, usage, and scaling functions are handled by an internal process called the Controller.
A "service" that acts as a scheduler
A service is how Kubernetes handles containers and pods at the management level. But as we mentioned above, it also abstracts functionally related or identical pods into services, and Kubernetes is at the service level when external clients and other elements in the application interact with the pods. Services have relatively stable IP addresses (used internally by Kubernetes). When a program needs to use functionality from a service, it makes a request to the service, not to a single pod. The service then acts as a scheduler, assigning a pod to handle the request.
Scheduling and Load Distribution
Seeing this, you may think, will load balancing be performed at the scheduling level? That's true. A Kubernetes service is a bit like a huge pool of devices, feeding identical machines into designated areas as needed. As part of the scheduling process, it needs to take full account of managing availability to avoid resource bottlenecks.
Let kube-proxy perform load balancing
The most basic type of load balancing in Kubernetes is actually load distribution, which is easy to implement at the scheduling level. Kubernetes uses two methods of load distribution, both performed through the kube-proxy function, which manages the virtual IPs used by the service. The default mode of Kube-proxy is iptables, which supports fairly sophisticated rule-based IP management. In iptables mode, the native method of load distribution is random selection - random selection of a pod in a service by an incoming request. The kube-proxy mode of earlier versions (and the original default mode) was userspace, which used round-robin load distribution to assign the next available pod on the IP list, and then replaced (or permuted) the list.
True Load Balancing: Ingress
We mentioned two methods of load balancing earlier, however, these are not true load balancing. In order to achieve true load balancing, the most popular, flexible, and widely used method is Ingress, which operates through a controller in a dedicated Kubernetes pod. The controller consists of an Ingress resource - a set of rules that govern traffic and a daemon that applies those rules. Controllers have their own built-in load balancing features, with some fairly sophisticated functionality. You can also allow Ingress resources to include more complex load balancing rules to meet the load balancing capabilities and needs of specific systems or providers.
Use a load balancer as an alternative
Instead of Ingress, you can use a load balancer type service instead. The service uses a cloud-based external load balancer. Load balancers can only be used with specific cloud service providers such as AWS, Azure, OpenStack, CloudStack, and Google Compute Engine, and the capabilities of the balancer vary by provider. Other load balancing methods are available from service providers and third parties.
Overall, I still recommend Ingress
Currently Ingress is the preferred method of load balancing. Because it executes inside Kubernetes as a pod-based controller, access to Kubernetes functionality is relatively unrestricted (unlike external load balancers, some of which may not be accessible at the pod level). The configurable rules contained in the Ingress resource support very detailed and highly granular load balancing, which can be customized according to the functional requirements of the application as well as the operating conditions.