K8S series (11) Kubernetes network concept and policy control

Four goals

The four goals are actually when designing a K8s system to provide services to the outside world, from the perspective of the network, how to connect the outside world to the application inside the container step by step?

• How is the communication between the outside world and the service? There is an Internet or a user outside the company, how to use the service? service specifically refers to the service concept in K8s.
• How does a service communicate with its backend pods?
• How do calls between pods communicate with each other?
• Finally, is the communication between containers inside the pod?

The ultimate goal is to connect the outside world to the innermost part and provide services to the container.

Explanation of Basic Constraints

For the basic constraints, some interpretations can be made: because the complexity of container network development lies in that it is actually parasitic on the Host network. From this perspective, container network solutions can be roughly divided into two factions: Underlay and Overlay :

• Underlay's standard is that it is on the same layer as the host network. One of the characteristics that can be seen from the outside is whether it uses the same network segment of the host network, the input and output infrastructure equipment, and whether the IP address of the container needs to be coordinated with the host network. (from the same central distribution or unified division). This is Underlay;
• The difference between Overlay is that it does not need to apply for an IP from the IPM management component of the Host network. Generally speaking, it only needs to not conflict with the Host network, and this IP can be freely allocated.

Why did the community propose such a simple and arbitrary model as perPodperIP? Personally, I think it brings a lot of benefits to manage the tracking performance monitoring of some services for the following services. Because an IP is consistent to the end, it will be of great benefit to the case or various small things.

2. Exploration of Netns

What exactly does Netns achieve

Let's briefly talk about the core basis of network implementation in Network Namespace. In a narrow sense, the runC container technology does not depend on any hardware. Its execution basis is its kernel. The kernel representative of a process is a task. If it does not need to be isolated, it uses the host space (namespace) and does not Spatially isolated data structures ( nsproxy-namespace proxy ) that require special setup.

On the contrary, if an independent network proxy, or mount proxy, it must be filled with real private data. The data structure it can see is shown above.

From the perspective of an isolated network space, it will have its own network card or network device. The network card may be virtual or physical, and it will have its own IP address, IP table and routing table, and its own protocol stack state. This specifically refers to the TCP/IP protocol stack, which will have its own status, and its own iptables and ipvs.

In terms of the whole sense, this is equivalent to having a completely independent network, which is isolated from the host network. Of course, the code of the protocol stack is still common, but the data structure is different.

This picture can clearly show the relationship between Netns in the pod. Each pod has an independent network space, and the pod net container will share this network space. Generally, K8s recommends using the Loopback interface to communicate between pod net containers, and all containers provide external services through the pod IP. In addition, for Root Netns on the host machine, it can be regarded as a special network space, but its Pid is 1

3. Introduction to mainstream network solutions

Typical container network implementation

Next, we briefly introduce typical container network implementation solutions. The container network solution may be the most flourishing field in K8s, and it has various implementations. The complexity of the container network lies in the fact that it needs to coordinate with the underlying Iass layer network, and needs to make some choices in terms of performance and IP allocation flexibility. There are various solutions.

The following briefly introduces several major solutions: Flannel, Calico, Canal, and finally WeaveNet. Most of the solutions in the middle adopt a strategy routing method similar to Calico.

• Flannel is a relatively large and unified solution that provides a variety of network backends. Different backends implement different topologies, which can cover a variety of scenarios;
• Calico mainly uses policy routing, and BGP protocol is used between nodes to synchronize routes. It is characterized by rich functions, especially the support for Network Point is better. We all know that Calico’s requirements for the underlying network generally require that the mac address can be directly connected and cannot cross the second layer domain;
• Of course, some students in the community will integrate the advantages of Flannel and Calico. We call it a grafted innovative project Cilium ;
• Finally, let’s talk about WeaveNet . If you need to encrypt data during use, you can choose to use WeaveNet. Its dynamic scheme can achieve better encryption.

Flannel scheme

The Flannel scheme is currently the most commonly used. As shown in the figure above, you can see a typical container network solution. The first thing it needs to solve is how to get the container's package to the Host. Here, it adopts the method of adding a Bridge. Its backend is actually independent, that is to say, how the package leaves the Host, which encapsulation method is used, or does not require encapsulation are all optional.

Now let's introduce the three main backends:

• One is udp in user mode, which is the earliest implementation;
• Then there is the kernel's Vxlan, both of which are overlay solutions. The performance of Vxlan will be better, but it has requirements for the version of the kernel, and the kernel needs to support the features and functions of Vxlan;
• If your cluster is not large enough and is in the same layer-2 domain, you can also choose to use the host-gw method. The backend of this method is basically started by a broadcast routing rule, and the performance is relatively high.

Fourth, the usefulness of Network Policy

Basic Concepts of Network Policy

Let's introduce the concept of Network Policy.

I just mentioned that the basic model of the Kubernetes network requires full interconnection between pods. This will cause some problems: in a K8s cluster, some call chains may not be called directly. For example, between two departments, then I hope that department A will not visit the services of department B. At this time, the concept of strategy can be used.

Basically, its idea is this: it uses various selectors (labels or namespaces), finds a set of pods, or finds the two ends of the communication, and then determines whether they can communicate with each other through the description of the characteristics of the flow , can be understood as a whitelist mechanism.

Before using Network Policy, please note that the apiserver needs to turn on these switches as shown in the figure above. Another more important thing is that the network plug-ins we choose need to support the implementation of Network Policy. Everyone should know that Network Policy is just an object provided by K8s, and there are no built-in components for implementation. It depends on whether the container network solution you choose supports or completes this standard. If you choose Flannel and the like, it If you haven't really implemented this policy, it won't help if you try it.

Next, let's talk about an example of configuration, or what should be done when designing a Network Policy? Personally, I feel that three things need to be decided:

• The first thing is to control the object, just like the spec part in this instance. In spec, through podSelector or namespace selector, you can choose to make a specific group of pods to accept our control;
• The second is to think clearly about the flow direction. Do you need to control the incoming or outgoing direction? Or is it controlled in both directions?
• The most important thing is the third part. If you want to add control objects to the selected direction to describe its flow, which streams can be put in or released? Analogous to the quintuple of the flow characteristics, some selectors can be used to determine which ones can be used as my remote end, which is the choice of the object; it is also possible to obtain which IPs can be released through the mechanism of IPBlock; the last is Which protocols or which ports. In fact, the flow characteristics are combined into a five-tuple, which will select specific acceptable flows.