Service, service, used to expose Pod for access.
Official documents and manuals
Kubernetes API v1.18/Service v1 core
Service ?
Pod will be created, and will disappear, which is controlled by ReplicaSets. Each Pod has its own IP address, but these IP addresses cannot be considered reliable.
So, if a part of the front-end Pods depend on the back-end Pods, how do these front-end Pods find and track the back-end Pods?
Service
Service is an abstraction that defines the logical collection of Pods and the policy of who can access them (sometimes called microservices). The set of Pods pointed to by the Service is determined by the Label Selector (there is no selector by the Write Service).
In a scenario, the back end runs three copies of a certain mirror, and the front end does not care which copy is accessed. The back-end Pod copy may change, but the front-end should not care about or track the changes in the back-end. The service abstraction provides this decoupling.
? ? ? For Kubernetes native applications, Kubernetes provides a simple Endpoints API, which will be updated whenever the Pod set in the service changes. For non-native applications, Kubernetes provides virtual IP-based service bridging, redirecting to the backend Pod
Defining a Service
Service is also a REST object, like a Pod. Submit the definition to the apiserver to create the object.
Assuming that a Pod exposes port 9376 , and the label is app: MyApp , the Service to the Pod set can be defined as follows:
# foo.yaml
# Define a service
kind: Service
apiVersion: v1
metadata:
# Service name
name: my-service
spec:
selector:
# Traffic is sent to the Pod labeled "app: MyApp"
app: MyApp
ports:
-protocol: TCP
port: 80
# The traffic is sent to this port of the Pod.
targetPort: 9376
? ? ? Service will be assigned an IP address (sometimes called Cluster IP), which will be used by the service agent. The Service's selector will be continuously evaluated and the results will be published to the Endpoints object named "my-service".
Note that Service can map an entry port to any targetPort. The default port and targetPort are the same.
The targetPort can also be a string that points to the port name of the backend Pod ( defined by containers.ports.name ). The port number assigned to this string can be different in each Pod. This is very helpful to deploy and upgrade Service. For example, the next time the Pod version is upgraded, the port can be modified without breaking the client.
The TCP protocol is used by default . Refer to the supported protocol manual for the supported protocols . Some services need to expose multiple ports. You can define multiple ports in the Service and use different protocols.
Services without selectors
In addition to abstracting Pods, other types of backends can also be abstracted, such as:
The production environment has an external database cluster, but you want to use your own database in the test. You want to point the Service to a Service in another Namespace (or other clusters) to migrate the workload to the k8s cluster, but some back-end services run outside of k8s
For the above scenarios, you can do this:
# service-foo.yaml
# Define a back-end service without a selector, so the corresponding EndPoints object
kind: Service
apiVersion: v1
metadata:
name: my-service
spec:
ports:
-protocol
: TCP
port: 80 targetPort : 9376
# and then map the Service manual to a particular backend
# Endpoints-bar.yaml
kind: Endpoints
apiVersion: v1
the Metadata:
same as # name of the Service
name: My-Service
Subsets:
- Addresses:
- ip: 1.2.3.4
the ports :
-port: 9376
#! ! ! The IP of Endpoints may not be loopback (127.0.0.0/8), link local (169.254.0.0/16) or link local multicast (224.0.0.0/24).
#! ! ! They cannot be the cluster IP of other Kubernetes services, because the kube-proxy component does not support VIP as the target address.
Traffic arriving at the Service will be sent to the address defined by Endpoints (1.2.3.4:9376).
ExternalName
It is a special case of Service. It does not have a selector but uses a DNS name. For details, see the ExternalName section later in this document.
Endpoint Slices
Refer to EndpointSlice section
Virtual network address / service agent
Each node in the cluster has a kube-proxy service, which is responsible for implementing a form of virtual network address for Service (except ExternalName).
In Kubernetes v1.0, Service is a "layer 4" (IP-based TCP/UDP) structure, and the proxy is purely in the user space.
In Kubernetes v1.1, the Ingress API (beta) was added to represent the "layer 7" (HTTP) service, and the iptables proxy was also added, and it became the default operation mode since Kubernetes v1.2.
In Kubernetes v1.8.0-beta.0, the ipvs proxy has been added.
The following are the three modes of kube-proxy operation:
Proxy mode: userspace
In this mode, kube-proxy observes the Kubernetes master node, then deletes and adds Service and Endpoints objects. Open a port (random) for each Service on the local node, and any connection that accesses the "proxy port" will be proxied to the backend Pod of the Service. As for which backend Pod to use, it is based on the SessionAffinity of the Service. Finally, it installs firewall rules, captures the traffic of the Service's Cluster IP (virtual) and port, and then redirects the traffic to the "proxy port".
The default backend is cyclically selected.
Client => ClusterIP:Port(iptables) => proxy port(kube-proxy) => Backend Pod 1 ~ N
("Open a port on the local node for each Service", the port number exists on the Cluster IP)
Proxy mode: iptables
In this mode, kube-proxy observes the Kubernetes master node, then deletes and adds Service and Endpoints objects. For each Service, it installs firewall rules, captures the service's Cluster IP (virtual) and port traffic, and then redirects the traffic to the Service's backend collection. For each Endpoints object, it installs firewall rules to select the backend Pod.
The default backend is cyclically selected.
Client => ClusterIP:Port(iptables) => Backend Pod 1 ~ N ^ | kube-proxy <- apiserver
Proxy mode: ipvs
In this mode, kube-proxy observes Kubernetes Service and Endpoints, then calls the netlink interface to create ipvs rules, and periodically synchronizes ipvs rules with k8s Service and Endpoints to ensure that the ipvs status meets expectations. When accessing the service, the traffic will be redirected to one of the backend Pods.
Similar to iptables, ipvs is based on netfilter hook functions, but uses hash tables as the underlying data structure and works in the kernel space. This means that ipvs redirects traffic faster and is more efficient when synchronizing rules. And ipvs provides a better load balancing algorithm:
rr: round-robin lc: least connection dh: destination hashing sh: source hashing sed: shortest expected delay nq: never queue
To use the ipvs mode, you need to install the IPVS kernel module on the node, and then run the kube-proxy service. When kube-proxy is started in ipvs proxy mode, kube-proxy will verify whether the IPVS module is installed on the node. If it is not installed, kube-proxy will fall back to iptables proxy mode.
Client => ClusterIP:Port(Virtual Server(ipvs)) => Backend Pod 1 ~ N ^ | kube-proxy <- apiserver
Summary of the agency model
In any proxy mode, the traffic accessing IP:Port is proxied to the corresponding backend, without the Client knowing any k8s Service, Endpoints or Pod. Session affinity based on Client-IP can be achieved by setting service.spec.sessionAffinity to ClientIP . Based on this, you can also set service.spec.sessionAffinityConfig.clientIP.timeoutSeconds to achieve the maximum Session paste time (default 10800).
Use kubectl exec -n kube-system -it "<dsname>" - cat /var/lib/kube-proxy/config.conf command to view the current mode of kube-proxy.
# TODO !!
How to set kube-proxy settings using kubectl on AKS
Manage kube-proxy by using IPVS
If you want to modify the configuration file, you can use: kubectl -n kube-system edit configmap kube-proxy
Multi-Port Services
Multi-port Service needs to specify the port name:
kind: Service
apiVersion: v1
metadata:
name: my-service
spec:
selector:
app: MyApp
ports:
#
name- name: http
protocol: TCP
port: 80
targetPort: 9376
#
name- name: https
protocol: TCP
port: 443
targetPort : 9377
#! ! ! The name can only use lowercase alphanumerics and horizontal lines, and must start and end with alphanumeric
#! ! ! Valid: 123-web or web
#! ! ! Invalid: 123_abc
Choosing your own IP address
When the service is created, the Cluster IP address can be specified. By setting the .spec.clusterIP field. For example, you have a DNS entry that you want to reuse, or an IP address is used in the system and is difficult to configure. The IP address selected by the user must be the CIDR range set in the service-cluster-ip-range of the API Server. If the IP address is invalid, the API Server returns a 442 status code.
Why not use round robin DNS?
Why use IPVS to implement Round-Robin, etc., instead of Round-Robin DNS:
The DNS library has never followed the DNS TTL and does not cache the results of name lookups. Many applications perform a DNS lookup once and cache the results. Even if the application and library are properly re-parsed, the load of each client's repeated re-resolution of DNS is difficult to manage.
The official also said that if there are many people who want to use circular DNS, they will also implement this function.
Discovering services
Two ways to find services: (1) environment variables; (2) DNS;
Environment variable
When running Pod on Node, kubelet will add a series of environment variables to the active Server. It supports Dokcer link compatible environment variables, {SVCNAME}_SERVICE_HOST and {SVCNAME}_SERVICE_PORT variables. The service name will be converted to uppercase, and the horizontal line will be replaced by an underline.
For example, the service "redis-master", port "6379", cluster IP address "10.0.0.11", will generate the following environment variables:
REDIS_MASTER_SERVICE_HOST=10.0.0.11 REDIS_MASTER_SERVICE_PORT=6379 REDIS_MASTER_PORT=tcp://10.0.0.11:6379 REDIS_MASTER_PORT_6379_TCP=tcp://10.0.0.11:6379 REDIS_MASTER_PORT_6379_TCP_PROTO=tcp REDIS_MASTER_PORT_6379_TCP_PORT=6379 REDIS_MASTER_PORT_6379_TCP_ADDR=10.0.0.11
Implied requirement: The service accessed by the Pod must be created first, otherwise the environment variables of the service will not be written to the Pod (there is no such restriction when using DNS).
DNS
By installing a cluster plug-in that provides DNS services.
The cluster-aware DNS Server will observe the service creation of the K8s API, and then generate a DNS record for each service. If DNS is enabled in the entire cluster, all Pods should be able to automatically perform name resolution on services.
For example, if the service is "my-service" and the namespace is "my-ns", the DNS is "my-service.my-ns". Pods in the same namespace can be accessed directly using "my-service", and services in different namespaces can be accessed using "my-service.my-ns". The result of the analysis is the IP address of the cluster.
In addition, it also supports DNS SRV records. For example, the Service of "my-service.my-ns" has a port named "http", using the "tcp" protocol, you can perform DNS SRV query "_http._tcp.my-service.my-ns" to discover The port number.
Kubernetes DNS is the only way to access ExternalName services. For more information, refer to the instructions in DNS Pods and Services .
Headless Services
Sometimes load balancing or service network address is not needed, you can set .spec.clusterIP to "None" to create a headless service.
This option allows developers to "reduce the coupling with the Kubernetes system by allowing them the freedom to discover in their own way". Applications can still use the self-registration mode, and can easily build adapters for other discovery systems on this API.
For such services, no Cluster IP is assigned, kube-proxy does not handle these services, and the platform does not perform load balancing or proxying for them. How to configure DNS automatically depends on whether the service has a selector defined:
Selector defined
If you have defined a selector's headless service, the Endpoints Controller will create an Endpoints record in the API and modify the DNS configuration to return an A record, which directly points to the Pod at the back end of the Service.
Undefined selector
If no selector is defined, Endpoints Controller will not create Endpoints records. However, the DNS system will find and configure:
1) CNAME records
for services of type ExternalName 2) Records for any Endpoints of all other types that share names with the service
Publishing Service-Service Type
Sometimes we want to expose the service for external access.
You can use ServiceTYpes to specify the type of service. The default is ClusterIP
The behavior of the type is as follows:
ClusterIP-Expose the service on the IP address inside the cluster, which can only be accessed from inside the cluster.
NodePort-Expose a port (NodePort) on each Node's IP address. **A ClusterIP service will be created automatically**, and NodePort services will be routed to this service. You can use NodeIP: NodePort to access the service.
LoadBalancer-Use cloud provider's load balancer to expose services externally. The "**NodePort** and **ClusterIP** services to which the "external load balancer will be routed" will be created automatically.
ExternalName-maps the service to the content of the externalName field by returning a CNAME record with its value (for example, foo.bar.example.com). No proxy of any kind is set up. This requires kube-dns version 1.7 or higher
NodePort
Assign a port from the range (30000-32767) specified by the --service-node-port-range flag, and each node will proxy the port to the service. Check the port in the .spec.ports[*].nodePort field of the service.
If you want to specify a specific IP address for the proxy port, you can set the --nodeport-addresses flag in kube-proxy to a specific IP block (supported since Kubernetes v1.10). A comma-separated list of IP blocks (for example, 10.0.0.0/8, 1.2.3.4/32) is used to filter the local address of this node. For example, if you use the flag --nodeport-addresses=127.0.0.0/8 to start kube-proxy, kube-proxy will only select the loopback interface for the NodePort service. --nodeport-addresses is empty by default ([]), which means that all available interfaces are selected and conform to the current NodePort behavior.
If you need a specific port number, you can specify a value in the nodePort field, and the system will assign the port for you, otherwise the API transaction will fail (that is, you need to deal with possible port conflicts yourself). The value you specify must be within the configuration range of the node port.
This allows developers to freely set up their own load balancers, configure environments that Kubernetes does not fully support, and even directly expose the IP of one or more nodes.
Please note that this service will be displayed as <NodeIP>:spec.ports[*].nodePort and .spec.clusterIP:spec.ports[*].port at the same time . (If the --nodeport-addresses flag in kube-proxy is set, NodeIP will be filtered.)
LoadBalancer
On cloud vendors that support the provision of external load balancers, setting the type to LoadBalancer will specify a load balancer for your Service. The actual creation of the load balancer occurs asynchronously, and the information about the configured balancer will be published in the .status.loadBalancer field of the Service . E.g:
kind: Service
apiVersion: v1
metadata:
name: my-service
spec:
selector:
app: MyApp
ports:
- protocol: TCP
port: 80
targetPort: 9376
clusterIP: 10.0.171.239
loadBalancerIP: 78.11.24.19
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: 146.148.47.155
Traffic from the external load balancer is redirected directly to the Pod, although how it works depends on the cloud provider.
Some cloud providers allow the loadBalancerIP field to be specified. In these cases, in these cases, the load balancer IP specified by the user will be used to create a load balancer. If the loadBalancerIP field is not specified, a temporary IP will be assigned to loadBalancer. If loadBalancerIP is specified, but the cloud provider does not support this function, this field will be ignored.
# TODO Load balancing type Service service
ExternalName
This type of service maps the service to a DNS name instead of a selector. You can use spec.externalName to specify parameters:
kind: Service apiVersion: v1 metadata: name: my-service namespace: prod spec: type: ExternalName externalName: my.database.example.com # ExternalName accepts an IPv4 address string, but is interpreted as a DNS name composed of numbers, and Not a network address. # ExternalNames similar to IPv4 addresses will not be resolved by CoreDNS or ingress-nginx, because ExternalName is designed to specify canonical DNS names. # To hard-code the IP address, consider headless services.
When looking up the my-service.prod.svc.cluster.local domain name, the DNS service will return a CNAME with a value of my.database.example.com. When accessing this service, there is no difference from accessing other services. The key is that this occurs at the DNS level, which is proxy or forwarded from time to time. Later, if you want to migrate external services to k8s, you only need to create the corresponding Pod, add selectors or Endpoints, and then modify the type field.
External network address
If there are external IPs routed to one or more cluster nodes, Kubernetes services can be exposed on these external IPs. Traffic entering the cluster using the external IP (as the destination IP) on the Service port will be routed to one of the service Endpoints. externalIPs are not managed by Kubernetes and are the responsibility of the cluster administrator.
In the Service Spec, externalIPs can be specified together with any ServiceTypes. In the following example, the client can access "my-service" on "80.11.12.10:80" (externalIP: port):
kind: Service
apiVersion: v1
metadata:
name: my-service
spec:
selector:
app: MyApp
ports:
- name: http
protocol: TCP
port: 80
targetPort: 9376
externalIPs:
- 80.11.12.10
Disadvantage
Agents in the user space using VIP will work on small and medium scales, but will not scale to very large clusters with thousands of services. For more information, please refer to the original design proposals portal .
Using a user space proxy will obscure the source IP of the data packets accessing the service, which makes certain types of firewalls impossible. The iptables proxy will not mask the source IP in the cluster, but it will still affect clients passing through the load balancer or node port.
The "type" field is designed as a nested function-each level is added to the previous level. Not all cloud providers strictly require this (for example, Google Compute Engine does not need to assign a NodePort to make LoadBalancer work, but AWS will do it) but the current API requires it.
future job
In the future, we envision that proxy strategies can be more subtle than simple round-robin balances, such as main selection or sharding. We also envision that some services will have "real" load balancers, in which case the VIP will simply transmit packets there.
We intend to improve the support for L7 (HTTP) services.
We intend to provide a more flexible Ingress mode for Service, including the current ClusterIP, NodePort and LoadBalancer modes, etc.
More details about virtual IP
For many people who just want to use the service, the previous information should be enough. However, there are many things that may be worth understanding behind the scenes
avoid collision
One of the main philosophies of Kubernetes is that users should not be exposed to situations that may cause their actions to fail, rather than their own fault. In this case, we are looking for a network port-if the choice may conflict with another user, the user does not have to choose a port number. This is an isolation failure.
In order to allow users to choose port numbers for their services, we must ensure that no two services can conflict. We do this by assigning its own IP address to each service.
In order to ensure that each service receives a unique IP, the internal allocator atomically updates the global allocation map in etcd before creating each service. The mapping object must exist in the registry to obtain the service of the IP, otherwise the creation will fail and a message will be displayed indicating that the IP cannot be assigned. The backend controller is responsible for creating this mapping (migrating from the old version of Kubernetes used in memory lock) and checking for invalid allocations due to administrator intervention, and clearing any IPs that have been allocated but not currently used by services.
IP and VIP
Unlike the Pod IP address that is actually routed to a fixed destination, the service IP is not actually answered by a single host. Instead, we use iptables (packet processing logic in Linux) to define virtual IP addresses that are transparently redirected as needed. When the client connects to the VIP, its traffic is automatically transmitted to the appropriate endpoint. The environment variables and DNS of the service are actually filled in according to the VIP and port of the service.
We support three proxy modes-userspace, iptables and ipv, they operate in slightly different ways
Userspace
as an example, consider the above-mentioned mirroring application. When creating a back-end service, the Kubernetes master server will assign a virtual IP address, such as 10.0.0.1. Assuming the service port is 1234, all kube-proxy instances in the cluster will observe the service. When the agent sees a new service, it opens a new random port, establishes an iptables redirection from the VIP to this new port, and starts accepting connections on it.
When the client connects to the VIP, the iptables rule is activated and the data packet is redirected to the service agent's own port. The service proxy selects the backend and starts proxying the traffic from the client to the backend.
This means that service owners can choose any port they want without conflict. Clients can simply connect to the IP and port without knowing which Pods they are actually accessing.
Iptables
again, consider the above-mentioned mirroring application. When creating a back-end service, the Kubernetes master server will assign a virtual IP address, such as 10.0.0.1. Assuming the service port is 1234, all kube-proxy instances in the cluster will observe the service. When the agent sees a new service, it installs a series of iptables rules, which redirect from VIP to per-service rules. The per-service rule is linked to the per-endpoint rule, which redirects (target NAT) to the backend.
When the client connects to the VIP, the iptables rule starts. Select the backend (based on session affinity or random) and redirect the packet to the backend. Unlike user space proxy, data packets are never copied to user space, so there is no need to run kube-proxy to make VIP work, and the client IP will not be changed.
When traffic enters through a node port or through a load balancer, the same basic process is performed, but in these cases, the client IP will indeed be changed
Ipvs
Iptables operations have slowed down significantly in large-scale clusters, such as 10,000 services. IPVS aims to achieve load balancing and is based on a hash table in the kernel. Therefore, we can achieve performance consistency for a large number of services from the IPVS-based kube-proxy. At the same time, kube-proxy based on IPVS has a more complex load balancing algorithm (minimal conns, locality, weighting, persistence)
API object
Service is the top-level object in the Kubernetes REST API. For more details on API objects, please refer to the Service API object manual.
Supported protocol
TCP
Kubernetes v1.0 TCP
UDP
Kubernetes v1.0 UDP can be used for most services. For services of type=LoadBalancer, whether it is supported depends on the facilities provided by the cloud provider.
HTTP
Kubernetes v1.1 HTTP If your cloud provider supports it, you can use the service in LoadBalancer mode to set up an external HTTP/HTTPS reverse proxy and forward it to the service endpoint.
PROXY
Kubernetes v1.1 PROXY If your cloud provider supports it (for example, AWS), you can use the service in LoadBalancer mode to configure a load balancer outside of Kubernetes, and it will forward connections with the PROXY protocol prefix. The load balancer will send a series of initial octets describing the incoming connection, similar to this example PROXY TCP4 192.0.2.202 10.0.42.7 12345 7\r\n, and then the data from the customer
SCTP
Kubernetes v1.12 SCTP Kubernetes supports SCTP as the protocol value defined in Service, Endpoint, NetworkPolicy, and Pod as an alpha function. To enable this feature, the cluster administrator needs to enable the SCTPSupport feature gate on the apiserver, for example "--feature-gates=SCTPSupport=true,...". After enabling the function, users can set the protocol fields of Service, Endpoint, NetworkPolicy, and Pod to SCTP. Kubernetes sets up the network for SCTP associations accordingly, just like it does for TCP connections.
Support for multi-homed SCTP association Support for multi-homed SCTP association
requires that the CNI plug-in can support the assignment of multiple interfaces and IP addresses to Pods.
The NAT used for multi-homed SCTP association requires special logic in the corresponding kernel module.
Service with type=LoadBalancer
can only create a service with type LoadBalancer and protocol SCTP when the cloud provider's load balancer implementation supports SCTP as the protocol. Otherwise, the service creation request will be rejected. The current cloud load balancer provider sets (Azure, AWS, CloudStack, GCE, OpenStack) do not support SCTP.
Windows Windows-
based nodes do not support SCTP.
User space kube-proxy
when kube-proxy mode in user space, it does not support management SCTP association.
Related Links
About Service network: Understanding kubernetes networking: services
related articles
"Kubernetes Objects"-CronJob (study notes)
"Kubernetes"-Use storage (study notes)
"Kubernetes Objects"-Pod (study notes)
"Kubernetes Objects"-ConfigMap (study notes)
"Kubernetes Objects"-Managing Compute Resources (study) notes)
references
kubernetes/CONCEPTS/Services
in json schema for service port is a number, but containerPort is a string #2123
Get a Shell to a Running Container
kube-proxy