Service Mesh's hottest project Istio layered architecture, do you really understand?

Author | Wang Xining Alibaba Senior Technical Expert

Participate in the message interaction at the end of the "Alibaba Cloud Native" public account, that is, have the opportunity to receive book donation benefits!

This article is taken from the book "Analysis and Practice of Istio Service Grid Technology" written by Alibaba Cloud senior technical expert Wang Xining. The article starts with basic concepts and introduces what is the service grid and Istio. The big development trend introduced systematically and comprehensively the relevant knowledge of Istio service grid. You just have to be happy to participate in the end of the article interaction, we are responsible for paying the bill! Technical man essential book "Istio service grid technology analysis and practice" free collar ~

Istio is an open source service grid that provides the basic operational and management elements required for a distributed microservices architecture. As organizations increasingly adopt cloud platforms, developers must use microservices to design architectures for portability, while operations personnel must manage large distributed applications that include hybrid cloud deployments and multi-cloud deployments. Istio uses a consistent way to protect, connect, and monitor microservices, reducing the complexity of managing microservice deployments.

From the perspective of architectural design, the Istio service grid is logically divided into two parts: the control plane and the data plane. Among them, the control plane Pilot is responsible for managing and configuring agents to route traffic, and configure Mixer to implement policies and collect telemetry data; the data plane is composed of a set of intelligent agents (Envoy) deployed in Sidecar mode, these agents can regulate and control microservices And all network communication between Mixer.

(Istio architecture)

As a proxy, Envoy is very suitable for service grid scenarios, but to maximize Envoy's value, it needs to make it work well with the underlying infrastructure or components. Envoy constitutes the data plane of the service mesh, and the supporting component provided by Istio creates the control plane.

(Pilot and Envoy data plane)

On the one hand, we saw in Envoy that a static configuration file or a set of discovery services can be used to configure a set of service agents to discover listeners, endpoints, and clusters at runtime. Istio implemented the xDS API of these Envoy proxies in Pilot.

On the other hand, Envoy's service discovery relies on a service registry to discover service endpoints. Istio Pilot implements this API, but also abstracts Envoy from any particular service registration implementation. When Istio is deployed on Kubernetes, the service registry of Kubernetes is used by Istio for service discovery. Other registries can also be used like HashiCorp's Consul. The Envoy data plane is completely unaffected by these implementation details.

(Mixer architecture)

In addition, the Envoy proxy can send out many indicators and telemetry data, and where to send these telemetry data depends on Envoy's configuration. Istio provides the telemetry receiver Mixer as part of its control plane, and the Envoy proxy can send this data to the Mixer. Envoy also sends distributed tracking data to an open tracking engine (following the Open Tracing API). Istio can support a compatible open tracking engine and configure Envoy to send its tracking data to that location.

Anatomy of the Istio control plane

Together, Istio's control plane and Envoy's data plane constitute a compelling service mesh implementation. Both have a thriving and vibrant community, and are geared towards the next-generation service architecture. Istio is platform independent and can run in a variety of environments, including cross-cloud, on-premises, Kubernetes, Mesos, etc. You can deploy Istio on Kubernetes or Nomad with Consul. Istio currently supports services deployed on Kubernetes, services registered with Consul, and services deployed on virtual machines.

Among them, the control plane part includes four components of Pilot, Mixer, Citadel and Galley. See a picture of Istio architecture.

1. Pilot

Istio's Pilot component is used to manage traffic. It can control the flow of traffic and API calls between services. Through Pilot, you can better understand the traffic so that you can find problems before they occur. This makes the call more reliable, the network is more robust, even if the adverse conditions are encountered, the application can be stabilized. With Istio's Pilot, you can configure service-level attributes such as fuses, timeouts, and retries, and set up common continuous deployment tasks, such as canary release, A / B testing, and phased release based on split traffic. Pilot provides service discovery functions for Envoy agents and traffic management functions for intelligent routing and resilience capabilities (such as timeout, retry, fuse, etc.). Pilot translates advanced routing rules that control traffic behavior into Envoy proxy-specific configurations and propagate them to Envoy at runtime. In addition, Istio provides powerful out-of-box failure recovery capabilities, including timeouts, retry mechanisms that support timeout budgets and variable jitter, concurrent connections to upstream services and limit on the number of requests, each member of the load balancing pool Conduct regular active health checks and passive health checks.

Pilot abstracts and synthesizes the platform-specific service discovery mechanism into a standard format, which can be used by any sidecar that conforms to the data plane API. This loose coupling allows Istio to run in multiple environments (such as Kubernetes, Consul, Nomad) while maintaining the same user interface for traffic management.

2. Mixer

Istio's Mixer component provides policy control and telemetry collection functions, isolating the rest of Istio from the implementation details of each back-end infrastructure back-end. Mixer is a platform-independent component that is responsible for performing access control and usage policies on the service grid, and collecting telemetry data from Envoy agents and other services. The agent extracts the request-level attributes and sends them to Mixer for evaluation.

Mixer includes a flexible plug-in model that enables it to connect to various host environments and back-end infrastructure, abstracting Envoy proxy and Istio managed services from these details. With Mixer, you can finely control all interactions between the grid and the backend infrastructure backend.

Unlike the Sidecar proxy, which must save memory, Mixer runs independently, so it can use considerable cache and output buffers to serve as Sidecar's highly scalable and highly available secondary cache.

Mixer is designed to provide high availability for each instance. Its local cache and buffers can reduce latency and also help shield back-end infrastructure back-end failures, even if the back-end is not responding.

3. Citadel

Istio Citadel security features provide powerful authentication, powerful policies, transparent TLS encryption, and authentication, authorization, and audit (AAA) tools to protect services and data. Envoy can terminate or initiate TLS to services in the grid flow. For this, Citadel needs to support the creation, signing and rotation of certificates. Istio Citadel provides application-specific certificates that can be used to establish mutual TLS to protect traffic between services.

(Istio Citadel architecture)

With Istio Citadel, you can ensure that services containing sensitive data can only be accessed from strictly authenticated and authorized clients. Citadel provides powerful service room and end user authentication through built-in identity and credential management. It can be used to upgrade unencrypted traffic in the service grid and provide operations personnel with the ability to enforce policies based on service identification rather than network control. Istio's configuration policy configures platform authentication on the server side, but does not enforce this policy on the client side, while allowing you to specify the authentication requirements of the service. Istio's key management system can automatically generate, distribute, rotate, and revoke keys and certificates.

Istio RBAC provides namespace-level, service-level, and method-level access control for services in the Istio grid, including easy-to-use role-based semantics, service-to-service and end-user-to-service authorization, and binding on roles Provide flexible custom attribute support in certain aspects.

Istio can enhance the security of microservices and their communication (including service-to-service and end-user-to-service communication) without changing the service code. It provides a powerful role-based identity mechanism for each service to achieve cross-cluster and cross-cloud interactive operations.

4. Galley

Galley is used to verify user-written Istio API configuration. Over time, Galley will take over Istio's top responsibility for obtaining configuration, processing, and distribution components. It is responsible for isolating other Istio components from the details of getting user configuration from the underlying platform (such as Kubernetes).

All in all, with Pilot, Istio can help you simplify traffic management as your deployment scales up. With Mixer, problems can be detected and repaired quickly and effectively with the help of robust and easy-to-use monitoring functions. With Citadel, the security burden is reduced, allowing developers to focus on other critical tasks.

There are several key goals in Istio's architectural design. These goals are critical for the system to handle large-scale traffic and high-performance service processing.

• Maximize Transparency: To adopt Istio, O & M and developers should be able to get real value from it with little cost. For this reason, Istio automatically injects itself into all network paths between services. Istio uses Envoy proxies to capture traffic and, if possible, automatically programs the network layer to route traffic through these proxies without having to make too many changes to the deployed application code, or even any changes. In Kubernetes, the Envoy proxy is injected into the pod and captures traffic through iptables rules. Once the Envoy proxy is injected into the pod and the routing rules are modified, Istio is able to regulate all traffic. This principle also applies to performance. When Istio is used for deployment, operation and maintenance personnel can find that the additional resource overhead for providing these functions is very small. All components and APIs must be designed with performance and scale in mind.

• Scalability: As operation and maintenance personnel and developers increasingly rely on the functions provided by Istio, the system must grow with their needs. While we continue to add new functions, what we need most is to be able to extend the strategy system, integrate other strategies and control sources, and propagate the grid behavior signals to other systems for analysis. The policy runtime supports standard extension mechanisms for insertion into other services. In addition, it allows the vocabulary to be extended to allow the enforcement of strategies based on new signals generated by the grid.

• Portability: The ecosystem that uses Istio is different in many ways. Istio must be able to run in any cloud or local environment with minimal cost. Porting Istio-based services to a new environment should be a breeze, and using Istio to deploy a service to multiple environments at the same time is also feasible, for example, it can be deployed on a hybrid cloud to achieve redundant disaster recovery.

• Policy consistency: The policy is applied to API calls between services, which can well control grid behavior. But for resources that do not need to be expressed at the API level, applying strategies to resources is equally important. For example, applying quotas to the amount of CPU consumed by machine learning training tasks is more useful than applying quotas to calls that initiate this job. Therefore, Istio maintains the policy system as a unique service with its own API instead of putting it in a proxy, which allows the service to directly integrate with it as needed.

Anatomy of the Istio data plane

When introducing the concept of service mesh, the concept of service agent and how to use agent to build a service mesh are mentioned to regulate and control all network communication between microservices. Istio uses Envoy proxies as the default out-of-the-box service proxies. These Envoy proxies run with all application instances participating in the service mesh, but not in the same container process, forming the data plane of the service mesh. As long as the application wants to communicate with other services, it will go through the service agent Envoy. This shows that the Envoy proxy is a key component of the data plane and the entire service mesh architecture.

1. Envoy Agent

Envoy was originally developed by Lyft to solve some complex network problems that arise when building distributed systems. It was provided as an open source project in September 2016, and joined the Cloud Native Computing Foundation (CNCF) a year later. Envoy is implemented in C ++ language and has high performance. More importantly, it is also very stable and reliable when running under high load. The network should be transparent to the application. When there is a problem with the network and the application, it should be easy to determine the source of the problem. Based on such a design concept, Envoy is designed as a service-oriented seven-layer proxy and communication bus.

In order to better understand Envoy, we need to first clarify a few related basic terms:

• Out of Process architecture: Envoy is an independent process. Envoy forms a transparent communication grid. Each application sends messages to or receives messages from the local host, but does not need to care about the network topology.
• Single-process multi-threaded model: Envoy uses a single-process multi-threaded architectural model. A main thread manages various trivial tasks, and some work sub-threads are responsible for performing monitoring, filtering and forwarding functions.
• Downstream: The host connected to Envoy and sending requests and receiving responses is called the downstream host, which means that the downstream host represents the host sending the request.
• Upstream: As opposed to downstream, the host receiving the request is called the upstream host.
• Listener: Listener is a named network address, including port, unix domain socket, etc., which can be connected by downstream hosts. Envoy exposes one or more listeners to downstream host connections. Each listener is independently configured with some network-level (that is, Layer 3 or Layer 4) filters. When the listener receives a new connection, the configured local filter will be instantiated and begin to process subsequent events. Generally speaking, the listener architecture is used to perform most of the different proxy tasks, such as speed limit, TLS client authentication, HTTP connection management, MongoDB sniff? Ing, raw TCP proxy, etc.
• Cluster: A cluster refers to a group of upstream hosts with the same logic connected by Envoy.
• xDS protocol: In Envoy, the xDS protocol represents multiple discovery service protocols, including cluster discovery service (CDS,
  Cluster Discovery Service), listener discovery service (LDS, Listener Discovery Service), and route discovery service (RDS, Route Discovery Service ), Endpoint Discovery Service (EDS, Endpoint Discovery Service), and Key Discovery Service (SDS, Secret Discovery Service).

(Agent for Envoy)

The Envoy proxy has many functions that can be used for inter-service communication. For example, it exposes one or more listeners to downstream host connections, and exposes them to external applications through ports; it handles the traffic transmitted in the listener by defining routing rules, and sends the traffic Direct to the target cluster, etc. Subsequent chapters will further analyze the roles and functions of these discovery services in Istio.

After understanding Envoy terminology, you may want to know what role Envoy plays as soon as possible?

First, Envoy is a proxy that acts as an intermediary in the network architecture and can add additional functions to the traffic management in the network, including providing security, privacy protection, or policies. In the scenario of invocation between services, the agent can hide the topology details of the service backend from the client, simplify the complexity of the interaction, and protect the backend service from overload. For example, a back-end service is actually a set of identical instances running, and each instance can handle a certain amount of load.

Second, the cluster in Envoy essentially refers to the logically identical set of upstream hosts that Envoy is connected to. So how does the client know which instance or IP address to use when interacting with the back-end service? Envoy acts as a proxy for routing. Through service discovery (SDS, Service Discovery Service), the Envoy proxy discovers all members in the cluster, and then determines the health status of the cluster members through active health checks, and passes the load according to the health status The balancing strategy determines which cluster member to route the request to. While the Envoy proxy handles the load balancing process across service instances, the client does not need to know any details of the actual deployment.

2. Envoy startup configuration

Envoy currently provides two versions of the API, namely v1 and v2. Since Envoy 1.5.0, there is the v2 API. In order to enable users to smoothly migrate to the v2 version of the API, Envoy sets a parameter when starting-v2 -conf? ig-only. Through this parameter, you can explicitly specify the protocol that Envoy uses the v2 API. Fortunately, the v2 API is a superset of v1 and is compatible with the v1 API. In the current version of Istio 1.0 and later, the API that supports v2 is clearly specified. By looking at the container startup command that uses Envoy as the Sidecar proxy, you can see similar startup parameters as follows, where the parameter --v2-conf? Ig-only is specified:

$ /usr/local/bin/envoy -c
/etc/istio/proxy/envoy-rev0.json --restart-epoch 0 --drain-time-s 45
--parent-shutdown-time-s 60 --service-cluster ratings --service-node
sidecar~172.33.14.2~ratings-v1-8558d4458d-ld8x9.default~default.svc.cluster.local
--max-obj-name-len 189 --allow-unknown-fields -l warn --v2-config-only

Among them, the parameter -c indicates the path of the boot configuration file based on version v2, the format is JSON, and other formats such as YAML, Proto3, etc. are also supported. It will be first parsed as the boot configuration file of version v2. If the analysis fails, it will be determined whether to parse as a JSON configuration file of version v1 according to the [--v2-conf? Ig-only] option. The other parameters are explained as follows, so that the reader can understand the configuration information when the Envoy agent is started in a timely manner:

• restart-epoch represents a warm restart cycle, which defaults to 0 for the first start and should be increased after each warm restart.
• service-cluster defines the name of the local service cluster that Envoy runs.
• service-node defines the name of the local service node where Envoy runs.
• drain-time-s indicates the time (seconds) that Envoy will drain the connection during a warm restart. The default is 600 seconds (10 minutes). Usually the exhaustion time should be less than the shutdown time of the parent process set by the --parent-shutdown-time-s option.
• parent-shutdown-time-s indicates the time (in seconds) Envoy waits before shutting down the parent process during a warm restart.
• max-obj-name-len describes the maximum length of the name field in the cluster cluster, routing configuration route_conf? ig, and listener, in bytes. This option is usually used in scenarios where cluster names are automatically generated, and usually exceeds the 60-character internal limit. The default is 60.
• Envoy's startup configuration file is divided into two ways: static configuration and dynamic configuration. The specific performance is:
• Static configuration is to put all information in the configuration file and load it directly at startup.
• Dynamic configuration needs to provide an Envoy server, which is used to dynamically generate the service discovery interface required by Envoy, which is commonly referred to as xDS. The configuration information is dynamically adjusted through the discovery service. Istio implements the v2 xDS API.

3. Envoy static and dynamic configuration

Envoy is an intelligent agent driven by configuration files in JSON or YAML format. For users who are already familiar with Envoy or Envoy configuration, I believe that they should already know that Envoy's configuration also has different versions. The initial version v1 is the original way to configure Envoy when Envoy starts. This version has been deprecated to support the v2 version of Envoy configuration. Envoy's reference documentation ( https://www.envoyproxy.io/docs ) also provides documentation that clearly distinguishes v1 and v2. This article will only focus on v2 configuration because it is the latest version and the one used by Istio.

The configuration API of Envoy version v2 is built on gRPC. An important feature of the v2 API is that it can use the streaming function when calling the API to reduce the time required for the Envoy proxy to aggregate configuration. In fact, this also eliminates the drawbacks of polling the API, allowing the server to push updates to the Envoy proxy instead of periodically polling the proxy.

Envoy's architecture makes it possible to use different types of configuration management methods. The method used in deployment will depend on the needs of the implementer. Simple deployment can be achieved through fully static configuration, and more complex deployments can incrementally add more complex dynamic configurations. Mainly divided into the following situations:

• Full static: In a full static configuration, the implementer provides a set of listener and filter chains, clusters, and optional HTTP routing configuration. Dynamic host discovery can only be discovered through DNS-based services. Configuration reload must be done through the built-in hot restart mechanism.
• SDS / EDS only: On a static configuration, Envoy can discover members in the upstream cluster through this mechanism.
• SDS / EDS and CDS: Envoy can discover the upstream cluster used by this mechanism.
• SDS / EDS, CDS, and RDS: RDS can discover the entire routing configuration for the HTTP connection manager filter at runtime.
• SDS / EDS, CDS, RDS and LDS: LDS can discover the entire listener at runtime. This includes all filter stacks, including HTTP filters with applications embedded in RDS.

Static configuration

We can use Envoy's configuration files to specify listeners, routing rules, and clusters. The following example provides a very simple Envoy configuration:

static_resources:
 
listeners:
 
- name: httpbin-demo
   
address:
     
socket_address: { address: 0.0.0.0, port_value: 15001 }
   
filter_chains:
   
- filters:
     
- name: envoy.http_connection_manager
       
config:
          stat_prefix: egress_http
          route_config:
            name: httpbin_local_route
            virtual_hosts:
            - name: httpbin_local_service
              domains: ["*"]
              routes:
              - match: { prefix: "/"
}
                route:
                  auto_host_rewrite: true
                  cluster: httpbin_service
          http_filters:
          - name: envoy.router
 
clusters:
   
- name: httpbin_service
     
connect_timeout: 5s
     
type: LOGICAL_DNS
     
# Comment out the following line to test on v6 networks
     
dns_lookup_family: V4_ONLY
     
lb_policy: ROUND_ROBIN
     
hosts: [{ socket_address: { address: httpbin, port_value: 8000 }}]

In this simple Envoy configuration file, we declare a listener that opens a socket on port 15001 and attaches a filter chain to it. The filter http_connection_manager uses routing instructions in the Envoy configuration (the simple routing instructions seen in this example are wildcards that match all virtual hosts) and routes all traffic to the httpbin_service cluster. The last part of the configuration defines the connection properties of the httpbin_service cluster. In this example, we specify that the type of endpoint service discovery is LOGICAL_DNS, and the load balancing algorithm when communicating with the upstream httpbin service is ROUND_ROBIN.

This is a simple configuration file used to create incoming traffic to the listener and route all traffic to the httpbin cluster. It also specifies the settings of the load balancing algorithm to be used and the connection timeout configuration to be used.

You will notice that many configurations are explicitly specified, such as which listeners are specified, what routing rules are, and which clusters we can route to. This is an example of a completely static configuration file.

For more information about these parameters, please refer to Envoy's documentation (www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/service_discovery#logical-dns).
In the previous section, we pointed out that Envoy can dynamically configure its various settings. The following will introduce the dynamic configuration of Envoy and how Envoy uses the xDS API for dynamic configuration.

Dynamic configuration

Envoy can utilize a set of APIs for configuration updates without any downtime or restart. Envoy only needs a simple boot configuration file, which points the configuration to the correct discovery service API, and the rest is dynamically configured. Envoy's dynamically configured APIs are generally collectively referred to as xDS services, which include the following:

• Listener Discovery Service (LDS): A mechanism that allows Envoy to query the entire listener. By calling this API, you can dynamically add, modify, or delete known listeners; each listener must have a unique name. If no name is provided, Envoy will create a UUID.
• Route Discovery Service (RDS): Envoy's mechanism for dynamically obtaining routing configuration. The routing configuration includes HTTP header modification, virtual hosts, and individual routing rules contained in each virtual host. Each HTTP connection manager can independently obtain its own routing configuration through the API. RDS configuration is part of the listener discovery service LDS and is a subset of LDS. It is used to specify when static and dynamic configuration should be used and which route to use.
• Cluster Discovery Service (CDS): An optional API that Envoy will call to dynamically acquire cluster management members. Envoy will also coordinate cluster management based on API responses, adding, modifying, or deleting known clusters as needed. Any clusters statically defined in the Envoy configuration cannot be modified or deleted through the CDS API.
• Endpoint Discovery Service (EDS): A mechanism that allows Envoy to obtain cluster members. It is based on gRPC or RESTJSON APIs. It is a subset of CDS; cluster members are called Endpoints in Envoy terminology. For each cluster, Envoy obtains the endpoint from the discovery service. EDS is the preferred service discovery mechanism.
• Key Discovery Service (SDS): API for distributing certificates; the most important benefit of SDS is to simplify certificate management. Without this feature, in a Kubernetes deployment, the certificate must be created as a key and mounted in the Envoy proxy container. If the certificate expires, the key needs to be updated and the agent container needs to be redeployed. Using the key discovery service SDS, the SDS server will push the certificate to all Envoy instances. If the certificate expires, the server only needs to push the new certificate to the Envoy instance, and Envoy will immediately use the new certificate without redeployment.
• Aggregation Discovery Service (ADS): A serialized stream of all the changes of the other APIs mentioned above; you can use this single API to get all the changes in sequence; ADS is not a real xDS, it provides a convergent function when needed When multiple simultaneous xDS accesses, ADS can be done in one stream.

The configuration can use one or a combination of several of the above services without having to use them all. One thing to note is that Envoy's xDS API is built on the premise of final consistency, and the correct configuration will eventually converge. For example, Envoy may eventually use a new route to get RDS updates, which routes traffic to clusters that have not been updated in CDS. This means that routing may introduce routing errors until the CDS is updated. Envoy introduced the aggregation discovery service ADS to solve this problem, and Istio implemented the aggregation discovery service ADS, and used ADS to make proxy configuration changes.

For example, if the Envoy proxy needs to discover the listener dynamically, you can use the following configuration:

dynamic_resources:
 
lds_config:
   
api_config_source:
     
api_type: GRPC
     
grpc_services:
       
- envoy_grpc:
            cluster_name: xds_cluster
clusters:
- name: xds_cluster
 
connect_timeout: 0.25s
 
type: STATIC
 
lb_policy: ROUND_ROBIN
 
http2_protocol_options: {}
 
hosts: [{ socket_address: { address: 127.0.0.3, port_value: 5678 }}]

With the above configuration, we don't need to explicitly configure each listener in the configuration file. We told Envoy to use the LDS API to find the correct listener configuration value at runtime. However, we need to explicitly configure a cluster, which is where the LDS API is located, which is the cluster xds_cluster defined in this example.

Based on the static configuration, the information provided by each discovery service is more intuitively represented.

(XDS service information)

Based on the static configuration, the information provided by each discovery service is more intuitively represented.

This article is excerpted from "Istio Service Grid Analysis and Actual Combat" and published with permission from the publisher. This book was written by Alibaba Cloud senior technical expert Wang Xining. It introduces the basic principles of Istio and the actual development in detail. It contains a large number of selected cases and reference codes that can be downloaded to quickly start Istio development. Gartner believes that the service grid will become the standard technology for all leading container management systems in 2020. This book is suitable for all readers who are interested in microservices and cloud native. It is recommended that you read this book in depth.

Participate in the public account message interaction, that is, have the opportunity to obtain the following benefits!

" Alibaba Cloud Native focuses on microservices, serverless, containers, service mesh and other technical fields, focuses on cloud native popular technology trends, cloud native large-scale landing practices, and is the public number that understands cloud native developers best."