Those common cloud server load balancing knowledge

Table of contents

Since the dimension is limited, why do some scenes still use time as the dimension?

Load Balancer Listener

A load balancing listener is a component in a load balancer that receives requests from clients and distributes them to backend servers. It listens to network traffic and, based on predefined rules , forwards requests to the server best suited to handle the request.

The load balancing listener has the following functions:

Distributing requests : The listener receives requests from clients and distributes requests to backend servers according to certain algorithms (such as polling, IP hashing, weighting, minimum connection, shortest connection, sticky polling, etc.). It can avoid performance degradation caused by some servers being overloaded, and improve the throughput and response speed of the overall system.
Health check : You can regularly check the health status of the back-end server, such as whether the server is online and whether it can respond to requests normally. If a server fails or is unavailable, the listener automatically forwards requests to other healthy servers to ensure availability and stability.
SSL/TLS encryption : The listener can provide SSL/TLS encryption function for encrypting the transmitted data. By configuring the certificate and encryption algorithm on the listener, the communication between the client and the server can be secured.
Session persistence : Some applications need to maintain the state of the user's session to ensure the continuity of the user between multiple requests. The listener can realize the function of session persistence by identifying the session identifier in the request and forwarding subsequent requests to the same server.

Virtual IP (Virtual IP, VIP)

A VIP is a single logical address used on a network to identify and access a group of computers or network services . It is an abstraction provided by load balancer or cluster management software.

The main role of virtual IP: to achieve high availability and load balancing . Binding multiple servers or network services to the same VIP can distribute client requests to these servers, thereby improving system availability and performance. When one of the servers fails or the load is too high, the load balancer can automatically transfer the traffic to other normal servers to ensure the continuity of services.

In addition, the virtual IP can also hide the real IP address of the back-end server to increase security.

Virtual IP provides the following important uses:

Realize high availability: Bind multiple servers to the same virtual IP to ensure that even if a server fails, the service is still available.
Load balancing: Improve system performance and throughput by distributing client requests to multiple servers.
Increase network security: hide the real IP address of the back-end server, reducing the attacker's direct access to the system.

Load Balancer instance (Load Balancer)

A load balancing instance is a device or service used to distribute network traffic, which can evenly distribute traffic to multiple servers to improve system reliability, stability and performance.

The functions of the load balancing instance include:

Improve system reliability : When a server fails or is unavailable, the load balancing instance can automatically transfer traffic to other available servers to ensure system continuity and availability. ( failover )
Improve system stability : Traffic is distributed to multiple servers to avoid overloading of a single server and reduce the risk of system crashes.
Improve system performance : Load balancing instances can dynamically allocate traffic to the most idle server according to the load of the server , thereby improving the response speed and processing capacity of the system.
Simplified system management : The load balancing instance can centrally manage multiple servers, and manage and monitor servers through a unified entry point, which simplifies system configuration and maintenance.

Note: The load balancing instance and the load balancing listener are two different concepts in the load balancing service.

Similarities and differences between Load Balancer and Load Balancer Listener

Load Balancer is a virtual server ( does not store data and does not have its own computing resources. Readers may say that NGINX is deployed on a machine? Load balancers can be hardware devices or software applications, but in many cloud computing environments, they Usually implemented in the form of a virtual server ). It receives requests from clients and distributes these requests to multiple servers on the backend for load balancing purposes. The load balancing instance usually has a public IP address through which clients can access the application.

Load Balancer Listener is a configuration item on the load balancing instance , which defines how the load balancing instance receives requests. Listeners typically contain the following information:

Protocol: Specifies the protocol used by the listener, such as HTTP, HTTPS, TCP, etc.
Port: Specifies the port number that the listener listens on.
Forwarding rules: Specify how the load balancing instance forwards requests to backend servers, such as polling, minimum number of connections, etc.

By configuring the load balancing listener, you can flexibly control how the load balancing instance receives requests to meet your business needs. Therefore, the load balancing instance and the load balancing listener are interrelated and need to be configured and used together .

Metrics and Dimensions

Metrics and dimensions are two important concepts in data analysis for understanding and interpreting data.

Metrics: Metrics are the values used to measure things , which reflect the performance or status of a certain aspect. In data analysis, indicators are usually quantifiable data, such as sales, visits, conversion rates, etc.
Dimensions: Dimensions are attributes used to describe data and provide context for metrics . Dimensions are typically categorical data such as region, time, product type, and so on.

Suppose we want to analyze the sales data of an e-commerce website. In this scenario, sales is an indicator that reflects the sales performance of the website. Dimensions such as region, time, and product type are dimensions that help us understand how sales change across regions, time, and product types. Through the analysis of indicators and dimensions, you can find out which regions have higher sales, which time periods are the peak sales periods, and which product types are the most popular, so as to provide a basis for decision-making.

Note: The value of the index can be unlimited, but the dimensions (Dimensions) must be limited. When the possible values of a dimension are limited, it is easier to group, filter, and sort data to discover potential patterns and trends in the data.

Metrics and dimensions in prometheus

Metrics: Metrics are metrics used to describe system or application performance . Metrics in Prometheus consist of a name and a value, usually a descriptive string, indicating the meaning of the metric, such as http_requests_total(total HTTP requests) or node_cpu_seconds_total(total CPU usage time). The metric value is a floating point number representing the actual value of the metric.
Dimensions: Dimensions in Prometheus are represented by labels . Tags are a set of key-value pairs used to describe the attributes of indicators, which can be used to filter, aggregate and group indicators . For example, for http_requests_total metrics, you can use method(request method) and status(response status code) as labels to analyze the number of different types of requests.

In PromQL, metric names and labels can be used to query and analyze metric data. To query the number of HTTP error requests per minute in the past 5 minutes, you can use the following PromQL query:

rate(http_requests_total{status=~"5.."}[5m])

http_requests_total It is the name of the indicator, and status=~"5.." it is a tag matcher (for example, 500 is a tag value, and the corresponding tags ( status code, controllable, limited ) such as 500, 502, etc. constitute a dimension), which means that only requests with a status code of 5xx are queried.

rate(http_requests_total{status="500"}[5m])

The result of the query represents the rate of bad requests with status code 500 generated per second over the past 5 minutes

Since the dimension is limited, why do some scenes still use time as the dimension?

Dimensions are limited, which means that in practical applications and analysis, the value range of dimensions is often limited so that they are within a manageable and analyzable range . Although time itself is continuous, in actual analysis, time is often divided into discrete intervals, such as year, quarter, month, day, hour, etc., thereby limiting the time dimension to a limited range.

When analyzing data with time as the dimension, we often pay attention to the changing trend and law of data within a specific period of time. For example, sales per month in the past year, visits per day in the past week, etc. In these scenarios, the value of the time dimension is limited, because only a specific time range and division granularity are concerned.

Of course, in some special cases, the value of the time dimension may become relatively large, such as the need to analyze data every minute or even every second. However, even in these cases, time frames are often limited to facilitate analysis and presentation of the data.

In short, although time itself is continuous, in practical applications, the time dimension is often limited to a limited range by limiting the time range and dividing granularity , so as to perform better data analysis.