In 2019, Berkeley predicted that Serverless will replace Serverful computing as the new paradigm of cloud computing. Serverless provides a new system architecture for application development. Taking advantage of the AIGC trend brought by OpenAI in 2023, Serverless represented by Alibaba Cloud Function Computing FC and AWS Lambda has many features such as higher cost-effectiveness, simpler back-end code & scalability, and more extreme elasticity. It liberates developers from heavy manual resource management and performance cost optimization, and stimulates developers' vigorous imagination and creativity again. More and more developers and enterprises in China are beginning to try how to apply Serverless to actual business or scenarios.
But before using Serverless elegantly, there are still many small problems that need to be solved in advance. Since the expansion and contraction of the serverless platform is based on request processing/event-driven concurrency, developers who are used to horizontal pod scaling based on CPU indicators will encounter the following difficulties, such as concurrency What is the relationship between the parameters , the minimum number of instances, and the maximum number of instances? Another example is how to set the maximum concurrency of a single instance to meet your business needs?
01 Consideration Dimensions for Serverless Parameter Configuration
Serverless can provide certain general capabilities, but appropriate configurations are required for different business logics to better leverage the value of Serverless. However, how to evaluate the optimal configuration of a function involves multivariate collaborative optimization. Although Function Compute FC provides request estimates based on the application's "total number of daily requests" and "average request response time", based on the server currently used by the application" "Specification" and "Utilization" and other methods to assist in parameter configuration. But for better configuration, we recommend evaluating and configuring Serverless service parameters from the following three dimensions.
(1) Make a trade-off between cost and performance
How to choose performance priority or cost priority according to business preferences is the first difficulty that needs to be considered in parameter configuration. In the case of a relatively fixed multi-concurrency number of single instance, the number of parallel processing requests of a single instance can be increased, the number of instances can be reduced, and the cost can be reduced. When the number of concurrency is too high, it will increase resource competition, resulting in increased performance delay, thereby increasing costs; if the sensitivity to delay is relatively low, you can choose a lower instance specification, and the unit price cost is lower. On the contrary, you want to be shorter Delay, you can choose a higher instance specification, but the unit price is higher.
(2) Combining the complexity of business logic with different functions
In addition to cost and performance trade-offs, for different types of function logic, the effects of different configuration parameters also have huge differences. The business logic of many functions is complex, and only specific configuration for a single logic branch does not mean that the overall performance is optimal; improper configuration may generate a large amount of unexpected operation and maintenance costs. In this regard, we test different types of CPU-intensive and IO-intensive functions in order to better explore the relationship between different specifications and TPS of different types of functions.
- Pressure testing of CPU-intensive functions under different specifications
It can be seen that the higher the CPU-intensive specification is, the larger the maxTPS is, and there is an obvious linear relationship between the specification and maxTPS. The larger the specification, the lower the maxRT, indicating that the CPU-intensive function, increasing the resource specification can significantly reduce the RT. However, after the specifications are increased to 4G and 8G, the effect on RT reduction will have diminishing marginal effects.
- Pressure testing of IO-intensive functions under different specifications
It can be seen that the performance improvement of the IO-intensive type is very limited by the specification improvement, and the correlation between the specification and maxRT and maxTPS is limited. Especially after expanding to high specifications, the improvement of maxTPS is small.
With the help of the above pressure test, we can see the conclusion that for CPU-intensive functions, the increase in specifications can greatly improve the performance of a single instance. However, for IO-intensive functions, there is a diminishing marginal effect of increasing the specification on the performance improvement of a single instance. When a certain specification is exceeded, the specification increase has little effect on the performance improvement.
(3) Taking into account the impact of function configuration on computing resource configuration
Since configurations such as function concurrency, minimum number of instances, and maximum number of instances will affect the resource allocation of the serverless platform, ensuring the rigid delivery of single-function resources and isolation of multi-function resources at the same time, making reasonable use of the platform's flexible scheduling capabilities and improving resource utilization is the last Questions to consider.
Take the scenario of processing x concurrent requests as an example. When the instance concurrency is set to 1, each instance can only process 1 request at a time, and Function Compute needs to create x instances to process the x requests. When the instance concurrency is set to X*10, each instance can handle X*10 requests at the same time, and Function Compute only needs to create 1 instance to handle the x requests.
Single-instance multi-concurrency is suitable for scenarios where the function spends a lot of time waiting for responses from downstream services. Waiting for a response generally does not consume resources, and concurrent processing within an instance can save costs. However, when the function traffic fluctuates and changes with low single-instance concurrency, the upper limit of single-instance concurrency will be reached in advance, resulting in more frequent instance expansion and contraction and cold start. At the same time, more instances need to be created and maintained, resulting in low overall resource utilization.
02 Evaluate the rationality of parameter configuration
Combining the above three dimensions, we can see that evaluating the parameter configuration of Serverless is no easy task. When many users start to try to use Serverless, they only configure relevant parameters through the guidance of the document. After the function is officially launched, it will soon be found that the previous configuration is unreasonable, the resulting cost exceeds expectations, and the performance is not as expected, etc., and try to modify the function configuration repeatedly for verification. Senior developers will choose to perform stress testing in order to test the best function configuration. However, the configuration of stress test scripts and the interpretation of stress test data reports require some practical experience, and developers cannot be sure whether the configuration conclusions obtained from stress tests are the optimal choice that meets business expectations. After counting the actual configuration and usage of a large number of users, we found that the actual resource usage of the users is low, and the actual configuration specifications are too large, resulting in a certain waste.
In order to better verify the rationality of configuration parameters, Function Compute FC provides a function performance detection function based on the performance test PTS capability to evaluate the performance upper limit of a single instance of a function under different specifications, so as to recommend the best concurrency and Function specification configuration, the detection method is based on little's law [1] queuing theory (concurrent number = average delay of requests * TPS), as shown in the figure:
(The abscissa is the number of concurrency, the ordinate on the left is TPS, and the ordinate on the right is the delay)
Since the processing capacity of each server is limited, as the number of concurrency increases, the throughput will first increase and then flatten, and may decline, that is, the performance will deteriorate; when the concurrency is too high, the delay will become higher, or even worsen sharply . Through performance detection, we will get the key performance data of each specification, that is, the highest acceptable QPS of each specification. On the premise of knowing the scale of business traffic, we can obtain the minimum number of instances and the most appropriate function. The maximum number of instances is the best specification and the best concurrency under the specification. We can only pressure test a single instance, because in a system with stable performance, the performance of multiple instances is a linear superposition of the performance of a single instance, so we only need to measure the performance of a single instance to calculate the performance of multiple instances.
For example, the user expects the end-to-end delay of the function call to be 1000 ms. According to the delay limit of 1000 ms, select the best specification and the best concurrency under this specification, that is, the corresponding concurrency of the highest QPS that meets the delay limit.
Currently, performance detection only supports pressure testing of HTTP functions, and does not support pressure testing of event functions. Only single-instance stress testing is supported, and multi-instance stress testing is not supported. Therefore, we provide performance detection (single instance) and performance test PTS (multiple instances) for verification.
- About performance probing
As one of the functions of Function Compute FC, in order to further reduce the use threshold of performance detection, the function adopts process guidance, and the performance detection function is completely free. Users only need to pay for the request traffic undertaken by the function, and do not need to pay for the pressure measurement function. .
- About Performance Test PTS
As a performance testing tool for Taobao Double 11 of Alibaba Group, the performance testing PTS supports on-demand initiation of pressure testing tasks, can provide millions of concurrent, tens of millions of TPS traffic initiation capabilities, and is 100% compatible with JMeter. Provided functions such as scene arrangement, API debugging, traffic customization, and traffic recording can quickly create business stress test scripts, accurately simulate different levels of users accessing business systems, and help businesses quickly improve system performance and stability. Currently, a free trial quota of 5000VUM is provided for new users.
03 For a single instance, how to verify the configuration parameters of a single instance through performance detection
Next, we briefly introduce the configuration process of performance detection, which can quickly initiate performance detection in only three steps.
1. Log in to the Function Compute Console [2] , select Services and Functions in the left navigation bar , and select the target service on the service list page.
2. On the function management page, select the target function and create a pressure test task on the performance detection tab .
3. On the Single Instance Performance Pressure Test Evaluation tab, enter the necessary pressure test API information (see the table below), and click Execute Pressure Test .
(Before performing the pressure test, please click API Test to verify whether the HTTP request parameters of the API are configured correctly and whether the function can be executed successfully.)
4. View the stress test result analysis report: View the stress test results on the single instance stress test result analysis tab. During the stress test, the interface will prompt the estimated time of the stress test. The more specifications of the stress test, the longer the stress test time and the higher the cost of the stress test. The stress test results include recommended instance specifications and the optimal number of concurrent requests. You can also view the analysis process from the two analysis graphs below the stress test results.
Note: The pressure testing function of Function Compute only supports single-instance pressure testing. If you need to configure multi-instance pressure testing, click the multi-instance elastic capacity pressure test on the right side of the single-instance pressure test result analysis tab to jump to the PTS console [3] configuration.
5. View the details of the stress test data of a single instance: Click View the details of the stress test data of a single instance to jump to the details page of the stress test data of a single instance , and view the details of the stress test of each specification. In the stress test details of each specification, you can see the changes in latency and maximum QPS as the concurrency increases.
However, it should be noted that the function configuration recommended by the performance detection first ensures that the performance requirements are met and the highest resource utilization rate is achieved. However, to truly achieve the lowest cost configuration, it is necessary to combine the historical traffic data analysis of the function online to make recommendations. When recommending specifications for cost optimization, it is not only necessary to achieve the purpose of saving costs, but also to ensure that the QoS of existing services is not damaged, that is, the performance will not increase the delay due to the reduction of instance specifications.
04 For multi-instance, how to use performance test PTS to configure multi-instance parameters
Next, we briefly introduce the configuration process of the PTS for performance testing. You only need to configure the API to quickly initiate a pressure test.
1. Go to the performance testing PTS console. In the left navigation bar, select Performance Test > Create Scenario .
2. On the Create Scenario page, click PTS Pressure Test .
3. On the Create PTS Scenario page, set the scenario name , such as Function Compute Serverless Application Pressure Test, and then on the Scenario Configuration tab, enter the pressure test API name
, such as demo, and click the icon to the right of the Add Pressure Test node . In the pressure test URL area, enter the access domain name of the serverless application copied and saved in the previous section. After the configuration is complete, you can click Debug API to verify whether the API interface configuration meets expectations. When the debugging response status code is 200, it means that the API can be accessed normally.
4. Click the Pressure Configuration tab on the Create PTS Scenario page to configure relevant pressure measurement parameters. Select Auto-increment for the increment mode , enter 500 for the maximum concurrency , enter 20 for the increment percentage , enter 1 for the single-level duration , and enter the total duration of the stress test. 5. Enter 100 for the maximum concurrent weight of serial link 1, and input 20 for the initial percentage of serial link 1 . After confirming the parameters, click Save to start the pressure test . In the prompt dialog box, confirm that the estimated consumption of the task does not exceed the free quota. After confirming , click OK to start the pressure test . (The above parameters are just examples, please adjust according to actual business)
5. Wait for about 3 minutes, and you can view the real-time stress measurement data of the current application on the stress measurement page.
6. On the stress testing page, you can see overview data such as success rate, RT, and TPS.
7. After the pressure test is completed, it will automatically jump to the edit scene page, and then click the pressure test report . On the Pressure Test Report tab, find your pressure test report, and click View under the Operation column on the right .
8. Learn about the detailed content of the stress test report in the stress test report, and view the detailed performance data of this stress test. For more details about the pressure test report, see Viewing the PTS Pressure Test Report [4] . At this point, the pressure test of the serverless application through the performance test PTS has been realized.
05 Developer Scenario Experience
Currently, the scenario of "Performance stress testing of serverless applications through performance testing PTS" has been launched in Yunqi Lab. While providing relevant free trial quotas, relevant operating procedures and templates are also provided so that you can quickly experience creating applications through FC and performing stress testing through PTS.
Portal: https://developer.aliyun.com/adc/scenario/186a9f71954b42419ebfa1f27fd8b042
Related Links:
[1] little's law
https://en.wikipedia.org/wiki/Little%27s_law
[2] Function Compute Console
[3] PTS Console
[4] View PTS pressure test report
https://help.aliyun.com/document_detail/65304.html
Author: Fu Yi, Cong Xiao
Click to try cloud products for free now to start the practical journey on the cloud!
This article is the original content of Alibaba Cloud, and shall not be reproduced without permission
Ministry of Industry and Information Technology: Do not provide network access services for unregistered apps Go 1.21 officially releases Linus to personally review the code, hoping to quell the "infighting" about the Bcachefs file system driver ByteDance launched a public DNS service 7-Zip official website was identified as a malicious website by Baidu Google releases AI code editor: Project IDX Tsinghua Report: Wenxin Yiyan firmly sits first in China, beyond the ChatGPT Vim project, the future meditation software will be launched, ChatGPT was founded by "China's first Linux person" with a daily cost of about 700,000 US dollars , OpenAI may be on the verge of bankruptcy