Overview
This is a series. Earlier, we testedthe performance/price comparison between Alibaba Cloud Economic Edition ("ARM") and the Standard Edition;Performance/price comparison between Huawei Cloud x86 specifications and ARM (Kunpeng enhanced) version. Now, let’s take a look at the ARM version of RDS in AWS
In 2018, AWS launched Graviton EC2 instances for the first time. In July 2020, AWS RDS officially supported Graviton 2 instances. Just two days ago, at the latest AWS re:Invent conference, AWS had launched the fourth-generation Graviton 4 instances. . Now, AWS Graviton is relatively mature and is widely used by a large number of enterprises and applications. AWS officials also claim that RDS instances using Graviton 2 can achieve a 52% cost-effective improvement (reference). Here, let’s test it through the standard Sysbench test to see the effect of the actual Graviton 2 instance.
The same as the last Alibaba Cloud test, this time we still used the same test tools and scenarios. For the more commonly used 4c16gb, namely db.m6g.xlarge and db.m5.xlarge instances, the number of concurrency was 2 4 8 16 respectively. 24 32 48 64 96 128 tests.
Test conclusion
In terms of performance, on average, x86 instance performance is about 12.7% higher than Graviton instance performance; x86 specification latency is 15% lower than Graviton specification. I also noticed that in the case of ultra-high concurrency (when the concurrency exceeds 96), Graviton instances and x86 instances are relatively close in terms of performance and latency. However, at this time, the system pressure is too great and the latency is too high. It does not have much reference value for actual use.
In order to make a more intuitive cost-effective comparison, here we select the data at 16 concurrency times for comparison. Under 16 concurrency, the TPS of the Graviton instance is 275, and the price of db.m6g.xlarge is $0.836/hour; the TPS of the x86 instance is 341, and the price is $0.94/hour. Then per 100TPS, the Graviton price is 0.304 and x86 is 0.275. Therefore, at 16 concurrency times, x86 specifications are more cost-effective in comparison, 10% higher than Graviton instances. This result is still very different from what was expected before the test, and also completely different from what AWS claimed.
Only under ultra-high concurrency conditions (96/128 concurrency), the throughput of Graviton instances is close to that of x86. At this time, Graviton instances show a cost-effective advantage of close to 10%. However, this kind of high concurrency is not the norm in actual scenarios, so it does not have a strong reference value.
The test results were quite different from what was expected, so I did another test later and the results were basically the same as this time. Therefore, in the end, you have to actually test the performance yourself, because the publicized data are usually very extreme scenarios for adapting the product, and are not real scenarios.
The following two pictures are comparisons of TPS and average latency:
The abscissa is the number of concurrent threads of sysbench, and the ordinate is tps and average latency respectively.
Test model description
The mixed read-write model (oltp_read_write) of sysbench is used here for testing. The single table size is 1 million, and there are ten tables in total. The single test duration is 300 seconds. The following concurrency performance is tested: 2, 4, 8, 10, 12, 14, 16, 24, 32.
Instance configuration and price
Here we choose the more commonly used 4c16gb instance for testing. Try to choose the default option for each option to more closely simulate the actual user scenario. Specifically, the version is AWS default 8.0.33, multi-availability zone version, storage default encryption, and gp3 storage , 100GB space, 3000 IOPS, and Performance Insight are also enabled by default. The complete options are as follows:
AWS prices are divided into computing node prices (CPU and memory), storage prices, and IOPS prices. Here we only focus on computing node prices. Storage and IOPS are the same for ARM and x86 instances. The prices chosen here are the Tokyo region and multi-AZ instances, as follows:
In the future, RDS based on io1 (Provisioned IOPS SSD) storage will also be tested.
Detailed test data reference
AWS RDS Graviton
（db.m6g.xlarge/gp3/100gb/3000iops/8.0.33)
threads|transactions| queries| time |avg/Latency|95%/Latency2| 11951| 239020|300.03| 50.21| 55.824| 23322| 466440|300.04| 51.45| 57.878| 43654| 873080|300.05| 54.98| 65.6516| 82519| 1650380|300.05| 58.17| 70.5524| 120541| 2410820|300.06| 59.74| 73.1332| 156680| 3133600|300.07| 61.28| 74.4648| 218709| 4374180|300.06| 65.85| 81.4864| 269430| 5388600|300.08| 71.27| 90.7896| 329366| 6587320|300.07| 87.45| 121.08128| 351579| 7031580|300.11| 109.24| 164.45
AWS x86 instances
（db.m5.xlarge/gp3/100gb/3000iops/8.0.33）
threads|transactions| queries| time |avg/Latency|95%/Latency2| 13357| 267140|300.03| 44.92| 112.674| 27539| 550780|300.03| 43.57| 50.118| 55330| 1106600|300.04| 43.38| 51.9416| 102408| 2048160|300.05| 46.87| 56.8424| 145718| 2914360|300.05| 49.41| 61.0832| 186619| 3732380|300.05| 51.44| 63.3248| 260415| 5208300|300.04| 55.30| 69.2964| 306939| 6138780|300.08| 62.56| 82.9696| 330131| 6602620|300.09| 87.25| 123.28128| 348095| 6961900|300.12| 110.34| 155.80
summary
When AWS RDS released Graviton 2 instances, it promoted that Graviton 2 instances had a 52% cost-effectiveness improvement. However, in the Sysbench mixed read and write test scenario here, the cost-effective advantage of x86 is more obvious. At 16 concurrency, the cost-effectiveness of x86 is 10% higher than that of the Graviton instance. And only in ultra-high concurrency, the cost performance of Graviton instances is 10% higher than that of x86. However, generally we will not let 4c16 instances run under such high pressure, so the reference significance of the latter situation is not strong.
Although AWS has promoted a large number of Graviton instances, there is no cost-effective advantage in actual testing. Therefore, in database application scenarios, the necessity of using AWS Graviton instances does not seem to be high. In addition, I also noticed that the instance of RDS Graviton 3 has not been launched, maybe this is one of the reasons.
reference
1. AWS Graviton Processors
2. Achieve up to 52% better price/performance with Amazon RDS using new Graviton2 instances
3. AWS Graviton – Wikipedia, the free encyclopedia
About the author: orczhou, cloud database expert, co-founder of NineData, Oracle ACE (MySQL), translator of the third and fourth editions of "High Performance MySQL", and former senior expert on Alibaba Cloud database.