Link layer of the TPC-C benchmark Optimization
Author: Yi Yan Jian Liang Wang Hongwei tree
Define the TPC-C standard, the test system is divided into RTE (Remote Terminal Emulator) and two portions SUT. In the actual TPC-C testing process, not just a test of the ability of DB end, all components are present in the great links of resource consumption and pressure. 60,880,000 tpmC at the test results, we run on a total of 64 64C128G cloud server 960 RTE client, to simulate a total of 47,942,400 users Terminal, and finally also the need for consistency so much based on statistical results and lasting RTE of audit verification. The SUT has split into three parts: WAS (Web Application Server), OceanBase Proxy (OBProxy) and OceanBase Server (OBServer). RTE's request to WAS, WAS and end customers to access OBProxy by OceanBase, OBProxy forwards the request to the back-end cluster OceanBase best ObServer to execute the request. WAS and OBProxy a bridge between RTE and OBServer, the bridge plays a vital role for bearing pressure. The TPC-C benchmark, mainly related to two assembly OceanBase access link:
- ODBC interface and driver: TPC-C test, WAS OceanBase request using the ODBC interface. ODBC (Open Database Connectivity) is proposed by Microsoft Data Access, ODBC can be used on most DBMS, OceanBase ODBC interface also provides access capability. Interested users can access ODBC API instructions quickly get started with using ODBC users can use this interface to directly access OceanBase seamless migration. ODBC interface and driver integrated within the WAS, as customer requests OceanBase end.
- OBProxy Agent: OceanBase realized OBProxy proxy server routing and disaster recovery to solve the problem of database link. OBProxy will address sensory data copies and zoning rules, does not participate in SQL engine involved in generating scheduling execution plan, SQL is responsible for routing and forwarding. This architecture design, OBProxy undertaken based routing and disaster recovery capabilities, and database functionality is paid in full
Implemented by OBServer. This is more simple and clear division of labor performance of each component can be made more extreme, OBProxy also done completely stateless, just add nodes to achieve the level of expansion agent capabilities, OceanBase can do the highest overall performance of the database.
TPC-C benchmark FIG Link Access OceanBase
TPC-C is a very demanding benchmark test model, the test of the ability to link the whole of a complete relational database system, any link bottlenecks are unable to maximize the performance of the database, the next article will respectively in terms of performance, cost and services continued under three aspects to illustrate how to optimize the components on the OceanBase link.
Link performance optimization
In SQL optimization of the TPC-C benchmark has already been mentioned, from the point of view of the entire link, SQL execution time required is very short, it takes a lot of time during the interaction with the client, resulting in waste and consumption of resources increases, for which provide Prepared Statement, stored procedures, and ARRAY BINDING ability OBServer. Client and OBProxy be support for the ability to make it really work. And the client itself has some optimization to enhance the link performance, introduces the next part of the link performance optimization points:
- Asynchronous interface provides the ability to: use the database access is usually synchronous interface, synchronous interface to facilitate the development, but the client by consuming large network interactions affect, concurrent capacity is limited. Multi-threaded approach can help improve concurrency, but the thread resources of the machine is valuable, too many threads will increase the cost of the machine thread switching, limiting concurrency. To WAS can achieve higher throughput, based on event-driven mechanism to increase our support asynchronous interfaces within the ODBC interface. Using an asynchronous interface, WAS within a single thread can execute without waiting for the results after sending a request to continue to send requests on other Session, through the full use of resources and thus significantly improve the thread handling capacity. Asynchronous interface itself reference ODBC interface specification, the user invokes an asynchronous interface will return immediately, if not already done SQL_STILL_EXECUTING complete returns, the user can return to poll the interface until execution is completed successfully (SQL_SUCCESS) or failure (SQL_ERROR), it can also be driven based on network events, when there are results returned again call interface get results. Using an asynchronous interface, it can easily support a large number of resources in a small number of threads of concurrent connections, greatly enhance the ability of concurrent WAS, the utilization of machine resources has also improved, significantly reducing the pressure test costs.
- The ability to provide Prepared Statement: Prepared Statement is a binary request interaction protocol, a PS SQL text transmission, multiple times, OBProxy SQL engine caches PS SQL text and parsing the results, each PS SQL need only be performed once Prepare operation, follow-up each execution on all Session only need to pass the corresponding Statement Id, you can find the corresponding SQL analysis result from the cache, combined with the incoming parameters, you can quickly calculate the routing information OBServer forwarding performance more efficient . Meanwhile, as OBProxy agent layer is also very good support for the distributed nature of OBServer when the need to switch Client Session Server Session, without sending the data type of PS SQL and Execute stage again, OBProxy can judge for themselves and decide whether to synchronize PS SQL plus or data type. Prepared Statement by ability, can effectively reduce the interaction between the system cost, improve performance, interactive way compared to ordinary SQL text, eliminating CPU overhead transmission and requesting a large number of text parsing SQL text.
- Stored Procedures: For stored procedures, OBProxy done a lot of optimization, the stored procedure typically contains multiple SQL, SQL usually require different routing execution to different OBServer, produce large amounts of remote execution. Remote execution will not only increase RT, will take up more CPU resources, therefore, OBProxy SQL engine parses SQL stored procedure, calculate the optimal strategy, stored procedure calls sent to the most appropriate OBServer execution, as far as possible reduce the number of remote execution. And analytical results will be cached routing information OBProxy stored procedures, eliminating the need for each CPU overhead caused by hard analysis.
- Complex types: do we re-expanded on the original OceanBase transfer protocol, so that the entire link supports complex types of transport. Meanwhile, OBProxy added parsing complex data types, and the route can be calculated according to the complex type cutting data partition. By supporting complex types, can improve data information carried by each transmission, effectively reducing the number of interactions, it is possible to calculate the optimal routing strategy based on complex types of data, as much as possible of the route to the partition more OBServer, reducing the number of remote execution. It was with OBProxy For complex types of arrays, and support, which makes the client can better ability to use stored procedures and ARRAY BINDING of.
Acting resource consumption
OBProxy agent multithreaded asynchronous frame forwarding and transparent flow design, to ensure high performance data forwarding (50,000 mononuclear QPS, forwarding RT 30 ~ 50 us), and a minimum consumption of the machine's own resources. In the TPC-C benchmark, we also used locally in the form of deployment to WAS end, so you can maximize the client to reduce network overhead proxy layer. In this test, OBProxy agent layer 64 deployed on the client machine, each machine using 10 to Core, occupying a total of 640 Cores, only about 7% of the overall test cost CPU, a little use of storage resources, small footprint ratio and the level of support expansion.
Continuous service capability
OceanBase provides a highly available database services, in the scene appears the machine is not available, the database does not require any manual intervention is still able to continue to provide services in this TPC-C tests do Durability forced off operation, OceanBase on the show the ability of the database without human intervention under continuous service, well beyond expectations auditors.
OceanBase forced off for this single failure scenarios, OBProxy have a gray list and black list contains two mechanisms for handling peak load shifting OBServer merger, upgrades, downtime, start / stop, network partitioning and other states. Blacklist take refreshed periodically maintained by OBServer feedback which server node can not provide services. Gray list the trigger to take the initiative to maintain, when OBProxy forward the request to OBServer, if found OBServer return a specific system error, or OBServer there many times in a row is not available, the OBServer added to the gray list for some time. Blacklist OBServer not be accessed, the gray list OBServer time to time will try again to check whether the need to whiten, to avoid long will OBServer downgrade. Through mechanisms such OceanBase, TPC-C database can ensure continued service capabilities Durability testing process.
to sum up
OceanBase link layer provides customers with a complete end to end solution for their self-development of transport protocols can be very flexible support SQL features and interaction protocols to achieve a standard database access interface and support for Oracle-compatible mode, you can easily reach the database use, high performance, maximum service continuity of balance. Follow-up will continue to optimize the transport protocol to achieve maximum transmission efficiency and interaction, improve database access standard interface to provide users with a more mature database services.