Typical scenario | How PolarDB-X supports SaaS multi-tenancy

SaaS multi-tenant background

For many platform applications or systems (such as e-commerce CRM platforms, warehouse order platforms, etc.), their service models are developed around the user dimension (the user dimension here can be a seller or brand, a warehouse, etc.) . Therefore, for this type of platform business, in order to support the horizontal scalability of the business system, the business database is usually horizontally segmented according to the user dimension. However, when some users of platform applications slowly grow into big users (such as big brands, big sellers, big warehouses, etc.), these big users will easily cause the following problems because their data volume or traffic is obviously much larger than other users. The phenomenon

The shard where large users are located will become a hot spot in the business system, occupying a large amount of database resources, and its service quality may easily become unstable due to resource constraints;
Other small users are easily affected by the resource consumption of large users, and service quality is also affected.

Finally, hot spots in the business system of the entire platform appeared frequently, database access was unstable, and business services were affected. The SaaS multi-tenant model, as an application architecture, is often used to solve the above-mentioned business problems. In the SaaS multi-tenant model, the business system will need to serve multiple users, and each user (or each batch of users) can be regarded as a tenant. These different tenants will use common infrastructure and platforms to run within the business system, but data from different tenants will still be independently isolated. Therefore, tenants usually have their own physical resources to store and manage data separately. Therefore, the core idea of SaaS multi-tenancy to solve the problem of business system stability and flexible customization of tenant resources is the isolation of resources and data between tenants. In different actual application scenarios, there are two common SaaS multi-tenant solutions:

Schema-level SaaS multi-tenancy

Schema-level SaaS multi-tenancy means that one tenant corresponds to a Schema containing multiple Table definitions (or a Database, in MySQL, the Schema concept is equivalent to Database). Schemas of different tenants will be distributed on different machines (as shown in Figure 1 below) ) to achieve resource isolation. This solution is suitable for scenarios where different tenants need to run using independent Schema;

Partition-level SaaS multi-tenancy

Partition-level SaaS multi-tenancy means that one tenant will correspond to one or more partitions of a Table (or part of the rows of a Table), and the Parittions of different tenants will be distributed on different machines (as shown in Figure 2 above). To achieve resource isolation, this solution is more suitable for scenarios where different tenants need to use a unified Schema to run.

From the perspective of isolation level, Schema-level SaaS multi-tenancy is more completely isolated than Partition-level SaaS multi-tenancy. However, because the former needs to maintain many Schemas, it will bring higher operation and maintenance costs and query analysis costs than the latter. However, Partition-level SaaS multi-tenancy usually relies on middleware sub-database and table partitioning or distributed database partitioning functions (otherwise a single-machine database cannot achieve resource isolation) to operate, while Schema-level SaaS multi-tenancy does not require that. Users can build several Stand-alone MySQL can also be operated, and the entry threshold is lower.

business issues

Business multi-tenant scenario

Just talking about the application architecture may be a bit abstract. In order to make it easier for readers to understand how SaaS multi-tenancy can help business solve problems, this article will illustrate it with a real case. Zhengma Software’s Banniu platform is China’s leading seller order management platform that provides full-cycle e-commerce customer service (hereinafter referred to as Company B). Its business system requires maintaining numerous sellers of multiple different brands. Usually a brand has multiple sellers (for example, a brand may open multiple online stores), so the brand and sellers have a one-to-many relationship. Currently, Company B's order management platform manages more than 50T of order data, with a daily QPS of nearly 3,000+. The order volume of different brands will vary greatly (the order volume of a large brand may be nearly a hundred times or higher than that of a small brand). In addition to having much larger order volumes than other brands, some big brands will also use more advanced paid VIP services: for example, requiring exclusive resources and data isolation for order data, allowing independent statistical analysis of their own brand's order data, etc. In order to solve the resource usage and service differences of different brands of data, Company B will divide its sellers by brand (equivalent to a brand being a tenant):

Big brand demands:
- The order volume is large (for example, the order data storage size exceeds 1T or 2T), and the data storage volume is large
- An exclusive set of storage resources and the need for independent access to analysis data
- All merchants of the brand must have the same set of storage resources
- There are 150+ big sellers of big brands, and there will be more in the future.
Small brand demands:
- The order list is small and the number of merchants is large (6W+ sellers)
- Share a set of storage resources
- Require all seller data to be evenly distributed on storage

The core question now is, how should the database of Company B's order management platform be designed to meet the demands of the many different brands and their big sellers for different resource usage and data isolation?

Common middleware solutions and their problems

For the above business scenario, if Company B does not use a distributed database, but simply uses stand-alone MySQL and some open source sub-database and table middleware to build a SaaS multi-tenant solution (for example, segmenting the brand and its sellers for the tenant) to isolate the tenant’s resources. On the surface, this seems feasible; but in fact, the business will face more and more difficult problems as a result.

The first is the issue of cross-machine distributed transactions . The vast majority of sharding middleware cannot provide strong consistent distributed transaction capabilities, or can only provide transaction compensation solutions based on final consistency, which means that the business needs to do a lot of additional application transformation costs to avoid cross-border transactions as much as possible. Computer transactions lead to business errors.

Then there is the issue of Schema consistency . Based on the middleware sub-database and table, whether it adopts Schema-level multi-tenancy or Partition-level multi-tenancy, Company B's order platform has to maintain the metadata consistency of each tenant's Schema or Table. For example, for MySQL's common DDL operations such as creating and deleting tables, adding and subtracting columns, and adding and subtracting indexes, the middleware solution cannot guarantee that the tables of all tenants of the platform can take effect at the same time. Once the execution is interrupted, manual intervention must be relied upon to correct it. high cost.

Then there is the issue of tenant data migration. Based on the SaaS multi-tenant solution, if Company B wants to allocate new independent resources to a big brand, it will naturally inevitably migrate the tenant data from the original machine to the new machine. This data migration operation can only be completed by relying on additional synchronization tools to build synchronization links. This intermediate cutting process may even require business downtime. This means that the basic operation of adding a new tenant will also bring very high operation and maintenance costs.

Based on the above analysis, Company B's SaaS multi-tenant solution based directly on stand-alone MySQL and some middleware is not a low-cost solution.

SaaS multi-tenant PolarDB-X solution

In fact, in the distributed database PolarDB-X 2.0, Company B can already solve the above-mentioned problems faced by its business by combining the two capabilities of non-templated secondary partitioning and Locality. In order to make it easier for readers to understand, the following is a brief introduction to the non-templated secondary partitioning and Locality functions of PolarDB-X 2.0.

Non-templated secondary partition

PolarDB-X supports creating partitioned tables using secondary partitioning starting from 5.4.17. Different from other distributed databases (such as TiDB/ CockroachDB, etc.), PolarDB-X's secondary partitioning is not only fully compatible with the native MySQL secondary partitioning syntax, but also extends many secondary partitioning capabilities, such as supporting User-defined non-templated secondary partitions (native MySQL only supports templated secondary partitions). The so-called non-templated secondary partition means that the number of secondary partitions and their boundary value definitions under each primary partition are allowed to be inconsistent, as shown below:

/* 一级分区 LIST COLUMNS + 二级分区HASH分区 的非模板化组合分区 */
CREATE TABLE t_order /* 订单表 */ (
 id bigint not null auto_increment, 
 sellerId bigint not null, 
 buyerId bigint not null,
 primary key(id)
) 
PARTITION BY LIST(sellerId/*卖家ID*/) /*  */
SUBPARTITION BY HASH(sellerId) 
(
  PARTITION pa VALUES IN (108，109) 
    SUBPARTITIONS 1 /* 一级分区 pa 之下有1个哈希分区, 保存大品牌 a 所有卖家数据 */,
  PARTITION pb VALUES IN (208，209) 
    SUBPARTITIONS 1 /* 一级分区 pb 之下有1个哈希分区, 保存大品牌 b 所有卖家数据 */,
  PARTITION pc VALUES IN (308，309，310)
    SUBPARTITIONS 2 /* 一级分区 pc 之下有2个哈希分区, 保存大品牌 c 所有卖家数据 */,
  PARTITION pDefault VALUES IN (DEFAULT)
    SUBPARTITIONS 64 /* 一级分区 pDefault 之下有64个哈希分区, 众多小品牌的卖家数据 */
);

Based on the above-mentioned LIST+HASH non-templated secondary partition, the direct effects it can bring to the application are:

For large brand sellers (equivalent to a tenant), data can be routed to a separate set of partitions;
For small and medium-sized brands, data can be automatically balanced to multiple different partitions according to a hash algorithm to avoid access hotspots.

When the merchant data of big brands and small and medium-sized brands are separated at the partition level according to LIST partitions, it is natural to realize the physical isolation of the storage resources of big brands and small and medium-sized brands. In PolarDB-X 2.0, users can easily implement resource isolation between different partitions with the help of Locality capabilities.

LOCALITY resource binding

PolarDB-X supports specifying the actual storage resource location of the database partition through the LOCALITY keyword (storage resources in PolarDB-X are composed of multiple data nodes (DN nodes), and the location can be allocated through the ID of the DN) to achieve data isolation or Uniform distribution of data. Its specific syntax is as follows:

ALTER TABLE #tableName 
MODIFY (SUB)PARTITION #(sub)partName 
SET LOCALITY='dn=dn1[, dn2,...]'

For example, Company B can use the following SQL command to move all the data of the big brand pa in t_order to a storage node dn4:

ALTER TABLE t_order MODIFY PARTITION pa SET LOCALITY='dn=dn4'

In actual use, users can query the list of all DN node instance IDs of PolarDB-X through SHOW STORAGE, for example:

mysql> show storage;
+----------------------------+----------------+------------+-----------+----------+-------------+--------+-----------+------------+--------+
| STORAGE_INST_ID            | LEADER_NODE    | IS_HEALTHY | INST_KIND | DB_COUNT | GROUP_COUNT | STATUS | DELETABLE | DELAY      | ACTIVE |
+----------------------------+----------------+------------+-----------+----------+-------------+--------+-----------+------------+--------+
| polardbx-storage-0-master  | 10.0.x.1:3306 | true       | MASTER    | 41       | 66          | 0      | false     | null       | null   |
| polardbx-storage-1-master  | 10.0.x.1:3307 | true       | MASTER    | 41       | 53          | 0      | true      | null       | null   |
| ......                     | ......        | true       | META_DB   | 2        | 2           | 0      | false     | null       | null   |
+----------------------------+----------------+------------+-----------+----------+-------------+--------+-----------+------------+--------+

Designing a SaaS multi-tenant solution

Going back to the previous example of Company B, the core requirement of Company B is to isolate the seller data and storage resources of large brands and small and medium-sized brands. Then, Company B can add the corresponding LOCALITY definition to each first-level partition based on the above-mentioned partition table of the second-level partition to specify the storage resources allowed to be used by the first-level partition and all second-level partitions. Then the business can directly implement the isolation of multi-tenant (i.e. brand side) storage resources at the SaaS layer during the table creation stage, as shown below:

/* 一级分区：list columns，二级分区：key 的非模板化组合分区 */
CREATE TABLE t_orders /* 订单表 */ (
 id bigint not null auto_increment, 
 sellerId bigint not null, 
 buyerId bigint not null,
 primary key(id)
) 
PARTITION BY LIST(sellerId /* 卖家ID */ ) 
SUBPARTITION BY HASH(sellerId) 
(
  PARTITION pa VALUES IN (108，109,....) 
    LOCALITY='dn=dn16' /* 大品牌 pa 独占一个DN dn4 */
    SUBPARTITIONS 1,
  PARTITION pb VALUES IN (208，209,....) 
    LOCALITY='dn=dn17' /* 大品牌 pb 独占一个DN dn5 */
    SUBPARTITIONS 1 ,
  PARTITION pc VALUES IN (308，309，310,...) 
    LOCALITY='dn=dn18,dn19' /* 大品牌 pc 独占两个DN: dn6 与 dn7 */
    SUBPARTITIONS 2,
  PARTITION pDefault VALUES IN (DEFAULT) 
    /* 一级分区 pDefault 占用 dn0 ~ dn15 共16个DN, 中小品牌共享 */
    LOCALITY='dn=dn0,dn1,...,dn2,dn15' 
    SUBPARTITIONS 64 
);

As shown in the figure above, through Locality binding to the DN node resources of each first-level partition, the tenants of the three major brands, pa, pb, and pc, are allocated node resources of 3 groups: DN16, DN17, and DN18~DN19 respectively. The pDefault partition of the small and medium seller pool is bound to 16 DN nodes.

SaaS multi-tenant operation and maintenance management

When the secondary partitioning and Locality capabilities solve Company B's isolation of multi-tenant resources of different brands, the immediate question that needs to be faced is naturally: How will users manage these multi-tenants effectively and conveniently? The answer is PolarDB-X 2.0's partition management capabilities. PolarDB-X 2.0 provides a complete series of flexible and powerful partition management commands for partition tables (as shown in the figure below), allowing users to implement different operation and maintenance changes in multi-tenant scenarios through simple SQL commands. .

Next, we will use the example of Company B to separately introduce common operation and maintenance changes in SaaS multi-tenant scenarios based on partition management.

Scenario 1: Add new sellers to tenants based on modifying the LIST partition

Take Company B as an example. One tenant of Company B corresponds to a brand. A brand usually has multiple sellers on Company B’s platform. Therefore, when a brand opens a new store, it needs to add the new seller ID to the brand's corresponding tenant resources. With the MODIFY PARTITION ADD/DROP VALUES function of PolarDB-X, you can easily add a new seller ID to the LIST partition, as shown below:

/* 给品牌 pb 增加新的卖家 205 */
ALTER TABLE t_orders MODIFY PARTITION pb ADD VALUES (205);

During the execution of this DDL, PolarDB-X will automatically extract all data with sellerId=205 from the DEFAULT partition of the LIST (if there is an explicit DEFAULT partition defined) and migrate it to the pb partition. The entire DDL process is Online and the business application is Almost imperceptible.

Scenario 2: Add new tenants and allocate new storage resources based on adding LIST partitions

For order management platforms such as Company B, sellers of various brands on the platform usually go through the process of starting from scratch and growing from small sellers to big sellers. Therefore, when a small seller of a brand develops into a large seller, the brand may ask Company B to extract its sellers from the seller pool of small and medium brands (for example, DEFAULT partition) and make them independent tenants. VIP and allocate separate storage resources to it.

With the help of PolarDB-X's ADD/DROP PARTITION and its Locality function, Company B can easily complete the change operation of the above scenario online. For example, Company B wants to extract the big seller 301 of the new big brand pe from the DEFAULT partition and make it exclusive to the new storage resource new_dn, as shown below:

/* 1.B公司在管控购买新的 CN/DN 的节点资源... */
/* 2.增加新的大卖家，创建新分区并放置到特定的DN节点 */
ALTER TABLE t_orders ADD PARTITION (
  /* pDefault 分区里再抽取出新的大卖家 301 , 并命名为 pe, 并将其数据放置新节点 new_dn */
    PARTITION pe VALUES IN (301) LOCALITY='dn=new_dn' SUBPARTITIONS 1,
);

Similar to MODIFY PARTITION, these ADD/DROP PARTITION change operations also belong to Online DDL, and the data migration operations in between are almost transparent to business applications.

Scenario 3: Support rebalancing of secondary partition data within tenants based on partition-level Locality

PolarDB-X's LIST + KEY non-templated secondary partitions can provide users with an important feature in multi-tenant scenarios, that is, it allows different tenants to have different numbers of secondary hash partitions. This means that different tenants are allowed to use different amounts of storage resources by defining different numbers of secondary partitions. For example, in the LIST partition definition of company B's t_orders table, the number of secondary partitions under the first-level LIST partition of the big brand pc is 2, and 2 DN nodes are exclusively used to store order data (that is, each of the pc partitions) DN nodes are assigned a secondary partition). In addition, there are 64 secondary partitions under the DEFAULT partition shared by its small and medium-sized brand sellers, and it also exclusively occupies a total of 16 DN nodes from dn0 to dn15 (as shown below):

PARTITION pDefault VALUES IN (DEFAULT) 
    /* 一级分区 pDefault 占用 dn0 ~ dn3 共16个DN, 中小品牌共享 */
    LOCALITY='dn=dn0,dn1,...,dn2,dn15' 
    SUBPARTITIONS 64

However, many small and medium-sized sellers in the DEFAULT partition may also have some hot spots (for example, 20% of the top sellers may account for 80% of the orders). If these hot sellers are not distributed reasonably, they may also lead to 16 DN nodes within DEFAULT. The load is unbalanced among them. Therefore, the problem that Company B needs to face is: How to manage the order data of many small and medium-sized sellers in these 64 secondary partitions so that it can be relatively evenly distributed to these 16 DN nodes and ensure the overall load balance of the system? This is why you need to use the partition-level Rebalance capability of PolarDB-X. The partition-level Rebalance function of PolarDB-X allows users to automatically schedule multiple secondary partitions within a primary partition according to the Locality of the primary partition, so that these secondary partitions can be located on the DN node defined by the Locality. Maintain a balanced distribution. Users only need to execute a SQL command (as shown below) to complete the above balanced changes:

REBLANCE TABLE t_orders PARTITIONS=pDefault;

Scenario 4: Support tenants’ data query and data security based on partition selection and view functions

PolarDB-X's partition table and Locality's SaaS-level multi-tenant capabilities can not only meet the data isolation and resource isolation requirements of order management platforms such as Company B, but also provide more services to the business. Data query capabilities. For example, big brands on company B's platform occasionally need to use VIP services such as independent query and analysis of their own order data. These brands will use some Web SQL tools provided by Company B to directly query and analyze their order data (for example, query the number of orders from important customers, etc.). However, as a platform system, Company B needs to ensure data security and isolation between different tenants: that is, tenants who query order data can only see their own data and cannot see any data of other tenants. So, based on PolarDB-X's partition table, how does Company B solve the problem of data isolation for different tenants? The answer is with partition selection and view definition. For example, if Company B wants to authorize its tenant pb to independently query and analyze its own order data, its Web SQL tool will automatically use SQL commands similar to the following to create the corresponding database for the tenant pb on PolarDB-X in advance. Read-only view t_order_pb_view:

CREATE VIEW t_order_pb_view AS 
SELECT * 
FROM t_orders PARTITION（pb) /*  t_orders 表的数据只会返回 pb分区以及下所有二级分区 */ ;

Then, after the platform performs automated authorization operations on the tenant pb account information, the tenant pb will only be allowed to see the read-only view t_order_pb_view after logging in in the Web SQL tool provided by it. So, if the tenant wants to execute a view query that counts the total number of orders as shown below:

/* 大租户 pb  查询订单数据的SQL：统计订单数目 */
SELECT COUNT(1) FROM t_order_pb_view;

PolarDB-X will automatically replace the view t_order_pb_view with the corresponding subquery:

/* 大租户 pb  查询订单数据的SQL：统计订单数目 */
SELECT COUNT(1) FROM 
(
   SELECT * 
     FROM 
   t_orders PARTITION（pb)
) as t_order_pb_view;

As a result, based on the limitation of the partition selection syntax, the view t_order_pb_view will only be allowed to return data of the pb partition. In this way, tenant pb cannot query the seller order data of other tenants, thereby achieving the effect of data isolation.

Practice summary

PolarDB-X partition tables and their supporting flexible management syntax can package various business models in different business scenarios. For example, the SaaS multi-tenant based on non-templated secondary partitioning + Locality capabilities introduced in this article is one of the classic uses. In fact, the merchant order management system of Company B in the real case mentioned in this article has been successfully launched based on the above-mentioned PolarDB-X 2.0 SaaS multi-tenant solution (its application architecture is shown in the figure below). The platform it currently provides is responsible for Manages more than 50T of order data.

However, the case of Company B is obviously a case that can be replicated and generalized. For example, its tenant dimension - brand, can be easily associated with other business dimensions and can build similar practices, such as logistics order management in major warehouses, data management of gifts given by viewers in each live broadcast room on the live broadcast platform, and major Urban traffic monitoring data management, meteorological monitoring data collection in major provinces, etc. Briefly summarize the best practices, if the business scenario exists

The data needs to be horizontally segmented according to a certain dimension (for example, region, warehouse, merchant or brand, etc.) and divided into multiple business units;
It is also necessary to carry out different resource configurations and physical isolation of data for the segmented business units;

For the above points, users can refer to the SaaS multi-tenant solution in this article for database design.

Author: Chengbi

Click to try the cloud product for free now to start your practical journey on the cloud!

Original link

This article is original content from Alibaba Cloud and may not be reproduced without permission.