I. Background
With the improvement of the complexity of the business, the size of data grows, more and more companies choose to split vertically or horizontally on its online business database, or even select a different database types to meet their business needs. Originally in the same database instance in SQL queries can be achieved, and now needs across multiple database instances to complete. Data services are "scattered" in various places, how easily these aggregated data related inquiries, has become a major problem plagued users.
For such problems, traditional solutions require the user to advance to all instances of data collection in advance to the same place, then do offline analysis. To do this, users need to maintain link data migration, storage pooling resources to buy the machine data takes a lot of resources and operation and maintenance costs. Moreover, data migration also means that data latency, online business data generated just need to "wait a minute" or even "wait a day" in order to do the analysis, can not meet the real-time requirement.
In order to solve the problem of cross-database queries promptly instance, Ali cloud DMS (Data Management) launched a cross-instance inquiry service.
Second, what is the cross-instance inquiry service
Examples inquiry services online across heterogeneous data sources in different environments, provide timely service related inquiries. Whether the database is MySQL, SQLServer, PostgreSQL or Redis, regardless of which database instances deployed in the cloud Ali region, do not need data collection, will be able to realize the link between these database instances only by a query SQL.
Not only that, your database instance can be deployed in different resource settings, in addition to the RDS, we also support self-built database on the ECS, the public network ip with a self-built database, the user's local IDC self-built database, or even in the other cloud vendors deploy database.
Third, the functional characteristics
3.1 online data in timely fashion
At present most of the data analysis solutions need to export data to offline data OLTP database system before being analyzed, but this approach is difficult to meet real-time requirements, while export data are also at risk of data loss when the system to offline.
DMS instances of cross-tracking, without user migration task, a direct write SQL, you can achieve more directly related to the online database analysis. Since no data synchronization, reducing the complexity of business architecture, but also saves the user holds the offline computing resources, budget and operational costs.
3.2 DBLink
Oracle familiar people should know that we can Oracle currently logged on to establish a DBLink point to another remote Oracle database tables. In cross-instance inquiry service, we have redefined the concept of DBLink, which is a connection point to the virtual instance of any database user, the database instance is an alias. For example, for MySQL is, DBLink and ip / port correspondence. With DBLink, to enable access to any SQL data source.
3.3 supports a variety of relational databases
Currently supports MySQL, SQLServer, PostgreSQL and other RDBMS.
3.4 supports SQL access to NoSQL
In addition to a relational database, queries across instances also supports SQL access, etc. Redis NoSQL database. With support for the SQL syntax, it can also be achieved correlation between the RDBMS and the query NoSQL. Yes, you read that right, it can achieve a SQL query association between MySQL and Redis.
3.5 supports cross-regional and hybrid cloud query
Enterprise development to a certain stage, the number of users, business volume continued to rise, the original stand-alone room capacity can not meet the needs of business development, combined with disaster recovery, high availability and other factors, often choose to deploy across the region, also known as unit deployment. At the same time, many companies need to expand its business overseas, deployed by the nearest local, providing a better experience for users abroad. This split level similar problem is caused by how the global business data associated with a unified summary queries.
With DMS cross-instance tracking, regardless of your database instances deployed in the region which Ali cloud, without the need for data migration across the region, the region can achieve a unified query all data.
In addition to Ali cloud RDS, we also support a variety of database users deploy cloud Ali on the ECS. Not only that, if your database is deployed in local IDC room, and even other cloud vendors, are available through the inquiry service across instances, to achieve these hybrid cloud scenarios across instances associated with the query.
3.6 cross-instance data import and export
insert into b select * from a;
As we all know, this SQL statement can export data to a table a table (b), but if the table is not a table and b on the same database instance, then this sql can not do anything.
跨实例查询服务的出现，打破了实例与实例之间数据导入导出的边界。它可以将数据从一个MySQL实例的表导出到另外一个MySQL实例的表中；也可以将SQLServer表和PostgreSQL表关联查询的结果，导出到MySQL实例的表中，就是这么灵活。
3.7 兼容标准SQL
通过标准的SQL语句，即可实现跨实例查询。同时跨实例查询服务高度兼容MySQL，支持MySQL协议，以及各种常用函数和语法。您可通过JDBC/ODBC驱动连接到跨实例查询服务；也可以使用各种MySQL GUI工具来管理各种数据源；当然，您也可以在DMS跨实例查询控制台上（https://dsql.console.aliyun.com）直接使用。
3.8 Serverless架构
跨实例查询是无服务器化的在线数据库关联查询服务。用户无需预购计算资源、无需维护资源、没有运维和升级成本，随时随地使用。
3.9 高性能低延迟
跨实例查询服务底层基于强大的MPP计算引擎，持续不断地对SQL查询进行优化，包括pushdown、join算法、执行计划缓存、Meta缓存、本地调度、连接池等技术。目前单表查询以及跨实例的多表关联查询，都能在毫秒级完成。
四、技术架构
用户可以在应用程序中，直接使用MySQL JDBC驱动连接跨实例查询服务，进行跨实例查询。当然，我们也提供了控制台页面（https://dsql.console.aliyun.com），直接输入SQL即可执行。
五、应用场景
5.1 垂直拆分后的跨数据库查询
某电商公司原先将会员、订单、商品等数据都存放在一个数据库实例中，但业务发展迅猛，访问量极速增长，导致数据库容量及性能遭遇瓶颈，因此用户决定对架构进行垂直拆分，将会员、商品、订单数据垂直拆分至三个数据库实例中。此时业务上需要展示某个品类商品的售卖订单量，原本在同一数据库里的查询，要变成跨两个数据库实例的查询。业务上要怎么进行关联查询？
用户首先想到的方法是，对现有业务代码进行重构，分别从两个数据库查询数据，然后在业务代码中进行join关联。那么问题来了，如果采用这个解决方案，业务上那么多查询改造起来，拆分难度极大，操作起来过于复杂。跨库join操作又没有非常高效的办法，需要从各个业务库迭代查询，查询效率也会有一定影响。
我们发现用户遇到的其实就是典型的跨实例查询问题。目前，阿里云跨实例查询服务已经支持跨多个数据库实例的SQL查询的能力，用户利用一条SQL即可解决上述难题。不仅能够满足“跨库Join”这一核心诉求，还能极大地简化用户的技术方案。
5.2 水平拆分后的跨数据库查询
某酒店在多个城市都有对应的门店，其数据库在每个城市也会单独部署一套，业务上有对多个城市全局数据查询的诉求。同样，现在越来越多的互联网行业开始引入单元化架构，在每个城市会单独部署机房和数据库，进行多单元数据汇总查询的需求也越来越强。
为了满足云上这些跨单元、跨region的数据库查询需求，跨实例查询服务打通region之间的屏障，用户通过一条SQL就能实现这些需求。
5.3 异构数据库的关联查询
某公司考虑成本和未来可扩展性，正在将业务数据从SQLServer迁移到MySQL上。在这期间，必然存在某些业务子系统仍然在SQLServer上，另外一些业务子系统已经全部迁移到MySQL上，这时两个子系统之间的联合查询，就可以借助阿里云的跨实例查询服务实现。不仅如此，在迁移过程中，还可以通过跨实例查询服务，来校验SQLServer和MySQL上的数据是否一致。
5.4 混合云场景的关联查询
某游戏公司，由于各种原因，同时保有阿里云、腾讯、UCloud、AWS等环境的数据库实例，同时在自己自建的IDC也部署了部分数据库。业务的数据如此分散，单是统计一下当前游戏在线用户数，都要分别到各个环境去查询一遍再做汇总。借助阿里云跨实例查询服务，一条SQL就能实现跨云厂商和IDC之间的关联查询。
六、小结
Ali cloud DMS (Data Management) across instances inquiry service, covering not only the scene of heterogeneous data sources associated with the query, but also to solve the cross-region, problems associated with cross-cloud database queries. Not only that, we had a substantial query performance optimization, so that most queries can be completed in milliseconds. Users without going through data collection, we can achieve cross-searching across instances through standard SQL.
Original Address: https: //www.csdn.net/article/a/2019-01-14/15968748