MySQL introduced middleware

360 Atlas

Atlas was developed by Qihoo 360, Web infrastructure team to maintain the platform part of a MySQL-based intermediate layer protocol data item. It is based on mysql-proxy 0.8.2 version on, it is optimized, added some new features. Atlas 360 Internal mysql service operation, read and write requests carried per day up of billions.

Altas architecture:

Atlas is located between the application and MySQL, which implements the MySQL client-server protocol, as communication with the application server, as both a client and MySQL communication. It shields the details of the application DB, and in order to reduce the burden of MySQL, it maintains a connection pool.

 

  The main function:

        1. separate read and write

        2. From the Library Load Balancing

        3. IP filtering

        4. SQL statements black list

        The automatic table

(1) sub-table with a required field.

(2) supports SELECT, INSERT, UPDATE, DELETE, REPLACE statement.

(3) Support merge and sort multiple sub-query results table.

 

 

 

     The main function of the official written logic 6. Before the use lua script, low efficiency, Atlas rewritten in C, QPS increase, latency is reduced.

     7. The security upgrade

      (1) the user is connected by Atlas pwds parameter configuration file access control.

     (2) filtering the ip has permission to connect Atlas by client-ips parameters.

     (3) the log records all SQL statements Altas process, including the IP client, DB actual execution of the statement, execution success, the time spent performing 

     8. GR

Set lvs-ips parameters to achieve a smooth restart function through the configuration file, otherwise restart Altas moment those SQL requests will fail. ip NIC physical parameter of the front-mounted lvs, attention not virtual ip. GR condition that at least two of the same configuration and Atlas hanging after lvs.

https://github.com/Qihoo360/Atlas

Cobar 

        Cobar Alibaba (B2B) division develops distributed processing system for relational data, it may look like a traditional database, like to provide you with massive data services in a distributed environment.

Cobar distributed mainly achieved through the table into different libraries:

        A horizontal table supported split into a plurality of parts into different libraries each table to achieve a horizontal resolution

        Support the different tables into different libraries

        In most cases, the user will be a mixture of two or more ways.

        Cobar does not support a table, for example, test table split into test_1, test_2, test_3 ..... in the same library, you must watch after the split into separate libraries to implement distributed.

        In the case where the user profile MySQL heartbeat, of Cobar can automatically connect to the rear end of sending the heartbeat MySQL, MySQL determined operating conditions, once the abnormal operation, preparation of Cobar can automatically switch to the working machine.

 Precautions:

        1. Prepared Cobar main switch trigger has two, one is the user manually triggers a heartbeat Cobar statement is automatically triggered after the abnormality is detected. Then, when an abnormal heartbeat to the host, switching to the standby machine, if the host recovery, require users to manually cut the work back to the host, of Cobar does not automatically switch back to the host when the host restored, unless the heartbeat preparation machine also returns an exception.

        2. Cobar only checks MySQL standby abnormal, do not care about data synchronization between the primary, so the user needs before using Cobar standby configuration in the two-way synchronization MySQL, the details can refer to the MySQL Reference Manual.

 Cobar functional constraint:

        1. does not support join, paging, sorting, child under cross-database query operation situation.

        2. SELECT statement executes is ignored, except for the transaction and character sets.

        3. Under the sub-library case, insert statement must include split fields column name.

        4. sub-library case, update statement can not update the value of the split field.

        5. SAVEPOINT operation is not supported.

        6. only supports MySQL data nodes.

 

 

 

 

https://github.com/alibaba/cobar

Mycat

前身 Cobar,开源,较为活跃。

特点:

1.遵守Mysql原生协议,跨语言,跨数据库的通用中间件代理。

2.基于心跳的自动故障切换,支持读写分离,支持 MySQL 一双主多从,以及一主多从

3.有效管理数据源连接,基于数据分库,而不是分表的模式。

4.基于 NIO 实现,有效管理线程,高并发问题。

5.支持数据的多片自动路由与聚合,支持 sum , count , max 等常用的聚合函数。

6.支持2表 join,甚至基于 caltlet 的多表 join。

7.支持通过全局表,ER 关系的分片策略,实现了高效的多表 join 查询。

8.支持多租户方案。

9.支持分布式事务(弱xa)

10.支持全局序列号,解决分布式下的主键生成问题。

11.分片规则丰富,插件化开发,易于扩展。

12.强大的 web,命令行监控。

13.支持前端作为 MySQL 通用代理,后端 JDBC 方式支持 Oracle、DB2、SQL Server 、 mongodb 。

集群基于 ZooKeeper 管理,在线升级,扩容,智能优化,大数据处理(2.0开发版)。

 

 

 

https://github.com/myCATApache

TDDL

已经不再使用,下一代是 DDRS。

        淘宝根据自己的业务特点开发了 TDDL (Taobao Distributed Data Layer 外号:头都大了 )框架,主要解决了分库分表对应用的透明化以及异构数据库之间的数据复制,它是一个基于集中式配置的 jdbc datasource 实现,具有主备,读写分离,动态数据库配置等功能。

        TDDL 通过部署在客户端的 jar 包,将用户的 SQL 路由到指定的数据库中。

 功能:

数据库主备和动态切换

带权重的读写分离

单线程读重试

集中式数据源信息管理和动态变更

剥离的稳定 jboss 数据源

支持 MySQL 和 Oracle 数据库

基于 JDBC 规范,很容易扩展支持实现 JDBC 规范的数据源

无 Server,Client,Jar 形式存在,应用直连数据库

读写次数,并发度流程控制,动态变更

可分析的日志打印,日志流控,动态变更

 

 

 

TDDL动态数据源使用示例说明:http://rdc.taobao.com/team/jm/archives/1645

简介和快速使用:http://jm.taobao.org/tag/diamond%E4%B8%93%E9%A2%98/

TDDL源码:https://github.com/alibaba/tb_tddl 

TDDL复杂度相对较高。当前公布的文档较少,只开源动态数据源,分表分库分还未开源,还需要依赖diamond,不推荐使用。

DRDS

        整合云服务,收费、Cobar、TDDL整合,商用,首选。

        阿里分布式关系型数据库服务(Distribute Relational Database Service,简称DRDS)是一种水平拆分、可平滑扩缩容、读写分离的在线分布式数据库服务。前身为淘宝 TDDL,是近千个应用首选组件,已稳定服务了七年以上。

 

 

 

Heisenberg

        整合 Cobar、TDDL,目前并不活跃,由百度员工个人编写。

 功能:

分库分表与应用脱离,分库表如同使用单库表一样

减少 DB 连接数压力

热重启配置

可水平扩容

遵守 MySQL 原生协议

无语言限制,MySQLClient,C,Java等都可以使用

      Heisenberg服务器通过管理命令可以查看,如连接数,线程池,结点等,并可以调整

 

 

https://github.com/songwie/heisenberg

 

Oceanus

      用得较少,58 Oceanus 致力于打造一个功能简单、可依赖、易于上手、易于扩展、易于集成的解决方案,甚至是平台化系统。拥抱开源,提供各类插件机制集成其他开源项目,新手可以在几分钟内上手编程,分库分表逻辑不再与业务紧密耦合,扩容有标准模式,减少意外错误的发生 datanode:数据源节点。为一个数据源命名,配置链接属性、报警实现。

      主要概念:

namenode:数据源的簇。为一组数据源命名,指定这组数据源的负载方式、访问模式、权重

table:映射表。匹配解析sql中的table名称,命中table标签的name属性值后,会执行约定的路由逻辑

bean:实体。由其他标签引用,实体类必须有无参的构造函数

tracker:监控埋点。涉及到计算和IO的功能点都有监控点,自定义一个埋点实现类,当功能耗时超出预期时会执行其中的回调函数,便于监控和优化系统

 

 

 

Vitess

    较全 高大上,和开发中的 Mycat2.0 类似。谷歌开发的数据库中间件,集群基于 ZooKeeper 管理,通过 RPC 方式进行数据处理,总体分为,server,command line,gui监控 3部分。

 

 

 

https://github.com/youtube/vitess

OneProxy 

        商业、收费。

        OneProxy 分布式中间件,是平民软件完全自主开发的分布式数据访问层,帮助用户在 MySQL/PostgreSQL 集群上快速搭建支持分库分表的分布式数据库中间件,也是一款具有 SQ L白名单(防 SQL 注入)及 IP 白名单功能的 SQL 防火墙软件。采用与 MySQL Proxy 一致的反向协议输出模式,对应用非常简单和透明易用,让用户畏惧的分库分表(Horizontal Partitioning)工作变得极其简单可控。基于 Libevent 机制实现,单个实例可以实现25万的 SQL 转发能力,用一个 OneProxy 节点可以带动整个 MySQL 集群。  

 

Guess you like

Origin www.cnblogs.com/FireLL/p/11671221.html