Foreword
Rookie like me, there are always a variety of questions, beginning on JDK API is in doubt, the question of the NIO, questions about the JVM, when after several years working on service availability, scalability is also a new doubt, any doubt of it? In fact, a common theme: the expansion of service problems.
Service evolution of the road under normal circumstances
Let's start from the beginning.
Each monomer applications from start-up companies are basically similar to the architecture up, nothing to talk about, basically every programmer SSM and SSH have experienced.
RPC applications when business is growing, we need to service level expansion, expansion is very simple, as long as the service is stateless on it, as shown below:
When business and growing our service relations are complicated, and there are many services do not need to access the DB connection, only need to connect to the cache, it can be made into a separate, reducing valuable connection DB. As shown below:
I believe that most companies are at this stage. Dubbo is to solve this problem students.
If your company's products are popular, business continued rapid development, more and more data, SQL operations more slowly, then the database will become a bottleneck, then you will certainly think of sub-library sub-table, either by ID hash or range the methods can be. As shown below:
This time should be no problem, right. Let your users more, complicated by high again, I just unlimited expansion of the database, unlimited expansion applications on it.
This is also the title of this article, sub-library sub-table can solve unlimited expansion it?
In fact, like the above infrastructure, and can not solve.
In fact, the problem and the problem is somewhat similar to RPC: Database connection too! ! !
通常，我们的 RPC 应用由于是使用中间件进行访问数据库，应用实际上是不知道到底要访问哪个数据库的，访问数据库的规则由中间件决定，例如 sharding JDBC。这就导致，这个应用必须和所有的数据库连接，就像我们上面的架构图一样，一个 RPC 应用需要和 3 个 mysql 连接，如果是 30 个 RPC 应用，每个 RPC 的数据库连接池大小是8 ，每个 mysql 需要维护 240 个连接，我们知道，mysql 默认连接数是 100，最大连接数是 16384，也就是说，假设每个应用的连接池大小是 8 ，超过 2048 个应用就无法再继续连接了，也就无法继续扩容了。
注意，由于每个物理库有很多逻辑库，再加上微服务运动如火如荼， 2048 并没有看起来那么大。
也许你说，我可以通过前面加一个 proxy 来解决连接数的问题，实际上，代理的性能也会成为问题，为什么？代理的连接数也是不能超过 16384 的，如果并发超过 16384，变成 163840，那么 proxy 也解决不了问题。
怎么办？让我们再看看上面的架构图：
我们发现，问题是出在“每个 RPC 应用都要连所有的库”，导致扩容应用的同时，每个数据库连接数就要增加。就算增加数据库，也不能解决连接数的问题。
那怎么办呢？
欢迎大家关注我的公种浩【程序员追风】，文章都会在里面更新，整理的资料也会放在里面。
单元化
单元化，听起来高大上，通常在一些 XXX 大会上，分享“关于两地三中心”，“三地五中心”，“异地多活”等等牛逼的名词的时候，单元化也会一起出现。
这里我们不讨论那么牛逼的，就只说“数据库连接数过多” 的问题。
实际上，思路很简单：我们不让应用连接所有的数据库就可以了。
假设我们根据 range 分成了 10 个库，现在有 10 个应用，我们让每个应用只连一个库，当应用增多变成 20个，数据库的连接不够用了，我们就将 10 个库分成 20 个库，这样，无论你应用扩容到多少个，都可以解决数据库连接数过多的问题。
Note: do this premise is: you must ensure that you access the database application request must be requested in this application. s
Put another way, when users come in from the DNS was there, that he is going that application, so the rules before the DNS on the set well, although it's a bit exaggerated, but certainly before the application into which you know to go to the library.
So, this usually requires a rule, for example, by user ID hash, the hash rule configuration broadcast center. Thus, all components can maintain consistent rules to correct access to the database. As shown below:
Here, we finally solved the problem of infinite expansion.
to sum up
From the single application, and gradually tells the story of the evolution process of a normal background, knows the sub-library sub-table will not solve the "infinite expansion" of the problem, only a unit in order to solve this problem. The unit of the bring more complexity. But the benefits of self-evident.
The unit brings more ideas.
With a unit to solve the problem of infinite expansion, but we have not considered the issue of a single point, namely the availability of services. You know, our database here are single point.
Finally,
we welcome the exchange, like the point of a praise yo remember the article, thanks for the support!