Mysql study concluded (74) - Slow SQL! The team crushed the last straw!

background

Today to discuss only one application model, is the most popular front-end back-end Web service calls in real time, the server through additions and deletions to change search DB's application to respond. As for off-line data analysis, online template rules engine execution, flow calculation is not in the scope of this discussion.

First, the heavy weight of SQL or Java development scenarios presentation

Look at an example, the demand is: check out the city where each student's name and the score display to the front. The classical Controller, Service, DAO development model description, design database table is as follows:

Write pictures described here

(1) Weight SQL schema sample code:

Write pictures described here

(2) weight Java mode sample code:

Write pictures described here 
Write pictures described here 
Write pictures described here

It can be seen using heavy SQL model for development is indeed very very soon, just need to get away basic SQL developed, but looking at the model with a heavy Java developers need to write a bunch of code, so it seems like SQL victory edge.

Well, PD suddenly said, and I want the city called "plantain", the fraction multiplied by 2 displayed. Grip the grass, how do this? ?

(1) Weight SQL schema sample code:

Write pictures described here

All right. . The SQL has become very complicated basic can not read. .

(2) weight Java mode sample code:

Write pictures described here

Hey Well it seems not many changes.

PD again this time I'm going to the city called "plantain", and the city Code of less than 10086, multiplied by the fraction 2 displayed. Grip the grass, finished, before all SQL, this needs to be how do? ? Continue up the overlay continue CASE WHEN? Do not think clearly, suddenly DBA fly over the phone, your brother dei SQL is too slow, and now caused the collapse of the entire library, you is not no indexed?

I: Index added ah. . . Is it not come? It was resolved before I would first slow SQL development needs? Demolition of the library is not possible, the logic so complicated demolition library devil ran absolutely can not, add CPU and memory ah DBA big brother! ! !

[DBA Daily] slow SQL 180+, 10 have been resolved.

And on a version

[DBA Daily] slow SQL 200+, 15 have been resolved.

And on a version

[DBA Daily] slow SQL 250+, 30 have been resolved.

Slowly, development and operation of both fatigue and DBA to monitor these daily SQL. . . .

Second, the above-described exemplary Consideration

观察了一下,传统企业以及绝大部分转型中的企业的 Java 应用中,很神奇的是,他们的开发人员包括我自己以前,大家都非常非常希望使用一个 SQL 来完成所有的逻辑的编写,非常多企业更是把数据库的存储过程和数据库自定义函数来完成。这些关于逻辑应该写在哪里的争论从来没有停止过,不仅仅发生在后端和数据库端,连前后端都经常会发生这种争论,现在只讨论后端和数据库端的纠结。我将从这五个方面分别对比一下两种模式的异同:

  •     出现场景
  •     开发效率
  •     缺陷排查
  •     架构升级
  •     系统维护

三、出现场景

1、SQL

我们绝大多数的历史代码都是用存储过程来实现的啊,如果有新需求不往上面做的话,很难兼容原来的逻辑啊啊。前面的人呢是这样写的,我来了看大家都这样写就这样写了。

2、Java

新应用嘛,我想怎么样写就怎样写。监控和埋点写起来简单吖,排查问题可方便了。前面的人呢是这样写的,我来了看大家都这样写就这样写了。

四、开发效率

1、SQL

这样写起来很快啊,而且写 Java 代码多难受啊,写 SQL 我自己在数据库开发环境跑一下结果正确我就直接丢到代码中提交了,多爽啊。老实说,这样子确实会提高开发的效率,因为不用写那么多查库聚合的操作,一切都在 SQL 中搞定了。另一方面来看,这确实会让 Java 代码看起来很鸡肋,好像只是把数据从 web 层到数据层的一个管道而已,一切 if else 能写在 SQL 中的都写在 SQL 中了。

但是新需求来或者需求变更的时候,我经常要重新写SQL,如果变动不多我可能要改动到原来的 SQL,但是我又不敢改,所以只好 copy 重新写一个,改 SQL 的风险好大,一报错又要重启好难受。

2、Java

一次要写N个类,有点烦。新需求来或者需求变更的时候,如果逻辑比较复杂,我直接抽成方法或者改成一些设计模式,维护起来效率还是可以接受的。

五、缺陷排查

1、SQL

开发排查问题的时候,除了看日志,直接把SQL和参数丢到 PL/SQL 或者 其他工具里跑一下,基本就能知道数据问题出现在哪了。测试同学在进行测试的时候,如果发现有不对的东西,直接跟开发同学一样的思路,把SQL 跑一下,问题基本就定位得七七八八了。

但是呢,一旦遇到跑 SQL 无法一眼看出问题的 bug 或者 SQL 实在是太长太长了的的时候,就蒙圈了。我曾经就维护了一个几千行的存储过程,一旦发生问题,排查问题的过程巨艰难。但是呢直接用一个数据库一个功能搞定所有功能未尝不是一件很爽的事情,因为关系型数据库实在是实在是太太太稳定了,一次编写永久运行。

2、Java

看日志看监控。根据报错的代码位置 check 一下代码逻辑。一些入参分支肉眼 check 不出来,只能远程 debug 慢慢看数据流向。测试的同学基本无法帮忙 check 缺陷,只能靠程序的表现来判断。

六、架构升级

1、SQL

SQL 慢没关系,它稳定啊,慢就把机器垂直扩展一下好啦,加cpu,加内存,换SSD,加加加绝对可以解决事情的。SQL 有各种索引和优化策略,说不定跑起来比我们自己写逻辑还快呢。加加加,加内存加cpu垂直升级。也没有其他招数了,除了前置缓存,但是如果查询都很个性化SQL很复杂,前置缓存也基本没啥乱用。。。

如果你的逻辑全部写在 SQL 中,那完蛋了,你这个表基本就没法分表了,因为你的业务逻辑跟数据库的数据完整性是强耦合的,需要一切数据基本都在一个数据库中,这是一件很难受很难受的事情,不信你去问问那些所有业务逻辑全写在 SQL 中的小伙。数据库中非常复杂的表关联会极大程度拖慢数据库处理每条 SQL 的平均时间,极大程度拖慢数据库 RT,降低了数据库的 RT ,如果逻辑都写在 SQL 中,那么只能进行垂直升级。因为一旦进行水平扩展,那么多机器的非常复杂的分布式表关联,RT 基本不是一个高并发的业务应用的能容忍的。

2、Java

如果是数据库瓶颈,加数据库机器,分库分表一下,应用层基本不用改,在DAO层进行路由一下。如果是服务器cpu瓶颈,多加几台机器就好了。如果还有瓶颈,增加一下查询缓存。在应用快速发展的过程中一般都会分库分表的拆分或者自动水平扩展,这时候其实只需要数据库层面做好自己的数据迁移和同步就好了,对于业务层来说是完全无感知的。即使业务非常非常复杂,需要拆应用,其实也非常简单,只需要把对应的 DAO 层的操作拆分出去,换成 RPC 或者其他方式的调用就好了。

七、系统维护

1、SQL

旧SQL完全不敢动,来一个需求加一个 SQL。慢SQL日益增加,应对疲乏。

2、Java

SQL写完一次基本不用动,来一个需求加一个方法聚合一下数据操作即可。应用维护比较简单,只要监控做好了,定位到问题基本都能很快解决。逻辑越来越复杂,没有好的开发框架的话,代码维护起来也是挺要命,因为完全不知道跑哪个分支去了。但是现在已经有很多优秀的开源框架来更好地维护代码了,比如 Spring 的全家桶。

八、怎么破!

旧的重 SQL 逻辑暂时不要动,新的逻辑都基于 Java 模式开发,先保证慢 SQL 不增加,旧的 SQL 稳定运行,毕竟业务稳定是第一要素。如果业务初期需要非常非常快速开发,那么使用重 SQL 模式也是可以理解的,但是还是要抽时间重构成 Java 模式。

九、结论

I support the application logic written in Java and other systems. Actually, the reason described above, the basic finished, the first is associated with complex SQL table is actually a very big relationship with the individual's ability, if a poorly written SQL, it is very slow very slow very easy to slow down the entire database . The second is to maintain these SQL is a very difficult subject matter, because you do not know the data transfer SQL behind what, you can go see in SQL bug, Java application based on his guess whatever the outcome, can debug it RBI data is not there to see? If the logic written in Java, so you really need to write a DAO layer only once, but can be used permanently, basic does not waste a lot of time at this level (used ibatis knows that changed the SQL required to restart the application, right?). The third is the logic written in SQL, is a very sad thing, really uncomfortable for a sub-library sub-table splitting, and applications for.

Reprinted from: https://mp.weixin.qq.com/s/62fTZoAU_ThqA50v9iY1TQ

Guess you like

Origin blog.csdn.net/u012562943/article/details/93198712