Remember a BUG modification! ! ! ! ! Questions about data pool connection
A microservice project based on springcloud, detailed configuration: SpringCloud + SpringMVC+SpringData JPA+ MySql+Postgresql
The project is configured with multiple data sources, and the pre-development test is no problem, but when the project is tested on the server, the database connection abnormality occurred in the morning the next day. After checking the log, I found the following exception:
Note the exception information: No operations allowed after connection closed.
That is to say, the connection obtained by jpa has been closed, and an operation on a closed link caused an exception.
the reason:
The reason for this exception is that MySQL 5.0 will deal with long-term DB connections afterwards, that is, if a DB connection has passed 8 hours without any operation (Mysql server default "wait_timeout" Is 8 hours), Mysql will automatically close the connection. This is the problem. If the connections in the connection pool are idle for more than 8 hours, mysql will disconnect them, and the connection pool itself does not know that the connection has failed. If there is a Client requesting connection at this time, the connection pool will be invalid Connection provided to Client will cause the above exception.
Therefore, when configuring the datasource, you need to configure the corresponding connection pool parameters to check the validity of the connection and clean up invalid connections regularly.
Solution
Add the following connection pool configuration (red font part) under the configuration of the two data sources in application.yml:
eureka:
client:
service-url:
defaultZone: http://localhost:8761/eureka
server:
port: 9013
spring:
application:
name: api
datasource:
druid:
#数据库连接1
mysql:
name: mysql
type: com.alibaba.druid.pool.DruidDataSource
driver-class-name: com.mysql.jdbc.Driver
url: jdbc:mysql://localhost:3306/datacube?useUnicode=true&characterEncoding=utf8&useSSL=false
username: root
password: rootrot
#数据库连接2
greenplum:
name: greenplum
type: com.alibaba.druid.pool.DruidDataSource
driver-class-name: org.postgresql.Driver
url: jdbc:postgresql://localhost:5432/datacube?useUnicode=true&characterEncoding=utf8&useSSL=false
username: root
password: rootroot
# The following is the supplementary setting of the connection pool, which is applied to all the above data sources.
# Initialize size, minimum, maximum
initialSize: 5
minIdle: 10
maxActive: 1000 #Configure
the time to wait for the connection to get timeout
maxWait: 60000 #How
long is the configuration interval to be tested , Detect idle connections that need to be closed, the unit is milliseconds
timeBetweenEvictionRunsMillis: 60000 #Configure the
minimum survival time of a connection in the pool, the unit is milliseconds
minEvictableIdleTimeMillis: 300000 #Verify
whether the connection is valid. This parameter must be set to a non-empty string, the following three items are set to true to take effect
validationQuery: SELECT 1 #Specify whether the connection is checked
by the idle connection collector (if any). If the detection fails, the connection will be removed from the pool.
testWhileIdle: true #Specify
whether to check before removing the connection from the pool, if If the check fails, remove the connection from the pool and try to take out another
testOnBorrow: true #Specify
whether to check before returning to the pool
testOnReturn: false
#Open PSCache and specify the size of the
PSCache on each connection poolPreparedStatements: true
maxPoolPreparedStatementPerConnectionSize: 20 #Configure
the filters for monitoring statistics interception. After removing the monitoring interface sql can not make statistics.'wall' is used for firewall
filters: stat,wall,log4j
#Turn on the mergeSql function through the connectProperties property; slow SQL recording
connectionProperties: druid.stat.mergeSql=true ;druid.stat.slowSqlMillis=1000;druid.stat.logSlowSql=true
#Merge monitoring data of multiple DruidDataSource
useGlobalDataSourceStat: true
# WebStatFilter:
# exclusions: "*.js,*.gif,*.jpg,*.png,*.css,*.ico,/druid/*"
# stat-view-servlet:
# login-username: admin
# login-password: admin
jpa:
database-platform: org.hibernate.dialect.MySQL5InnoDBDialect
show-sql: true
hibernate:
#dialect: org.hibernate.dialect.MySQL5Dialect
ddl-auto: none
naming:
strategy: org.hibernate.cfg.ImprovedNamingStrategy
In this project, Alibaba's druid connection pool is used. The configuration is simple. Except for the database address, driver type, user name and password, they are all defaults. At the beginning, because the project is updated more frequently, there are not too many problems. The library was changed later. As a result, the previous link failed, and the request was good and bad. I followed the code and the configuration of other projects. One of the properties testOnBorrow is set to false (the default setting is false), and testOnBorrow=false will not detect the connection in the pool. Availability,
So if the connection in the connection pool is closed by the database, the application may get these unavailable connections through the connection pool getConnection, and if these connections are not recycled by other threads, they will not be abolished by the connection pool. Will not be re-created, occupying the quota of the connection pool, the project itself as the server, the database link is closed, a large number of timeouts will appear when the client calls the server, the client has set a timeout, but actively disconnected, the server Close_wait will inevitably appear. Because the default maximum number of threads in tomcat is 200, it will soon hang up. Although most sources and data sources without problems, the link will also die when they are concurrently sent, so increase the tomcat default thread (server.tomcat.max -threads=3000) but other data source links will not die in a short time.
The default configuration does not apply to all scenarios, so you need to use it in conjunction with the scenario.
Since testOnborrow = true consumes a lot of performance, in order to ensure the stability of the server, you can use other configurations to avoid this. Use testWhileIdle = true (but the default is false) and timeBetweenEvictionRunsMillis to avoid this problem, so it is possible to set testOnborrow = false Efficient
Reference link 1: https://blog.csdn.net/lx348321409/article/details/76095751
Reference link 2: https://soberchina.iteye.com/blog/2355289
Reference link 3: https://blog.csdn.net/blueheart20/article/details/52384032
---------------------
Author: BUG daily fight
Source: CSDN
Original: https: //blog.csdn.net/qq_38023253/article/details/80815618
copyright notice : This article is the original article of the blogger, please attach a link to the blog post if you reprint it!