Spring Boot (four): Druid connection pooling password encryption and monitoring

In the previous article "Spring Boot (three): ORM framework with connection pooling JPA Hikari" We introduced the JPA integration using connection pooling Hikari in the country use more of a connection pool is also open source Ali Druid. In this article we have to use some chat Druid posture.

1. Druid What is that?

We first look at the official answer:

Druid is the best Java language database connection pool. Druid can provide a powerful monitoring and extensions.

Said the Druid is the best Java language database connection pool, this I personally feel a little bragging, Hikari at least in terms of performance and our previous introduction is not better than, the relevant performance tests can be found on the Internet a lot, I not listed here. But the Druid in some other aspects to do relatively well, and is very rich:

• Can monitor database access performance, Druid built provides a powerful StatFilter plugin that detailed statistics on the performance of SQL execution, which is helpful for online access performance analysis database.
• Database password encryption. Write directly to the database password in the configuration file, which is not good behavior, easily lead to security problems. DruidDruiver and DruidDataSource support PasswordCallback.
• SQL execution log, Druid offers different LogFilter, to support Common-Logging, Log4j and JdkLog, you can press the need to select the appropriate LogFilter, database monitoring visits your application.
• Extended JDBC, if you want to have a demand programming for JDBC layer, Filter mechanism provided by the Druid, it is easy to write extensions JDBC layer.

2. Spring Boot application how to use

Druid official currently offers two ways for us to use dependency, one is based dependencies of traditional Java project to provide, maven following coordinates:

<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>druid</artifactId>
    <version>1.1.20</version>
</dependency>

Another is based on dependencies Spring Boot provided, maven following coordinates:

<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>druid-spring-boot-starter</artifactId>
    <version>1.1.20</version>
</dependency>

This dependence in addition to the following package contains the above kind of Druid basic package, also includes the Spring Boot automatic configuration dependencies and sl4j-api, we use a Druid in Spring Boot, of course, the reader is recommended you use the second way the introduction of dependency.

3. combat engineering

3.1 Creating a parent project spring-boot-jpa-druid

Parent project pom.xml as follows:

Listing: the Spring-the Boot-JPA-Druid / pom.xml
***

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.8.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.springcloud</groupId>
    <artifactId>spring-boot-jpa-druid</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>spring-boot-jpa-druid</name>
    <description>spring-boot-jpa-druid</description>

    <properties>
        <druid.version>1.1.20</druid.version>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>

        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid-spring-boot-starter</artifactId>
            <version>${druid.version}</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

• The author Druid dependencies used here is the druid-spring-boot-starterversion number is 1.1.20.

3.2 database password is not encrypted configuration file application-pass.yml as follows:

Listing: the Spring-the Boot-JPA-Druid / src / main / Resources / the Application-pass.yml
***

spring:
  datasource:
    type: com.alibaba.druid.pool.DruidDataSource
    url: jdbc:mysql://192.168.0.128:3306/test?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=UTF-8&useSSL=false
    username: root
    password: 123456
    driverClassName: com.mysql.cj.jdbc.Driver
    druid:
      # 连接池的配置信息
      # 初始化时建立物理连接的个数
      initial-size: 3
      # 连接池最小连接数
      min-idle: 3
      # 连接池最大连接数
      max-active: 20
      # 获取连接时最大等待时间，单位毫秒
      max-wait: 60000
      # 申请连接的时候检测，如果空闲时间大于timeBetweenEvictionRunsMillis，执行validationQuery检测连接是否有效。
      test-while-idle: true
      # 既作为检测的间隔时间又作为testWhileIdel执行的依据
      time-between-connect-error-millis: 60000
      # 销毁线程时检测当前连接的最后活动时间和当前时间差大于该值时，关闭当前连接
      min-evictable-idle-time-millis: 30000
      # 用来检测连接是否有效的sql 必须是一个查询语句
      # mysql中为 select 'x'
      # oracle中为 select 1 from dual
      validation-query: select 'x'
      # 申请连接时会执行validationQuery检测连接是否有效,开启会降低性能,默认为true
      test-on-borrow: false
      # 归还连接时会执行validationQuery检测连接是否有效,开启会降低性能,默认为true
      test-on-return: false
      # 是否缓存preparedStatement,mysql5.5+建议开启
      pool-prepared-statements: true
      # 当值大于0时poolPreparedStatements会自动修改为true
      max-pool-prepared-statement-per-connection-size: 20
      # 合并多个DruidDataSource的监控数据
      use-global-data-source-stat: false
      # 配置扩展插件
      filters: stat,wall,slf4j
      # 通过connectProperties属性来打开mergeSql功能；慢SQL记录
      connect-properties: druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000
      # 定时输出统计信息到日志中，并每次输出日志会导致清零（reset）连接池相关的计数器。
      time-between-log-stats-millis: 300000
      # 配置DruidStatFilter
      web-stat-filter:
        enabled: true
        url-pattern: '/*'
        exclusions: '*.js,*.gif,*.jpg,*.bmp,*.png,*.css,*.ico,/druid/*'
      # 配置DruidStatViewServlet
      stat-view-servlet:
        # 是否启用StatViewServlet（监控页面）默认值为false（考虑到安全问题默认并未启动，如需启用建议设置密码或白名单以保障安全）
        enabled: true
        url-pattern: '/druid/*'
        # IP白名单(没有配置或者为空，则允许所有访问)
        allow: 127.0.0.1,192.168.0.1
        # IP黑名单 (存在共同时，deny优先于allow)
        deny: 192.168.0.128
        # 禁用HTML页面上的“Reset All”功能
        reset-enable: false
        # 登录名
        login-username: admin
        # 登录密码
        login-password: admin

• Meaning configuration has been written in the comments, there is one thing to talk about, when we want to configure statistics (including monitoring information) time-between-log-stats-millisoutput to the log, the monitoring data merge multiple DruidDataSource's use-global-data-source-statnot open, otherwise it will start error.
• spring.datasource.druid.filters: Because Druid expansion is to open in the form of Filter plug-in, where we have opened statand wall, maybe has to monitor each injection attack and defense SQL. Druid also provides some other default Filter, as follows:

Filter class name	Aliases
default	com.alibaba.druid.filter.stat.StatFilter
stat	com.alibaba.druid.filter.stat.StatFilter
mergeStat	com.alibaba.druid.filter.stat.MergeStatFilter
encoding	com.alibaba.druid.filter.encoding.EncodingConvertFilter
Log4j	com.alibaba.druid.filter.logging.Log4jFilter
log4j2	com.alibaba.druid.filter.logging.Log4j2Filter
slf4j	com.alibaba.druid.filter.logging.Slf4jLogFilter
commonlogging	com.alibaba.druid.filter.logging.CommonsLogFilter
wall	com.alibaba.druid.wall.WallFilter

Can be seen from the name, are mainly related Filter some coding and logs.

3.3 database password encryption

In a production environment, direct exposure in the configuration file in plain text password is a very dangerous thing, for two considerations: Foreign, even if the application service is compromised, the database is safe; internal, password database theory production environment only dba should know, but the code is placed in the code repository, if the password is not encrypted, each release before dba will need to manually modify the configuration file and then packaged compilation.

First, we need to generate the database password ciphertext, you need to execute the following command at the command line:

java -cp druid-1.0.16.jar com.alibaba.druid.filter.config.ConfigTools you_password

Output is as follows:

privateKey:MIIBVAIBADANBgkqhkiG9w0BAQEFAASCAT4wggE6AgEAAkEAh12hnaZuMe76Yb4pi7ogSAEMOcavmz7Blo8DYxeipxeZQhnrXngxc0gAQ6ORlofLWtDm6S7bI7wfDT2EFy/2DwIDAQABAkABMRjYK3vy4pi/vY3eFhBssd2qsI4hPsczjSTJfY7IC9Dc1f7g0axTM6Cx68tRUwv0rSnUiJ5EcDEhuD0JusSZAiEAwX1HpCTq8QgBV1WriHQC7Cd/9Qqp1V4yJeA/jdvXhbsCIQCzGS6wdTQCXDZKLvjRLeSUyTmmIqV/wckqdnpMUZ2BvQIgBIamr1tBt6OlTGKvoYB9NQLzhkrakCgk6ifltK7IytMCIBIbf67zipiafhqt+RYdD7lDRwLXCeiKzS3v4JmKvuP5AiEAr+zqD6sdXv7rWjqu50n+LXbWtNP/M4JzzO1mJOHEhoE=

publicKey:MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAIddoZ2mbjHu+mG+KYu6IEgBDDnGr5s+wZaPA2MXoqcXmUIZ6154MXNIAEOjkZaHy1rQ5uku2yO8Hw09hBcv9g8CAwEAAQ==

password:Y464AerH8tabxQg5DlkUej6gQ64KY73ahgiPyaB0vguLBLjUEEkVu6VBueiXxcnMfVjh1Nbd+lJNUTnS1a3/xg==

Here we need to generate a public key publicKeyand password passwordto join the configuration file, application-decrypt.ymlas follows:

Listing: the Spring-the Boot-JPA-Druid / src / main / Resources / the Application-decrypt.yml
***

spring:
  datasource:
    type: com.alibaba.druid.pool.DruidDataSource
    url: jdbc:mysql://192.168.0.128:3306/test?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=UTF-8&useSSL=false
    username: root
    # 加密后密文，原密码为 123456
    password: Y464AerH8tabxQg5DlkUej6gQ64KY73ahgiPyaB0vguLBLjUEEkVu6VBueiXxcnMfVjh1Nbd+lJNUTnS1a3/xg==
    driverClassName: com.mysql.cj.jdbc.Driver
    druid:
      filter:
        config:
          enabled: true
      connection-properties: config.decrypt=true;config.decrypt.key=MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAIddoZ2mbjHu+mG+KYu6IEgBDDnGr5s+wZaPA2MXoqcXmUIZ6154MXNIAEOjkZaHy1rQ5uku2yO8Hw09hBcv9g8CAwEAAQ==
    # 剩余配置省略

• Has been omitted part of the configuration, there is a need readers can get access Github repository.

3.4 configuration file application.ymlas follows:

Listing: the Spring-the Boot-JPA-Druid / src / main / Resources / application.yml
***

server:
  port: 8080
spring:
  application:
    name: spring-boot-jpa-druid
  profiles:
    active: decrypt
  jpa:
    database: mysql
    show-sql: true
    generate-ddl: true
    database-platform: org.hibernate.dialect.MySQL5InnoDBDialect
    hibernate:
      ddl-auto: update
    properties:
      hibernate:
        format_sql: true

The rest of the test code above article "Spring Boot (three): ORM framework with connection pooling JPA Hikari" , interested readers can get access Github repository, I will not list them here.

4. Test

We in the main configuration file, select the password encryption configuration file starts, spring.profiles.activeconfigure decrypt, start clicking, you can see the project start properly, view the console output log, which has such a sentence:

2019-09-22 21:21:54.501  INFO 16972 --- [-Log-1465691120] c.a.d.p.DruidDataSourceStatLoggerImpl    : {"url":"jdbc:mysql://192.168.0.128:3306/test?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=UTF-8&useSSL=false","dbType":"mysql","name":"DataSource-1465691120","activeCount":0,"poolingCount":3,"poolingPeak":3,"poolingPeakTime":"2019-09-22 21:21:54","connectCount":0,"closeCount":0,"physicalConnectCount":3}

You can see, we configure the monitor information output will first output once when the system starts, we configured in the configuration file is output every 5 minutes, such as ten minutes to look at the console output results are as follows:

2019-09-22 21:26:54.503  INFO 16972 --- [-Log-1465691120] c.a.d.p.DruidDataSourceStatLoggerImpl    : {"url":"jdbc:mysql://192.168.0.128:3306/test?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=UTF-8&useSSL=false","dbType":"mysql","name":"DataSource-1465691120","activeCount":0,"activePeak":1,"activePeakTime":"2019-09-22 21:21:54","poolingCount":3,"poolingPeak":3,"poolingPeakTime":"2019-09-22 21:21:54","connectCount":2,"closeCount":2,"connectionHoldTimeHistogram":[0,0,2]}

2019-09-22 21:31:54.505  INFO 16972 --- [-Log-1465691120] c.a.d.p.DruidDataSourceStatLoggerImpl    : {"url":"jdbc:mysql://192.168.0.128:3306/test?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=UTF-8&useSSL=false","dbType":"mysql","name":"DataSource-1465691120","activeCount":0,"poolingCount":3,"connectCount":0,"closeCount":0}

2019-09-22 21:36:54.505  INFO 16972 --- [-Log-1465691120] c.a.d.p.DruidDataSourceStatLoggerImpl    : {"url":"jdbc:mysql://192.168.0.128:3306/test?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=UTF-8&useSSL=false","dbType":"mysql","name":"DataSource-1465691120","activeCount":0,"poolingCount":3,"connectCount":0,"closeCount":0}

As can be seen from the time, the output will indeed every 5 minutes.

Open a browser to access: HTTP: // localhost: 8080 / Druid / , see Druid monitoring page, the results shown:

We can make some interface testing, monitoring viewing the page, you can see all of the SQL are normally recorded as:

At the same time, we look at the background of the print log, a log record is normal play, intercepting print part, as follows:

2019-09-22 21:51:54.506  INFO 16972 --- [-Log-1465691120] c.a.d.p.DruidDataSourceStatLoggerImpl    : {"url":"jdbc:mysql://192.168.0.128:3306/test?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=UTF-8&useSSL=false","dbType":"mysql","name":"DataSource-1465691120","activeCount":0,"activePeak":1,"activePeakTime":"2019-09-22 21:47:28","poolingCount":3,"poolingPeak":3,"poolingPeakTime":"2019-09-22 21:47:28","connectCount":4,"closeCount":4,"executeCount":4,"commitCount":4,"pstmtCacheHitCount":2,"pstmtCacheMissCount":2,"startTransactionCount":4,"transactionHistogram":[0,1,2,1],"connectionHoldTimeHistogram":[0,1,0,3],"sqlList":[{"sql":"insert into user (age, nick_name, id) values (?, ?, ?)","executeCount":2,"executeMillisMax":1,"executeMillisTotal":2,"executeHistogram":[1,1],"executeAndResultHoldHistogram":[1,1],"concurrentMax":1,"updateCount":2,"updateCountMax":1,"updateHistogram":[0,2],"inTransactionCount":2},{"sql":"select usermodel0_.id as id1_0_, usermodel0_.age as age2_0_, usermodel0_.nick_name as nick_nam3_0_ from user usermodel0_ order by usermodel0_.id desc","executeCount":2,"executeMillisMax":3,"executeMillisTotal":4,"executeHistogram":[0,2],"executeAndResultHoldHistogram":[2],"concurrentMax":1,"fetchRowCount":4,"fetchRowCountMax":2,"fetchRowHistogram":[0,2],"inTransactionCount":2}]}

It can be seen printed in the Journal of SQL-related information on our execution, monitoring information, and we see exactly the same page.

So far, the test is successful, space reasons, some of the testing process is not listed, you can interested readers to try it yourself.

5. Sample Code

Sample Code -Github

Sample Code -Gitee

6. References

"Druid official documents."