Automatically execute multiple SQL scripts when the SpringBoot project starts

background

There is a project fyk-config, which needs to create a configuration table (FYK_PROPERTIES) when configuring, and insert the configuration records of each microservice into the table.

solution

In SpringBoot, there is a DataSourceInitializer class, which will execute the initialization script when the project starts. The specific code is as follows:
First, in the resources directory, create the folder scritp/db, and then put the sql file in the db folder:

Then, in the project, write a configuration class:

@Slf4j
@Configuration
public class DbScriptInit {
    
    

    @Bean
    public DataSourceInitializer dataSourceInitializer(final DataSource dataSource) throws IOException {
    
    
        final DataSourceInitializer initializer = new DataSourceInitializer();
        initializer.setDataSource(dataSource);
        initializer.setDatabasePopulator(this.databasePopulator());
        return initializer;
    }

    /**
     * 初始化数据资源
     * 
     * @author FYK
     * @return org.springframework.core.io.Resource[] 资源对象
     */
    private Resource[] getResources() throws IOException {
    
    
        ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
        Resource[] resources = resolver.getResources("classpath*:script/db/*.sql");
        log.info("加载初始化脚本文件---------start");
        for (Resource resource : resources) {
    
    
            log.info(resource.getFilename());
        }
        log.info("加载初始化脚本文件---------end");
        return resources;
    }

    /**
     * 初始化数据策略
     * 
     * @author FYK
     * @return org.springframework.jdbc.datasource.init.DatabasePopulator 策略对象
     */
    private DatabasePopulator databasePopulator() throws IOException {
    
    
        final ResourceDatabasePopulator populator = new ResourceDatabasePopulator();
        populator.addScripts(this.getResources());
        return populator;
    }
}

Notice

There are two points to explain:

1. There may be multiple scripts under the db folder, so here we need to use PathMatchingResourcePatternResolver to read multiple .sql files;
2. The path to obtain the file is classpath*:script/db/*.sql , and an asterisk is added after the classpath, which means that even the files in the jar package that meet the rules can be obtained, because the project will eventually be packaged as a jar package To run, if you do not use this asterisk, it will appear in the development, no problem, wait until the test or production environment, there will be a problem that the sql file cannot be found . Of course, the file rules here need to be defined by yourself, so as not to scan files in other packages that should not be loaded.

Supplementary Note

According to the above operation, you should be able to complete the required requirements.
However, the following issues may require attention:

1. SQL scripts are executed sequentially. For example, in my project, I need to build a table and then insert initialization data into the table, so there are two scripts, one is to build the table, and the other is to insert the initialization data (of course, if All your SQL statements are in one file, then there is no such problem). Then you should execute the table creation statement first, and then execute the initialization statement.
   Give a solution: the naming rules of all script files: serial number-table name-operation type-remarks.sql, where the serial number indicates the execution order of these script files. as follows:
   
2. These script files will be executed when the project starts. Then after the project is restarted, it will be executed again. How should this be avoided?
   Here is a reference: First, for each script file, configure a SQL statement to verify whether it is executed, and then when loading these configuration files, first execute the SQL statement to determine whether to execute the script. The code is roughly as follows: in
   application In the configuration file, configure the verification sql corresponding to each file. Here, each of my sql will eventually return 0 or 1. Returning 0 indicates that the script file is to be executed, and returning 1 indicates that it is not executed:

fyk.db-script.check-sql={\
  "1-FYK_PROPERTIES-DQL":"select case when exists(select 1 from all_tables t where t.TABLE_NAME = upper('fyk_properties')) then 1 else 0 end as result from dual",\
  "2-FYK_PROPERTIES-DML-fyk-oauth":"select case when exists(select 1 from fyk_properties t where t.application='fyk-oauth') then 1 else 0 end as result from dual"\
  }

Then modify the image of the DbScriptInit class, and modify the getResources method inside:

    @Value("#{${fyk.db-script.check-sql}}")
    private Map<String, String> checkSql;

    @Autowired
    private JdbcTemplate jdbcTemplate;   
     
   /**
     * 初始化数据资源
     * <p>
     * 首先，每个脚本文件，都要对应一个验证sql，只有验证SQL返回0的时候，才执行该脚本文件，否则不执行。
     * </p>
     * 
     * @author FYK
     * @return org.springframework.core.io.Resource[] 资源对象
     */
    private Resource[] getResources() throws IOException {
    
    
        ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
        Resource[] resources = resolver.getResources("classpath*:script/db/*.sql");
        List<Resource> resultList = new LinkedList<>();
        log.info("加载初始化脚本文件---------start");
        String checkSqlStr;
        Integer resutlNum;
        String dqlSqlMatch = "";
        for (Resource resource : resources) {
    
    
            String fileFullName = resource.getFilename();
            // 如果DQL语句都没有执行，则默认要执行DML语句
            if (fileFullName.matches(".*DML.*") && StringUtil.isNotBlank(dqlSqlMatch)
                && fileFullName.matches(dqlSqlMatch)) {
    
    
                resultList.add(resource);
                continue;
            }
            // 获得验证脚本
            checkSqlStr = this.getCheckSql(fileFullName);
            if (StringUtil.isNotBlank(checkSqlStr)) {
    
    
                resutlNum = jdbcTemplate.queryForObject(checkSqlStr, Integer.class);
                if (resutlNum != null && resutlNum == 0) {
    
    
                    resultList.add(resource);
                    if (fileFullName.matches(".*DQL.*")) {
    
    
                        dqlSqlMatch += this.buildDqlSqlMatch(fileFullName, dqlSqlMatch);
                    }
                } else {
    
    
                    log.info("sql初始化脚本文件[{}]验证结果为1，跳过该脚本", fileFullName);
                }
            }
        }
        log.info("加载初始化脚本文件---------end");
        return resultList.toArray(new Resource[0]);
    }

    /**
     * 获取去掉后缀的文件名
     * 
     * @author FYK
     * @param fileFullName
     *            资源文件全名
     * @return java.lang.String 文件名
     */
    private String getCheckSql(@NonNull String fileFullName) {
    
    
        String fileName = fileFullName.replace(".sql", "");
        log.info("{}.sql", fileName);
        String checkSqlStr = checkSql.get(fileName);
        if (StringUtil.isBlank(checkSqlStr)) {
    
    
            log.warn("sql脚本文件[{}.sql]的执行验证语句未配置~~~~~不执行该SQL脚本", fileName);
        }
        return checkSqlStr;
    }

    /**
     * 构建DQL语句匹配规则
     * 
     * @author FYK
     * @param fileFullName
     *            全文件名
     * @param dqlSqlMatch
     *            DQL语句匹配规则
     * @return java.lang.String 构建结果
     */
    private String buildDqlSqlMatch(String fileFullName, String dqlSqlMatch) {
    
    
        String[] fileFullNameSplit = fileFullName.split("-");
        StringBuilder sb = new StringBuilder();
        if (StringUtil.isBlank(dqlSqlMatch)) {
    
    
            sb.append(".*\\-");
        } else {
    
    
            sb.append("|.*\\-");
        }
        sb.append(fileFullNameSplit[1]);
        sb.append("\\-.*");
        return sb.toString();
    }

for reference only! ! !