Text/Zhu Jiqian
I stayed up late to finish writing, and there are still some shortcomings, but I am still studying hard and summarizing. Your likes and attention are the greatest encouragement to me!
In the development of some localization projects, there is a demand that the developed project needs to be able to build the database required by the system and its corresponding database tables by itself when the first deployment is started.
To solve such needs, there are actually many open source frameworks that can automatically generate database tables, such as mybatis plus, spring JPA, etc., but have you ever thought about building a more complex table structure by yourself? At this time, can this open source framework also satisfy the requirements? If not, how can it be achieved?
I wrote an Activiti workflow study note (3) before - an analysis of the underlying principles of automatically generating 28 database tables . In it, I analyzed the underlying principles of the workflow Activiti's automatic construction of 28 database tables. In my opinion, one of the reasons for learning the underlying principles of open source frameworks is to learn something that can be used by me. Therefore, after analyzing and understanding the underlying principles of the workflow to automatically build 28 database tables, I decided to also write a demo of self-created databases and tables based on the Springboot framework. I referred to the logic of the underlying table creation implementation of the workflow Activiti6.0 version. Based on the Springboot framework, the implementation project can automatically build various complex databases and tables such as multi-table associations when it is first started.
The overall implementation idea is not complicated, it goes something like this: first design a set of database sql scripts that completely create multi-table associations, put them in resources, and automatically execute the sql scripts during the springboot startup process.
First, let’s design a set of feasible multi-table association database scripts at once. Here I mainly refer to using Activiti’s own tables as an implementation case. Because it has designed many table associations internally, no additional design is needed.
The statement of the sql script is the ordinary create table statement, similar to the following:
1 create table ACT_PROCDEF_INFO ( 2 ID_ varchar(64) not null, 3 PROC_DEF_ID_ varchar(64) not null, 4 REV_ integer, 5 INFO_JSON_ID_ varchar(64), 6 primary key (ID_) 7 ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE utf8_bin;
Add external primary keys and indexes——
1 create index ACT_IDX_INFO_PROCDEF on ACT_PROCDEF_INFO(PROC_DEF_ID_); 2 3 alter table ACT_PROCDEF_INFO 4 add constraint ACT_FK_INFO_JSON_BA 5 foreign key (INFO_JSON_ID_) 6 references ACT_GE_BYTEARRAY (ID_); 7 8 alter table ACT_PROCDEF_INFO 9 add constraint ACT_FK_INFO_PROCDEF 10 foreign key (PROC_DEF_ID_) 11 references ACT_RE_PROCDEF (ID_); 12 13 alter table ACT_PROCDEF_INFO 14 add constraint ACT_UNIQ_INFO_PROCDEF 15 unique (PROC_DEF_ID_);
The whole thing is to design a set of SQL statements that meet the demand scenarios, save them in the .sql script file, and finally store them in the resource directory, similar to the following:
The next step is to implement the CommandLineRunner interface, rewrite its run() bean callback method, and develop functions that can automatically build database and table logic in the run method.
At present, I have uploaded the developed demo to my github. Interested children can download it by themselves. Currently, it can be directly downloaded and run in the local environment. It can be used as a reference according to your actual needs.
First of all, when solving such needs, the first thing to solve is how to implement the table creation method only once after Springboot is started.
Here you need to use a CommandLineRunner interface, which comes with Springboot. The class that implements this interface, and its overridden run method will be automatically executed after Springboot is started. The source code of this interface is as follows:
1 @FunctionalInterface
2 public interface CommandLineRunner {
3
4 /**
5 *Callbacks used to run beans
6 */
7 void run(String... args) throws Exception;
8
9 }
To expand, in Springboot, you can define multiple classes that implement the CommandLineRunner interface, and you can sort these implementation classes. You only need to add @Order, and its overridden run method can be executed in order. Code case verification:
1 @Component
2 @Order(value=1)
3 public class WatchStartCommandSqlRunnerImpl implements CommandLineRunner {
4
5 @Override
6 public void run(String... args) throws Exception {
7 System.out.println("The first Command is executed");
8 }
9
10
11 @Component
12 @Order(value = 2)
13 public class WatchStartCommandSqlRunnerImpl2 implements CommandLineRunner {
14 @Override
15 public void run(String... args) throws Exception {
16 System.out.println("Second Command execution");
17 }
18 }
19
The information printed by the console is as follows:
1 The first Command is executed 2 The second Command is executed
Based on the above verification, we can implement the CommandLineRunner interface and rewrite its run() bean callback method to implement the table creation method only once after Springboot starts. To implement the function of creating tables at project startup, you may also need to determine whether there is already a corresponding database. If not, you should create a new database first. At the same time, you must consider the situation that there is no corresponding database. Therefore, we connect to MySQL for the first time through jdbc. , you should connect an existing library that comes with it. After each MySql installation is successful, there will be a mysql library. When establishing a jdbc connection for the first time, you can connect to it first.
code show as below:
Class.forName("com.mysql.jdbc.Driver");
String url="jdbc:mysql://127.0.0.1:3306/mysql?useUnicode=true&characterEncoding=UTF-8&ueSSL=false&serverTimezone=GMT%2B8";
Connection conn= DriverManager.getConnection(url,"root","root");
After establishing a connection with the MySql software, first create a Statement object, which is an object in jdbc that can be used to execute static SQL statements and return the results it generates. It can be used here to perform the functions of searching for libraries and creating libraries.
1 //Create Statement object
2 Statement statment=conn.createStatement();
3 /**
4 Use the statment query method executeQuery("show databases like \"fte\"")
5 Check whether MySql has the fte database
6 **/
7 ResultSet resultSet=statment.executeQuery("show databases like \"fte\"");
8 //If resultSet.next() is true, prove it exists;
9 //If false, it proves that the library does not exist yet, then execute statment.executeUpdate("create database fte") to create the library
10 if(resultSet.next()){
11 log.info("Database already exists");
12 }else {
13 log.info("The database does not exist, create the fte database first");
14 if(statment.executeUpdate("create database fte")==1){
15 log.info("New database successfully created");
16 }
17 }
After the automatic creation of the database FTE is completed, you can create tables in the FTE library.
I encapsulated all the related methods of creating tables into the SqlSessionFactory class. The related methods of creating tables also need to use the Connection of jdbc to connect to the database. Therefore, the connected Connection reference variables need to be passed as parameters to the initial constructor of SqlSessionFactory:
1 public void createTable(Connection conn,Statement stat) throws SQLException {
2 try {
3
4 String url="jdbc:mysql://127.0.0.1:3306/fte?useUnicode=true&characterEncoding=UTF-8&ueSSL=false&serverTimezone=GMT%2B8";
5 conn=DriverManager.getConnection(url,"root","root");
6 SqlSessionFactory sqlSessionFactory=new SqlSessionFactory(conn);
7 sqlSessionFactory.schemaOperationsBuild("create");
8 } catch (SQLException e) {
9 e.printStackTrace();
10 }finally {
11 stat.close();
12 conn.close();
13 }
14 }
After initializing new SqlSessionFactory(conn), you can use the Connection object that has been connected in this object.
1 public class SqlSessionFactory{
2 private Connection connection ;
3 public SqlSessionFactory(Connection connection) {
4 this.connection = connection;
5 }
6 ......
7 }
There are two situations where parameters can be passed here, namely "create" represents the function of creating a table structure, and "drop" represents the function of deleting the table structure:
1 sqlSessionFactory.schemaOperationsBuild("create");
Entering this method, you will first make a judgment——
1 public void schemaOperationsBuild(String type) {
2 switch (type){
3 case "drop":
4 this.dbSchemaDrop();break;
5 case "create":
6 this.dbSchemaCreate();break;
7 }
8 }
If it is this.dbSchemaCreate(), perform the table creation operation:
1 /**
2 * Add new database table
3 */
4 public void dbSchemaCreate() {
5
6 if (!this.isTablePresent()) {
7 log.info("Start executing create operation");
8 this.executeResource("create", "act");
9 log.info("Execution of create completed");
10 }
11 }
this.executeResource("create", "act") represents creating a database table named act——
1 public void executeResource(String operation, String component) {
2 this.executeSchemaResource(operation, component, this.getDbResource(operation, operation, component), false);
3 }
Among them, this.getDbResource(operation, operation, component) is the path to get the sql script. Enter the method and you can see——
1 public String getDbResource(String directory, String operation, String component) {
2 return "static/db/" + directory + "/mysql." + operation + "." + component + ".sql";
3 }
Next, read the sql script under the path and generate the input stream byte stream:
1 public void executeSchemaResource(String operation, String component, String resourceName, boolean isOptional) {
2 InputStream inputStream = null;
3
4 try {
5 //Read sql script data
6 inputStream = IoUtil.getResourceAsStream(resourceName);
7 if (inputStream == null) {
8 if (!isOptional) {
9 log.error("resource '" + resourceName + "' is not available");
10 return;
11 }
12 } else {
13 this.executeSchemaResource(operation, component, resourceName, inputStream);
14 }
15 } finally {
16 IoUtil.closeSilently(inputStream);
17 }
18
19 }
Finally, the core implementation of the entire sql script is implemented in this.executeSchemaResource(operation, component, resourceName, inputStream) method——
1 /**
2 * Execute sql script
3 * @param operation
4 * @param component
5 * @param resourceName
6 * @param inputStream
7 */
8 private void executeSchemaResource(String operation, String component, String resourceName, InputStream inputStream) {
9 //SQL statement concatenates strings
10 String sqlStatement = null;
11 Object exceptionSqlStatement = null;
12
13 try {
14 /**
15 * 1.jdbc connects to mysql database
16 */
17 Connection connection = this.connection;
18
19 Exception exception = null;
20 /**
21 * 2. Read the sql script data in "static/db/create/mysql.create.act.sql" in separate lines
22 */
23 byte[] bytes = IoUtil.readInputStream(inputStream, resourceName);
24 /**
25 * 3. Convert the data lines in the sql file into strings, and replace the line breaks with the escape character "\n"
26 */
27 String ddlStatements = new String(bytes);
28 /**
29 * 4. Read string data in the form of character stream
30 */
31 BufferedReader reader = new BufferedReader(new StringReader(ddlStatements));
32 /**
33 * 5. Read in separate lines according to the escape character "\n" in the string
34 */
35 String line = IoUtil.readNextTrimmedLine(reader);
36 /**
37 * 6. Loop through each line read
38 */
39 for(boolean inOraclePlsqlBlock = false; line != null; line = IoUtil.readNextTrimmedLine(reader)) {
40 /**
41 * 7. If there is still data in the next line, it proves that not all of them have been read, and the read can still be executed.
42 */
43 if (line.length() > 0) {
44 /**
45 8. When there is not enough splicing to create a complete table statement,! line.endsWith(";") will be true,
46 That is to say, it keeps looping for splicing, and when it encounters ";", it jumps out of the if statement.
47 **/
48 if ((!line.endsWith(";") || inOraclePlsqlBlock) && (!line.startsWith("/") || !inOraclePlsqlBlock)) {
49 sqlStatement = this.addSqlStatementPiece(sqlStatement, line);
50 } else {
51 /**
52 9. If you encounter the symbol ";" during loop splicing, it means that it has been spliced to form a complete SQL table creation statement, for example
53 create table ACT_GE_PROPERTY (
54 NAME_ varchar(64),
55 VALUE_ varchar(300),
56 REV_ integer,
57 primary key (NAME_)
58 ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE utf8_bin
59 In this way, you can first execute the table creation statement into the database through code, as follows:
60 **/
61 if (inOraclePlsqlBlock) {
62 inOraclePlsqlBlock = false;
63 } else {
64 sqlStatement = this.addSqlStatementPiece(sqlStatement, line.substring(0, line.length() - 1));
65 }
66 /**
67 * 10. Pack the table creation statement string into a Statement object
68 */
69 Statement jdbcStatement = connection.createStatement();
70
71 try {
72 /**
73 * 11. Finally, execute the table creation statement into the database
74 */
75 log.info("SQL: {}", sqlStatement);
76 jdbcStatement.execute(sqlStatement);
77 jdbcStatement.close();
78 } catch (Exception var27) {
79 log.error("problem during schema {}, statement {}", new Object[]{operation, sqlStatement, var27});
80 } finally {
81 /**
82 * 12. At this step, it means that the execution of the previous SQL table creation statement has ended.
83 * If no errors occur, it has been proven that the first database table structure has been created.
84 * You can start splicing the next table creation statement,
85 */
86 sqlStatement = null;
87 }
88 }
89 }
90 }
91
92 if (exception != null) {
93 throw exception;
94 }
97 } catch (Exception var29) {
98 log.error("couldn't " + operation + " db schema: " + exceptionSqlStatement, var29);
99 }
100 }
The main function of this part of the code is to first read the data in the sql script in the form of a byte stream, convert it into a string, and replace the line breaks with the escape character "/n". Then convert the string into a character stream BufferedReader to read, divide the reading of each line according to the "/n" match, loop to splice each line of string read, when the loop reaches a certain line and encounters ";" , which means that it has been spliced into a complete create table statement, similar to this form——
1 create table ACT_PROCDEF_INFO ( 2 ID_ varchar(64) not null, 3 PROC_DEF_ID_ varchar(64) not null, 4 REV_ integer, 5 INFO_JSON_ID_ varchar(64), 6 primary key (ID_) 7 ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE utf8_bin;
At this time, you can first put the spliced create table string into the database through the jdbcStatement.execute(sqlStatement) statement. When the execution is successful, the ACT_PROCDEF_INFO table means that it has been successfully created, and then the next row is continued to be read in the form of a BufferedReader character stream to build the next database table structure.
The whole process is probably based on this logic. On this basis, you can design SQL statements for more complex table building structures. When the project starts, you can execute the corresponding SQL statements to build the table.
The demo code has been uploaded to git and can be downloaded and run directly: GitHub - z924931408/Springboot-AutoCreateMySqlTable: imitates the automatic table creation mechanism of the workflow engine Activity to realize Springboot automatically generates database and table demos at startup