Article directory
1. Data preparation
In database interaction, batch operations are definitely the most efficient in terms of functionality. Compared with doing things individually in a loop, it will greatly improve interface performance and efficiency.
This article introduces how to perform batch operations on data based on Mybatis + Postgres database.
Let's take a book as an example, which contains three attributes: title, author, and price, and prepare data for this.
1.1 Create table SQL
CREATE TABLE book (
id bigserial,
name varchar(20),
author varchar(20),
price int4,
CONSTRAINT book_pk PRIMARY KEY (id)
);
1.2 Java objects
@Data
public class Book {
/** 主键 */
@TableId(type = IdType.AUTO)
private Long id;
/** 书名 */
private String name;
/** 作者 */
private String author;
/** 单价（分） */
private Integer price;
}
2. Batch operations
2.1 Batch insert
2.1.1 Mapper interface
/**
* 批量插入
*
* @param books 对象集合
* @return 成功插入条数
*/
Integer batchInsert(List<Book> books);
2.1.2 Mapper.xml
<!-- 批量插入数据 -->
<insert id="batchInsert">
INSERT INTO book (name, author, price) VALUES
<foreach collection="list" item="item" separator=",">
(#{item.name}, #{item.author}, #{item.price})
</foreach>
</insert>
2.1.3 Data simulation
Book book1 = new Book();
book1.setName("三国演义");
book1.setAuthor("罗贯中");
book1.setPrice(1980);
Book book2 = new Book();
book2.setName("红楼梦");
book2.setAuthor("曹雪芹");
book2.setPrice(2180);
List<Book> bookList = new ArrayList<>();
bookList.add(book1);
bookList.add(book2);
bookMapper.batchInsert(bookList);
2.1.4 Result display
2.2 Batch update
2.2.1 Mapper interface
/**
* 批量修改（根据id进行修改）
*
* @param books 修改对象集合
* @return 成功修改条数
*/
Integer batchUpdateById(List<Book> books);
2.2.2 Mapper.xml
<!-- 根据ID批量修改对象 -->
<update id="batchUpdateById">
UPDATE book b SET name = a.name, author = a.author, price = a.price FROM (SELECT
UNNEST(ARRAY<foreach collection="list" open="[" close="]" item="item" separator=","> #{item.id} </foreach>) AS id,
UNNEST(ARRAY<foreach collection="list" open="[" close="]" item="item" separator=","> #{item.name} </foreach>) AS name,
UNNEST(ARRAY<foreach collection="list" open="[" close="]" item="item" separator=","> #{item.author} </foreach>) AS author,
UNNEST(ARRAY<foreach collection="list" open="[" close="]" item="item" separator=","> #{item.price} </foreach>) AS price
) AS a WHERE b.id = a.id
</update>
2.2.3 Data simulation
Book book1 = new Book();
book1.setId(1L);
book1.setName("西游记");
book1.setAuthor("吴承恩");
book1.setPrice(1780);
Book book2 = new Book();
book2.setId(2L);
book2.setName("水浒传");
book2.setAuthor("施耐庵");
book2.setPrice(1680);
List<Book> bookList = new ArrayList<>();
bookList.add(book1);
bookList.add(book2);
bookMapper.batchUpdateById(bookList);
2.2.4 Result display
2.3 Add batches (ignore conflicts)
2.3.0 Causes of conflict
The so-called conflict means that when adding new entities in batches, the newly added entities may overlap with the unique index of the database, resulting in batch insertion errors (as shown below).
Cause: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint “unique_name_index”
Generally speaking, if data conflicts, it must either be ignored or data overwritten and updated. Here, we first demonstrate how to ignore conflicts.
2.3.1 Create a unique index
First, if there is a conflict, a unique index is required. Assume that our book title is unique. As shown in the SQL below, create a unique index for the name field in the database (the index name is unique_name_index).
CREATE UNIQUE INDEX unique_name_index ON book (name);
2.3.2 Mapper interface
/**
* 批量新增（忽略冲突），即插入的数据如果和唯一性索引冲突，则不进行处理
*
* @param books 对象集合
* @return 成功插入条数
*/
Integer batchInsertIgnoreConflict(List<Book> books);
2.3.3 Mapper.xml
As you can see, the SQL writing here only has one more ON CONFLICT (name) DO NOTHING at the end than the SQL for batch insertion.
<!-- 批量新增，并忽略冲突（存在和唯一性索引重复的，则不再进行插入） -->
<insert id="batchInsertIgnoreConflict">
INSERT INTO book (name, author, price) VALUES
<foreach collection="list" item="item" separator=",">
(#{item.name}, #{item.author}, #{item.price})
</foreach>
ON CONFLICT (name) DO NOTHING
</insert>
2.3.4 Data simulation
Before simulating the data, we can first see that there are only four classics and four books in the database. Now we have added two new books. One is Romance of the Three Kingdoms that already exists in the database, and the other is Strange Stories from a Chinese Studio that does not exist in the data. .
// 这个对象数据库中已经存在（插入时存在冲突，但会被忽略）
Book book1 = new Book();
book1.setName("三国演义");
book1.setAuthor("罗贯中");
book1.setPrice(1888);
// 这个对象数据库中不存在（成功插入到数据库）
Book book2 = new Book();
book2.setName("聊斋志异");
book2.setAuthor("蒲松龄");
book2.setPrice(2180);
List<Book> bookList = new ArrayList<>();
bookList.add(book1);
bookList.add(book2);
bookMapper.batchInsertIgnoreConflict(bookList);
2.3.5 Result display
2.4 Add batches (update if conflict occurs)
To put it simply, it means adding or updating in batches, that is, adding non-conflicting data and updating conflicting data.
First, you can look at 2.3.1 and 2.3.2 to understand the cause of the conflict and create a unique index that causes the conflict.
2.4.1 Mapper
/**
* 批量新增或更新（和唯一性索引冲突的数据进行字段更新）
*
* @param books 对象集合
* @return 插入&更新的成功条数
*/
Integer batchInsertOrUpdate(List<Book> books);
2.4.2 Mapper.xml
Compared with the SQL that ignores new conflicts in batches, the last DO NOTHING is replaced by DO UPATE , and the excluded object behind it refers to the conflict object.
<!-- 批量新增或更新（和唯一性索引冲突的数据进行字段更新） -->
<insert id="batchInsertOrUpdate">
INSERT INTO book (name, author, price) VALUES
<foreach collection="list" item="item" separator=",">
(#{item.name}, #{item.author}, #{item.price})
</foreach>
ON CONFLICT (name) DO UPDATE SET author = excluded.author, price = excluded.price
</insert>
2.4.3 Data simulation
Now there is the following data in our data. Now we are going to add a book Historical Records (no conflict data) and a Romance of the Three Kingdoms (conflict data, we changed the book price to 1888 this time).
// 这个对象数据库中已经存在（插入时存在冲突，但数据会被更新）
Book book1 = new Book();
book1.setName("三国演义");
book1.setAuthor("罗贯中");
book1.setPrice(1688);
// 这个对象数据库中不存在（成功插入到数据库）
Book book2 = new Book();
book2.setName("史记");
book2.setAuthor("司马迁");
book2.setPrice(1980);
List<Book> bookList = new ArrayList<>();
bookList.add(book1);
bookList.add(book2);
2.4.4 Result display
