[EasyExcel import and export (test with millions of data) paste and use]

Import process: User uploads files --> Backend gets file stream --> Listener reads files --> Batch inserts into database
Export process: User clicks button --> Calls backend interface --> Paging query needs to be exported Data-->Export to excel file

Because it is a test project, in order to make it more convenient and clear, all operations are placed in the controller. When using it formally, be careful to put it in the interface implementation layer impl.

If you are using other frameworks, please ignore the "①②③④⑤ operations in the backend directory" and use your own framework's method instead.

1. Overall project directory

2. Database

1.Create table

CREATE TABLE test_excel (
  user_id varchar(255) NOT NULL COMMENT '表主键id',
  user_name varchar(255) NOT NULL COMMENT '用户姓名，不能为空',
  user_age varchar(255) DEFAULT NULL COMMENT '用户年龄，允许为空',
  user_cardid varchar(255) NOT NULL COMMENT '身份证号，不能为空，不允许重复',
  PRIMARY KEY (user_id)
);

3. Backend

1. pom.xml file

The main dependencies are:

<dependency>
     <groupId>com.alibaba</groupId>
     <artifactId>easyexcel</artifactId>
     <version>2.2.10</version>
</dependency>

All dependencies (select dependencies according to your own needs, no need to import them all):

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.4.3</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.bug</groupId>
    <artifactId>bug</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>my_springboot1</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>com.google.zxing</groupId>
            <artifactId>core</artifactId>
            <version>3.3.0</version>
        </dependency>
        <dependency>
            <groupId>com.google.zxing</groupId>
            <artifactId>javase</artifactId>
            <version>3.3.3</version>
        </dependency>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>RELEASE</version>
            <scope>compile</scope>
        </dependency>
        <!-- ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ itextpdf依赖包 ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ -->
        <dependency>
            <groupId>com.itextpdf</groupId>
            <artifactId>itextpdf</artifactId>
            <version>5.5.6</version>
        </dependency>
        <dependency>
            <groupId>com.itextpdf</groupId>
            <artifactId>itext-asian</artifactId>
            <version>5.2.0</version>
        </dependency>
        <!-- ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ itextpdf依赖包 ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ -->
        <!-- ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ 用于测试依赖包 ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13</version>
        </dependency>
        <!-- ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ 用于测试依赖包 ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ -->
        <!-- ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ slf4j+logback包 ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.30</version>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.2.3</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
        </dependency>
        <!-- ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ slf4j+logback包 ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ -->
        <!-- ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ json包 ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.76</version>
        </dependency>
        <!-- ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ json包 ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ -->
        <!-- ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ 过滤器 ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ -->
        <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.4</version>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.4</version>
        </dependency>
        <!-- ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ 过滤器 ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ -->
        <!-- ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ swagger ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ -->
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger2</artifactId>
            <version>2.9.2</version>
        </dependency>
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger-ui</artifactId>
            <version>2.9.2</version>
        </dependency>
        <!-- ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ swagger ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ -->
        <!-- ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ 连接数据库-MySQL-mybatis-plus ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
            <version>2.5.3</version>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.38</version>
        </dependency>
        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-boot-starter</artifactId>
            <version>3.3.2</version>
        </dependency>
        <!-- ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ 连接数据库-MySQL-mybatis-plus  ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ -->
        <!-- ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ Easy Excel ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>easyexcel</artifactId>
            <version>2.2.10</version>
        </dependency>
        <!-- ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ Easy Excel ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ -->
        <!-- ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ lombok ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.12</version>
            <scope>provided</scope>
        </dependency>
        <!-- ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ lombok ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑ -->

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <version>2.4.3</version>
            </plugin>
        </plugins>

        <resources>
            <resource>
                <directory>src/main/java</directory>
                <excludes>
                    <exclude>**/*.java</exclude>
                </excludes>
            </resource>
            <resource>
                <directory>src/main/resources</directory>
                <includes>
                    <include>**/*.*</include>
                </includes>
            </resource>
        </resources>

    </build>

</project>

2. yml file (import and export shared code)

spring:
  devtools:
    restart:
      enabled: true  #设置开启热部署
      additional-paths: src/main/java #重启目录
      exclude: WEB-INF/**
  freemarker:
    cache: false    #页面不加载缓存，修改即时生效
  servlet:
    multipart: #这里必须要加这配置，不然一百万条数据文件太大会拒绝导入
      max-request-size: 100MB
      max-file-size: 100MB

  #数据库的配置，这里有个关键点，因为用的是mybatis-plus的saveBatch()方法批量插入数据库的，
  #所以这里必须加rewriteBatchedStatements=true，不然插入会非常的慢，
  #使用其它插入方式的可以忽略此配置
  datasource:
    driver-class-name: com.mysql.jdbc.Driver
    url: jdbc:mysql://ip地址:端口号/数据库名称?rewriteBatchedStatements=true
    username: 用户名
    password: 密码

3. Shared code files (if you use other frameworks, you can ignore the ①②③④⑤ operations )

Start coding

① Entity object TestExcel (common code for import and export)

package com.bug.entity;

import com.alibaba.excel.annotation.ExcelProperty;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;

@Data
@TableName("test_excel")
public class TestExcel {

  /**
   * 可以通过index和name去匹配excel里的列
   * 注意name,index值不要重复
   */
  @TableId
  private String userId;

  @ExcelProperty(value = "姓名",index = 0)
  private String userName;

  @ExcelProperty(value = "年龄",index = 1)
  private String userAge;

  @ExcelProperty(value = "身份证号",index = 2)
  private String userCardid;


}

② mapper layer (import and export shared code)

package com.bug.mapper.excel;

import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.bug.entity.TestExcel;
import org.apache.ibatis.annotations.Mapper;

@Mapper
public interface TestExcelMapper extends BaseMapper<TestExcel> {
}

③ Service layer (import and export shared code)

package com.bug.service.excel;

import com.baomidou.mybatisplus.extension.service.IService;
import com.bug.entity.TestExcel;

public interface TestExcelService extends IService<TestExcel> {

}

④ impl implementation layer (import and export common code)

package com.bug.service.excel.impl;

import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import com.bug.entity.TestExcel;
import com.bug.mapper.excel.TestExcelMapper;
import com.bug.service.excel.TestExcelService;
import org.springframework.stereotype.Service;

@Service
public class TestExcelServiceImpl extends ServiceImpl<TestExcelMapper, TestExcel> implements TestExcelService {

}

⑤ Paging tool (import and export shared code)

package com.bug.config;

import com.baomidou.mybatisplus.extension.plugins.PaginationInterceptor;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class PageConfig {

    /**
     * 分页工具
     * @return PaginationInterceptor
     */
    @Bean
    public PaginationInterceptor paginationInterceptor(){
        return new PaginationInterceptor();
    }

}

⑥ EasyExcel tool class (the most important file, if you use other frameworks, you can ignore the above code, and the comments will be as detailed as possible)

package com.bug.util.excel;

import com.alibaba.excel.context.AnalysisContext;
import com.alibaba.excel.metadata.CellData;
import com.alibaba.excel.metadata.CellExtra;
import com.alibaba.excel.read.listener.ReadListener;
import com.alibaba.fastjson.JSON;
import com.bug.entity.TestExcel;
import com.bug.service.excel.TestExcelService;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;

/**
 * 小bug
 * excel导入的工具类
 * 是不是很奇怪这里是实现的ReadListener接口而不是继承的AnalysisEventListener类
 * 因为源码里AnalysisEventListener类也是继承的ReadListener接口，何必多此一举呢
 */
public class ExcelUpload implements ReadListener<TestExcel> {

    private TestExcelService testExcelService;
    /**
     * 注意这里是不能交给spring管理的，就是不能使用@Resource,或者@Autowired注入
     * 可以使用构造方法的方式获取你想要的对象
     * @param testExcelService testExcelService
     */
    public ExcelUpload(TestExcelService testExcelService){
        this.testExcelService = testExcelService;
    }

    /**
     * 无参构造方法，不能省略
     */
    public ExcelUpload(){}

    public static final Logger log = LoggerFactory.getLogger(ExcelUpload.class);

    private static final int count = 10000;//设置读取的条数，每次达到指定条数时就保存入数据库
    private List<TestExcel> testExcelListTure = new ArrayList<>();//校验正确的数据集合，数量达到设定值后插入数据库，然后再清空一直循环
    private List<TestExcel> testExcelListFalse = new ArrayList<>();//校验失败、保存数据库失败的数据集合，可以插入到一个失败数据表，或者显示在前端提醒用户哪些数据导入失败
    /**
     * 很明显，onException这个就是用来处理异常的，当其它侦听器出现异常时会触发此方法
     * @param e Exception
     * @param analysisContext analysisContext
     * Exception 默认是直接抛出异常的，要注意处理异常
     */
    @Override
    public void onException(Exception e, AnalysisContext analysisContext) {
        log.info("兄嘚，你的代码出现异常了！");
        e.printStackTrace();
    }

    /**
     * 获取excel的第一行head数据
     * @param map 数据map
     * @param analysisContext analysisContext
     */
    @Override
    public void invokeHead(Map<Integer, CellData> map, AnalysisContext analysisContext) {
        log.info("第一列:{} 第二列:{} 第三列:{}",map.get(0).getStringValue(),map.get(1).getStringValue(),map.get(2).getStringValue());

    }

    /**
     * 读取正文数据，一次只读取一行
     * @param testExcel 实体类对象
     * @param analysisContext analysisContext
     */
    @Override
    public void invoke(TestExcel testExcel, AnalysisContext analysisContext) {
        log.info("读取到一条数据:{}", JSON.toJSONString(testExcel));
        //因为是测试，这里只做一些简单的为空判断，正式的可以根据业务需求自己写校验条件
        if(testExcel == null){//对象为空直接跳出
            return;
        }else if(StringUtils.isBlank(testExcel.getUserName())){//判断名字是否为空
            testExcelListFalse.add(testExcel);//放入错误集合列表
            return;
        }else if(StringUtils.isBlank(testExcel.getUserCardid())){//判断身份证是否为空
            /**
             * 身份证号的判断，以前碰到过一个需求，就是不能和数据库已有的数据重复，数据库存在这个身份证号则表示数据已导入/不能再次导入
             * 我使用的方式是，先把那个表的”身份证号“字段全部查询出来加载到内存里，然后这里直接和查询出来的身份证号进行对比，存在的就不
             * 导入，并记录到错误集合列表标注为”重复导入“，不存在的才存入正确的集合列表，并把这个身份证号也存入内存，给后面的数据校验
             * 是否有重复的数据，这样所有的校验都在内存里进行，优点是：速度会很快、数据库压力也会很小，但是缺点也很明显：很占内存。
             * 不过通过测试：数据在百万级的，只查询身份证号的话，内存的占用是很少的，即使是微型服务器也能满足需求，而如果是千万级数据，相信
             * 能有这个数据量的，服务器也差不了，上亿数据量的还没处理过，以后有机会碰到了再进行测试吧，这里不进行身份证号相同的校验
             */
            testExcelListFalse.add(testExcel);//放入错误集合列表
            return;
        }
        testExcelListTure.add(testExcel);//校验通过的方法正确集合列表
        if(count <= testExcelListTure.size()){//集合数据大于设定的数量时，提交数据库保存
            testExcelService.saveBatch(testExcelListTure);//批量存入数据库
            testExcelListTure = new ArrayList<>();//清空正确列表数据，再次循环
        }

    }

    /**
     * 额外单元格返回的数据，这个方法还没详细了解过，一直没用到过
     * @param cellExtra cellExtra对象
     * @param analysisContext analysisContext
     */
    @Override
    public void extra(CellExtra cellExtra, AnalysisContext analysisContext) {
        log.info("extra:{}",JSON.toJSONString(cellExtra));
    }

    /**
     * 当读取完所有数据后就会执行次方法
     * @param analysisContext analysisContext
     */
    @Override
    public void doAfterAllAnalysed(AnalysisContext analysisContext) {
        log.info("兄嘚，所有数据读取完了哦！");
        //读取完excel后再次判断是否还要未存入数据库的数据
        if(testExcelListTure.size() > 0){
            testExcelService.saveBatch(testExcelListTure);//不为空，则存入数据库
        }
        //这里也可以处理错误列表，保存入错误列表数据库，或者显示到前端给用户查看
    }

    /**
     * 这个方法最坑，默认返回的是false，一定要记得改成true，为false时会只返回一条数据，
     * 意思是只会执行invokeHead方法
     * @param analysisContext analysisContext
     * @return 是否读取下一行
     */
    @Override
    public boolean hasNext(AnalysisContext analysisContext) {
        return true;
    }
}

4. Use a for loop to create data and export it to an excel file (all the dependencies of the controller are placed here, and will not be written again later)

Since 1 million pieces of data is too much, we first use code to generate 1 million pieces of data, and then test the import and export.

package com.bug.controller;

import com.alibaba.excel.EasyExcel;
import com.alibaba.excel.ExcelReader;
import com.alibaba.excel.ExcelWriter;
import com.alibaba.excel.read.metadata.ReadSheet;
import com.alibaba.excel.write.metadata.WriteSheet;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.bug.entity.TestExcel;
import com.bug.entity.WxConfig;
import com.bug.mapper.excel.TestExcelMapper;
import com.bug.service.excel.TestExcelService;
import com.bug.service.wxconfig.WxConfigService;
import com.bug.util.excel.ExcelUpload;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;

import javax.annotation.Resource;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.OutputStream;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

/**
 * EasyExcel 普通导入excel
 */
@RequestMapping("/public")
@RestController
public class publicController {

    public static final Logger log = LoggerFactory.getLogger(publicController.class);
    @Resource
    private WxConfigService wxConfigService;
    @Resource
    private TestExcelService testExcelService;
    @Resource
    private HttpServletRequest request;
    @Resource
    private HttpServletResponse response;
    @Resource
    private TestExcelMapper testExcelMapper;

    /**
     * 1、EasyExcel excel导出（for循环生成数据导出，百万级数据测试）
     * 相当于一次查询出一百万条数据，然后直接导出（此方式只适合少量数据，会很消耗内存）
     */
    @GetMapping("/exportForExcel")
    public void exportForExcel() {
        log.info("for循环导出excel。。。");
        //需要导出的数据集合
        List<TestExcel> testExcelList = new ArrayList<>();
        long number = 100000000000000000L;
        for(int i=1; i<=1000000; i++){
            TestExcel testExcel = new TestExcel();
            testExcel.setUserAge(String.valueOf(i));
            testExcel.setUserCardid(String.valueOf(number+i));
            testExcel.setUserName("张三"+i);
            testExcelList.add(testExcel);
        }

        try {
            //下方使用了用户自己选择文件保存路径的方式，所以需要配请求参数，如果使用固定路径可忽略此代码
            String filename = URLEncoder.encode(System.currentTimeMillis()+".xlsx","UTF-8");
            response.setCharacterEncoding("UTF-8");
            response.setHeader("Content-disposition", "attachment; filename=" + filename);//设定输出文件头
            response.setContentType("application/x-xls");// 定义输出类型

            //不需要导出的字段userId，如果没有不需要导出的字段，可以忽略这个方法
            //只需要导入的字段，用includeColumnFiledNames()方法，写法是一样的
            Set<String> excludeColumnFiledNames = new HashSet<String>();
            excludeColumnFiledNames.add("userId");
            /*系统指定文件的输出路径
                String fileName = "D:/test/"+System.currentTimeMillis()+".xlsx";
                指定路径使用写法EasyExcel.write(fileName,TestExcel.class)
                EasyExcel.write()有很多种方法，其它方法可以直接查看源码
            */
            //这里采用用户自己选择文件保存路径的方式
            OutputStream out = response.getOutputStream();
            //开始导出，
            EasyExcel.write(out,TestExcel.class).excludeColumnFiledNames(excludeColumnFiledNames)
                    .sheet("测试模板")
                    .doWrite(testExcelList);
        }catch (Exception e){
            log.info("兄嘚，你代码又出错啦");
            e.printStackTrace();
        }

    }


}

5. Excel import

/**
     * 2、EasyExcel excel导入(百万级数据测试)
     * @return
     */
    @PostMapping("/submitExcel")
    public void submitExcel(MultipartFile file){
        log.info("开始导入excel。。。");
        /**
         * EasyExcel.read有很多方法，也可以直接传入路径
         * String fileUrl = "D:/test/test.xlsx";
         */
        //创建reader
        ExcelReader excelReader = null;
        try {
            excelReader = EasyExcel.read(file.getInputStream(), TestExcel.class, new ExcelUpload(testExcelService)).build();
            // 构建sheet，可以指定是第几个sheet
            ReadSheet readSheet = EasyExcel.readSheet(0).build();
            // 读取sheet
            excelReader.read(readSheet);
        }catch (Exception e){
            e.printStackTrace();
        }finally {
            if (excelReader != null) {
                //这里不能省略
                excelReader.finish();
            }
        }
    }

6. excel export

    /**
     * 3、EasyExcel excel导出（分页查询数据导出，百万级数据测试）
     * 采用分页查询的方式导出excel，内存占用很少，数据量大的时候推荐使用此方式
     */
    @GetMapping("/exportSqlExcel")
    public void exportSqlExcel() {
        log.info("sql分页导出excel。。。");
        //分页查询，每次查询量，这个插件最大只允许一次查500或者直接查全部，要想查更多的就只能去改源码了
        int pageNum = 500;
        ExcelWriter excelWriter = null;
        try {
            //下方使用了用户自己选择文件保存路径的方式，所以需要配请求参数，如果使用固定路径可忽略此代码
            String filename = URLEncoder.encode(System.currentTimeMillis()+".xlsx","UTF-8");
            response.setCharacterEncoding("UTF-8");
            response.setHeader("Content-disposition", "attachment; filename=" + filename);//设定输出文件头
            response.setContentType("application/x-xls");// 定义输出类型

            //不需要导出的字段userId，如果没有不需要导出的字段，可以忽略这个方法
            //只需要导入的字段，用includeColumnFiledNames()方法，写法是一样的
            //Set<String> excludeColumnFiledNames = new HashSet<String>();
            //excludeColumnFiledNames.add("userId");
            //这里采用用户自己选择文件保存路径的方式
            OutputStream out = response.getOutputStream();
            //这里其实就是把上面的方法分开写，写入同一个sheet
            excelWriter = EasyExcel.write(out, TestExcel.class).build();
            WriteSheet writeSheet = EasyExcel.writerSheet("测试模板呀").build();
            //重点是这里的循环调用---分页查询，先查询出需要导出的总数
            long count = testExcelService.count();
            int num = (int)(count / pageNum) + 1;
            for (int i = 1; i < num; i++) {
                //分页查询
                Page<TestExcel> page = new Page<>(i,pageNum);
                IPage<TestExcel> testExcelIPage = testExcelService.page(page);
                List<TestExcel> testExcelList = testExcelIPage.getRecords();
                if(testExcelList.size() > 0){
                    //导出
                    excelWriter.write(testExcelList, writeSheet);
                }
            }
        }catch (Exception e){
            log.info("兄嘚，你代码又出错啦");
            e.printStackTrace();
        }finally {
            if (excelWriter != null) {
                excelWriter.finish();
            }
        }

    }

4. Summary

The reading and writing speed of EasyExcel itself is very fast. As shown above: importing 1 million pieces of data into 3 fields only takes 3-4 minutes to complete, and exporting 1 million pieces of data through a for loop only takes 2 minutes.

What really affects the speed is: your batch insertion method and the speed of your paging query. For example, export the top paging query and loop through 1 million paging queries.

Only checking 500 items at a time, the entire export takes about 15 minutes, and the query only takes about 12-13 minutes. Let’s take a look at the final effect!

Note: It is not easy to create the original text. If you forward it, please bring a link to the original text and indicate the source.