Effectively improve the efficiency of writing large amounts of data into excel

In the development process, there is often a need to export data to excel. When the amount of data is large, reaching the level of tens of thousands or even hundreds of thousands or millions, how to speed up the generation of excel?
First of all, popularize the knowledge background:
Excel2003 and below versions support a maximum of 65536 rows and 256 columns of data, so Excel2003 cannot be used to generate 100,000 or 100 million level data;
Excel2007 version supports a maximum of 1048576 rows and 16384 columns for a table , which can basically meet the data level below a million level.

Generally, the way to generate excel through poi is as follows: (Forgive me, I have not studied poi before, so I can only use this way, and many demos on the Internet are also in this way)

 1 public static void exportDataToExcel1(String[] header, List<String[]> datas, String path) {
 2         File file = new File(path);
 3         Workbook workbook = null;
 4         if (path.endsWith(EXCEL_2003)) {
 5             workbook = new HSSFWorkbook();
 6         }
 7         if (path.endsWith(EXCEL_2007)) {
 8             workbook = new XSSFWorkbook();
 9         }
10         Sheet sheet = workbook.createSheet();
11         Row firstRow = sheet.createRow(0); //第一行
12         for (int i = 0; i < header.length; i++) {
13             Cell cell = firstRow.createCell(i);
14             cell.setCellValue(header[i]);
15         }
16         if (datas != null && datas.size() > 0) {
17             for (int i = 0; i < datas.size(); i++) {
18                 Row row = sheet.createRow(i + 1);
19                 String[] d = datas.get(i);
20                 for (int j = 0; j < d.length; j++) {
21                     Cell cell = row.createCell(j);
22                     cell.setCellValue(d[j]);
23                 }
24             }
25         }
26         try {
27             OutputStream outputStream = new FileOutputStream(file);
28             workbook.write(outputStream);
29             outputStream.flush();
30             outputStream.close();
31         } catch (FileNotFoundException e) {
32              e.printStackTrace ();
33          } catch (IOException e)
 34              34.printStackTrace ();
35          }
 36      }

Using the above method to generate an excel sheet with 100,000 rows and 30 columns takes about 40 seconds on my computer

Data ready for 1731 ms
The export took 46795 ms

Check the poi official website http://poi.apache.org/spreadsheet/index.html and find that the SXSSF api has been added since version 3.8 beta3 to solve large data volume scenarios

This method adds a mechanism for automatically flushing data. You can set a data volume threshold. When the threshold is reached, the data will be flushed to the disk, which relieves the pressure during runtime.

The modified code is as follows:

public static void exportDataToExcel(String[] header, List<String[]> datas, String path) {
        File file = new File(path);
        SXSSFWorkbook sxssfWorkbook = new SXSSFWorkbook(100);
        Sheet sheet = sxssfWorkbook.createSheet();
        Row firstRow = sheet.createRow(0);         //第一行
        for (int i = 0; i < header.length; i++) {
            Cell cell = firstRow.createCell(i);
            cell.setCellValue(header[i]);
        }
        if (datas != null && datas.size() > 0) {
            for (int i = 0; i < datas.size(); i++) {
                Row row = sheet.createRow(i + 1);
                String[] d = datas.get(i);
                for (int j = 0; j < d.length; j++) {
                    Cell cell = row.createCell(j);
                    cell.setCellValue(d[j]);
                }
            }
        }
        try {
            OutputStream outputStream = new FileOutputStream(file);
            sxssfWorkbook.write(outputStream);
            outputStream.flush();
            outputStream.close();
        } catch (FileNotFoundException e) {
            e.printStackTrace ();
        } catch (IOException e) {
            e.printStackTrace ();
        }finally {
            sxssfWorkbook.dispose();
        }
    }

Using this method to test data of the same magnitude, exporting excel is shortened to 6 or 7 seconds. It can be seen that this improvement is still obvious.

Data ready for 1096 ms
The export took 6784 ms