Foreword
Before meeting a demand, export data as excel files. First thought is for export with poi.
Learn all know poi, poi there are two classes, HSSFWorkbook used to derive .lsx excel file, which is excel2003 previous version, XSSFWorkbook used to derive .xlsx excel file, which is more than excel2007 version. (Poi specific use would not elaborate, online many examples.)
When developed using hundreds of pieces of data, written test, no problem.
Later, will increase the amount of data to 10000 on the problem of time, memory overflow always occurs. Analysis of the reasons, poi when exporting excel file, 10,000 one-time data is written or objects XSSFWorkbook HSSFWorkbook of this class, and then output to the front end in the form of streams. In other words, from the beginning to write the first data memory is increasing, after all finish until all the data output to the front end, the memory will be released. The jvm memory is fixed, usually 512 bar. So memory overflow will occur.
solution
SXSSFWorkbook This class can solve this problem.
Look under the official description of the class
Streaming version of XSSFWorkbook implementing the "BigGridDemo" strategy. This allows to write very large files without running out of memory as only a configurable portion of the rows are kept in memory at any one time. You can provide a template workbook which is used as basis for the written data. See https://poi.apache.org/spreadsheet/how-to.html#sxssf for details. Please note that there are still things that still may consume a large amount of memory based on which features you are using, e.g. merged regions, comments, ... are still only stored in memory and thus may require a lot of memory if used extensively. SXSSFWorkbook defaults to using inline strings instead of a shared strings table. This is very efficient, since no document content needs to be kept in memory, but is also known to produce documents that are incompatible with some clients. With shared strings enabled all unique strings in the document has to be kept in memory. Depending on your document content this could use a lot more resources than with shared strings disabled. Carefully review your memory budget and compatibility needs before deciding whether to enable shared strings or not.
Probably means that these may not need to put all the data in memory, but only the data, some mergers and other forms of information or to comment in memory. As to why only part of the data can be stored in memory, did not specify how.
Look constructor
SXSSFWorkbook objects when generating excel, it will generate a side edge record row, row when a certain amount of time, it will be written before the row to a temporary path of the hard disk, and then memory row will be re-recorded, and so forth until all finished.
The number of control parameters rowAccessWindowSize is held in memory, the default is 100.
Parameters compressTmpFiles is to say whether a temporary file on your hard disk needs to be compressed
Parameters workbook is XSSFWorkbook object, you can not fill
Use Demo
SXSSFWorkbook wb = new SXSSFWorkbook(100); // 在内存当中保持 100 行 , 超过的数据放到硬盘中
Sheet sh = wb.createSheet();
for (int rownum = 0; rownum < 10000; rownum++) {
Row row = sh.createRow(rownum);
for (int cellnum = 0; cellnum < 10; cellnum++) {
Cell cell = row.createCell(cellnum);
String address = new CellReference(cell).formatAsString();
cell.setCellValue(address);
}
}
FileOutputStream out = new FileOutputStream("C:/Users/infodba/Desktop/sxssf.xlsx");
wb.write(out);
out.close();
// dispose of temporary files backing this workbook on disk
wb.dispose();
System.out.println("");We can see, the constructor of many SXSSFWorkbook, choose one of the Simply fill rowAccessWindowSize.
Other consistent usage and XSSFWorkbook.
The last execution of a dispose method is to delete temporary files on your hard disk.