Today everyone Amway import a excel file artifact, easyexcel, official address :( https://github.com/alibaba/easyexcel ).
In the official website of the document it has presented its performance.
From the above it can be seen easyexcel performance tests on some analytic Processed weaker than poiuserModel mode. The main reason is the use of internal reflection I do field mapping model, the middle and I added a cache, but feel this gap may be acceptable. But on the memory consumption difference is more obvious, easyexcel increase again later in the file, memory consumption hardly increased. But poi userModel not the same, simply it will explode. Think of a parsing excel 200M, while there are an estimated 20 people and then hung up on a machine.
How to use it
1, rely on the introduction of maven
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>easyexcel</artifactId>
<version>2.0.5</version>
</dependency>Since the change jar made some poi package on package, and therefore need to comment poi rely referenced project, otherwise there will be a version conflict.
2, create entity receives excel data
@Data
public class Person {
@ExcelProperty(value = "姓名",index = 1)
private String name;
@ExcelProperty("性别")
private String sex;
@ExcelProperty("年龄")
private int age;
}@ExcelProperty this annotation is used to specify which columns corresponding to the attribute data excel file. There are two properties, a value is, the other is the index (starting from 0), it is not advisable at the same time with the index and name, or an object only index, or only an object name to match.
3, increasing the listener person
@Slf4j
public class PersonListener extends AnalysisEventListener<Person> {
/**
* 每隔5条存储数据库，实际使用中可以3000条，然后清理list ，方便内存回收
*/
private static final int BATCH_COUNT = 5;
List<Person> list = new ArrayList();
@Override
public void invoke(Person person, AnalysisContext analysisContext) {
log.info("解析到一条数据:{}",person);
if (list.size() >= BATCH_COUNT) {
saveData();
list.clear();
}
}
@Override
public void doAfterAllAnalysed(AnalysisContext analysisContext) {
saveData();
log.info("所有数据解析完成！");
}
@Override
public void onException(Exception exception, AnalysisContext context) {
log.error("解析失败，但是继续解析下一行", exception);
}
/**
* 加上存储数据库
*/
private void saveData(){
log.info("{}条数据，开始存储数据库！", list.size());
log.info("存储数据库成功！");
}
}4, file upload method
@RestController
@Slf4j
public class PersonController {
@PostMapping("importFile")
public String readPerson(MultipartFile file){
PersonListener personListener = new PersonListener();
try {
// headRowNumber(2) 这里可以设置1，因为头就是一行。如果多行头，可以设置其他值。不传入也可以，他没有指定头，也就是默认1行
EasyExcel.read(file.getInputStream(), Person.class,personListener).sheet().headRowNumber(1).doRead();
}catch (IOException ioe){
log.info("读取excel异常={}",ioe);
}
return "";
}
}So that the code base is complete. Next we look at how
Use spring framework
We know that in the spring when the file storage may need to call service, service call dao storage. How do we call it in the service of this PersonListener.
First, the injection service in PersonController in.
@Autowired
private PersonService personService;Then, the increase in PersonListener a constructor parameter.
private PersonService personService;
public PersonListener(PersonService personService){
this.personService = personService;
}Finally PersonController in new PersonListener when the service will come to pass.
@Autowired
private PersonService personService;
@PostMapping("importFile")
public String readPerson(MultipartFile file){
PersonListener personListener = new PersonListener(personService);
try {
// headRowNumber(2) 这里可以设置1，因为头就是一行。如果多行头，可以设置其他值。不传入也可以，他没有指定头，也就是默认1行
EasyExcel.read(file.getInputStream(), Person.class,personListener).sheet().headRowNumber(2).doRead();
}catch (IOException ioe){
log.info("读取excel异常={}",ioe);
}
return "";
}Next we look, there is a demand like this, we need to import statistics about the number of successes and a few failures and the number of write or else fail to excel in a return to the page, how to achieve?
We adjust PersonListener
@Slf4j
public class PersonListener extends AnalysisEventListener<Person> {
private int successCount = 0; // 成功量
private int exceptionCount = 0; // 异常量
private List<Person> exceptionList = new ArrayList<>(); // 异常数据
/**
* 每隔5条存储数据库，实际使用中可以3000条，然后清理list ，方便内存回收
*/
private static final int BATCH_COUNT = 5;
List<Person> list = new ArrayList();
private PersonService personService;
public PersonListener(PersonService personService){
this.personService = personService;
}
@Override
public void invoke(Person person, AnalysisContext analysisContext) {
log.info("解析到一条数据:{}",person);
successCount++;
list.add(person);
if (list.size() >= BATCH_COUNT) {
saveData();
list.clear();
}
}
@Override
public void doAfterAllAnalysed(AnalysisContext analysisContext) {
saveData();
log.info("所有数据解析完成！");
}
@Override
public void onException(Exception exception, AnalysisContext context) {
log.error("解析失败，但是继续解析下一行", exception);
Person person = (Person)context.readRowHolder().getCurrentRowAnalysisResult();
exceptionList.add(person);
exceptionCount++;
}
/**
* 加上存储数据库
*/
private void saveData(){
log.info("{}条数据，开始存储数据库！", list.size());
log.info("存储数据库成功！");
}
/**
* 插入结果返回
* @return
*/
public Map<String,Object> getData(){
Map<String,Object> map = new HashMap<>();
map.put("success",successCount);
map.put("exception",exceptionCount);
return map;
}
/**
* 失败数据返回
* @return
*/
public List<Person> getExceptionList(){
return exceptionList;
}
}Abnormality data written to the file in the PersonController.
@RestController
@Slf4j
public class PersonController {
@Autowired
private PersonService personService;
@PostMapping("importFile")
public String readPerson(MultipartFile file){
PersonListener personListener = new PersonListener(personService);
try {
// headRowNumber(1) 这里可以设置1，因为头就是一行。如果多行头，可以设置其他值。不传入也可以，他没有指定头，也就是默认1行
EasyExcel.read(file.getInputStream(), Person.class,personListener).sheet().headRowNumber(1).doRead();
}catch (IOException ioe){
log.info("读取excel异常={}",ioe);
}
Map<String, Object> data = personListener.getData();
String exception = data.get("exception") + "";
int exceptionCount = Integer.parseInt(exception);
if(exceptionCount>0){
String fileName = System.currentTimeMillis() + ".xlsx";
// 这里 需要指定写用哪个class去读，然后写到第一个sheet，名字为模板 然后文件流会自动关闭
EasyExcel.write("E://upload/file/"+fileName, Person.class).sheet("模板").doWrite(personListener.getExceptionList());
data.put("fileName",fileName);
}
return data.toString();
}
}We write a test page
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>文件上传</title>
</head>
<body>
<form action="/importFile" method="post" enctype="multipart/form-data">
<input type="file" name="file"/>
<button type="submit">提交</button>
</form>
</body>
</html>Here is the test log files
2019-10-20 11:21:59.809 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 解析到一条数据:Person(name=执偕, sex=男, age=18)
2019-10-20 11:21:59.811 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 解析到一条数据:Person(name=执偕2, sex=男, age=19)
2019-10-20 11:21:59.811 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 解析到一条数据:Person(name=执偕3, sex=男, age=20)
2019-10-20 11:21:59.811 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 解析到一条数据:Person(name=执偕4, sex=男, age=21)
2019-10-20 11:21:59.812 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 解析到一条数据:Person(name=执偕5, sex=男, age=22)
2019-10-20 11:21:59.812 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 5条数据，开始存储数据库！
2019-10-20 11:21:59.812 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 存储数据库成功！
2019-10-20 11:21:59.812 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 解析到一条数据:Person(name=执偕6, sex=男, age=23)
2019-10-20 11:21:59.812 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 解析到一条数据:Person(name=执偕7, sex=男, age=24)
2019-10-20 11:21:59.813 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 解析到一条数据:Person(name=执偕8, sex=男, age=25)
2019-10-20 11:21:59.813 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 解析到一条数据:Person(name=执偕9, sex=男, age=26)
2019-10-20 11:21:59.813 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 4条数据，开始存储数据库！
2019-10-20 11:21:59.813 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 存储数据库成功！
2019-10-20 11:21:59.813 INFO 17748 --- [nio-8080-exec-1] c.zhixie.easyexcel.demo.PersonListener : 所有数据解析完成！Today on the introduction to this, the text of the code I have uploaded to the cloud code,
address: https://gitee.com/javaXiaoCaiJi/zhixie-code-example/tree/master/easyexcel
If the article helpful, please remember to focus on points like yo ~
Welcome to my public concern number <Love it>, pushing daily technical articles for them to learn.