Split the List by the specified number, and mybatis saves the data in batches

 

Requirement: read the data in excel, and then store it in the database, because the amount of data in excel is relatively large

1-2w, it will be very slow to store to the database at one time, consider splitting the List into multiple Lists of 1000 size

Now there is a List with 110 pieces of data, and I want to split it into 3 small Lists evenly

 

 
int sum = list.size(); total number of data
  System.out.println( "[File data volume]=" + sum);
        
 int pagesize = 1000;
 int page = (int)Math.ceil((double)sum/pagesize);

Each List has 1000 items, similar to paging, first calculate the total number of copies (pages)
Then call the method of splitting the List

 

public static <T> List<List<T>> averageAssign(List<T> source,int n){  
        List <List<T>> result= new ArrayList<List<T>> ();  
         int remaider=source.size()%n;   // (calculate the remainder first)   
        int number=source.size()/n;   // then the quotient   
        int offset=0; // offset   
        for ( int i=0;i<n;i++ ){  
            List<T> value=null;  
            if(remaider>0){  
                value=source.subList(i*number+offset, (i+1)*number+offset+1);  
                remaider - ;  
                offset++;  
            }else{  
                value=source.subList(i*number+offset, (i+1)*number+offset);  
            }  
            result.add(value);  
        }  
        return result;  
    }

 

Test, List five pieces of data, split into 3 Lists is 2 - 2 -1

 

 List<Integer> integers=new ArrayList<>();  
        integers.add(1);  
        integers.add(2);  
        integers.add(3);  
        integers.add(4);  
        integers.add(5);  
 List<List<Integer>> lists=averageAssign(integers, 3);  

结果
[[1, 2], [3, 4], [5]]

 

2. Mybatis saves data in batches

int batchInsert(List<Sku> list);

    <insert id="batchInsert"  parameterType="java.util.List">  
        insert into sku (  
        SKU_NO,  
        SOLD_TO,  
        HUB_ID,  
        SOLD_ALLOCATION,
        HUB_ALLOCATION  
        )  
        values  
        <foreach collection="list" item="sku" index="index" separator="," >  
          (  
            # {sku.skuNo, jdbcType = VARCHAR},  
            # {sku.soldTo, jdbcType = VARCHAR},  
            # {sku.hubId, jdbcType = VARCHAR},
            # {sku.soldAllocation, jdbcType = VARCHAR},  
            # {sku.hubAllocation, jdbcType = VARCHAR}
         )  
        </foreach>  
    </insert>