Spring Boot JPARepository performance on save()

teamerMan :

I have an issue where my spring boot application performance is very slow when inserting data.

I am extracting a large subset of data from one database and inserting the data into another database.

The following is my entity.

@Entity
@Table(name = "element")
public class VXMLElementHistorical {

@Id
@Column(name = "elementid")   
private long elementid;

@Column(name = "elementname")
private String elementname; 

Getter/Setter methods...

I have configured a JPA repository

public interface ElementRepository extends JpaRepository<Element, Long> {

}

and call the save() method with my object

@Transactional 
public void processData(List<sElement> hostElements) 
throws DataAccessException { 

List<Element> elements = new ArrayList<Element>();    

for (int i = 0; i < hostElements.size(); i++) {
        Element element = new Element();
        element.setElementid(hostElements.get(i).getElementid());
        element.setElementname(hostElements.get(i).getElementname());
        elements.add(element);
    }

   try{
   elementRepository.save(elements);{
   //catch etc...

}

What is happening is that for each item, it is taking between 6 and 12 seconds to perform an insert. I have turned on hibernate trace logging and statistics and what is happening when I call the save function is that hibernate performs two queries, a select and an insert. The select query is taking 99% of the overall time.

I have ran the select query direct on the database and the result returns in nanoseconds. Which leads me to believe it is not an indexing issue however I am no DBA.

I have created a load test in my dev environment, and with similar load sizes, the over all process time is no where near as long as in my prod environment.

Any suggestions?

M. Deinum :

Instead of creating a list of elements and saving those, save the individual elements. Every now an then do a flush and clear to prevent dirty checking to become a bottleneck.

@PersistenceContext
private EntityManager entityManager;

@Transactional 
public void processData(List<sElement> hostElements) 
throws DataAccessException {     

for (int i = 0; i < hostElements.size(); i++) {
        Element element = new Element();
        element.setElementid(hostElements.get(i).getElementid());
        element.setElementname(hostElements.get(i).getElementname());
        elementRepository.save(element)
        if ( (i % 50) == 0) {
            entityManager.flush();
            entityManager.clear();
        }
}
entityManager.flush(); // flush the last records.

You want to flush + clear every x elements (here it is 50 but you might want to find your own best number.

Now as you are using Spring Boot you also might want to add some additional properties. Like configuring the batch-size.

spring.jpa.properties.hibernate.jdbc.batch_size=50

This will, if your JDBC driver supports it, convert 50 single insert statements into 1 large batch insert. I.e. 50 inserts to 1 insert.

Spring Boot JPARepository performance on save()

Guess you like