Build Spring Batch by yourself

the goal:

Build a batch job: import data from CVS, transform it, and then transfer it to the specified location. In this example, the log is printed.

Source download address

original

The business data is placed in the CVS file (sample-data.csv):

Jill,Doe
Joe,Doe
Justin,Doe
Jane,Doe
John,Doe

The SQL script creates a table to store data (schema-all.sql):

DROP TABLE people IF EXISTS;

CREATE TABLE people  (
    person_id BIGINT IDENTITY NOT NULL PRIMARY KEY,
    first_name VARCHAR(20),
    last_name VARCHAR(20)
);

Spring Batch and HyperSQL database dependencies (pom.xml):

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.2.2.RELEASE</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.example</groupId>
	<artifactId>batch-processing</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>batch-processing</name>
	<description>Demo project for Spring Boot</description>

	<properties>
		<java.version>1.8</java.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-batch</artifactId>
		</dependency>

		<dependency>
			<groupId>org.hsqldb</groupId>
			<artifactId>hsqldb</artifactId>
			<scope>runtime</scope>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
			<exclusions>
				<exclusion>
					<groupId>org.junit.vintage</groupId>
					<artifactId>junit-vintage-engine</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-test</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>

</project>

选择Gradle(build.gradle):

plugins {
    
    
	id 'org.springframework.boot' version '2.2.2.RELEASE'
	id 'io.spring.dependency-management' version '1.0.8.RELEASE'
	id 'java'
}

group = 'com.example'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '1.8'

repositories {
    
    
	mavenCentral()
}

dependencies {
    
    
	implementation 'org.springframework.boot:spring-boot-starter-batch'
	runtimeOnly 'org.hsqldb:hsqldb'
	testImplementation('org.springframework.boot:spring-boot-starter-test') {
    
    
		exclude group: 'org.junit.vintage', module: 'junit-vintage-engine'
	}
	testImplementation 'org.springframework.batch:spring-batch-test'
}

test {
    
    
	useJUnitPlatform()
}

Create a business class:

Use the first name and last name to instantiate the class (Person.java) through the constructor or by setting properties:

package com.example.batchprocessing;

public class Person {
    
    

  private String lastName;
  private String firstName;

  public Person() {
    
    
  }

  public Person(String firstName, String lastName) {
    
    
    this.firstName = firstName;
    this.lastName = lastName;
  }

  public void setFirstName(String firstName) {
    
    
    this.firstName = firstName;
  }

  public String getFirstName() {
    
    
    return firstName;
  }

  public String getLastName() {
    
    
    return lastName;
  }

  public void setLastName(String lastName) {
    
    
    this.lastName = lastName;
  }

  @Override
  public String toString() {
    
    
    return "firstName: " + firstName + ", lastName: " + lastName;
  }

}

Create an intermediate processor:

Converter to convert the name to uppercase (PersonItemProcessor.java):

package com.example.batchprocessing;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import org.springframework.batch.item.ItemProcessor;

public class PersonItemProcessor implements ItemProcessor<Person, Person> {
    
    

  private static final Logger log = LoggerFactory.getLogger(PersonItemProcessor.class);

  @Override
  public Person process(final Person person) throws Exception {
    
    
    final String firstName = person.getFirstName().toUpperCase();
    final String lastName = person.getLastName().toUpperCase();

    final Person transformedPerson = new Person(firstName, lastName);

    log.info("Converting (" + person + ") into (" + transformedPerson + ")");

    return transformedPerson;
  }

}

Aggregate batch job

Reader, processor and writer (BatchConfiguration.java):

@Configuration
@EnableBatchProcessing
public class BatchConfiguration {
    
    

	@Autowired
	public JobBuilderFactory jobBuilderFactory;

	@Autowired
	public StepBuilderFactory stepBuilderFactory;
	// end::setup[]

	// tag::readerwriterprocessor[]
	@Bean
	public FlatFileItemReader<Person> reader() {
    
    
		return new FlatFileItemReaderBuilder<Person>()
			.name("personItemReader")
			.resource(new ClassPathResource("sample-data.csv"))
			.delimited()
			.names(new String[]{
    
    "firstName", "lastName"})
			.fieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {
    
    {
    
    
				setTargetType(Person.class);
			}})
			.build();
	}

	@Bean
	public PersonItemProcessor processor() {
    
    
		return new PersonItemProcessor();
	}

	@Bean
	public JdbcBatchItemWriter<Person> writer(DataSource dataSource) {
    
    
		return new JdbcBatchItemWriterBuilder<Person>()
			.itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>())
			.sql("INSERT INTO people (first_name, last_name) VALUES (:firstName, :lastName)")
			.dataSource(dataSource)
			.build();
	}
	// end::readerwriterprocessor[]

	// tag::jobstep[]
	@Bean
	public Job importUserJob(JobCompletionNotificationListener listener, Step step1) {
    
    
		return jobBuilderFactory.get("importUserJob")
			.incrementer(new RunIdIncrementer())
			.listener(listener)
			.flow(step1)
			.end()
			.build();
	}

	@Bean
	public Step step1(JdbcBatchItemWriter<Person> writer) {
    
    
		return stepBuilderFactory.get("step1")
			.<Person, Person> chunk(10)
			.reader(reader())
			.processor(processor())
			.writer(writer)
			.build();
	}
	// end::jobstep[]
}

Detailed explanation about the above code:

Spring Batch provides many utility classes, which reduce the need to write custom code. You can focus on business logic.

The first part of the code defines the input, processor and output.

• reader() creates an ItemReader. It will look for a file named sample-data.csv and parse enough information for each order item to convert it to Person.
• processor() creates the instance of PersonItemProcessor you defined earlier to convert the data to uppercase.
• The writer (DataSource) creates an ItemWriter. This is for JDBC targets and automatically obtains a copy of the dataSource created by @EnableBatchProcessing. It contains the SQL statement required to insert a single Person driven by Java bean properties.
  
  The first method defines a job, and the second method defines a step. The job is constructed in steps, and each step can involve
  the prefix <Person,Person> of the reader, processor, and writer chunk(), indicating the type of input and output processed by each "chunk".
  How to get notified when the job is completed:

（JobCompletionNotificationListener.java):

package com.example.batchprocessing;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.BatchStatus;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.listener.JobExecutionListenerSupport;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Component;

@Component
public class JobCompletionNotificationListener extends JobExecutionListenerSupport {
    
    

  private static final Logger log = LoggerFactory.getLogger(JobCompletionNotificationListener.class);

  private final JdbcTemplate jdbcTemplate;

  @Autowired
  public JobCompletionNotificationListener(JdbcTemplate jdbcTemplate) {
    
    
    this.jdbcTemplate = jdbcTemplate;
  }

  @Override
  public void afterJob(JobExecution jobExecution) {
    
    
    if(jobExecution.getStatus() == BatchStatus.COMPLETED) {
    
    
      log.info("!!! JOB FINISHED! Time to verify the results");

      jdbcTemplate.query("SELECT first_name, last_name FROM people",
        (rs, row) -> new Person(
          rs.getString(1),
          rs.getString(2))
      ).forEach(person -> log.info("Found <" + person + "> in the database."));
    }
  }
}

(BatchProcessingApplication.java):

package com.example.batchprocessing;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class BatchProcessingApplication {
    
    

  public static void main(String[] args) throws Exception {
    
    
    System.exit(SpringApplication.exit(SpringApplication.run(BatchProcessingApplication.class, args)));
  }
}

@SpringBootApplication is a convenience annotation that adds all of the following:

• @Configuration: Mark the class as the source of the bean definition of the application context.
• @EnableAutoConfiguration: Tell Spring Boot to start adding beans based on the classpath settings, other beans and various property settings. For example, if spring-webmvc is on the classpath, this annotation marks the application as a web application and activates key behaviors, such as setting DispatcherServlet.
• @ComponentScan: Tell Spring to look for other components in the package, configure and serve com/example, and let it find the controller.

run

• Right-click on the two maven files to select:
  
• start up:
  
• In maven, type the jar package corresponding to the complete file: (The judgment method is that the jar is finally in the complete folder)
  -execute java -jar target / jar package name (java -jar target / batch-processing-0.0.1.jar)
  

result:

You can see the execution result printed through the log, and convert the name to all uppercase: