SpringBoot batch framework
Spring Batch is a lightweight, comprehensive batch framework designed to enable the development of robust batch applications that are vital for the daily operations of enterprise systems.
A typical batch program generally:
Spring Batch automates this basic batch iteration, providing the capability to process similar transactions as a set, typically in an offline environment without any user interaction. Batch jobs are part of most IT projects, and Spring Batch is the only open source framework that provides a robust, enterprise-scale solution.
The diagram below is a simplified version of the batch reference architecture that has been proven through decades of implementations on many different platforms. It introduces the key concepts and terms relevant to batch processing, as used by Spring Batch.
As shown :
1- A batch process is typically encapsulated by a Job consisting of multiple Steps.
2- Each Step typically has a single ItemReader, ItemProcessor, and ItemWriter.
3- A Job is executed by a JobLauncher, and metadata about configured and executed jobs is stored in a JobRepository.
4- Each Job may be associated with multiple JobInstances, each of which is defined uniquely by its particular JobParameters that are used to start a batch job. Each run of a JobInstance is referred to as a JobExecution.
5- Each JobExecution typically tracks what happened during a run, such as current and exit statuses, start and end times, etc.
A Step is an independent, specific phase of a batch Job, such that every Job is composed of one or more Steps. Similar to a Job, a Step has an individual StepExecution that represents a single attempt to execute a Step.
StepExecution stores the information about current and exit statuses, start and end times, and so on, as well as references to its corresponding Step and JobExecution instances.
6- An ExecutionContext is a set of key-value pairs containing information that is scoped to either StepExecution or JobExecution. Spring Batch persists the ExecutionContext, which helps in cases where you want to restart a batch run (e.g., when a fatal error has occurred, etc.). All that is needed is to put any object to be shared between steps into the context and the framework will take care of the rest. After restart, the values from the prior ExecutionContext are restored from the database and applied.
7- JobRepository is the mechanism in Spring Batch that makes all this persistence possible. It provides CRUD operations for JobLauncher, Job, and Step instantiations. Once a Job is launched, a JobExecution is obtained from the repository and, during the course of execution, StepExecution and JobExecution instances are persisted to the repository.
pom.xml
领英推荐
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
Job configuration along with Custom reader , custom processor and custom writer :
@Configuration
public class BatchConfig {
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Bean
public Job processJob() {
return jobBuilderFactory
.get("processJob")
.incrementer(
new RunIdIncrementer())
.listener(listener())
.flow(orderStep1()).end().build();
}
@Bean
public Step orderStep1() {
return stepBuilderFactory
.get("orderStep1").<String, String> chunk(1)
.reader(new Reader()).processor(new Processor())
.writer(new Writer()).build();
}
@Bean
public JobExecutionListener listener() {
return new JobCompletionListener();
}
}
public class Reader implements ItemReader<String> {
private String[] messages = { "javainuse.com",
"Welcome to Spring Batch Example",
"We use H2 Database for this example" };
private int count = 0;
@Override
public String read() throws Exception, UnexpectedInputException,
ParseException, NonTransientResourceException {
if (count < messages.length) {
return messages[count++];
} else {
count = 0;
}
return null;
}
}
public class Processor implements ItemProcessor<String, String> {
@Override
public String process(String data) throws Exception {
return data.toUpperCase();
}
}
public class Writer implements ItemWriter<String> {
@Override
public void write(List<? extends String> messages) throws Exception {
for (String msg : messages) {
System.out.println("Writing the data " + msg);
}
}
}
public class JobCompletionListener extends JobExecutionListenerSupport {
@Override
public void afterJob(JobExecution jobExecution) {
if (jobExecution.getStatus() == BatchStatus.COMPLETED) {
System.out.println("BATCH JOB COMPLETED SUCCESSFULLY");
}
}
}
Running Jobs from the Command Line
@SpringBootApplication
public class App implements CommandLineRunner
{
@Autowired
JobLauncher jobLauncher;
@Autowired
Job job;
public static void main(String[] args)
{
SpringApplication.run(App.class, args);
}
@Override
public void run(String... args) throws Exception
{
JobParameters params = new JobParametersBuilder()
.addString("JobID", String.valueOf(System.currentTimeMillis()))
.toJobParameters();
jobLauncher.run(job, params);
}
<bash$ java CommandLineJobRunner io.spring.BatchConfig processJob\ schedule.date=2023-10-10,java.time.LocalDate,true \ vendor.id=123,java.lang.Long,false
Running Jobs from within a Web Container
@Controller
public class JobLauncherController {
@Autowired
JobLauncher jobLauncher;
@Autowired
Job job;
@RequestMapping("/jobLauncher")
public void handle() throws Exception{
JobParameters jobParameters = new JobParametersBuilder()
.addLong("time", System.currentTimeMillis())
.toJobParameters();
jobLauncher.run(job, new JobParameters());
}
}
Running jobs schduled
// run every 5000 msec (i.e., every 5 secs)
@Scheduled(fixedRate = 5000)
public void run() throws Exception {
JobExecution execution = jobLauncher
run(job(),new JobParametersBuilder().toJobParameters()
);
}
Refrences :