If you are working with chunk based steps, ItemReader is responsible to provide input to the chunk based step.
What is Chunk?
Chunk encapsulate list of items to be processed. Chunk based step is item based. There are 3 main components in Chunk based step.
a. ItemReader: Provide input to the step
b. ItemProcessor: Process the information. It is optional.
c. ItemWriter: Provide output to the step.
Suppose you have 10000 items to process, you can specify chunk size as 1000. On every read Spring Batch read 1000 items and process and commit them. Once the items are processed and committed, Spring Batch read next 1000 items and proceed. This process continue until all the items are processed.
Let’s create custom ItemReader and process the items with chunk size of 3.
Step 1: Create Custom ItemReader by implementing ItemReader interface.
MyItemReader.java
public class MyItemReader implements ItemReader<String> {
private Iterator<String> items;
public MyItemReader(Iterator<String> items) {
this.items = items;
}
@Override
public String read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
if (items.hasNext()) {
String item = items.next();
System.out.println("Reading item : " + item);
return item;
}
return null;
}
}
Step 2: Define ItemReader bean.
@Bean
public MyItemReader myItemReader() {
List<String> data = Arrays.asList("ONE", "TWO", "THREE", "FOUR", "FIVE", "SIX", "SEVEN", "EIGHT", "NINE",
"TEN");
return new MyItemReader(data.iterator());
}
Step 3: Create chunk based step using ItemReader defined in step 2.
@Bean
public Step step1() {
return this.stepBuilderFactory.get("step1").<String, String>chunk(3).reader(myItemReader()).writer(list -> {
for (String item : list) {
System.out.println("Writing : " + item);
}
System.out.println("\n");
}).build();
}
Find the below working application.
Step 1: Create new maven project ‘item-reader-demo’.
Step 2: Update pom.xml with maven dependencies.
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.sample.app</groupId>
<artifactId>item-reader-demo</artifactId>
<version>1</version>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.6.RELEASE</version>
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<!-- https://mvnrepository.com/artifact/org.springframework.batch/spring-batch-core -->
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-core</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
</dependency>
</dependencies>
</project>
Step 3: Create new package ‘com.sample.app.readers’ and define MyItemReader.
MyItemReader.java
package com.sample.app.readers;
import java.util.Iterator;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.NonTransientResourceException;
import org.springframework.batch.item.ParseException;
import org.springframework.batch.item.UnexpectedInputException;
public class MyItemReader implements ItemReader<String> {
private Iterator<String> items;
public MyItemReader(Iterator<String> items) {
this.items = items;
}
@Override
public String read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
if (items.hasNext()) {
String item = items.next();
System.out.println("Reading item : " + item);
return item;
}
return null;
}
}
Step 4: Create package ‘com.sample.app.configuration’ and define JobCOnfiguration.
JobConfiguration.java
package com.sample.app.configuration;
import java.util.Arrays;
import java.util.List;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.transaction.PlatformTransactionManager;
import com.sample.app.readers.MyItemReader;
@Configuration
@EnableBatchProcessing
public class JobConfiguration {
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Bean
public MyItemReader myItemReader() {
List<String> data = Arrays.asList("ONE", "TWO", "THREE", "FOUR", "FIVE", "SIX", "SEVEN", "EIGHT", "NINE",
"TEN");
return new MyItemReader(data.iterator());
}
@Bean
public Step step1() {
return this.stepBuilderFactory.get("step1").<String, String>chunk(3).reader(myItemReader()).writer(list -> {
for (String item : list) {
System.out.println("Writing : " + item);
}
System.out.println("\n");
}).build();
}
@Bean
public Job myJob(JobRepository jobRepository, PlatformTransactionManager platformTransactionManager) {
return jobBuilderFactory.get("My-First-Job").start(step1()).build();
}
}
Step 5: Define main Application class.
App.java
package com.sample.app;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@EnableBatchProcessing
@SpringBootApplication
public class App {
public static void main(String[] args) {
SpringApplication.run(App.class, args);
}
}
Step 6: Create ‘application.properties’ file under src/main/resources folder.
application.properties
logging.level.root=ERROR logging.level.org.hibernate=ERROR ## H2 specific properties spring.h2.console.enabled=true spring.h2.console.path=/h2 spring.datasource.url=jdbc:h2:file:~/db/myOrg.db;DB_CLOSE_ON_EXIT=FALSE;DB_CLOSE_DELAY=-1; spring.datasource.username=krishna spring.datasource.password=password123 spring.datasource.driverClassName=org.h2.Driver ## JPA specific properties # Creates the schema, destroying previous data. spring.jpa.hibernate.ddl-auto=create-drop spring.jpa.database-platform=org.hibernate.dialect.H2Dialect spring.jpa.show-sql=false spring.jpa.properties.hibernate.format_sql=false ## Database connection pooling properties # Number of ms to wait before throwing an exception if no connection is available. spring.datasource.max-wait=10000 # Maximum number of active connections that can be allocated from this pool at the same time. spring.datasource.tomcat.max-active=10 spring.datasource.tomcat.max-idle=5 spring.datasource.tomcat.min-idle=3
Total project structure looks like below.
Run App.java, you will see below messages in console.
Reading item : ONE Reading item : TWO Reading item : THREE Writing : ONE Writing : TWO Writing : THREE Reading item : FOUR Reading item : FIVE Reading item : SIX Writing : FOUR Writing : FIVE Writing : SIX Reading item : SEVEN Reading item : EIGHT Reading item : NINE Writing : SEVEN Writing : EIGHT Writing : NINE Reading item : TEN Writing : TEN
You can download complete working application from this link.
https://github.com/harikrishna553/springboot/tree/master/batch/item-reader-demo
No comments:
Post a Comment