Spring Batch: JdbcPagingItemReader: Read data from table

In this post, I am going to explain how to read data from a table using JdbcPagingItemReader.

 

Step 1: Define RowMapper that maps ResultSet to an Employee instance.

public class EmployeeRowMapper implements RowMapper<Employee> {

	@Override
	public Employee mapRow(ResultSet rs, int rowNum) throws SQLException {

		Employee emp = new Employee();

		emp.setId(rs.getInt("id"));
		emp.setFirstName(rs.getString("first_name"));
		emp.setLastName(rs.getString("last_name"));

		return emp;
	}

}

 

Step 2: Define JdbcPagingItemReader bean.

@Bean
public JdbcPagingItemReader<Employee> jdbcPagingItemReader() {
	JdbcPagingItemReader<Employee> pagingItemReader = new JdbcPagingItemReader<>();

	pagingItemReader.setDataSource(dataSource);
	pagingItemReader.setFetchSize(3);
	pagingItemReader.setRowMapper(new EmployeeRowMapper());

	MySqlPagingQueryProvider mySqlPagingQueryProvider = new MySqlPagingQueryProvider();
	mySqlPagingQueryProvider.setSelectClause("id, first_name, last_name");
	mySqlPagingQueryProvider.setFromClause("FROM employee");

	Map<String, Order> orderByKeys = new HashMap<>();
	orderByKeys.put("first_name", Order.ASCENDING);

	mySqlPagingQueryProvider.setSortKeys(orderByKeys);

	pagingItemReader.setQueryProvider(mySqlPagingQueryProvider);

	return pagingItemReader;
}

 

As you see we set the query to execute and EmployeeRowMapper to the reader.

 

Step 3: Define ItemWriter that writes the chunk to console.

@Bean
public ItemWriter<? super Object> itemWriter() {
	return emps -> {
		System.out.println("\nWriting chunk to console");
		for (Object emp : emps) {
			System.out.println(emp);
		}
	};
}

 

Step 4: Define step with reader and writer.

@Bean
public Step step1() {
	return this.stepBuilderFactory.get("step1").chunk(3).reader(jdbcPagingItemReader()).writer(itemWriter()).build();
}

Step 5: Define job.

@Bean
public Job myJob(JobRepository jobRepository, PlatformTransactionManager platformTransactionManager) {
	return jobBuilderFactory.get("My-First-Job").start(step1()).build();
}

 

That’s it you are done.

 

Find the below working application.

 

Step 1: Create new maven project ‘jdbcCursorItemReader-demo’.

 

Step 2: Update pom.xml with maven dependencies.

 

pom.xml

 

<project xmlns="http://maven.apache.org/POM/4.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.sample.app</groupId>
	<artifactId>jdbcPagingItemReader-demo</artifactId>
	<version>1</version>

	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.1.6.RELEASE</version>
	</parent>

	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	</properties>

	<dependencies>

		<!-- https://mvnrepository.com/artifact/org.springframework.batch/spring-batch-core -->
		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-core</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-jpa</artifactId>
		</dependency>

		<dependency>
			<groupId>com.h2database</groupId>
			<artifactId>h2</artifactId>
		</dependency>
	</dependencies>
</project>

 

Step 3: Create application.properties file under src/main/resources folder.

 

application.properties

 

logging.level.root=ERROR
logging.level.org.hibernate=ERROR

## H2 specific properties
spring.h2.console.enabled=true
spring.h2.console.path=/h2

spring.datasource.url=jdbc:h2:file:~/db/myOrg.db;DB_CLOSE_ON_EXIT=FALSE;DB_CLOSE_DELAY=-1;

spring.datasource.username=krishna
spring.datasource.password=password123

spring.datasource.driverClassName=org.h2.Driver

## JPA specific properties
# Creates the schema, destroying previous data.
spring.jpa.hibernate.ddl-auto=create-drop

spring.jpa.database-platform=org.hibernate.dialect.H2Dialect

spring.jpa.show-sql=false
spring.jpa.properties.hibernate.format_sql=false

## Database connection pooling properties
# Number of ms to wait before throwing an exception if no connection is available.
spring.datasource.max-wait=10000

# Maximum number of active connections that can be allocated from this pool at the same time.
spring.datasource.tomcat.max-active=10
spring.datasource.tomcat.max-idle=5
spring.datasource.tomcat.min-idle=3

 

Step 4: Create data.sql file in src/main/resources folder. It is used to insert data to employee table.

 

data.sql

 

INSERT INTO employee (id, first_name, last_name) VALUES
  (1, 'Sowmya', 'Krishna'),
  (2, 'Mridhu', 'Latha'),
  (3, 'Gowthami','Prasad'),
  (4, 'Sailu', 'Venkat'),
  (5, 'Chamu', 'Krishna'),
  (6, 'Rama', 'Lakshman'),
  (7, 'Saranya', 'Nataraj'),
  (8, 'Suneetha', 'Surendra');

 

Step 5: Create package ‘com.sample.app.entity’ and define Employee class.

 

Employee.java

 

package com.sample.app.entity;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import javax.persistence.Table;

@Entity
@Table(name = "employee")
public class Employee {

	@Id
	@GeneratedValue
	@Column(name = "id")
	private int id;

	@Column(name = "first_name")
	private String firstName;

	@Column(name = "last_name")
	private String lastName;

	public int getId() {
		return id;
	}

	public void setId(int id) {
		this.id = id;
	}

	public String getFirstName() {
		return firstName;
	}

	public void setFirstName(String firstName) {
		this.firstName = firstName;
	}

	public String getLastName() {
		return lastName;
	}

	public void setLastName(String lastName) {
		this.lastName = lastName;
	}

	@Override
	public String toString() {
		StringBuilder builder = new StringBuilder();
		builder.append("Employee [id=").append(id).append(", firstName=").append(firstName).append(", lastName=")
				.append(lastName).append("]");
		return builder.toString();
	}

}

 

Step 6: Create package ‘com.smaple.app.mappers’ and define EmployeeRowMapper.

 

EmployeeRowMapper.java

 

package com.smaple.app.mappers;

import java.sql.ResultSet;
import java.sql.SQLException;

import org.springframework.jdbc.core.RowMapper;

import com.sample.app.entity.Employee;

public class EmployeeRowMapper implements RowMapper<Employee> {

	@Override
	public Employee mapRow(ResultSet rs, int rowNum) throws SQLException {

		Employee emp = new Employee();

		emp.setId(rs.getInt("id"));
		emp.setFirstName(rs.getString("first_name"));
		emp.setLastName(rs.getString("last_name"));

		return emp;
	}

}

 

Step 7: Create package ‘com.sample.app.configuration’ and define JobConfiguration.

 

JobConfiguration.java

 

package com.sample.app.configuration;

import java.util.HashMap;
import java.util.Map;

import javax.sql.DataSource;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.database.JdbcPagingItemReader;
import org.springframework.batch.item.database.Order;
import org.springframework.batch.item.database.support.MySqlPagingQueryProvider;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.transaction.PlatformTransactionManager;

import com.sample.app.entity.Employee;
import com.smaple.app.mappers.EmployeeRowMapper;

@Configuration
@EnableBatchProcessing
public class JobConfiguration {
	@Autowired
	private JobBuilderFactory jobBuilderFactory;

	@Autowired
	private StepBuilderFactory stepBuilderFactory;

	@Autowired
	private DataSource dataSource;

	@Bean
	public JdbcPagingItemReader<Employee> jdbcPagingItemReader() {
		JdbcPagingItemReader<Employee> pagingItemReader = new JdbcPagingItemReader<>();

		pagingItemReader.setDataSource(dataSource);
		pagingItemReader.setFetchSize(3);
		pagingItemReader.setRowMapper(new EmployeeRowMapper());

		MySqlPagingQueryProvider mySqlPagingQueryProvider = new MySqlPagingQueryProvider();
		mySqlPagingQueryProvider.setSelectClause("id, first_name, last_name");
		mySqlPagingQueryProvider.setFromClause("FROM employee");

		Map<String, Order> orderByKeys = new HashMap<>();
		orderByKeys.put("first_name", Order.ASCENDING);

		mySqlPagingQueryProvider.setSortKeys(orderByKeys);

		pagingItemReader.setQueryProvider(mySqlPagingQueryProvider);

		return pagingItemReader;
	}

	@Bean
	public ItemWriter<? super Object> itemWriter() {
		return emps -> {
			System.out.println("\nWriting chunk to console");
			for (Object emp : emps) {
				System.out.println(emp);
			}
		};
	}

	@Bean
	public Step step1() {
		return this.stepBuilderFactory.get("step1").chunk(3).reader(jdbcPagingItemReader()).writer(itemWriter())
				.build();
	}

	@Bean
	public Job myJob(JobRepository jobRepository, PlatformTransactionManager platformTransactionManager) {

		return jobBuilderFactory.get("My-First-Job").start(step1()).build();
	}

}

 

Step 8: Define App.java

 

App.java

 

package com.sample.app;

import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@EnableBatchProcessing
@SpringBootApplication
public class App {

	public static void main(String[] args) {
		SpringApplication.run(App.class, args);
	}
}

  Total project structure looks like below.

 

Run App.java, you will see below messages in console.

Writing chunk to console
Employee [id=5, firstName=Chamu, lastName=Krishna]
Employee [id=3, firstName=Gowthami, lastName=Prasad]
Employee [id=2, firstName=Mridhu, lastName=Latha]

Writing chunk to console
Employee [id=6, firstName=Rama, lastName=Lakshman]
Employee [id=4, firstName=Sailu, lastName=Venkat]
Employee [id=7, firstName=Saranya, lastName=Nataraj]

Writing chunk to console
Employee [id=1, firstName=Sowmya, lastName=Krishna]
Employee [id=8, firstName=Suneetha, lastName=Surendra]

 

You can download the application from this link.

https://github.com/harikrishna553/springboot/tree/master/batch/jdbcPagingItemReader-demo

 

 

 

 

Previous                                                    Next                                                    Home