As an example think of processing a large flat file composed of records and writing the output to a database. Each line in the file corresponds to an item: it will be read, transformed and output separately. Spring Batch provides a solid foundation to implement item-oriented jobs and includes a collection of ready-to-use objects for common.
In this tutorial, we will show you how to read items from multiple resources (multiple csv files), and write the items into a single csv file.
Tools and libraries used
P.S This example – 3 CSV files (reader) – combine into a single CSV file (writer).
Review the final project structure, a standard Maven project.
There are 3 csv files, later we will use MultiResourceItemReader
to read it one by one.
A job to read resources that matches this pattern csv/inputs/domain-*.csv
, and write it into a single cvs file domain.all.csv
.
Create a Java class and run the batch job.
Output. The content of three csv files is read and combine into a single csv file.
Batch processing is the execution of a series of programs (“jobs”) on a computer without manual intervention.
Spring Batch provides mechanisms for processing large amount of data like transaction management, job processing, resource management, logging, tracing, conversion of data, interfaces, etc.
These functionalities are available out of the box and can be reused by applications containing the Spring Batch framework.
In this tutorial, we will show you how to configure a Spring Batch job to read CSV file into a CSV file, and filter out the record before writing with ItemProcessor
. Its a very easy program for beginners.
Tools and libraries used
1. Create a maven project . I named my project as SpringBatchProject.
2. Project Dependencies –
3. Project Structure –
4. CSV file resources/files/input.csv
5. Read CSV file resources/jobs/job-report.xml
6. The csv file mapped to Pojo Report.java
7. Spring batch Core Settings
Define jobRepository
and jobLauncher
8. Spring batch Jobs
A Spring batch job, read the report.csv
file, map it to Report
object, and write it into a csv file.
9. Spring Batch – ItemProcessor
In Spring batch, the wired Processor
will be fired before writing to any resources, so, this is the best place to handle any conversion, filtering and business logic. In this example, the Report
object will be ignored (not write to csv file) if its’ age
is greater than equal to 30.
10. I have scheduled this process which will run in every 5 seconds through cron jobs.
RunScheduler.java
11. Run the Main class now
12. output csv file i.e. report.csv