CSV to MongoDB Spring Boot Batch Example

CSV to MongoDB Spring Boot Batch Example | In the previous example we have seen Spring Boot Batch CSV to MySQL database. Now we will see the Spring Boot batch CSV to MongoDB example. Also see:-

• Spring Boot Batch API Introduction
• CSV To MySQL Database Spring Boot Batch API Example
• MySQL to CSV Spring Boot Batch Example
• MongoDB to CSV Spring Boot Batch Example
• MySQL to XML Spring Boot Batch Example

The properties of MongoDB:-

spring.data.mongodb.host=localhost
spring.data.mongodb.port=27017
spring.data.mongodb.database=sample

For mongo database:-

• MongoItemWriter is the implementation class of the ItemWriter interface.
• MongoTemplate will be used which can be auto-configured.
• The collection name is used while creating and inserting the data.

CSV to MongoDB Spring Boot Batch

Create a Spring starter project with the following dependencies:- Lombok, Spring Batch, Spring Data MongoDB, H2 Database, Spring web. The H2 database is used for repository purposes.

Different Classes/files:-

• Model class
• Batch Config
• Properties file
• Controller class
• CSV file

In products.csv file:-

id,code,cost
10,PEN,200.0
11,BOOK,500.0
12,BOTTLE,600.0
13,MOBILE,1500.0
14,MOUSE,300.0
15,KEYBOAD,900.0
16,BAG,600.0

In application.properties file:-

spring.data.mongodb.host=localhost
spring.data.mongodb.port=27017
spring.data.mongodb.database=sample
spring.batch.job.enabled=false

Model class:-

package com.knowprogram.demo.model;

import org.springframework.data.annotation.Id;

import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;

@Data
@AllArgsConstructor
@NoArgsConstructor
public class Product {
    @Id
    private Integer prodId;
    
    private String prodCode;

    private Double prodCost;
    
    private Double prodTax;
    
    private Double prodDiscount;
}

In the BatchConfig file, compared to the previous example of CSV to MySQL, mainly writer() bean method logic gets changes and ProductRepository is replaced with MongoTemplate. The remaining code will be remain the same.

@Configuration
public class BatchConfig {

    @Autowired
    private MongoTemplate mongoTemplate;

    @Bean
    ItemReader<Product> reader() {
        FlatFileItemReader<Product> itemReader = new FlatFileItemReader<>();
        itemReader.setResource(new ClassPathResource("products.csv"));
        itemReader.setName("csv-reader");
        itemReader.setLinesToSkip(1); // to skip heading

        itemReader.setLineMapper(new DefaultLineMapper<>() {
            {
                setLineTokenizer(new DelimitedLineTokenizer() {
                    {
                        setDelimiter(DELIMITER_COMMA);
                        // variables name
                        setNames("prodId", "prodCode", "prodCost");
                        setStrict(false);
                    }
                });
                setFieldSetMapper(new BeanWrapperFieldSetMapper<>() {
                    {
                        setTargetType(Product.class);
                    }
                });
            }
        });

        return itemReader;
    }

    @Bean
    ItemProcessor<Product, Product> processor() {
        return item -> {
            item.setProdTax(item.getProdCost() * 0.12);
            item.setProdDiscount(item.getProdCost() * 0.08);
            return item;
        };
    }

    @Bean
    ItemWriter<Product> writer() {
        MongoItemWriter<Product> writer = new MongoItemWriter<>();
        writer.setTemplate(mongoTemplate);
        writer.setCollection("products"); // collection name
        return writer;
    }

    @Bean
    Step step(JobRepository repository, PlatformTransactionManager transactionManager) {
        return new StepBuilder("csv-step", repository)
                .<Product, Product>chunk(10, transactionManager)
                .reader(reader())
                .processor(processor()).writer(writer())
                .taskExecutor(new SimpleAsyncTaskExecutor() {
                    private static final long serialVersionUID = 1L;
                    {
                        setConcurrencyLimit(10);
                    }
                })
                .build();
    }

    @Bean
    JobExecutionListener listener() {
        return new JobExecutionListener() {
            @Override
            public void beforeJob(JobExecution jobExecution) {
                System.out.println("MyJobListener.beforeJob()");
            }

            @Override
            public void afterJob(JobExecution jobExecution) {
                System.out.println("MyJobListener.afterJob()");
            }
        };
    }

    @Bean(name = "csvJob")
    Job job(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
        return new JobBuilder("csv-job", jobRepository)
                .listener(listener())
                .flow(step(jobRepository, transactionManager))
                .end()
                .build();
    }
}

Controller class:-

@RestController
package com.knowprogram.demo.controller;

public class ProductBatchController {

    @Autowired
    private JobLauncher jobLauncher;

    @Autowired
    private Job job;

    @GetMapping("/startBatch")
    public String startBatch() throws JobExecutionAlreadyRunningException, 
        JobRestartException,
        JobInstanceAlreadyCompleteException, 
        JobParametersInvalidException {
        JobParameters params = new JobParametersBuilder()
             .addLong("time", System.currentTimeMillis())
             .toJobParameters();

        JobExecution run = jobLauncher.run(job, params);
        return run.getStatus().toString();
    }

}

In MongoDB:-

> use sample
> show collections
> db.products.find()
[
  {
    _id: 13,
    prodCode: 'MOBILE',
    prodCost: 1500,
    prodTax: 180,
    prodDiscount: 120,
    _class: 'com.knowprogram.demo.model.Product'
  },
  {
    _id: 15,
    prodCode: 'KEYBOAD',
    prodCost: 900,
    prodTax: 108,
    prodDiscount: 72,
    _class: 'com.knowprogram.demo.model.Product'
  },
  {
    _id: 12,
    prodCode: 'BOTTLE',
    prodCost: 600,
    prodTax: 72,
    prodDiscount: 48,
    _class: 'com.knowprogram.demo.model.Product'
  },
  {
    _id: 10,
    prodCode: 'PEN',
    prodCost: 200,
    prodTax: 24,
    prodDiscount: 16,
    _class: 'com.knowprogram.demo.model.Product'
  },
  {
    _id: 14,
    prodCode: 'MOUSE',
    prodCost: 300,
    prodTax: 36,
    prodDiscount: 24,
    _class: 'com.knowprogram.demo.model.Product'
  },
  {
    _id: 11,
    prodCode: 'BOOK',
    prodCost: 500,
    prodTax: 60,
    prodDiscount: 40,
    _class: 'com.knowprogram.demo.model.Product'
  },
  {
    _id: 16,
    prodCode: 'BAG',
    prodCost: 600,
    prodTax: 72,
    prodDiscount: 48,
    _class: 'com.knowprogram.demo.model.Product'
  }
]

If you enjoyed this post, share it with your friends. Do you want to share more information about the topic discussed above or do you find anything incorrect? Let us know in the comments. Thank you!

Search