- 浏览: 19220 次
- 性别:
- 来自: 广州
文章分类
最新评论
暂时还没完成,只供自己参考
参考链接:
http://www.yihaomen.com/article/java/433.htm
http://www.zuidaima.com/share/1732772811131904.htm
http://13146489.iteye.com/blog/1412295
实现的具体功能:
step1 : 从 A 文件夹中读取csv 文件,处理之后,写到数据库中保存
App.java
CustomItemProcessor.java
ReportFieldSetMapper.java
ReportWrite2Database.java
Report.java
report.csv
context.xml
database.xml
job-hello-world.xml
框架结构:
引用的jar包
运行结果:
参考链接:
http://www.yihaomen.com/article/java/433.htm
http://www.zuidaima.com/share/1732772811131904.htm
http://13146489.iteye.com/blog/1412295
实现的具体功能:
step1 : 从 A 文件夹中读取csv 文件,处理之后,写到数据库中保存
App.java
package yihaomen; import java.util.HashMap; import java.util.Map; import java.util.Random; import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameter; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; public class App { public static void main(String[] args) { String[] springConfig = { "spring/batch/jobs/job-hello-world.xml" }; ApplicationContext context = new ClassPathXmlApplicationContext(springConfig); JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher"); Job job = (Job) context.getBean("helloWorldJob"); try { Map<String,JobParameter> parameters = new HashMap<String,JobParameter>(); parameters.put("RUN_MONTH_KEY", new JobParameter(Math.random()));//因为一个job parameter只能用一次,所以为了多次测试使用,所以我直接用随机数 JobExecution execution = jobLauncher.run(job, new JobParameters(parameters)); System.out.println("Exit Status : " + execution.getStatus()); } catch (Exception e) { e.printStackTrace(); } System.out.println("Done"); } }
CustomItemProcessor.java
package yihaomen; import org.springframework.batch.item.ItemProcessor; import yihaomen.model.Report; public class CustomItemProcessor implements ItemProcessor<Report, Report> { @Override public Report process(Report item) throws Exception { System.out.println("Processing..." + item); return item; } }
ReportFieldSetMapper.java
package yihaomen; import java.text.ParseException; import java.text.SimpleDateFormat; import org.springframework.batch.item.file.mapping.FieldSetMapper; import org.springframework.batch.item.file.transform.FieldSet; import org.springframework.validation.BindException; import yihaomen.model.Report; public class ReportFieldSetMapper implements FieldSetMapper<Report> { private SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd"); @Override public Report mapFieldSet(FieldSet fieldSet) throws BindException { Report report = new Report(); report.setId(fieldSet.readInt(0)); report.setSales(fieldSet.readBigDecimal(1)); report.setQty(fieldSet.readInt(2)); report.setStaffName(fieldSet.readString(3)); //default format yyyy-MM-dd //fieldSet.readDate(4); String date = fieldSet.readString(4); try { report.setDate(dateFormat.parse(date)); } catch (ParseException e) { e.printStackTrace(); } return report; } }
ReportWrite2Database.java
package yihaomen; import java.util.List; import org.springframework.batch.item.ItemWriter; import yihaomen.model.Report; public class ReportWrite2Database implements ItemWriter<Report>{ @Override public void write(List<? extends Report> items) throws Exception { System.out.println("开始存储到数据库report表中......."); for (Report r : items ){ System.out.println(r.toString()); } System.out.println("存储完毕!......."); } }
Report.java
package yihaomen.model; import java.math.BigDecimal; import java.util.Date; import javax.xml.bind.annotation.XmlAttribute; import javax.xml.bind.annotation.XmlElement; import javax.xml.bind.annotation.XmlRootElement; @XmlRootElement(name = "record") public class Report { private int id; private BigDecimal sales; private int qty; private String staffName; private Date date; @XmlAttribute(name = "id") public int getId() { return id; } public void setId(int id) { this.id = id; } @XmlElement(name = "sales") public BigDecimal getSales() { return sales; } public void setSales(BigDecimal sales) { this.sales = sales; } @XmlElement(name = "qty") public int getQty() { return qty; } public void setQty(int qty) { this.qty = qty; } @XmlElement(name = "staffName") public String getStaffName() { return staffName; } public void setStaffName(String staffName) { this.staffName = staffName; } public Date getDate() { return date; } public void setDate(Date date) { this.date = date; } @Override public String toString() { return "Report [id=" + id + ", sales=" + sales + ", qty=" + qty + ", staffName=" + staffName + "]"; } }
report.csv
1001,"213,100",980,yihaomen, 2013-01-01 1002,"320,200",1080,staff 1, 2013-01-01 1003,"342,197",1200,staff 2, 2013-01-01
context.xml
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd"> <!-- stored job-meta in memory --> <!-- <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"> <property name="transactionManager" ref="transactionManager" /> </bean> --> <!-- stored job-meta in database --> <bean id="jobRepository" class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean"> <property name="dataSource" ref="dataSource" /> <property name="transactionManager" ref="transactionManager" /> <property name="databaseType" value="oracle" /> </bean> <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" /> <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name="jobRepository" ref="jobRepository" /> </bean> </beans>
database.xml
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:jdbc="http://www.springframework.org/schema/jdbc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.1.xsd"> <!-- connect to database --> <bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource"> <property name="driverClassName" value="oracle.jdbc.OracleDriver" /> <property name="url" value="jdbc:oracle:thin:@localhost:1521:orcl" /> <property name="username" value="evan" /> <property name="password" value="880823" /> </bean> <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" /> <!-- create job-meta tables automatically --> <!-- spring batch 2.2.3 里面的语句有点问题,在后面两个需要加"," 号,自己可以拷贝出来,修改执行. <jdbc:initialize-database data-source="dataSource"> <jdbc:script location="org/springframework/batch/core/schema-drop-oracle10g.sql" /> <jdbc:script location="org/springframework/batch/core/schema-oracle10g.sql" /> </jdbc:initialize-database> --> </beans>
job-hello-world.xml
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:batch="http://www.springframework.org/schema/batch" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd "> <import resource="../config/context.xml" /> <import resource="../config/database.xml" /> <bean id="report" class="yihaomen.model.Report" scope="prototype" /> <bean id="itemProcessor" class="yihaomen.CustomItemProcessor" /> <batch:job id="helloWorldJob"> <batch:step id="step1"> <batch:tasklet> <batch:chunk reader="cvsFileItemReader" writer="xmlItemWriter" processor="itemProcessor" commit-interval="10"> </batch:chunk> </batch:tasklet> </batch:step> </batch:job> <bean id="cvsFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"> <property name="resource" value="classpath:cvs/input/report.csv" /> <property name="lineMapper"> <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper"> <property name="lineTokenizer"> <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"> <property name="names" value="id,sales,qty,staffName,date" /> </bean> </property> <property name="fieldSetMapper"> <bean class="yihaomen.ReportFieldSetMapper" /> <!-- if no data type conversion, use BeanWrapperFieldSetMapper to map by name <bean class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper"> <property name="prototypeBeanName" value="report" /> </bean> --> </property> </bean> </property> </bean><!-- <bean id="xmlItemWriter" class="org.springframework.batch.item.xml.StaxEventItemWriter"> <property name="resource" value="file:xml/outputs/report.xml" /> <property name="marshaller" ref="reportMarshaller" /> <property name="rootTagName" value="report" /> </bean> --> <bean id="xmlItemWriter" class="yihaomen.ReportWrite2Database"> </bean> <bean id="reportMarshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller"> <property name="classesToBeBound"> <list> <value>yihaomen.model.Report</value> </list> </property> </bean> </beans>
框架结构:
引用的jar包
运行结果:
相关推荐
Spring Batch 是一个轻量级的、完善的批处理框架,旨在帮助企业建立健壮、高效的批处理应用。Spring Batch是Spring的一个子项目,使用Java语言并基于Spring框架为基础开发,使得已经使用 Spring 框架的开发者或者企业更...
基于Spring Batch的大数据量并行处理 基于Spring Batch的大数据量并行处理
NULL 博文链接:https://virusfu.iteye.com/blog/1150730
Spring Batch批处理框架Spring Batch批处理框架Spring Batch批处理框架
最近在研究springBoot+springbatch ,按照官网的实例做了一个实例。 最近在研究springBoot+springbatch ,按照官网的实例做了一个实例。
Spring Boot整合Spring Batch的一个小例子,在网上发现这方面的资源比较少,特此将其上传供大家学习。
1.本项目运行在tomcat容器中,主要功能为从spring_batch_left库的user_from表抓取数据,之后批量插入到spring_batch_right库的user_to表 2.应用quartz对job进行定时触发(目前设置的定时为每隔一分钟执行一次,目前...
资源名称:Spring Batch 批处理框架内容简介:《Spring Batch 批处理框架》全面、系统地介绍了批处理框架Spring Batch,通过详尽的实战示例向读者展示了Spring Batch框架对大数据批处理的基本开发能力,并对框架的...
spring batch批处理框架和对应的源码资源 rar 可以直接运行的
SpringBatch+SpringBoot构建海量数据企业批处理系统和性能优化,Spring Batch是一个基于Spring的企业级批处理框架,所谓企业批处理就是指在企业级应用中,不需要人工干预,定期读取数据,进行相应的业务处理之后,再...
Spring Batch是一个轻量级的,完全面向Spring的批处理框架,可以应用于企业级大量的数据处理系统。Spring Batch以POJO和大家熟知的Spring框架为基础,使开发者更容易的访问和利用企业级服务。Spring Batch可以提供...
主要给大家介绍了Spring Batch读取txt文件并写入数据库的方法,SpringBatch 是一个轻量级、全面的批处理框架。这里我们用它来实现文件的读取并将读取的结果作处理,处理之后再写入数据库中的功能。需要的朋友可以...
Spring Batch API(Spring Batch 开发文档).CHM。 官网 Spring Batch API,Spring Batch 开发文档
《Spring Batch 批处理框架》全面、系统地介绍了批处理框架Spring Batch,通过详尽的实战示例向读者展示了Spring Batch框架对大数据批处理的基本开发能力,并对框架的架构设计、源码做了特定的剖析;在帮助读者掌握...
spring-batch4.0.0 batch spring-batch集成 spring-batch.jar
本人花费多日时间,亲测有效。所需文件代码完整,只需导入常用开发IDE和mysql数据库即可正常使用。
spring batch官方文档:...简单的任务操作实现,当前项目只是一个demo,我也是初学,目前启动执行完任务boot自动关闭问题还没有解决,代码可供讨论,不能直接使用,哪位大佬解决了一起讨论一下
Work with all aspects of batch processing in a modern Java environment using a selection of Spring frameworks. This book provides up-to-date examples using the latest configuration techniques based on...
springbatch简单用