1 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Size: px

Start display at page:

Cornelius Webb
5 years ago
Views:

3 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. 3 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

4 Batch Applications for the Java Platform Standardizes batch processing for Java Non-interactive, bulk-oriented, long-running Data or computationally intensive Sequentially or in parallel Ad-hoc, scheduled or on-demand execution Led by IBM Spring Batch, WebSphere Compute Grid (WCG), z/os Batch Part of Java EE 7, can be used in Java SE 4 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

8 Reader, Processor, Writer public interface ItemReader<T> { public void open(externalizable checkpoint); public T readitem(); public Externalizable checkpointinfo(); public void close(); } public interface ItemProcessor<T, R> { public R processitem(t item); } public interface ItemWriter<T> { public void open(externalizable checkpoint); public void writeitems(list<t> items); public Externalizable checkpointinfo(); public void close(); } 8 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Batch Applications for the Java Platform Step Example using Job Specification Language (JSL) //For a job <step id= sendstatements > <chunk reader= accountreader processor=

.. { public Account readitem() { // read account using JPA @Named( accountprocessor")...implements ItemProcessor.

9 Batch Applications for the Java Platform Step Example using Job Specification Language (JSL) //For a job <step id= sendstatements > <chunk reader= accountreader processor= accountprocessor writer= writer item-count= 10 /> accountreader")...implements ItemReader... { public Account readitem() { // read account using accountprocessor")...implements ItemProcessor... { public Statement processitems(account account) { // read Account, return writer")...implements ItemWriter... { public void writeitems(list<statements> statements) { // use JavaMail to send 9 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

11 Checkpointing For data intensive tasks, long periods of time Checkpoint/restart is a common design requirement Basically saves Reader, Writer positions Naturally fits into Chunk oriented steps reader.checkpointinfo() and writer.checkpointinfo() are called The resulting Externalizable data is persisted When the Chunk restarts, the reader and writer are initialized with the respective Externalizable data 11 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

12 Reader, Processor, Writer public interface ItemReader<T> { public void open(externalizable checkpoint); public T readitem(); public Externalizable checkpointinfo(); public void close(); } public interface ItemProcessor<T, R> { public R processitem(t item); } public interface ItemWriter<T> { public void open(externalizable checkpoint); public void writeitems(list<t> items); public Externalizable checkpointinfo(); public void close(); } 12 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

13 Handling Exceptions <job id=...>... <chunk skip-limit= 5 retry-limit= 5 > <skippable-exception-classes> <include class="java.lang.exception"/> <exclude class="java.io.filenotfoundexception"/> </skippable-exception-classes> <retryable-exception-classes>... </retryable-exception-classes> <no-rollback-exception-classes>... </no-rollback-exception-classes> </chunk>... </job> 13 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

14 Partitioned Step A batch step may run as a partitioned step A partitioned step runs as multiple instances of the same step definition across multiple threads, one partition per thread <step id="step1" > <chunk...> <partition> <plan partitions= 10" threads="2"/> <reducer.../> </partition> </chunk> </step> 14 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

15 Partitioning Advanced Scenarios Partition Mapper Dynamically decide number of partitions (partition plan) Partition Reducer Demarcate logical unit of work around partition processing Partition Collector Sends interim results from individual partition to step's partition analyzer Partition Analyzer Collection point of interim results, single point of control and analysis 15 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

16 Flow and Split Flow defines a set of steps to be executed as a unit <flow id= flow-1" next= {flow, step, decision}-id > <step id= flow_1_step_1 > </step> <step id= flow_1_step_2 > </step> </flow> Split defines a set of flows to be executed in parallel <split > <flow./> <! each flow runs on a separate thread --> <flow./> </split> 16 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

17 Batchlet Task oriented unit of work Doesn't support resume Not item-oriented Simple Batchlet interface process(), stop() In job xml: <batchlet ref= {name} /> 17 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

19 Batch Runtime Specification JobOperator interface for programmatic lifecycle control of jobs Step level Metrics through the StepExecution object Batch Artifact loading through META-INF/batch.xml Job XML loading through META-INF/batch-jobs/my-job.xml Packaging jar, war, ejb-jar 19 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

20 Example: Programmatic Invocation of Jobs In a Servlet/EJB: import javax.batch.runtime.batchruntime;... JobOperator jo = BatchRuntime.getJobOperator(); jo.start("myjob", new Properties()); Copyright 2012, Oracle and/or its affiliates. All rights reserved.

21 Example: META-INF/batch-jobs/my-job.xml <job id="myjob" xmlns=" <step id="mystep"> <chunk item-count="3"> <reader ref="myitemreader"/> <processor ref="myitemprocessor"/> <writer ref="myitemwriter"/> </chunk> </step> </job> If in a WAR, this would be WEB-INF/classes/META-INF/batchjobs/my-job.xml 21 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

JBatch with Apache BatchEE. Mark Struberg / INSO, TU Vienna

JBatch with Apache BatchEE. Mark Struberg / INSO, TU Vienna JBatch with Apache BatchEE Mark Struberg / INSO, TU Vienna About the Speakers http://github.com/struberg http://people.apache.org/~struberg Apache Software Foundation member Apache OpenWebBeans, MyFaces,