Hinri Kerstens. NGS pipeline using Broad's Cromwell

Transcription

1 Hinri Kerstens NGS pipeline using Broad's Cromwell

2 Introduction Princess Máxima Center is a organization fully specialized in pediatric oncology. By combining the best possible research and care, we will be able to heal more children in the future. That is our mission! Currently about 150 cancer patients are treated on yearly basis. From 2018 onward we are expecting a total of ~700 patients a year: ~550 new cases and ~ relapses. For all patients we will perform genome sequencing on primary/ metastatic tumors and controls. Targeted sequencing depths range from 50X for normal to 150x for tumors.

3 Genomics environment External Data End User interfaces Curation / Morphing Repositories/Archival Lab Experiments LIMS / Metadata Clinical Data API calls Storage and workflow infrastructure

4 Workflow infrastructure Persistent analyses and scale out to the cloud API Workflow execution service + database

5 Analyses Workflows Until recently, analyses workflows had major shortcomings not readable for human beings backend specific (not very well portable) laborious to write/implement/maintain/debug With the advance of human readable Workflow Languages and their executers this has changed.. Now a audience can build robust pipelines that understand things like parallelism, dependencies of inputs and outputs between tasks, and resume intelligently if they get interrupted.

6 Supported Parallelisms Users are not required to understand parallelisms in detail, but just can make use of them.

7 Workflow and tasks Workflow = calls to set of tasks The order in which the workflow block, task definitions, or call task statements are arranged does not matter. The are inferred by the executer based in the task in and outputs. Task components

8 Task level variables A task can accept variables from a call Define variables in the workflow and apply them in the task call statements java -jar wdltool.jar validate myworkflow.wdl

9 Workflow inputs for a particular run { "myworkflowname.my_input": "~/path/to/input.bam, "myworkflowname.name": "NA12878, "myworkflowname.my_ref": ~/path/to/grch38.fa }

10 Validate syntax and run java jar wdltool.jar validate myworkflow.wdl java -jar Cromwell.jar run myworkflow.wdl myworkflow_inputs.json

11 Progress status log cromwell-executions 3096c720-a b-a348-ea4dd99118bb call-getbwaversion rc script script.submit stderr stdout version call-getpicardversion rc script call-samtofastqandbwamem

12 Progress and status log scatter cromwell-executions 3096c720-a b-a348-ea4dd99118bb call-getbwaversion rc.. call-getpicardversion rc call-samtofastqandbwamem shard-0 rc script script.submit stderr stdout shard-1 rc.. shard-2

13 Progress and status log inputs outputs cromwell-executions 3096c720-a b-a348-ea4dd99118bb call-getbwaversion call-getpicardversion call-samtofastqandbwamem shard-0 rc script script.submit stderr stdout C2CPVACXX_0_4_none.unmerged.bam C2CPVACXX_0_4_none.unmerged.bwa.stderr.log /path/to/c2cpvacxx_0_4_none.bam (ubam input) Homo_sapiens_assembly38.dict Homo_sapiens_assembly38.fasta These are copies! shard-1

14 Workflow options Persistent workflow results Enables resume of failed workflows (not getting to work yet) Lots of well structures workflow logging database { // This specifies which database to use config = main.mysql main { mysql { driver = "slick.driver.mysqldriver$" db { driver = "com.mysql.jdbc.driver" url = "jdbc:mysql://localhost:3306/cromwell_m" user = "cromwell" password = "cromwell" connectiontimeout = 5000 } }

15 Cromwell database (job store)

16 Adaptation to our HPC environment On big machines running the docker image provided by Broad Institute, example workflows will run without modifications Management of field specific software missing: LMOD environmental module system Job Scheduler, resources missing: Management of requestable resources: cores, memory, runtime, scratch

17 Software modules # Read unmapped BAM, convert on-the-fly to FASTQ and stream to BWA MEM for alignment task SamToFastqAndBwaMem { String module_java_version String module_picard_version String module_bwa_version String module_samtools_version File input_bam command <<< module load ${module_java_version} module load ${module_picard_version} module load ${module_bwa_version} module load ${module_samtools_version} java -Xmx4G -jar $PICARD \ SamToFastq \ INPUT=${input_bam} \ FASTQ=/dev/stdout \ INTERLEAVE=true \ NON_PF=true \ ${bwa_commandline} /dev/stdin - \ samtools view -1 - > ${output_bam_basename}.bam >>> # WORKFLOW DEFINITION Workflow myworkflownmae { String module_bwa_version String module_java_version String module_picard_version String module_samtools_version # Map reads to reference call SamToFastqAndBwaMem { input: module_java_version = module_java_version, module_picard_version = module_picard_version, module_bwa_version = module_bwa_version, module_samtools_version = module_samtools_version,

18 Software versions as workflow inputs myworkflowname.inputs.json { "##_COMMENT6": "MODULES", "TestFlow.module_bwa_version": "bwa/1.0", "TestFlow.module_java_version": "Java/1.8.0_60", "TestFlow.module_picard_version": "picardtools/2.5.0", "TestFlow.module_samtools_version": "samtools/1.3 }

19 Requestable resources # Sort BAM file by coordinate order and fix tag values for NM and UQ task SortAndFixSampleBam { String module_java_version String module_picard_version File input_bam Int tmp_space String wallclock command { module load ${module_java_version} module load ${module_picard_version} java -Djava.io.tmpdir=$TMPDIR -Xmx4G -jar $PICARD \ SortSam \ runtime { cpu: "2" memory: "6 GB" tmp_space: "${tmp_space}" wallclock: "${wallclock}" } Values for tmp_space and wallclock are task specific but might need modification with data size

20 Workflow inputs with task specific values { } myworkflowname.markduplicates.tmp_space": 200, myworkflowname.sortandfixsamplebam.tmp_space": 400, myworkflowname.baserecalibrator.tmp_space": 4 task MarkDuplicates { Int tmp_space command {... } } task BaseRecalibrator { Int tmp_space command {... } } task SortAndFixSampleBam { Int tmp_space command {... } } workflow myworkflowname { call MarkDuplicates {} call BaseRecalibrator {} call SortAndFixSampleBam {} }

21 Tell the executer about these backend resources backend { default = SGE SGE { config { runtime-attributes = """ Int? cpu Int? memory_gb String? tmp_space = "1" String? wallclock = "00:1:00" "" submit = """ qsub \ -terse -b n -N ${job_name} \ -wd ${cwd} \ -o ${out} \ -e ${err} \ -pe threaded ${cpu} -l h_vmem=${memory_gb}g,tmpspace=${tmp_space}g,h_rt=${wallclock} \ ${script} """

22 Cromwell server

23 Work in progress Succesfull resubmit/resumes Testing more recent version: tested: cromwell-0.20-ff3bb7a-snapshot

24 WDL features

25 Lineair chain call stepb { input: in=stepa.out } call stepc { input: in=stepb.out }

26 Multi-input/Multi-output call stepc { input: in1=stepb.out1, in2=stepb.out2 }

27 Branche & Merge call stepb { input: in=stepa.out } call stepc { input: in=stepa.out } call stepd { input: in1=stepc.out, in2=stepb.out }

28 Scatter-Gather Parallelism Array[File] inputfiles #explicit array scatter (onefile in inputfiles) { call stepa { input: in=onefile } } call stepb { input: files=stepa.out } #implicit array

29 Task Aliasing call stepa as firstsample { input: in=firstinput } call stepa as secondsample { input: in=secondinput } call stepb { input: in=firstsample.out } call stepc { input: in=secondsample.out }