PCJ-BLAST massively parallel sequence alignment using NCBI Blast and PCJ Java library

Transcription

1 PCJ-BLAST mssively prllel sequece ligmet usig NCBI Blst d PCJ Jv librry Piotr Bł Mrek Nowicki Dvit Bzhlv bl@icm.edu.pl ICM Uiversity of Wrsw frmir@mt.umk.pl ICM Uiversity of Wrsw N. Copericus Uiversity dvit.bzhlv@ki.se Krolisk Istituet

2 Blst Sequece ligmet is essetil for NGS There is umber of softwre pckges for sequece ligmet bsed o vrious similrity serch methods BLAST Bsic Locl Aligmet Serch Tool (1991) The heruistic lgorithm it uses is much fster th other pproches The serch time c be log (dys or weeks) for lrge dtsets NCBI blst is the most widely used implemettio There is strog iterest i usig lrge computer systems to ru blst Blst ruig o cloud or grid Prllel versios of blst ruig o HPC systems 2

3 Fst iput (2 reds out of 1 milio) >C _2.0 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAACACTTTGGGAGGCCAAGATGGGAAGATCTTTTGAGG CCAGGCGTTCAAGACCAGTCTGGGCAATATGGAGAGACCT GTCTCTACAAAAAAAATTAGCCAGACCTAGTGGCTGGCTGA GGCAGGAGGATCATCTGAGCTCAGGAGATTGAGAT >C _2.0 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAATTTTAAGTTTTTCCTTATAAGGGAAATAAGCTTATTTCT TTTAGAAGCAGAAACTGCAAATGGGGGTGTATGGAATTCTTT TTTTTTTTTTTTTTTTTGCGACAAGGTCTCACTGTGTTGCCCA GACTGGAGTGCAGTGGTGCAATCATAGCT 3

4 Time [s] NCBI blst performce (XC40 sigle ode) 1000 Time of processig 48 sequeces t oce usig 1 istce of differet versio of NCBI BLAST with differet umber of BLAST threds Number of BLAST threds cbi-blst cbi-blst cbi-blst cbi-blst _gcc_5.3 4

5 Blst processig time of reds chges 800 Processig time [s]

6 Prllel blst implemettios mpiblast uses old versio (2012) of blst prllel_blst.py limited to sigle ode, regulr split of iput Prcel BLAST (bsed o NCBI blst ) costly! The BlueGee implemettio limited to IBM BGQ systems pioblast prllel versio which uses MPIIO memory problems while referece dtbse is lrge SprkBLAST Apche Sprk implemettio ruig o AWS cloud (PhD thesis 2016) 6

7 Pcj-blst Prllel versio developed usig PCJ (Prllel Computig i Jv librry developed t ICM) Jv code is resposible for redig fst iput d execute NCBI-blst o set of sequeces (reds) The NCBI blst (curretly ) is executed with the -um_threds optio For 28 core odes: 4 PCJ threds with um_threds=7 The solutio c be executed o y multiode system with Jv 8 istlled The I/O performce is importt thousds of blst istces reds blst dtbse (52GB) cocurretly lustr filesystem performs worse th fs 7

8 Pcj-blst Reds iput sequece (fst) lie by lie Blocks of lies (reds) re used s iput for blst query Pcj-blst c strt hudreds of blst istces NCBI blst is used pcj-blst ot bouded to prticulr versio Oe or more NCBI blst istces re ru o the hrdwre ode Output from ech ode (text or XML) is gthered Output is postprocessed d is stored i sigle file Postprocessig is prllelized Pcj-blst c be ru o HPC systems, clusters Requires Jv 8 d NCBI blst istlled 8

9 pcj-blst 9

10 Multiple NCBI blst ruig o sigle ode (X86 cluster) 10

11 I/O performce (x86 sigle file red) 11

12 I/O performce (XC40 sigle file red) 12

13 pcj-blst sclbility (x86 cluster) 13

14 pcj-blst sclbility (Cry XC40) 14

15 PGAS lguges Idepedet threds Locl d shred (globl) vribles Globl vribles visible to ll threds Oe-sided commuictio Commuictio detils hidde to progrmmer Mi opertios: Sychroiztio (brrier) get put 15

16 PCJ Librry Jv librry pcj.icm.edu.pl Desiged bsed o PGAS (Prtitioed Globl Address Spce) prdigm Simple d esy to use Does ot itroduce extesios to the Jv lguge o ew compilers or pre-processor Does ot require dditiol librries Works with Jv 1.8, 1.9 Versio for Jv 1.7 vilble Good performce Good sclbility (lredy beyod 50k+ cores) 16

17 PCJ thred 0 PCJ thred 1 PCJ thred 2 PCJ thred 3 PCJ thred 4 PCJ thred 5 PCJ thred 6 PCJ thred 7 PCJ memory lyout Physicl ode Physicl ode JV M JV M Shred vribles Locl vribles CPU CPU CPU CPU CPU CPU CPU CPU 17

18 PCJ thred 0 PCJ thred 1 PCJ thred 2 PCJ thred 3 PCJ thred 4 PCJ thred 5 PCJ thred 6 PCJ thred 7 PCJ memory lyout d commuictio Physicl ode Physicl ode JV M JV M Shred vribles Locl vribles CPU CPU CPU CPU CPU CPU CPU CPU 18

19 PCJ Prllel Computtios i Jv Mi fuctiolity: Sychroiztio Get dt from other thred (get) Sed dt to other thred (put) Additiol fuctiolity: Brodcst Moitorig of the vrible chge (useful with put()) Prllel I/O Groups of threds 19

20 PCJ Hello world import org.pcj.* public clss PcjHelloWorld implemets StrtPoit public void mi() { System.out.pritl("Hello!"); } } public sttic void mi(strig[] rgs) { PCJ.deploy(PcjHelloWorld.clss, } ew NodesDescriptio("odes.txt")); 20

21 PCJ mi eum Shred {, rry } // Defie storge if (PCJ.myId()==0) c =(double) PCJ.get(3, Shred.); if (PCJ.myId()==0) PCJ.put(5.0, 3, Shred.); if (PCJ.myId()==0) PCJ.put(5.0, 3, Shred.rry, p); if (PCJ.myId()==0) PCJ.brodcst(5.0, Shred.); public sttic void PCJ.brrier(); public sttic itpcj.thredcout() public sttic itpcj.myid() 21 16/05/2017 Piotr Bł

22 PCJ red dt eum Shred { } double ; // Defie storge double l; if (PCJ.myId()==0) { Scer s = ew Scer(System.i); l = s.extdouble(); PCJ.brodcst(l., Shred.); } else { PCJ.witfor(Shred.); } PCJ.brier() 22 16/05/2017 Piotr Bł

23 PCJ Gther usig (PcjGther.clss) eum Shred { } double PcjFuture L[] = ew PcjFuture[PCJ.thredCout()]; double sum = 0.0; if (PCJ.myId() == 0) { for (it p = 0; p < PCJ.thredCout(); p++) { L[p] = PCJ.sycGet (p, Shred.); } for (it p = 0; p < PCJ.thredCout(); p++) { sum = sum + (double) L[p].get(); } } 23 16/05/2017 Piotr Bł

24 Grph500 kerel 2 (BFS) Cry XC40 24

25 Cofigurtios per secod [1/s] Neurl etwork optimiztio usig GA (coectom of C. Elgs) Eergy , Cofigurtios/Time Eergy 0,5 0,4 0,3 1 0, Number of threds 0,2 0,1 0 25

26 PCJ vs Apche Hdoop ICM) 26 Pi Estimte (blue), BFS (orge), WordCout (red)

27 PCJ-BLAST mssively prllel sequece ligmet usig NCBI Blst d PCJ Jv librry PCJ: pcj.icm.edu.pl Piotr Bł Mrek Nowicki Dvit Bzhlv bl@icm.edu.pl ICM Uiversity of Wrsw frmir@mt.umk.pl ICM Uiversity of Wrsw N. Copericus Uiversity dvit.bzhlv@ki.se Krolisk Istituet