Portability, Scalability, and Numerical Stability in Accelerated Kernels

Transcription

1 Portility, Slility, nd Numril Stility in Alrtd Krnls John Strtton Dotorl Cndidt: Univrsity of Illinois t Urn-Chmpign Snior Arhitt: MultiorWr In

2 Outlin Prformn Portility Wht CPU progrmmrs nd to lrn from GPU omputing Corollry/Tkwy: Most dvloprs should only writ thir od on Building roust high-prformn prlll ods Som things GPU progrmmrs nd to lrn from th rih CPU lirry dvlopmnt history Corollry/Tkwy: Prformn isn t vrything

3 High-prformn GPU Softwr Slility Thousnds of thrds Nds Algorithms w hoos now hv to vry prlll to lst for yrs to om Lolity Rndomly ssd glol mmory is slow Rgulrity SIMD mttrs lot

4 High-prformn CPU Softwr Nds Slility Numr of ors ontinus to grow Lolity Rndomly ssd glol mmory is still slow Rgulrity x86 SIMD widths trnding up in prtiulr 4

5 Simplisti Arhittur Comprison GPU CPU Chip Prossing Unit (SM) Lol Mm/Ch Rgistrs SIMD Unit Thrding & Control Thrd prlllism Vtor prlllism Chip Cor Lol Ch Rgistrs SIMD Unit Thrding & Control 5

6 So, why writ two vrsions of od? You ll do ttr jo th sond tim. A fst CPU vrsion is sy to writ so it dosn t rlly ount. I gt pid pr lin of od writtn, vn if it just implmnts duplit fturs. </srsm> GPUs nd CPUs hv inomptil ids out thrds, nd rltionships twn thrd- nd instrution-lvl prlllism. 6

7 Wht if thr wr tools to Lt th progrmmr nsur lolity, SIMDfrindlinss, t. Adpt tsk grnulrity to trgt rhittur nd tsk shdulr Mgi? Biggst portility hurdl Gnrts multithrdd C or x86 od just lik norml ompilr Squntil Trgt Cors-Grind Thrdd Trgt Multithrdd Vtor Trgt 7

8 Thr r OpnCL implmnttions for x86 (Intl, AMD, MultiorWr) PGI CUDA-x86 Compilr MCUDA Som r ttr thn othrs: don t judg th prinipl sd on on immtur tool 8

9 Admi Proof of Conpt Intl Prlll MKL Prformn (riprol runtim) of MCUDA-trnsltd pp, normlizd to hndprlllizd CPU od. 9

10 With sll prlllism Spdup ovr singl or Idl CP MRI LBM TPACF BLINN MM PNS Numr of ors utilizd

11 Why this works wll High-prformn progrmming is ll out Mssiv Prlllism Lolity nd Hirrhy Rgulrity Oviously tru for GPUs Oviously tru for lustrs Boming mor tru for CPUs h yr Good CUDA optimiztions r oftn good CPU optimiztions Good CPU lgorithms r oftn good GPU lgorithms

12 *Thnks to Li-Wn Chng & Ry Sung for som of th ontnt in this stion LEARNING TO WRITE LIBRARIES FROM THE EXPERTS

13 GPU Tridigonl Systm Solvr Cs Study Hyrid Mthods PCR-Thoms (Kim, Dvidson ) CR-PCR (CUSPARSE ) Et CPU lirris us non of ths: Numrilly unstl Thoms (squntil) Cyli Rdution ( stp) PCR ( stp)

14 Numril Stility An lgorithm is numrilly stl if it n lwys find n pproprit solution to th prolm for ny givn input vlus, ssuming on xists. Algorithms tht fll short of this rquirmnt r rfrrd to s numrilly unstl. 4

15 Exmpls of numril instility Algorithms tht don t hk for divid y zro nn nn Limitd ility to rprsnt prision nd sl - - inf 5

16 Pivoting: or stility thniqu Judiiously swp rows (or olumns) to void d ss - Swp rows Elimint - Inhrntly squntil lgorithm: w nd mor prlllism 6

17 SPIKE Prtitioning Algorithm A numrilly stl mthod for domposing ndd systm: A X = F Algrilly dompos A into D nd S: D S X = F Comput D - Solv y tils Solv S X = F 7

18 SPIKE Prtitioning lgorithm Crting S is just two mor tild invrs tridigonl systm prolms (DV=F nd DW=F) Cn solvd in prlll Solving S X = Y is muh sir prolm us of th mtrix strutur Tks t most 5% of totl solution tim 8

19 Put th stl squntil lgorithm insid h GPU thrd Eh thrd will pross on til y itslf with squntil, numrilly stl pivoting lgorithm Two prolms Eh thrd s first, sond, t. lmnt r fr wy from th nxt thrd s orrsponding lmnt of its own til, rsulting in lrg-stridd sss Eh thrd onsums dt from its til t diffrnt rt sd on its pivoting disions 9

20 Tils Prossd y Eh Thrd Eh til: Lyout of ll tils: (similr to n rry of struturs lyout) Lt s do trnspos! Out of pl? X mmory ovrhd. In pl? Gnuinly diffiult for ritrry sizs.

21 In-pl Trnspostion: simpl s // dt[w][h]-->dt[h][w] prlll for (j<w) prlll for (i<h) flot tmp = dt[j][i]; //offst = j*h + i

22 In-pl Trnspostion: First Attmpt // dt[w][h]-->dt[h][w] prlll for (j<w) prlll for (i<h) flot tmp = dt[j][i]; //offst = j*h + i rrir();

23 In-pl Trnspostion: First Attmpt // dt[w][h]-->dt[h][w] prlll for (j<w) prlll for (i<h) flot tmp = dt[j][i]; //offst = j*h + i rrir(); dt[i][j] = tmp; //offst = i*w + j Wht if th dtst is lrgr thn onhip mmory?

24 Anothr Dt Lyout Altrntiv divid into tils 4

25 ASTA Dt Lyout 5

26 AoS to ASTA Trnsformtion AoS to ASTA Mrshling Krnl Glol Mmory Throughput (GB/s) Fin Print Out-of-Pl 8 x Sp In-Pl Brrir Syn 95 Til Siz (tunl) < On-hip Mmory Wht if til siz > on-hip mmory pity? S Sung t l. DL: A Dt Lyout Trnsformtion Systm for Htrognous Computing, InPr 6

27 Dynmi tiling 7

28 Finl prformn rsults 8

29 Summry Lrn good lgorithms nd good optimiztions Stt-of-th-rt CPU lgorithms r grt pl to strt for writing roust GPU lirris Stt-of-th-rt GPU optimiztions r grt pl to strt for writing fst CPU od High-prformn prlll omputing mjor prolms nd thniqus for solving thm tht r prtty ommon ross rhitturs Mmory Lolity -> Tiling & Lyout Exution Slility -> Effiint, Prlll Algorithms Limitd Prision Computtion -> Stl Algorithms 9

30 Mor informtion? MCUDA: Strtton t l. Effiint Compiltion of Fin-grind SPMD-thrdd Progrms for Multior CPUs, CGO ' Not urrntly mintind. MxPA: S MultiorWr prss rls, ontt info@multiorwrin.om for mor informtion &id=74&itmid=86 Tridigonl Solvr Lirry: oming up in SC, Chng t l. A Sll, Numrilly Stl, High-prformn Tridigonl Solvr using GPUs