Programming the Cell BE

Size: px

Start display at page:

Download "Programming the Cell BE"

Opal Gordon
5 years ago
Views:

1 Programming the Cell BE Second Winter School Geilo, Norway André Rigland Brodtkorb SINTEF ICT Department of Applied Mathematics

2 Outline 1 Briefly about compilation. 2 Hello World: using one, using all s, using all s and Direct Memory Access (DMA). 3 Discrete Cosine Transform (JPEG). 4 Summary Workshop-like presentation with real code:) You can download the code from now! A. R. Brodtkorb Applied Mathematics 2/53

3 Compilation You need a PS3, a Cell blade server, a Mercury Cell card, or a Cell simulator. You need a special compiler (IBM XL C/C++ or modified GCC). code is embedded into PPE code. Embedded code is linked together with the PPE code to one executable. Keep it in your mind that the compiler can contain bugs. A. R. Brodtkorb Applied Mathematics 3/53

4 Hello World CPU OS Hello World! 1 Run PPE program 2 PPE program starts the 3 The writes Hello World A. R. Brodtkorb Applied Mathematics 4/53

5 Hello World on one. We need a datastructure to store program information: Entry point (address). Arguments. ID of context. We need code that will execute on the. We need to create the contexts, and start them on the PPE. A. R. Brodtkorb Applied Mathematics 5/53

6 HelloWorld.cpp (PPE) 0 typedef struct{ spe_context_ptr_t speid; //< Identifier for context void* argp; //< Pointer to arguments void* envp; //< Pointer to environment unsigned int entry; //< Entry point of the spe 5 } helloworldcontext; extern spe_program_handle_t HelloWorld_spe; i n t main ( i n t argc, char argv ) { 10 spe_stop_info_t stop_ info ; helloworldcontext context ; i n t sta tu s ; unsigned i n t f l a g s = _EVENTS_ENABLE ; 15 context.speid = spe_context_create(flags, NULL); context. argp = NULL; //Not used context. envp = NULL; //Not used context. en t ry = _DEFAULT_ENTRY; 20 status = spe_program_load(context.speid, &HelloWorld_spe); status = spe_context_run(context.speid, &context.entry, 0, context.argp, context.envp, &stop_info); spe_context_destroy(context.speid); } A. R. Brodtkorb Applied Mathematics 7/53

7 Running the program HelloWorld.cpp - excerpt (PPE) 0... s t a t u s = spe_context_run (context.speid, &context. entry, 0, context.argp, context.envp, &stop_info ) ;... HelloWorld_spe.cpp () 0 i n t main ( unsigned long long speid, unsig ned lo ng lo ng argp, unsig ned lo ng lo ng envp) { p r i n t f (" Hello world!\ n" ) ; r e turn 0; 5 } A. R. Brodtkorb Applied Mathematics 9/53

8 Summary one We have a struct that keeps track of the context. The PPE creates the context, and loads the program into that context. The PPE runs the context, and waits for it to complete. A. R. Brodtkorb Applied Mathematics 10/53

9 Hello world from All Hello World Hello World Hello World CPU Hello World OS Hello World Hello World 1 Run PPE program 2 PPE program starts all s using pthreads 3 All s write Hello World A. R. Brodtkorb Applied Mathematics 11/53

10 Hello World using all s Find out how many s are available. Use pthreads on the PPE to execute different contexts. Wait for the context to complete, and thus the PPE thread to join. HelloWorldFromAll.cpp - helloworldcontext (PPE) 0 typedef s t r u c t { spe_context_ptr_t speid ; //< I d e n t i f i e r for context pthread_t threadid; //< Identifier for PPE thread v o i d argp ; //< Po in t er to arguments sent to the spe v o i d envp ; //< Po in t er to environment sent to the spe 5 unsigned i n t entry ; //< Entry point of the spe } helloworldcontext ; A. R. Brodtkorb Applied Mathematics 13/53

11 HelloWorldFromAll.cpp - main (PPE) 0 i n t main ( i n t argc, char argv ) { unsigned i n t f l a g s = _EVENTS_ENABLE ; i n t sta tu s ; 5 int n = spe_cpu_info_get(_count_physical_s,-1); helloworldcontext* contexts = new helloworldcontext[n]; f or ( i n t i =0; i<n ; ++i ) { // Create context and load program contexts [ i ]. speid = spe_context_create ( flags, NULL); 10 contexts [ i ]. argp = NULL; //Not used contexts [ i ]. envp = NULL; //Not used contexts [ i ]. entry = _DEFAULT_ENTRY; status = spe_program_load(contexts[i].speid, &HelloWorld_spe); 15 } f or ( i n t i =0; i<n ; ++i ) { //Run the contexts status = pthread_create(&contexts[i].threadid, NULL, &create_spe_thread, &contexts[i]); } 20 f or ( i n t i =0; i<n ; ++i ) { //Wait for pthreads to j o i n status = pthread_join(contexts[i].threadid, NULL); } 25 } d e l e t e [ ] contexts ; A. R. Brodtkorb Applied Mathematics 15/53

12 Pthread function HelloWorldFromAll.cpp - create_spe_thread (PPE) 0 void create_spe_thread ( void context_ ) { spe_stop_info_t stop_ info ; helloworldcontext context ; i n t sta tu s ; 5 context = ( helloworldcontext ) context_ ; status = spe_context_run(context->speid, &context->entry, 0, context->argp, context->envp, &stop_info); 10 spe_context_destroy ( context >speid ) ; } pthread_exit (NULL); A. R. Brodtkorb Applied Mathematics 17/53

13 Summary all s 1 The PPE creates the contexts, and loads the programs. 2 Then, the PPE creates one pthread per physical. 3 The pthread executes the create_spe_thread function, which runs the context. 4 The PPE thread exits when the context has completed. A. R. Brodtkorb Applied Mathematics 18/53

14 Hello world with Direct Memory Access (DMA) Hello World from 6 Hello World from 5 CPU Hello World from 4 Hello World from 3 OS Hello World from 1 Hello World from 2 1 Run PPE program 2 PPE program starts all s 3 All s DMA an integer over to local store 4 All s write Hello World from %i A. R. Brodtkorb Applied Mathematics 19/53

15 Hello World with DMA. DMA - Direct Memory Access. Explicit data movement, or software cache. The s (16 long queue per ) are more efficient than the PPE (8 long queue). 16 byte alignment required (128 preferable). DMA size must be 1, 2, 4, 8 or n 16 byte (multiple of 128 preferable). Maximum DMA size is 16 Kbyte Assembly ( intrinsics). A. R. Brodtkorb Applied Mathematics 20/53

16 Allocating aligned data HelloWorldDMA.cpp - excerpt (PPE) 0 i n t main ( i n t argc, char argv ) {... f or ( i n t i =0; i<n ; ++i ) { // Create context and load program contexts [ i ]. speid = spe_context_create ( flags, NULL); contexts[i].argp = malloc_align(sizeof(int), 16); 5 contexts [ i ]. envp = NULL; //Not used contexts [ i ]. entry = _DEFAULT_ENTRY; } int* spe_argp = (int*) contexts[i].argp; *spe_argp = i; stat us = spe_program_load ( contexts [ i ]. speid, &HelloWorldDMA_spe ) ; }... A. R. Brodtkorb Applied Mathematics 22/53

17 Direct Memory Access HelloWorldDMA_spe.cpp () 0 i n t main ( unsigned long long speid, unsig ned lo ng lo ng argp, unsig ned lo ng lo ng envp ) { int i attribute ((aligned(16))); 5 spu_writech(mfc_wrtagmask,-1); void* lsa = (void*) &i; unsigned intea = argp; unsigned int size = sizeof(int); 10 unsigned int tagid = 0; unsigned int cmd = MFC_GET_CMD; 15 spu_mfcdma32(lsa, ea, size, tagid, cmd); spu_mfcstat(mfc_tag_update_all); p r i n t f (" Hello world from %d!\ n", i ) ; } r e turn 0; A. R. Brodtkorb Applied Mathematics 24/53

18 Summary DMA DMA on the Cell requires aligned memory! DMA does not stall the processor. The s can queue more DMA commands than the PPE. Compute whilst waiting for data. A. R. Brodtkorb Applied Mathematics 25/53

19 MJPEG compression using the Cell BE Algorithm 1 Divide the input image into 8 8 blocks 2 For each block 1 Convert from discrete pixels to discrete cosines (DCT). 2 Quantize (element division by a fixed 8 8 matrix). 3 Run through the result matrix in a zigzag pattern, and store nonzero coefficients. 4 Huffman code the vector. A. R. Brodtkorb Applied Mathematics 26/53

20 (M)JPEG algorithm (1)Original (2)DCT (3)Quantize A. R. Brodtkorb Applied Mathematics 27/53

21 DCT and quantization p(u,v) = α(u)α(v) 7 7 i=0 j=0 r(u,v) = p(u,v)/q(u,v) { 1/8,n = 0 α(k) = 2/8,n 0 [ π ] [ π ] g i,j cos 8 (i + 0.5) u cos 8 (j + 0.5) v Use precomputed cosine values (they are expensive and reused). Use SIMD instructions that operate on 4 floats. Quantization is elementwise division with a quantization matrix. A. R. Brodtkorb Applied Mathematics 28/53

22 Main idea Each computes one row of 8 8 blocks. Input data is unsigned char (1byte). Output data is signed short (2byte). Must take care that each gets proper alignment for DMA. DCT and quantization is trivial, but not when optimizing using SIMD. A. R. Brodtkorb Applied Mathematics 29/53

23 Creating contexts Constructor - excerpt (PPE) 0 spe_argp = ( dct_args ) malloc_align ( spes s i z e o f ( dct_args ), 16); dctcontext. r e s i z e ( spes ) ; f or ( i n t i =0; i<spes ; ++i ) { // Create contexts 5 dctcontext[i].speid = spe_context_create(flags, NULL); spe_program_load(dctcontext[i].speid, &CellDctQuant_spe); dctcontext [ i ]. argp = ( void ) &spe_argp [ i ] ; dctcontext [ i ]. envp = NULL; dctcontext [ i ]. entry = _DEFAULT_ENTRY; 10 // Set up the arguments f o r the spe thread spe_argp [ i ]. width = width ; spe_argp [ i ]. height = height ; spe_argp [ i ]. padwidth = padwidth ; 15 spe_argp [ i ]. padheight = padheight ; spe_argp [ i ]. quanttbl = qu ant iza tion ; } A. R. Brodtkorb Applied Mathematics 31/53

24 Running contexts DctQuantize - excerpt (PPE) 0 whi le ( remainding_block_lines > 0) { i n t spe_threads = min( spes, remainding_block_lines ) ; f or ( i n t i =0; i<spe_threads ; ++i ) { // Create threads offset_ to_ block_ line = BLOCK_SIZE ( computed_block_lines+i ) ; spe_argp[i].input = &(indata[width*offset_to_block_line]); 5 spe_argp [ i ]. output = &(outdata [ padwidth offset_to_block_line ] ) ; 10 } status = pthread_create(&dctcontext[i].threadid, NULL, &create_spe_thread, &dctcontext[i]); remainding_block_lines ; f or ( i n t i =0; i<spe_threads ; ++i ) { // Join threads pthread_join(dctcontext[i].threadid, NULL); computed_block_lines++; 15 } } A. R. Brodtkorb Applied Mathematics 33/53

25 Arguments to the CellDctQuant.h - excerpt (/PPE) 0 typedef struct{ intwidth attribute ((aligned(16))); intheight attribute ((aligned(16))); intpadwidth attribute ((aligned(16))); int padheight attribute ((aligned(16))); 5 unsigned char* input attribute ((aligned(16))); signed short* output attribute ((aligned(16))); unsigned char* quanttbl attribute ((aligned(16))); } dct_args; //To passargumentsto the 10 A. R. Brodtkorb Applied Mathematics 35/53

26 Acquiring the argument struct CellDctQuant_spe.c - excerpt () 0 5 i n t main ( unsigned long long speid, unsig ned lo ng lo ng argp, unsig ned lo ng lo ng envp ) { dct_args args attribute (( a lign ed (1 6 ) )) ;... // misc v a r i a b l e defs. l sa = ( void ) &args ; ea = argp ; 10 s i z e = s i z e o f ( dct_args ) ; tagid = 0; cmd = MFC_GET_CMD; spu_mfcdma32(lsa, ea, size, tagid, cmd); spu_mfcstat(mfc_tag_update_all); A. R. Brodtkorb Applied Mathematics 37/53

27 Acquiring the quantization table CellDctQuant_spe.c - cont excerpt () 0... unsigned char quanttbl [6 4 ] attribute (( a lign ed (1 6 ) )) ; //The qu ant iza tion t ab le s i z e = 64 s i z e o f ( unsigned char ) ; 5 l sa = ( void ) quanttbl ; ea = ( unsigned i n t ) args. quanttbl ; tagid = 1; cmd = MFC_GET_CMD; spu_mfcdma32(lsa, ea, size, tagid, cmd); A. R. Brodtkorb Applied Mathematics 39/53

28 Acquiring input data Width % 16 -> Input alignemnt issues (4 bpp / 1 byte). Width % 8 -> Output alignment issues (8 bpp / 2 byte). Outputsize is always % 8, we are lucky:) Must ensure start address is aligned, and size is a multiple of quadword. Quadword boundaries A. R. Brodtkorb Applied Mathematics 40/53

29 Acquiring input data (cont.) CellDctQuant_spe.c - excerpt cont. () 0... unsigned char input attribute (( aligned ( 1 6 ) ) ) ; o f f s e t = (( i n t ) args. input ) % 16; //< o f f s e t to 16 s i z e = args. width height //number of elements 5 s i z e o f ( unsigned char ) // bytes per element + o f f s e t ; // our 16 boundary o f f s e t s i z e = ( unsigned i n t ) c e i l ( s i z e /( f l o a t ) 16) 16; //Pad void input_tmp = malloc_align ( size, 16); 10 l sa = input_tmp ; ea = (unsigned int) args.input-offset; tagid = 2; cmd = MFC_GET_CMD; spu_mfcdma32 ( lsa, ea, size, tagid, cmd ) ; 15 input = input_tmp + offset;... A. R. Brodtkorb Applied Mathematics 42/53

30 Preparing for SIMD CellDctQuant.h - excerpt (/PPE) 0 typedef union{ vector float v; float s[4]; } v4f; //For intrinsics A. R. Brodtkorb Applied Mathematics 44/53

31 Precomputing sines using SIMD CellDctQuant_spe.c - excerpt cont. () 0 vector f l o a t cosuv [ 8 ] [ 8 ] [ 8 ] [ 2 ] attribute (( aligned ( 1 6 ) ) ); f or ( v = 0; v < 8; v++) { f or (u = 0; u < 8; u++) { vector float uvec = spu_splats((float) u); 5 f o r ( j = 0; j < 8; j++) { //v (2 j +1) pi /16 vector f l o a t vjvec = spu_splats (( f l o a t ) v (2 j +1) pi16 ) ; //(2 i +1) pi /16, i = vector f l o a t ivec1 = {1 pi16, 3 pi16, 5 pi16, 7 pi16 }; //(2 i +1) pi /16, i =4..7 vector f l o a t ivec2 = {9 pi16, 11 pi16, 13 pi16, 15 pi16 }; 15 // cos (u (2 i +1) PI /16) cos ( v (2 j +1) PI /16) cosuv[v][u][j][0] = spu_mul(cosf4(spu_mul(ivec1, uvec)), cosf4(vjvec)); cosuv[v][u][j][1] = spu_mul(cosf4(spu_mul(ivec2, uvec)), cosf4(vjvec)); 20 } } } A. R. Brodtkorb Applied Mathematics 46/53

32 CellDctQuant_spe.c - excerpt cont. () 0 f o r ( k=0; k<args. width ; k+=8) { // For each block f o r ( v=0; v<block_size ; ++v ) { // For each element in dct block f o r (u=0; u<block_size ; ++u) { v4f dctvec ; dctvec. s [ 0 ] = 0; dctvec. s [ 1 ] = 0; dctvec. s [ 2 ] = 0; dctvec. s [ 3 ] = 0; 5 f o r ( j =0; j<height ; ++j ) { // Calculate dct v4f inputvec1 ; v4f inputvec2 ; f o r ( i =0; i <4; ++i ) { // S h i ft the values ( unsigned >signed ) 10 inputvec1. s [ i ] = ( f l o a t ) input [ j args. width+k+i ] f ; inputvec2. s [ i ] = ( f l o a t ) input [ j args. width+k+i ] f ; } dctvec.v = spu_madd(inputvec1.v, cosuv[v][u][j][0], spu_madd(inputvec2.v, cosuv[v][u][j][1], 15 dctvec.v)); } f l o a t a1 =! u? i s q r t 2 : 1.0 f ; f l o a t a2 =! v? i s q r t 2 : 1.0 f ; output[v*args.padwidth+k+u] = (signed short) ((dctvec.s[1] + 20 dctvec.s[1] + dctvec.s[2] + dctvec.s[3])*a1*a2)/(4.0f*quanttbl[u*block_size+v]); } } 25 } A. R. Brodtkorb Applied Mathematics 48/53

33 DMAing the result to the PPE CellDctQuant_spe.c - excerpt cont. () 0 //DMA the data back to the CPU. l sa = ( void ) output ; ea = ( unsigned i n t ) args. output ; s i z e = args. padwidth 8 s i z e o f ( signed short ) ; tagid = 3; 5 cmd = MFC_PUT_CMD; spu_mfcdma32 ( lsa, ea, size, tagid, cmd ) ; spu_mfcstat (MFC_TAG_UPDATE_ALL) ; A. R. Brodtkorb Applied Mathematics 50/53

34 Summary DCT Create contexts in the constructor. Use pthreads to start the contexts. Issue DMA request for input data. Use a C union to couple vector float and float[4]. Precompute sines using SIMD instructions. Use precomputed sines to compute the DCT. Quantize. DMA result back to the PPE. A. R. Brodtkorb Applied Mathematics 51/53

35 Conclusion Using DMA can be frustrating (but easy after a couple of tries). SIMDification of code can be hard, but worth it. (Risc of compiler bugs). Great speedups (e.g., 25 faster than CPU for folding@home) The Cell processor is versatile (can be used from pipelining to embarassingly parallel problems), and readily available! A. R. Brodtkorb Applied Mathematics 52/53

36 Resources Source code Can be compiled and run on PS3, Cell blade, or Cell simulator Cell BE documentation Cell BE code samples /opt/ibm/cell-sdk/prototype/src/samples/ A. R. Brodtkorb Applied Mathematics 53/53

PS3 Programming. Week 2. PPE and SPE The SPE runtime management library (libspe)

PS3 Programming. Week 2. PPE and SPE The SPE runtime management library (libspe) PS3 Programming Week 2. PPE and SPE The SPE runtime management library (libspe) Outline Overview Hello World Pthread version Data transfer and DMA Homework PPE/SPE Architectural Differences The big picture