Parallel MATLAB at FSU: Task Computing

Transcription

1 Parallel MATLAB at FSU: Tak John Burkardt Department of Scientific Florida State Univerity... 1:30-2:30 Thurday, 07 April Dirac Science Library... jburkardt/preentation/... fu 2011 matlab tak.pdf Florida State Univerity 1 / 54

2 Parallel MATLAB: Tak Tak QUAD Example KNAPSAK Example ELL DETETION Example RANDOM WALK Example oncluion Florida State Univerity 2 / 54

3 Tak : not PARFOR or SPMD In lat week lecture, I talked about MATLAB parfor and pmd command. With parfor, a ingle program and a ingle et of hared data were involved. When the client reached a parallel loop, extra worker would ait. With pmd, we aw a ingle program, but one which wa divided between command to the client and command to the worker. Moreover, the client and worker had eparate memory pace. Data could be moved only by explicit command. Depite their difference, programming with parfor and pmd have omething in common; the client and the worker are imultaneouly executing a program. Florida State Univerity 3 / 54

4 Tak : Decription Today, we will conider a third technique, tak c, in which a big job i divided into very independent tak; Each tak run on the mallet addreable type of proceor: a ingle core on a dektop or a ingle node on a cluter. Tak run in any order, at any time, on any available proceor. We ll aume each tak execute the ame MATLAB function. Each tak ha it own memory. Each tak i given a et of input; it doe no further communication until it computation i complete, when it return it reult. When all the tak are completed, the collection of reult can be examined, analyzed or plotted. Florida State Univerity 4 / 54

5 Tak : Decription You could et up 100 tak, for intance, a a ingle job. For each tak, MATLAB locate an available proceor, enure that the tak receive it input, and recover the output. Once all the tak are completed, the output can be examined. Tak c allow you to organize a computation that ha many independent part. If many proceor are available, many tak will run at the ame time, but you don t worry about the precie chedule. Tak c i a handy way of filling up pare computer time with bite ized piece of a computation whoe final reult can be determined once all the piece have been completed. Tak can run on your dektop or on a cluter. Florida State Univerity 5 / 54

6 Tak : Example Tak c example: earch: divide the earch pace among tak; Monte arlo: give each tak a unique random number eed; ummation: any tak involving multiple ummation, multiplication, or other operation (quadrature) image proceing: operation that can be carried out over part of the image, or frame of an animation; ray tracing; Florida State Univerity 6 / 54

7 Parallel MATLAB: Tak Tak QUAD Example KNAPSAK Example ELL DETETION Example RANDOM WALK Example oncluion Florida State Univerity 7 / 54

8 QUAD: Approximate integration Here we ue evenly paced ample point and equal weight. Other cheme vary the pacing or weight, or randomize abcia. Florida State Univerity 8 / 54

9 QUAD: Approximate integration We recall our favorite problem, the approximation of an integral by a weighted um of function value: Z I = b f (x) dx a Q= N X wi f (xi ) i=1 We could eaily regard thi computation a, ay, 4 tak: Q = Q1 + Q2 + Q3 + Q4 where each Qi could be computed independently. The only communication required would come at the end, when the Qi mut be combined to form Q. Florida State Univerity 9 / 54

10 QUAD: Tak and Job Our baic tak etimate the integral over a ubinterval [ai, bi ] uing ni point. We can write a MATLAB function to do thi: qi = quad_tak ( ni, ai, bi ) Then our job i made up of four tak (aume [a, b] = [0,1]!): tak 1 integrate from 0/4 to 1/4 tak 2 integrate from 1/4 to 2/4 tak 3 integrate from 2/4 to 3/4 tak 4 integrate from 3/4 to 4/4 Each tak can be expreed by pecifying the appropriate input argument to quad tak. Florida State Univerity 10 / 54

11 QUAD: QUAD TASK function qi = quad_tak ( ni, ai, bi ) x = linpace ( ni, ai, bi ); qi = h * ( 0.5 * f ( x(1) )... + um ( f ( x(2:ni-1) ) ) * f ( x(ni) ) ); return end Florida State Univerity 11 / 54

12 QUAD: Local Execution of a Job On a dektop, we create a job, define it tak, and ubmit it: job = createjob ( configuration, local ); n = ; ni = 25001; for tak = 1 : 4 ai = ( tak - 1 ) / 4; bi = tak / 4; tak_id = createtak ( 1, { ni, ai, bi } ) end ubmit ( job ); wait ( job ); <-- Wait until all tak have completed. (The third argument to createtak, ( 1 ), report the number of output produced by quad tak). Florida State Univerity 12 / 54

13 QUAD: ollecting Reult Our final integral etimate Q i the um of the individual reult. If qi i the name of the output from the quad tak function, the load command can return a cell array containing the value of qi returned by each tak; qi = load ( job, qi ); qi = cell2mat ( qi ); q = um ( qi ); Florida State Univerity 13 / 54

14 QUAD: FSU HP luter To run on the FSU HP cluter, we run a a imple job, and we have to create the cell array arg of input argument: n = ; tak_num = 4; ni = 1 + floor ( ( n - 1 ) / tak_num ); arg = {}; for tak = 1 : tak_num ai = ( tak - 1 ) / tak_num; bi = tak / tak_num; arg{tak} = { ni, ai, bi }; end qi = fulutermatlab ( [], [],, w,... arg ) Florida State Univerity 14 / 54

15 QUAD: FSU HP luter We have aked MATLAB to execute the four command: qi{1} qi{2} qi{3} qi{4} = = = = quad_tak quad_tak quad_tak quad_tak ( ( ( ( 0.00, 0.25, 0.50, 0.75, 0.25, 0.50, 0.75, 1.00, ); ); ); ); in any order, on any available core, while we wait. Although thee command execute independently, when they are done, we have acce to the reult. In particular, the quadrature value i: qi = cell2mat ( qi ); q = um ( qi ); Florida State Univerity 15 / 54

16 QUAD: Our implet example QUAD demontrate the feature of tak programming. The calculation i thought of a a job. The job i thought of a multiple tak, with each tak executed by the ame tak function, uing different input. The job collect the output from each tak function. The uer can wait for the job to complete (the w option), or log out and check back in later (the n option). When the job i complete, the uer can reume an interactive MATLAB eion to analyze, manipulate or plot the reult. Florida State Univerity 16 / 54

17 Parallel MATLAB: Tak Tak QUAD Example KNAPSAK Example ELL DETETION Example RANDOM WALK Example oncluion Florida State Univerity 17 / 54

18 KNAPSAK: Problem Definition In the claical problem, the object have both weight and value. For our problem, we ll jut worry about the weight! Florida State Univerity 18 / 54

19 KNAPSAK: Problem Definition Suppoe we have a knapack with a limited capacity, and a number of object of varying weight. We want to find a ubet of the object which exactly meet the capacity of the knapack. (Thi i ometime called the greedy burglar problem!) Symbolically, we are given a target value t, and a et W of n weight. We want a ubet S W o that: X t= S We don t know if a given problem ha 0, 1, or many olution. Florida State Univerity 19 / 54

20 KNAPSAK: Encoding A olution of the problem i a ubet S W. Each n-digit binary tring from 0 to 2n 1 i a code for a poible olution. For weight W={15,11,10,8,3}, target t=24, we have: ode Binary ode Indice {} {1} {2} {2,1} {3} {3,1} {3,2}... {5,4,3,2,1} S {} {3} {8} {8,3} {10} {10,3} {10,8}... {15,11,10,8,3} P Florida State Univerity 20 / 54

21 KNAPSAK: Algorithm A imple earch chooe a value of code in the range 0 to 2n 1, decode the ubet S, add the weight, and compare to t. On the 23rd tep of the earch, we have a code of 22 = binary = ubet {5,3,2}, o a weight of =33, too high. The proce of checking one code i completely independent of checking any other. One program could check all code, or we could ubdivide the range, and check the ubrange in any order and at any time. Florida State Univerity 21 / 54

22 KNAPSAK: Program to Search Entire Range function [ i_chooe, w_chooe ] = knapack ( w, t ) i_chooe = []; w_chooe = []; n = length ( w ); for code = 0 : 2^n - 1 chooe = find ( bitget ( code, 1:n ) ); if ( um ( w(chooe) ) == target ) i_chooe = chooe; w_chooe = w(chooe); return end end Florida State Univerity 22 / 54

23 KNAPSAK: A ouple Myteriou MATLAB Function The MATLAB function bitget return a vector of 0 and 1 for poition 1 to n in the counter code. Each uch binary tring decribe a unique ubet. The function find indexe the 1 in the binary tring. Thi tring of indice, called chooe, elect the ubet of w that we mut compare to t. If the ubet ha the right weight, we found a olution, and return. Florida State Univerity 23 / 54

24 KNAPSAK: The alculation i Diviible It eay to ee that we could divide thi problem up into maller problem that are worked on independently. For thi problem, it clear that the key i to take the original range of code, from 0 to 2n 1, and break it into ubrange. A ingle function can work on the problem over the retricted ubrange. In fact, we only have to lightly modify our original code to make thi new verion. Florida State Univerity 24 / 54

25 KNAPSAK: Program to Search Selected Range function [ i_chooe, w_chooe ] = knapack ( w, t, rnge ) i_chooe = []; w_chooe = []; n = length ( w ); for code = rnge(1) : rnge(2) chooe = find ( bitget ( code, 1:n ) ); if ( um ( w(chooe) ) == target ) i_chooe = chooe; w_chooe = w(chooe); return end end Florida State Univerity 25 / 54

26 KNAPSAK: Set up for Local Execution Once we have mentally divided our calculation into independent ubcalculation, we need to be able to expre thi logical fact to MATLAB. Of coure, jut a for a equential calculation we need to define the general problem parameter; However, now, we alo need to pecify the number of tak, identify the function that will execute the tak and create a cell array containing a eparate copy of the input to each tak. We feed thi information to the fulutermatlab command. Florida State Univerity 26 / 54

27 KNAPSAK: Execution on FSU HP luter w = [ , , (...more...), ]; t = ; arg = {}; i2 = -1; for tak = 1 : 4 i1 = i2 + 1; i2 = floor ( ( 2^n - 1 ) * tak / 4 ); arg{tak} = { w, t, [ i1, i2 ] }; end reult = fulutermatlab ( [], [],, w,... arg ) Florida State Univerity 27 / 54

28 KNAPSAK: Examine the reult The output argument from each tak are returned a a cell array. Tak 3 econd output (the weight) i in reult{3,2}. for tak = 1 : 4 if ( iempty ( reult{tak,1} ) ) fprintf ( 1, Tak %d found no olution.\n, tak ); ele fprintf ( 1, Tak %d found a olution:\n, tak ); dip ( Indice hoen: ); dip ( reult{tak,1} ); dip ( Weight hoen: ); dip ( reult{tak,2} ); end end Florida State Univerity 28 / 54

29 KNAPSAK: Sample Run >> knapack_fu reult = [] [1x3 double] [] [] [] [1x3 double] [] [] Tak 1 did not find a olution. Tak 2 found the following olution: weight indice weight_value Tak 3 did not find a olution. Tak 4 did not find a olution. Florida State Univerity 29 / 54

30 Parallel MATLAB: Tak Tak QUAD Example KNAPSAK Example ELL DETETION Example RANDOM WALK Example oncluion Florida State Univerity 30 / 54

31 ELLS: Problem Definition Medical reearcher can film mall group of biological cell. In ome cae, an enormou number of uch record are created. Rather than having a lab worker view each frame of film, it i poible to automatically proce the image and, for mot part, detect the cell, determine the poition of any given cell over a equence of image, and monitor the area, hape and average eparation of the cell. We will look at a imple application in which it i deired to identify cell by urrounding each one with a white boundary. If we have many image file, each can be proceed independently. Florida State Univerity 31 / 54

32 ELLS: A Typical Image Florida State Univerity 32 / 54

33 ELLS: A Typical Image with ell Identified Florida State Univerity 33 / 54

34 ELLS: Problem Definition In the KNAPSAK problem, the input and output for each tak wa a hort lit of number. For the ELLS problem, the input i really a graphic file, and the output i a tranformed graphic file. Thi mean that each tak, when it run, ha to be able to determine which unique file it hould open; it need to be able to find that file; and it need to know how to name the output file and where to place it. So while the KNAPSAK tak ued the input and output argument of the MATLAB function for their communication, the ELLS example will almot entirely be dealing with external file. Florida State Univerity 34 / 54

35 ELLS: File Name Our 99 input file are indexed in a natural way, tarting with AT3 01.tif and running up to AT3 99.tif. The output file will follow a imilar convention, but their name will tart with BT3. The input filename for a given tak can be computed: filename_input = printf ( AT3_%02d.tif, tak ); Uing the format %02d mean that mall integer will be left-padded with zero. Florida State Univerity 35 / 54

36 ELLS: The Tak Function For thi problem, the tak function will have a imple interface: function cell_tak ( tak ) It only need to know the index of the file it hould open, and it doen t return any output - at leat, not via output argument. So the body of thi function will be: generate input filename baed on tak number open input file proce data generate output filename baed on tak number write output file and we won t worry about the detail of proceing the data! Florida State Univerity 36 / 54

37 ELLS: Initializing the Job Now we need to et up the tak. Notice that there are no output reult... The output of cell tak will be the modified verion of the image file. tak_num = 99; <-- there are 99 image to proce; arg = {}; for tak = 1 : tak_num arg{tak} = { tak }; end <-- tak i work on image i. fulutermatlab ( [], [],, w,... arg ); Florida State Univerity 37 / 54

38 ELLS: Defining the Tak To keep thing imple, we aume that the file cell tak.m and all the input image file are in the current directory. Then, once we iue the fulutermatlab command, each tak, when it execute, will tart in thi directory, be able to ee the input image, and will leave it modified verion here a well. Florida State Univerity 38 / 54

39 ELLS: Sample Output for 10 Image >> cell_fu ELL_FSU: Ue MATLAB tak c on the FSU HP cluter. Here, we want to apply an image proceing operation (identify edge) to each of 10 biological image. Here i a current directory liting: AT3_01.tif AT3_04.tif AT3_07.tif AT3_10.tif AT3_02.tif AT3_05.tif AT3_08.tif cell_fu.m AT3_03.tif AT3_06.tif AT3_09.tif cell_tak.m all fulutermatlab Florida State Univerity 39 / 54

40 ELLS: Sample Output for 10 Image After fulutermatlab execute, we ee: Here i a current directory liting: AT3_01.tif AT3_02.tif AT3_03.tif AT3_04.tif AT3_05.tif AT3_06.tif AT3_07.tif AT3_08.tif AT3_09.tif AT3_10.tif BT3_01.tif BT3_02.tif BT3_03.tif BT3_04.tif BT3_05.tif BT3_06.tif BT3_07.tif Job1.in.mat BT3_08.tif Job1.jobout.mat BT3_09.tif Job1.out.mat BT3_10.tif Job1.tate.mat cell_fu.m matlab_metadata.mat cell_tak.m Job1 Job1.common.mat Job file are MATLAB log and data information. Florida State Univerity 40 / 54

41 Parallel MATLAB: Tak Tak QUAD Example KNAPSAK Example ELL DETETION Example RANDOM WALK Example oncluion Florida State Univerity 41 / 54

42 WALKS: Problem Definition The claic example of a random walk place a peron at the origin. The peron then flip a coin, and take one tep to the left or right, repeating thi proce a long a deired. Surpriingly, random walk are model of many phyical procee, and their imulation and analyi can give inight to ituation where there are no exact method available. A variation of thi problem i the elf avoiding walk in 2D, in which the peron i allowed to move over a 2D lattice, but can never viit the ame place twice. Becaue the path never croe, thi walk can be thought of a a imple model of a protein folding. Self avoiding walk of a fixed length could repreent poible hape of the protein. Florida State Univerity 42 / 54

43 WALKS: A Self Avoiding Walk / Abtract Protein Florida State Univerity 43 / 54

44 WALKS: Sampling a Huge, Unruly Space Our idea i to generate lot of different elf avoiding walk. Since the et i extremely large, we rely on the random number generator to make our choice. A different initial eed hould almot alway give a different walk. The actual number of poible walk can eaily exceed the number of poible eed! Our ample might give u an etimate for the typical ditance between the tart and end; the typical ditance of the tarting point to the boundary ; number of empty lattice point in the convex hull of the walk; the likelihood a walk will terminate early; Florida State Univerity 44 / 54

45 WALKS: The Tak ode function [ tep_num, dit ] = walk_tak ( tep_max, eed ) The tak ue eed to initialize rand() by: rand ( twiter, eed ); Then it trie taking tep max elf-avoiding tep from the origin. The walk terminate early if all four neighbor have already been viited. Otherwie, we move to a random unviited neighbor. The function return the actual number of tep taken, and the final ditance from the origin. Florida State Univerity 45 / 54

46 WALKS: Setting up the Job The job code mut decide how many walk are to be generated, and how many tep they are to take. It chooe different random number eed for each tak. tak_num = 100; tep_max = 200; arg = {}; for tak = 1 : tak_num eed = tak; arg{tak} = { tep_max, eed }; end reult = fulutermatlab ( [], [],, w,... arg ); reult = cell2mat ( reult ); Florida State Univerity 46 / 54

47 WALKS: Plotting the Reult reult = cell2mat ( reult ); <-- make numeric tep_num = reult(:,1); dit2 = reult(:,2).^2; x = log ( tep_num ); y = log ( dit2 ); <-- X = log tep <-- Y = log ditˆ2 c = polyfit ( x, y, 1 ); <-- Seek linear fit log_fit = c(1) * x + c(2); plot ( x, y, bo,... x, log_fit, r- ) Florida State Univerity 47 / 54

48 WALKS: Dit Squared = c * Step Florida State Univerity 48 / 54

49 Parallel MATLAB: Tak Tak QUAD Example KNAPSAK Example ELL DETETION Example RANDOM WALK Example oncluion Florida State Univerity 49 / 54

50 ONLUSION: Summary of Example Our example ugget the computation uitable for tak programming. QUAD i o imple we have ued it with parfor, pmd and tak. In KNAPSAK, we had to figure out an encoding, and we gave each tak a ubrange of the full problem. We had thouand of code to check, but we aigned a large ubrange of code to each tak (rather than generating thouand of tiny tak!) In ELL, each tak worked on a file, and there wa no input or output, except that each tak knew it index. In RANDOM WALK, we had to enure that each tak received a unique random eed, and we had to gather the data up at the end and plot it. Florida State Univerity 50 / 54

51 ONLUSION: Summary of Tak Tak c i ignificantly different from the uual kind of parallel c, where the computation happen concurrently. Here, each tak i aigned a ingle proceor. Tak can run equentially on the ame proceor, imultaneouly on different proceor, or in many other combination. Tak do not communicate, except that they receive an initial input from the job, and end their output reult back to it. In each of our example, the ame MATLAB function executed each tak, but a job i free to collect an arbitrary et of tak. Florida State Univerity 51 / 54

52 oncluion: Dektop Experiment If you are intereted in parallel MATLAB, the firt thing to do i get acce to the Parallel Toolbox on your multicore dektop machine, o that you can do experimentation and practice run. You can begin with ome of the ample program we have dicued today. You hould then ee whether the job and tak approach would help you in your own programming need. Florida State Univerity 52 / 54

53 oncluion: FSU HP luter If you are intereted in eriou parallel MATLAB c, you hould conider requeting an account on the FSU HP cluter, which offer MATLAB on up to 16 core. To get an account, go to and look at the information under Apply for an Account. Account on the general acce cluter are available to any FSU faculty member, or to peron they ponor. Florida State Univerity 53 / 54

54 ONLUSION: Where i it? MATLAB Parallel Toolbox Uer Guide doc/ditcomp/... ditcomp.pdf jburkardt/preentation/... fu 2011 matlab tak.pdf thee note; Gaurav Sharma, Jo Martin, MATLAB: A Language for Parallel, International Journal of Parallel Programming, Volume 37, Number 1, page 3-36, February jburkardt/m rc/m rc.html quad tak knapack tak cell detection tak random walk 2d avoid tak Florida State Univerity 54 / 54