Performance Analysis. Space Complexity. Instruction Space. Data Structures and Programming 資料結構與程式設計. Topic 2 Complexity Analysis.

Transcription

1 Performace Aalysis Data Structures ad Programmig 資料結構與程式設計 Topic 2 Complexity Aalysis 課程編號 : EE 3011 科目名稱 : 資料結構與程式設計授課教師 : 黃鼎偉時間地點 : 一 678 電機二館 229 Performace of a Program Performace aalysis by aalytical methods Performace measuremet by experimet Complexity of a Program Space (Memory) complexity Importat for multiuser system, memory allocatio, selectig suitable compiler, ad estimatio of the largest problem that the program ca solve Time complexity Importat for systems with time limit, programs which require real-time respose, ad selectig amog differet programs that ca solve the same problem. 2 Space Complexity Istructio Space Space Complexity Istructio space Deped o the compiler, compiler optios, ad target computer Data space Number of bytes of differet data types Eviromet stack space Retur address Values of local variables ad formal parameters for recursive fuctios. 01 LOAD a 02 ADD b 03 STORE t1 04 LOAD b 05 MUL c 06 STORE t2 07 LOAD t1 08 ADD t2 09 STORE t3 10 LOAD a 11 ADD b 12 SUB c 13 STORE t4 14 LOAD a 15 ADD b 16 STORE t5 17 LOAD t4 18 DIV t5 19 STORE t6 20 LOAD t3 21 ADD t6 22 ADD 4 01 LOAD a 02 ADD b 03 STORE t1 04 SUB c 05 DIV t1 06 STORE t2 07 LOAD b 08 MUL c 09 STORE t3 10 LOAD t1 11 ADD t3 12 ADD t2 13 ADD 4 01 LOAD a 02 ADD b 03 STORE t1 04 SUB c 05 DIV t1 06 STORE t2 07 LOAD b 08 MUL c 09 ADD t2 10 ADD t1 11 ADD 4 (a+b)+(b*c)+(a+b c)/(a+b)+4 3 4

2 Data Space Space Allocated to C++ Data Types o a 32-bit/word Computer Type bool char usiged char short usiged short log usiged log it usiged it float double log double poiter poiter Space (bytes) double a[100]; it maze[rows][cols]; Rage {true, false} [ 127, 128] [0, 255] [ 32767, 32768] [0, 65535] [ 2 31, ] [0, ] [ 2 31, ] [0, ] ±3.4E±38 (7 digits) ±1.7E±308 (15 digits) ±1.2E±4932 (19 digits) (ear, _cs, _ds, _es, _ss poiters) (far, huge poiters) 800 bytes 4*rows*cols bytes 5 Eviromet Stack Space Eviromet stack space is geerally idepedet of the istace characteristics uless recursive fuctios are i use. 08 template<class T> 09 T rsum(t a[], it ) 10 {// Retur sum of umbers a[0: 1]. 11 if ( > 0) 12 retur rsum(a, 1) + a[ 1]; 13 retur 0; 14 } rsum(a,) rsum(a, 1) rsum(a, 2)... rsum(a,1) rsum(a,0) Note: The istace characteristics is usually defied as the size of the data ivolved i the program. rsum.cpp 6 Eviromet Stack Space The amout of stack space eeded by recursive fuctios is called the Recursive Stack Space: A fixed part that is idepedet of the istace characteristics. A variable part that cosists of dyamically allocated space. c + S P (istace characteristics) Example 2.1 abcit 03 #iclude<iostream> usig amespace std; it abc(it a, it b, it c) 08 { 09 retur a + b * c; 10 } it mai() 13 { 14 cout << abc(2,3,4) << edl; 15 retur 0; 16 } S abc (istace characteristics) = 0. c = 4 (a, b, c, &abc) 7 abcit.cpp 8

3 Example 2.2 sequetialsearch Example 2.3 sum 08 template<class T> 09 it sequetialsearch(t a[], it, cost T& x) 10 {// Search the uordered list a[0: 1] for x. 11 // Retur positio if foud; retur 1 otherwise. 12 it i; 13 for (i = 0; i < && a[i]!= x; i++); 14 if (i == ) retur 1; 15 else retur i; 16 } S sequetialsearch () = 0. c = 4 (a,, &x, &sequetialsearch) 07 template<class T> 08 T sum(t a[], it ) 09 {// Retur sum of the umbers a[0: 1]. 10 T thesum = 0; 11 for (it i = 0; i < ; i++) 12 thesum += a[i]; 13 retur thesum; 14 } S sum () = 0. c = 5 (a,, thesum, i, &sum) sequetialsearch1.cpp 9 sum.cpp 10 Example 2.4 rsum Example 2.5 factorial 08 template<class T> 09 T rsum(t a[], it ) 10 {// Retur sum of umbers a[0: 1]. 11 if ( > 0) 12 retur rsum(a, 1) + a[ 1]; 13 retur 0; 14 } a, : it (4 bytes) Retur address: far poiter (4 bytes) S rsum () = 12(+1). rsum(a,) rsum(a, 1) rsum(a, 2)... rsum(a,1) rsum(a,0) 07 it factorial(it ) 08 {// Compute! 09 if ( <= 1) retur 1; 10 else retur * factorial( 1); 11 } : it (4 bytes) Retur address: far poiter (4 bytes) S factorial () = 8*max(,1). factorial() factorial( 1) factorial( 1)... factorial(1) rsum.cpp 11 factorial.cpp 12

4 Exercise: rsequectialserach 10 it rsequetialsearch(t a[], it, cost T& x) 11 {// Search the uordered list a[0: 1] for x. 12 // Retur positio if foud; retur 1 otherwise. 13 if ( < 1) retur 1; 14 if (a[ 1] == x) retur 1; 15 retur rsequetialsearch(a, 1, x); 16 } S rsequetialsearch () =? Time Complexity Operatio Couts Step Couts Asymptotic complexity Compoets of Time Complexity tp caadd( ) cssub( ) c ` mmul( ) cddiv ( ) (2.1) where ca, cs, cm, cd ADD( ), SUB( ), MUL( ), DIV( ) : (type-depedet) time required for sigle +, -, *, / operatio : istace characteristics : umber of +, -, *, / operatios (Operatio Couts) Note: the time used for comparisos ad elemet moves may also be cosidered. rsequetialsearch.cpp Example 2.7 idexofmax 07 template<class T> 08 it idexofmax(t a[], it ) 09 {// Locate the largest elemet i a[0: 1]. 10 if ( <= 0) 11 throw illegalparametervalue(" must be > 0"); it idexofmax = 0; 14 for (it i = 1; i < ; i++) 15 if (a[idexofmax] < a[i]) 16 idexofmax = i; 17 retur idexofmax; 18 } Number of comparisos = max{ 1,0} Example 2.8a polyeval (Polyomial Eval.) 10 T polyeval(t coeff[], it, cost T& x) 11 {// Evaluate the degree polyomial with 12 // coefficiets coeff[0:] at the poit x. 13 T y = 1, value = coeff[0]; 14 for (it i = 1; i <= ; i++) 15 {// add i ext term 16 y *= x; 17 value += y * coeff[i]; 18 } 19 retur value; 20 } P( x) ci x i0 i Number of additios = Number of multiplicatios = 2 idexofmax.h 15 polyeval.cpp 16

5 Example 2.8b horer (Polyomial Eval.) 10 T horer(t coeff[], it, cost T& x) 11 {// Evaluate the degree polyomial with 12 // coefficiets coeff[0:] at the poit x. 13 T value = coeff[]; 14 for (it i = 1; i <= ; i++) 15 value = value * x + coeff[ i]; 16 retur value; 17 } i P( x) cix ( ( cxc 1) xc2) xc3) x) xc0 i0.., ( ) ((5 4) 1) 7 eg P x x x x x x x Number of additios = Number of multiplicatios = Example 2.9 rak (Rakig) 10 void rak(t a[], it, it r[]) 11 {// Rak the elemets a[0: 1]. 12 // Elemet raks retured i r[0: 1] 13 for (it i = 0; i < ; i++) 14 r[i] = 0; // iitialize // compare all elemet pairs 17 for (it i = 1; i < ; i++) 18 for (it j = 0; j < i; j++) 19 if (a[j] <= a[i]) r[i]++; 20 else r[j]++; 21 } Number of comparisos = ( 1)/2 horer.cpp 17 rak.cpp 18 Example: Sortig Rearrage a[0], a[1],, a[ 1] ito ascedig order. Whe doe, a[0] <= a[1] <= <= a[ 1] 8, 6, 9, 4, 3 3, 4, 6, 8, 9 Isertio Sort Bubble Sort Selectio Sort Cout Sort (Rak Sort) Shaker Sort Shell Sort Heap Sort Merge Sort Quick Sort Example 2.10 rearrage (Rak Sort) 10 template<class T> 11 void rearrage(t a[], it, it r[]) 12 {// Rearrage the elemets of a ito sorted order 13 // usig a additioal array u. 14 T *u = ew T []; // create additioal array // move to correct place i u 17 for (it i = 0; i < ; i++) 18 u[r[i]] = a[i]; // move back to a 21 for (it i = 0; i < ; i++) 22 a[i] = u[i]; delete [] u; 25 } Number of elemet moves = 2 19 raksort1.cpp 20

6 Example 2.11 selectiosort (Selectio Sort) 10 void selectiosort(t a[], it ) 11 {// Sort the elemets a[0: 1]. 12 for (it size = ; size > 1; size ) 13 { 14 it j = idexofmax(a, size); 15 swap(a[j], a[size 1]); 16 } 17 } Number of comparisos = ( 1)/2 Number of elemet moves = 3( 1) Example 2.12 bubble (Bubble Sort) 10 void bubble(t a[], it ) 11 {// Bubble largest elemet i a[0: 1] to right. 12 for (it i = 0; i < 1; i++) 13 if (a[i] > a[i+1]) swap(a[i], a[i + 1]); 14 } Number of comparisos = 1 16 template<class T> 17 void bubblesort(t a[], it ) 18 {// Sort a[0: 1] usig bubble sort. 19 for (it i = ; i > 1; i ) 20 bubble(a, i); 21 } Total umber of comparisos = ( 1)/2 raksort1.cpp 21 bubblesort.cpp 22 Best, Worst ad Average Operatio Couts Worst-case cout = maximum cout Best-case cout = miimum cout Average cout Operatio Couts sequetialsearch 10 it rsequetialsearch(t a[], it, cost T& x) 11 {// Search the uordered list a[0: 1] for x. 12 // Retur positio if foud; retur 1 otherwise. 13 if ( < 1) retur 1; 14 if (a[ 1] == x) retur 1; 15 retur rsequetialsearch(a, 1, x); 16 } Worst: comparisos (usuccessful search) Best: 1 compariso Average: (+1)/2 23 Because equal probability 1/ for each elemet 1 i ( 1)/2 i 1 sequetialsearch.cpp 24

7 Operatio Couts isert (to a sorted array) Step Couts 10 void isert(t a[], it&, cost T& x) 11 {// Isert x ito the sorted array a[0: 1]. 12 // Assume a is of size > 13 it i; 14 for (i = 1; i >= 0 && x < a[i]; i ) 15 a[i+1] = a[i]; 16 a[i+1] = x; 17 ++; // oe elemet added to a 18 } Worst: comparisos (usuccessful search) Best: 1 compariso Average: comparisos 2 1 Isert to a[i+1] requires i comparisos ad to a[0] requires comparisos ( 1) ( i) j i0 j1 A step is a amout of computig that does ot deped o the istace characteristic 10 adds, 100 subtracts, 1000 multiplies ca all be couted as a sigle step adds caot be couted as 1 step s/e: step/executio isert.cpp Step Couts sum s/e Freq. Total Steps 08 T sum(t a[], it ) {// Retur sum of the umbers a[0: 1] T thesum = 0; for (it i = 0; i < ; i++) thesum += a[i]; 1 13 retur thesum; } t ( ) 23, 0 sum Step Couts rsum s/e Freq. Total Steps 08 template<class T> T rsum(t a[], it ) {// Retur sum of umbers a[0: 1] if ( > 0) 1?? 12 retur rsum(a, 1) + a[ 1]; 1?? 13 retur 0; 1?? 14 } 0 0 0? trsum ( ) 2 trsum ( 1) 22 t ( 2) rsum 2 trsum (0) 22, 0 sum.cpp 27 rsum.cpp 28

8 Step Couts traspose s/e Freq. Total Steps 09 void traspose(t **a, it rows) {//traspose a[0:rows 1][0:rows 1] for (it i = 0; i < rows; i++) 1 rows + 1 rows+1 12 for (it j = i+1; j < rows; j++) 1 rows(rows + 1)/2 rows(rows + 1)/2 13 swap(a[i][j], a[j][i]); 1 rows(rows -1)/2 rows(rows -1)/2 14 } rows 2 + rows + 1 Step Couts s/e is t always 0 or 1 x = sum(a, ); ( is the istace characteristic) has a s/e cout of 2+3. t rows rows rows ( ) 2 1 traspose Note: Here, swap(,) oly takes 1 s/e, although it takes 3 moves. matrixtraspose.cpp Step Couts ief (Prefix Sums) s/e Freq. Total Steps 18 void ief(t a[], T b[], it ) {// Compute prefix sums for (it j = 0; j < ; j++) b[j] = sum(a, j + 1); 2j + 6 ( + 5) 22 } sum(a, j + 1) b[j] = 1 j0 2( j1) j6 (2 j6) ( 5) t ( ) ief Asymptotic Notatio O( 2 ), ( 3 ), ( log ), o() Practical Complexities ief.cpp 31 32

9 Big O (O), Omega (), Theta (), ad Little Oh (o) Notatio Big O (O), Omega (), Theta (), ad Little Oh (o) Notatio f() = O(g()) f() is asymptitically smaller or equal to g() f() = (g()) f() is asymptotically bigger tha or equal to g() f() = (g()) f() is asymptotically equal to g() f() = o(g()) f() is asymptotically smaller tha g() 33 2 lg lg 1 ( ) ( ) 1 < log < < log < 2 < 3 < 2 <! < 1: costat 2 : quadratic log : logarithmic 3 : cubic : liear 2 : expoetial log : log!: factorial Asymptotic Idetities Step Couts ief (Prefix Sums) ( 3 ) ca be ay oe of O,, ad s/e Freq. Total Steps 18 void ief(t a[], T b[], it ) 0 0 (0) 19 {// Compute prefix sums. 0 0 (0) 20 for (it j = 0; j < ; j++) () 21 b[j] = sum(a, j + 1); 2j + 6 ( 2 ) 22 } 0 0 (0) ( 2 ) sum(a, j + 1) b[j] = 1 j0 (2 j6) ( 5) t ( ) 61 ief 2( j1) j ief.cpp 36

10 Practical Complexities Impractical Complexities 10 9 istructios/secod log istructios/secod μsec 10 μsec 1 msec 1 sec mi 3.2 x years 3.2 x years μsec 130 μsec 100 msec 17 mi days?????? msec 20 msec 17 mi 32 years x 10 7 years???????????? Faster Computer vs. Better Algorithm Algorithmic improvemet more useful tha hardware improvemet. e.g., Reduce from 2 to 3 Some Uses Of Performace Aalysis Determie practicality of algorithm Predict ru time o large istace Compare 2 algorithms that have differet asymptotic complexity e.g., O() ad O( 2 ) 39 40

11 Limitatios of Aalysis Memory Hierarchy Does t accout for costat factors. However, costat factor may domiate 1000 vs. 2, ad we are iterested oly i < 1000 Moder computers have a hierarchical memory orgaizatio with differet access time for memory at differet levels of the hierarchy. ALU RAM CPU MAIN L2 1C 2C 10C 100C L1 R KB 512KB 512MB Limitatios of Aalysis Performace Measuremet Our aalysis does t accout for this differece i memory access times. Measure actual time o a actual computer. What do we eed? Programs that do more work may take less time tha those that do less work

12 Performace Measuremet Needs Programmig laguage Workig program Computer Compiler ad optios to use gcc -O2 /O2 i Visual Studio for Speed Optimizatio Choosig Istace Size Developig the Test Data Settig up the Experimet Performace Measuremet Needs Data to use for measuremet Worst-case data Best-case data Average-case data Timig mechaism --- clock Timig i C++ 01 double clockspermillis = double(clocks_per_sec) / 1000; 02 // clock ticks per millisecod 03 clock_t starttime = clock(); isertiosort(a, ); // code to be timed double elapsedmillis = (clock() starttime) / 08 clockspermillis; 09 // elapsed time i millisecods Shortcomig Clock accuracy Assume 100 ticks Examiig remaiig cases, we get trueelapsedtime = fiishtime starttime ± 100 ticks To esure 10% accuracy, require elapsedtime = fiishtime starttime >= 1000 ticks Repeat work may times to brig total time to be >= 1000 ticks timeisertiosort1.cpp 47 48

13 Accurate Timig 01 clock_t starttime = clock(); 02 log umberofrepetitios; 03 do { 04 umberofrepetitios++; 05 // put code to iitialize a[] here 06 isertiosort(a, ); 07 } while (clock() starttime < 1000) 08 double elapsedmillis = (clock() starttime) / 09 clockspermillis; 10 double timeforcode = elapsedmillis/umberofrepetitios; Accuracy Now accuracy is 10%. First readig may be just about to chage to starttime Secod readig (fial value of clock())may have just chaged to fiishtime So fiishtime starttime is off by 100 ticks timeisertiosort2.cpp Bad Way To Time 01 do { 02 couter++; 03 starttime = clock(); 04 isertiosort(a, ); 05 elapsedtime += clock() starttime; 06 } while (elapsedtime < 1000) Calculate the Overhead 01 do 02 { 03 umberofrepetitios++; 04 // iitialize with worst case data 05 for (it i = 0; i < ; i++) 06 a[i] = i; 07 // isertiosort(a, ); 08 } while (clock( ) starttime < 1000); timeisertiosort3.cpp 51 timeisertiosort4.cpp 52

14 Time Shared System Widows (Server 2003/2008 with plugis) timeit f MyProgram UNIX time MyProgram Effect of Cache Misses o Ru Time 08 template<class T> 09 void squarematrixmultiply(t **a, T **b, T **c, it ) 10 {// Multiply the x matrices a ad b to get c. 11 for (it i = 0; i < ; i++) 12 for (it j = 0; j < ; j++) 13 { 14 T sum = 0; 15 for (it k = 0; k < ; k++) 16 sum += a[i][k] * b[k][j]; 17 c[i][j] = sum; 18 } 19 } i j k order More Cache Misses = 2000, t P = sec Oe ested loop More efficiet 53 timematrixmultiply1.cpp 54 Effect of Cache Misses o Ru Time 08 void fastsquarematrixmultiply(it **a, it **b, it **c, it ) 09 { 10 for (it i = 0; i < ; i++) Two ested loops 11 for (it j = 0; j < ; j++) 12 c[i][j] = 0; Less efficiet for (it i = 0; i < ; i++) 15 for (it j = 0; j < ; j++) 16 for (it k = 0; k < ; k++) 17 c[i][j] += a[i][k] * b[k][j]; 18 } i j k order More Cache Misses = 2000, t P = sec Effect of Cache Misses o Ru Time 08 void fastsquarematrixmultiply(it **a, it **b, it **c, it ) 09 { 10 for (it i = 0; i < ; i++) Two ested loops 11 for (it j = 0; j < ; j++) 12 c[i][j] = 0; Less efficiet for (it i = 0; i < ; i++) 15 for (it k = 0; j < ; j++) 16 for (it j = 0; k < ; k++) 17 c[i][j] += a[i][k] * b[k][j]; 18 } i k j order Less Cache Misses = 2000, t P = 54.2 sec timefastsquarematrixmultiply1.cpp 55 timefastsquarematrixmultiply2.cpp 56