Determining Necessity of Optimization in Matrix Chain Multiplication. Quen Parson

Transcription

1 Determining Necessity of Optimization in Matrix Chain Multiplication Quen Parson

2 Introduction - What is a matrix? Mathematics - set of data Arranged in rows and columns Individual cells called elements Elements can be variables, constants, functions, matrices, etc.

3 Matrix Applications Machine learning (neural networks) Elements represent links between nodes Data mining Operations on large sets of data

4 Matrix Multiplication Associative - Grouping still produces same product Grouping can affect workload (number of multiplication operations) Noncommutative - Order can change product Result matrix dimensions - # ColA x # RowB Result matrix element Elementx,y = (ax,1*b1,y + ax,2*b2,y + ) Total number of operations (multiplication weight): # RowA * # ColA * # ColB Note: # RowA = # ColB 70px- Matrix_multiplication_diagram_2.svg.png

5 Matrix Chain Multiplication and Need for Optimization Matrix Chain Multiplication: Potentially very costly Larger chains can increase multiplication operations exponentially Recall weight calculation: product matrices can grow very quickly in chains Associative property: Matrix chains can be rearranged sometimes for faster calculation

6 Optimization Algorithm Dynamic programming: breaking larger problem into smaller problems that are easier to solve Memoization: algorithm saves intermediate products to memory to speed up future calculations Algorithm: Memoization Optimization Calculates cost of every possible chain grouping Uses dynamic programming to split chains into smaller chains for easier calculation Uses memoization to save products of chains to prevent redundant calculation 12/LCS recurrence- tre e.jp g

7 Optimization Inefficiency Algorithm complexity: O(2 n ), does not run in reasonable time Algorithm can speed up calculations of expensive chains, but can slow down the calculation of others (ex. chains that are already optimized) Solution speed: Optimization time + Solving time

8 Research Goal Determine criteria for when optimization is beneficial Independent Variables: chain length, distribution of weights Dependent Variable: calculation time Weight Distribution Weight distribution measured using average rate of change in weights from beginning to end of chain Calculated by first getting the matrix multiplication weights from left to right, and then getting the average rate of change Language and IDE: Visual C++ using Visual Studio

9 Procedure 1. Implement matrix class, relevant data and methods for project 2. Implement unoptimized solver, solves matrix chain as given without optimization step 3. Implement optimized solver, solves matrix chain after optimization step 4. Implement testing class, automatically runs test cases within given boundaries to speed up testing process 5. Analyze and interpret results

10 Results Test cases generated by test program to automate experimentation 48 million test cases planned Only approx. 20 million tested due to time constraints Chain length values tested: 2 to 6 Average weight slope tested: -30 to 30 Above red line: chain better unoptimized Below red line: chain better optimized

11 of Chain Weights

12 Discussion Time differences only significant when total chain weight is significant (sum of weights > 10,000) For chains with low number of total operations, time difference and total calculation time negligible Below this point, optimization is neither significantly detrimental or beneficial Optimization dependent on average weight slope Average weight slope < 2: chain better optimized Average weight slope > 2: chain better unoptimized

13 Applications Further speed up operations with large matrix chains Minimal change to existing algorithms, but significant performance gain Can be applied on other algorithms Same procedure can find criteria for other algorithms Determining necessity of optimization is O(n), low cost operation Much faster than time wasted in unnecessary optimization Easy to implement: simple conditional

14 Future Research Test using more hardware Determine if findings remain constant on alternative hardware (various industry standards) Run more tests to reduce variance in results More exact criteria Run procedure on other optimization algorithms

15 Acknowledgements Mrs. Yarbrough, Alabama School of Fine Arts Mrs. Chin, Alabama School of Fine Arts Dr. Pirkelbauer, University of Alabama in Birmingham