Computational Complexity Analysis

Transcription

1 Computational Complexity Analysis

2 Largely About Algorithm Performance Operating systems and database Sorting, searching, E.g., search engines Crawling, indexing, searching Key components Indexing algorithms Searching algorithms Often, it is about speed. Other measures in some areas.

3 Should We Care? Computers keep getting faster and can do more things. But life becomes more complicated too. Phones in 80s. Phones nowadays. The question is: Can computer keep up the pace with our needs?

4 How to Evaluate Algorithms? Speed and space Speed: can an algorithm process data quickly enough? Space: can an algorithm work with limited resources, say memory?

5 Memory Was a Big Concern! Remember the millennium bug? Saving two bytes was a big deal!

6 Big Data Era Data is getting larger and larger. Navteq: maps and traffic data (200G per day). Bill Gate s Basic in 4k memory vs. current applications (hundreds of Mega bytes)

7 But, Storage Becomes Cheaper and Cheaper.

8

9

10 Speed Is Still A Big Concern. CPU clock rates cannot go up forever!

11

12 Measure Algorithm Speed is Key! Algorithm complexity (computational complexity). We could compare different algorithms sideby-side. But lots of hassles: Implementation We need an easy way to evaluate algorithms.

13 Algorithm Analysis Estimates problem cost as a function of growth rate. Basic idea How many additional operations does an algorithm need to carry out a task when more data, say n, is added? Representation The big O: the order of complexity O(*)

14 Example: Sorting Sorting is a basic procedure in many computational task. Excel data, files, A sorting problem can be simplified as A dataset D={d 0,d 1,d 2,, d n } d 0,d 1,d 2,, d n are not ordered. Sort the data in D.

15 Bubble Sorting BubbleSort (d 0,d 1,d 2,, d n ) for i = 1 to n -1 for j = 1 to n-i if d j >d j+1 then interchange d j and d j+1

16 How many times to go through the core block? Only the two outside loops matter. The total times:(n-1) * n / 2 The big O: (n 2 -n)/2 O(n 2 )

17 8 passes Within the 1st pass: 3 vs. 7 no action 7 vs. 8 no action 8 vs. 5 swap 8 vs. 2 swap 8 vs. 1 swap 8 vs. 9 no action 9 vs. 5 swap 9 vs. 4 swap After 1 st pass: comparisons and 5 swaps

18 8 passes After 1 st pass: (8, 5) After 2 nd pass: (7, 5) After 3 rd pass: (6, 4) After 4 th pass: (5, 3) After 5 th pass: (4, 2) After 6 th pass: (3, 0) After 7 th pass: (2, 0) After 8 th pass: (1, 0)

19 Is This Good? Let s do a comparison.

20 Bubblesort vs. Quicksort Bubblesort: O(n 2 ) Quicksort: O(nlogn)

21 WXz5rF64

22 Good, Bad, and Ugly Good logarithmic: O(log n) linear: O(n) Subquadratic: O(n log n) Polynomial Algorithms Bad (relatively speaking) quadratic: O(n 2 ) cubic: O(n 3 ) Ugly exponential: O(a n ), a > 1 factorial: O(n!) Exponential Algorithms

23 Reasonable vs. Unreasonable Algorithms Polynomial algorithms are usually regarded as reasonable. Real time/online computation Search Offline computation Indexing, sorting, Exponential ones are unreasonable. Even offline is not practical.

24 Comparison n log n n n 2 2 n

25 Some Points Big O is about the average case. Worst case and best case are mentioned, but are not concerned. Constant is dropped. Constant becomes irrelevant when n becomes big. The big O of a complex algorithm can be the combination of the big Os of simpler algorithms. Algorithms piled up one above another: addition Algorithms nested in each other: multiplication Data structures matter. Algorithm analysis is very complicated.

26 Example: Quicksort Recursive two-step approach Partition: split a list by half Comparison/swap: put elements in two appropriate groups. Best case: pivots always split the target lists in half evenly. Number of partitions: log 2 n Comparison/swap: n/2 O(nlogn)

27 Example: Quicksort Worst case: pivots are always the smallest number in the target lists Partition: n Comparison/swap: n/2 O(n 2 ) General case: Discrete math methods to calculate O(1.38nlogn)

28 Example: Network Similarity

29

30 Space Complexity Time complexity and space complexity Trade off Fast algorithms usually require more memory space. Bubble sorting Time complexity: O(n 2 ) Space complexity: O(1) Quick sorting Time complexity: O(nlogn) Space complexity: O(nlogn)

31 Take Home Messages Algorithm complexity Big O: the order of complexity Where is the number n? Polynomial or exponential Reasonable vs. unreasonable algorithms Online computation: sub-linear. Offline: quadratic is OK. Trade-off between time complexity and space complexity.