Algorithms and Data Structures CS-CO-412

Size: px

Start display at page:

Download "Algorithms and Data Structures CS-CO-412"

Gavin Hamilton
5 years ago
Views:

1 Algorithms and Data Structures CS-CO-412 David Vernon Professor of Informatics University of Skövde Sweden Algorithms and Data Structures 1 Copyright D. Vernon 2014 Course Outline Motivation & Preview The importance of algorithms & data structures Complexity of algorithms Performance of algorithms Time and space tradeoff Worst case and average case performance Big O notation Recurrence relationships Analysis of complexity of iterative and recursive algorithms Tractable and intractable algorithmic complexity Algorithms and Data Structures 2 Copyright D. Vernon 2014

2 Course Outline Simple searching algorithms Linear search Binary search Simple sorting algorithms Bubblesort Quicksort Abstract Data Types (ADTs) Algorithms and Data Structures 3 Copyright D. Vernon 2014 Course Outline Lists, stacks, and queues ADT specification Array implementation Linked-list implementations Trees Binary trees Binary search trees Depth-first traversals Applications of trees (e.g. Huffman coding) Height-balanced trees (e.g. AVL Trees, Red-Black Trees) Algorithms and Data Structures 4 Copyright D. Vernon 2014

3 Course Outline Priority queues Priority queue data structure Binary heap Heapsort Graphs Graph data structure Breadth-first search traversal Depth-first search traversal Minimum spanning tree (e.g. Prim s and Kruskal s algorithms) Shortest-path algorithms (e.g. Dijkstra s and Floyd s algorithms) Topological sort Algorithms and Data Structures 5 Copyright D. Vernon 2014 Course Outline Algorithmic strategies Brute-force Divide-and-conquer Greedy Dynamic programming Combinatorial Search & Backtracking Combinatorics: subsets and permutations All paths in a graphs Branch-and-bound Algorithms and Data Structures 6 Copyright D. Vernon 2014

4 Motivation & Preview The Importance of Algorithms & Data Structures Algorithms and Data Structures 7 Copyright D. Vernon 2014 Muḥammad ibn Mūsā al-khwārizmī محمد بن موسى الخوارزمي Born approximately 780, died between 835 and 850 Persian mathematician and astronomer from the Khorasan province of present-day Uzbekistan The word algorithm is derived from his name Algorithms and Data Structures 8 Copyright D. Vernon 2014

Inventor of Pascal and Modula Inventor programming of languages Pascal

5 Algorithms and Data Structures 9 Copyright D. Vernon 2014 Algorithms + Data Structures = Programs Niklaus Wirth, 1976 Inventor of Pascal and Modula Inventor programming of languages Pascal and Modula Winner of Turing Award Algorithms and Data Structures 10 Copyright D. Vernon 2014

6 Algorithms + Data Structures = Programs Information Processing: Representation & Transformation Input Output Algorithms and Data Structures 11 Copyright D. Vernon 2014 The So6ware Development Process Computational Modelling System Specifica=on Software Design Problem Iden=fica=on Problem Modelling Requirements Elicia=on Validation Algorithms & Representa=ons Testing: Verification & Evaluation So6ware Coding Waterfall Model Software development life-cycle Algorithms and Data Structures 12 Copyright D. Vernon 2014

Marr s Hierarchy of Abstrac)on / Levels of Understanding Framework Computa)onal Theory Representa)on & Algorithm Hardware/So<ware Implementa)on Goal, logic, strategy, model Loose coupling I/O

Vernon 2014 Marr s Hierarchy of Abstrac)on / Levels of Understanding Framework Trying to understand percep)on by studying only neurons is like trying to understand bird flight by studying only

7 Marr s Hierarchy of Abstrac)on / Levels of Understanding Framework Computa)onal Theory Representa)on & Algorithm Hardware/So<ware Implementa)on Goal, logic, strategy, model Loose coupling I/O representa)on, transforma)on algorithm Loose coupling Physical realiza)on Algorithms and Data Structures 13 Copyright D. Vernon 2014 Marr s Hierarchy of Abstrac)on / Levels of Understanding Framework Trying to understand percep)on by studying only neurons is like trying to understand bird flight by studying only feathers: it just cannot be done. In order to understand bird flight, we have to understand aerodynamics; only then do the structure of feathers and the different shapes of birds wings make sense Marr, D. Vision, Freeman, Algorithms and Data Structures 14 Copyright D. Vernon 2014

8 Sorting a List Given a sequence of n keys a 1,, a n Computa)onal Theory Representa)on & Algorithm Find the permuta)on (reordering) such that a i a j 1 i, j n Hardware/So<ware Implementa)on Algorithms and Data Structures 15 Copyright D. Vernon 2014 Computa)onal Theory Bubble Sort Sorting a List Representa)on & Algorithm Hardware/So<ware Implementa)on Insertion Sort Quick Sort Merge Sort, Key point: different computational efficiency Algorithms and Data Structures 16 Copyright D. Vernon 2014

9 Computa)onal Theory Representa)on & Algorithm Sorting a List Hardware/So<ware Implementa)on Algorithms and Data Structures 17 Copyright D. Vernon 2014 Computa)onal Theory Representa)on & Algorithm Sorting a List Hardware/So<ware Implementa)on Algorithms and Data Structures 18 Copyright D. Vernon 2014

10 Fourier Transform Computa)onal Theory Representa)on & Algorithm Hardware/So<ware Implementa)on Algorithms and Data Structures 19 Copyright D. Vernon 2014 Computa)onal Theory Representa)on & Algorithm Hardware/So<ware Implementa)on Fourier Transform DFT: Discrete Fourier Transform FFT: Fast Fourier Transform FFTW: Fasted Fourier Transform in the West Key point: different computational efficiency Algorithms and Data Structures 20 Copyright D. Vernon 2014

Vernon 2014 Hardware/So<ware Implementa)on Algorithms and Data Structures

11 Computa)onal Theory Representa)on & Algorithm Fourier Transform Hardware/So<ware Implementa)on Algorithms and Data Structures 21 Copyright D. Vernon 2014 Computa)onal Theory Representa)on & Algorithm Fourier Transform Hardware/So<ware Implementa)on Algorithms and Data Structures 22 Copyright D. Vernon 2014

12 Marr s Levels of Understanding Framework updated 2012 by T. Poggio Learning & Development Calibra)ng & improving the model Computa)onal Theory CS- AI- 421 Ar)ficial Cogni)ve Systems CS- AI- 422 Machine Learning Representa)on & Algorithm Hardware/So<ware Implementa)on Algorithms and Data Structures 23 Copyright D. Vernon 2014 Marr s Levels of Understanding Framework updated 2012 by T. Poggio Evolu)on Genera)ng new models Learning & Development Calibra)ng & improving the model Computa)onal Theory Representa)on & Algorithm Hardware/So<ware Implementa)on Algorithms and Data Structures 24 Copyright D. Vernon 2014

13 Algorithms and Data- Structures Strategy Difficulty Sample of the core ADS topics from 2-3 standard undergraduate Algorithms and Data- Structures courses Gradually increase the slope of the learning curve Time Algorithms and Data Structures 25 Copyright D. Vernon 2014 Algorithms and Data- Structures Topics Difficulty Sample of the core ADS topics from 2-3 standard undergraduate Algorithms and Data- Structures courses Gradually increase the slope of the learning curve Graphs Heapsort Greedy Divide & Conquer Branch & Bound Backtracking Binary heaps Priority queues Height-balanced trees Binary search trees Complexity Searching Sorting ADTs Lists, Stacks, Queues Time Algorithms and Data Structures 26 Copyright D. Vernon 2014

14 Algorithms and Data- Structures Topics Complexity of algorithms: Difficulty Performance of algorithms Big O nota)on Recurrence rela)onships Analysis of complexity Itera)ve and recursive algorithms Tractable, intractable complexity Complexity Time Algorithms and Data Structures 27 Copyright D. Vernon 2014 Use Big O notation to determine an upper bound function g(n) that characterizes how the complexity grows with the size of the data set O(g(n)) e.g. O(n log 2 n) Algorithms and Data Structures 28 Copyright D. Vernon 2014

15 Algorithms and Data Structures 29 Copyright D. Vernon 2014 function/ n Exponential Polynomial n 2 1/10,000 second n 5 2 n n n 1/10 second 1/1000 second 2.8 hours 1/2,500 second 3.2 seconds 1 second 3.3 trillion years 1/400 second 5.2 minutes 35.7 years a 70 digitnumber of centuries 1/100 second 2.8 hours 400 trillion centuries a 185 digitnumber of centuries 9/100 second 28.1 days a 75 digitnumber of centuries a 728 digitnumber of centuries Algorithms and Data Structures 30 Copyright D. Vernon 2014

searching algorithms Difficulty linear search O(n) binary search

16 Algorithms and Data Structures 31 Copyright D. Vernon 2014 Algorithms and Data- Structures Topics Simple searching algorithms Difficulty linear search O(n) binary search O(log 2 n) Complexity Searching Time Algorithms and Data Structures 32 Copyright D. Vernon 2014

17 A B D F G J K M O P R Algorithms and Data Structures 33 Copyright D. Vernon 2014 A B D F G J K M O P R Algorithms and Data Structures 34 Copyright D. Vernon 2014

18 A B D F G J K M O P R Algorithms and Data Structures 35 Copyright D. Vernon 2014 A B D F G J K M O P R Algorithms and Data Structures 36 Copyright D. Vernon 2014

Algorithms and Data- Structures Topics Simple sor)ng algorithms: Difficulty Bubblesort (Itera)ve O(n 2 ) ) Quicksort (Recursive O(n log 2 n) ) Complexity Searching Sorting Time Algorithms and Data

19 Algorithms and Data- Structures Topics Simple sor)ng algorithms: Difficulty Bubblesort (Itera)ve O(n 2 ) ) Quicksort (Recursive O(n log 2 n) ) Complexity Searching Sorting Time Algorithms and Data Structures 37 Copyright D. Vernon 2014 /* Quicksort: assume A[R] contains a sentinel */ void quicksort (int A[], int L, int R) { int i, j, pivot; if ( R > L) { i = L; j = R; pivot = A[i]; do { while (A[i] <= pivot) i=i+1; while ((A[j] >= pivot) && (j>l)) j=j-1; if (i < j) { exchange(a[i],a[j]); /*between partitions*/ i = i+1; j = j-1; } } while (i <= j); exchange(a[l], A[j]); /* reposition pivot */ quicksort(a, L, j); } } quicksort(a, i, R); /*includes sentinel*/ Algorithms and Data Structures 38 Copyright D. Vernon 2014

4 9 8 10 11 99 Algorithms and Data Structures 39 Copyright D.

20 Algorithms and Data Structures 39 Copyright D. Vernon 2014 Algorithms and Data- Structures Topics Abstract Data Types (ADTs) Difficulty Informa)on hiding Encapsula)on Data- hiding Basis for object- orienta)on Complexity Searching Sorting ADTs Time Algorithms and Data Structures 40 Copyright D. Vernon 2014

Vernon 2014 Algorithms and Data- Structures Topics Lists, Stacks, and Queues Difficulty ADT specifica)on Array

21 Algorithms and Data- Structures Topics Difficulty David Parnas Advisory Board Innopolis University Complexity Searching Sorting ADTs Time Algorithms and Data Structures 41 Copyright D. Vernon 2014 Algorithms and Data- Structures Topics Lists, Stacks, and Queues Difficulty ADT specifica)on Array implementa)on Linked- list implementa)on Complexity Searching Sorting ADTs Lists, Stacks, Queues Time Algorithms and Data Structures 42 Copyright D. Vernon 2014

22 Example of List manipula6on w = Next(Last(L),L) Insert(e,w,L) w = Previous(w,L) Delete(w,L) Algorithms and Data Structures 43 Copyright D. Vernon 2014 Header node List element pointer NULL pointer Algorithms and Data Structures 44 Copyright D. Vernon 2014

23 temp List window To delete a node at this window position we re-arrange the links and free the node Algorithms and Data Structures 45 Copyright D. Vernon 2014 Algorithms and Data Structures 46 Copyright D. Vernon 2014

24 Head Tail Head Tail Algorithms and Data Structures 47 Copyright D. Vernon 2014 Algorithms and Data- Structures Topics Trees Difficulty Binary trees Binary search trees Depth- first traversal Breadth- first traversals Applica)ons of trees (e.g. Huffman coding) Height- balanced trees (e.g. AVL Trees, Red- Black Trees) Height-balanced trees Binary search trees Complexity Searching Sorting ADTs Lists, Stacks, Queues Time Algorithms and Data Structures 48 Copyright D. Vernon 2014

25 Algorithms and Data Structures 49 Copyright D. Vernon 2014 Sun Mon Tue Fri Sat Thur Wed Binary Search Tree Algorithms and Data Structures 50 Copyright D. Vernon 2014

26 Pointer Implementation Algorithms and Data Structures 51 Copyright D. Vernon C A B D E F G Inorder Traversal Algorithms and Data Structures 52 Copyright D. Vernon 2014

27 AVL Trees Height Balancing +2 A 0 B +1 B A R B L 0 A B L B R B R A R Algorithms and Data Structures 53 Copyright D. Vernon 2014 AVL Trees - RL rotation - 2 A 0 C A L B A 0 B - 1 C B L A L C L C R B L C L C R Algorithms and Data Structures 54 Copyright D. Vernon 2014

28 Red-Black Height-balanced Trees Algorithms and Data Structures 55 Copyright D. Vernon 2014 Algorithms and Data- Structures Topics Priority queues, heaps, graphs: Difficulty Topological sort Minimum spanning tree (e.g. Prim s and Kruskal s) Shortest- path algorithms (e.g. Dijkstra s & Floyd s) Graphs Heapsort Binary heaps Priority queues Height-balanced trees Binary search trees Complexity Searching Sorting ADTs Lists, Stacks, Queues Time Algorithms and Data Structures 56 Copyright D. Vernon 2014

29 Adjacency matrix and list implementa)ons of a graph Algorithms and Data Structures 57 Copyright D. Vernon 2014 Algorithms and Data Structures 58 Copyright D. Vernon 2014

30 Minimum Spanning Trees Tree formed from subset of graph edges that connects all ver)ces and minimizes the total path length Algorithms and Data Structures 59 Copyright D. Vernon 2014 Minimum Spanning Tree Shortest Path from Centre Spanning Tree Algorithms and Data Structures 60 Copyright D. Vernon 2014

Greedy algorithms Combinatorial Search & Backtracking Branch- and- bound Greedy Divide & Conquer Graphs Heapsort Branch &

31 Algorithms and Data Structures 61 Copyright D. Vernon 2014 Algorithms and Data- Structures Topics Difficulty Algorithmic strategies Brute- force Divide- and- conquer Greedy algorithms Combinatorial Search & Backtracking Branch- and- bound Greedy Divide & Conquer Graphs Heapsort Branch & Bound Backtracking Binary heaps Priority queues Height-balanced trees Binary search trees Complexity Searching Sorting ADTs Lists, Stacks, Queues Time Algorithms and Data Structures 62 Copyright D. Vernon 2014

32 Combinatorial Search and Heuris)c Methods Algorithms and Data Structures 63 Copyright D. Vernon 2014 Typical Input for HCP Hamiltonian cycle for the graph Another Hamiltonian cycle for the same graph in A Hamiltonian cycle for a given graph G=(V, E) consists on finding an ordering of the ver)ces of the graph G such that each vertex is visited exactly once Algorithms and Data Structures 64 Copyright D. Vernon 2014

Copyright D. Vernon 2014 See www.algorist.

33 Travelling Salesman Problem or, a variant, the vehicle rou)ng problem Algorithms and Data Structures 65 Copyright D. Vernon 2014 See for online resources Algorithms and Data Structures 66 Copyright D. Vernon 2014

` Algorithms and Data-Structures The foundation of all solutions to computational information processing problems Often unseen, but always there Algorithms and Data Structures hpp://www.wired.

34 ` Algorithms and Data-Structures The foundation of all solutions to computational information processing problems Often unseen, but always there Algorithms and Data Structures hpp:// towing- drinking- water/ 67 Copyright D. Vernon 2014 Marr s Hierarchy of Abstrac)on / Levels of Understanding Framework Computa)onal Theory Goal, logic, strategy, model Representa)on & Algorithm I/O representa)on, transforma)on algorithm Hardware/So<ware Implementa)on Physical realiza)on Algorithms and Data Structures 68 Copyright D. Vernon 2014

Introduction to Algorithms Third Edition

Thomas H. Cormen Charles E. Leiserson Ronald L. Rivest Clifford Stein Introduction to Algorithms Third Edition The MIT Press Cambridge, Massachusetts London, England Preface xiü I Foundations Introduction