Data Structures Week #9. Sorting

Transcription

1 Data Structures Week #9 Sortig

2 Outlie Motivatio Types of Sortig Elemetary (O( 2 )) Sortig Techiques Other (O(*log())) Sortig Techiques 21.Aralık.2010 Boraha Tümer, Ph.D. 2

3 Sortig 21.Aralık.2010 Boraha Tümer, Ph.D. 3

4 Motivatio Sortig is a fudametal task i may computer sciece problems. To sort a set of data is to put the data poits (or records) i some order with regard to some feature of data. I a studet registratio system, a example to sortig would be to put the studet records i ascedig order with respect to studets IDs. 21.Aralık.2010 Boraha Tümer, Ph.D. 4

5 Types of Sortig Sortig techiques may be classified based o whether the etirety of data fits i the mai memory. Sortig techiques for data that etirely fit i the mai memory are called the iteral sortig techiques. Those for data that do ot are called the exteral sortig techiques. We will discuss iteral sortig techiques. 21.Aralık.2010 Boraha Tümer, Ph.D. 5

6 Elemetary Sortig (O( 2 ))Techiques We discuss three (iteral) sortig techiques Selectio Sort Isertio Sort Bubble Sort 21.Aralık.2010 Boraha Tümer, Ph.D. 6

7 Selectio Sort void selectiosort(usiged it *a, usiged it ) //sorts itegers i ascedig order (i.e., a[]<a[+1]) { it i, j, mi, t; for (i=1; i<; i++) { // i th smallest umber gets placed i its correct positio. mi=i; for (j=i+1; j<=; j++) if (a[j] < a[mi]) mi=j; t=a[mi]; a[mi]=a[i]; a[i]=t; } } 21.Aralık.2010 Boraha Tümer, Ph.D. 7

8 Selectio Sort Example i mi j a[j] < a[mi] for (i=1; i<; i++) { mi:=i; for (j=i+1; j<=; j++) if (a[j] < a[mi]) mi:=j; t:=a[mi]; a[mi]=a[i]; a[i]=t; } 21.Aralık.2010 Boraha Tümer, Ph.D. 8

9 Selectio Sort Example i mi j a[j] < a[mi] for (i=1; i<; i++) { mi:=i; for (j=i+1; j<=; j++) if (a[j] < a[mi]) mi:=j; t:=a[mi]; a[mi]=a[i]; a[i]=t; } etc. 21.Aralık.2010 Boraha Tümer, Ph.D. 9

10 Algorithm Aalysis of Selectio Sort Barometer statemet (or piece of code): the coditio of if (a[j] < a[mi]) We fid how may times it is executed The outer loop turs -1 times. Ier loop turs -i times at the i th tur of the outer loop. 1 1 = ( i) = i= 1 j= i ( + 1) 2 i= 1 i= 1 i= 1 2 = 2 O( 2 21.Aralık.2010 Boraha Tümer, Ph.D ) i =

11 Isertio Sort Isertio sort keeps the left portio of the array sorted. The isertio sort algorithm places the curret (i.e., i th ) elemet at its correct positio at the i th tur of the outer loop. 21.Aralık.2010 Boraha Tümer, Ph.D. 11

12 Isertio Sort Algorithm void isertiosort() { it i, j, v; it smallest; // boolea variable for (i=2; i<=; i++) { // a sigle elemet array is sorted as is. v=a[i]; j=i; smallest=0; while (a[j-1] > v &&!smallest) { a[j] =a[j-1]; j=j-1; if (j <= 1) smallest=1; } a[j]=v; } } 21.Aralık.2010 Boraha Tümer, Ph.D. 12

13 Isertio Sort Example Iitial sequece After fourth tur of for loop v=16<a[j-1]=48 After first tur of for loop v=24<a[j-1]=48 After secod tur of for loop v=20<a[j-1]={24,48} After third tur of for loop v=8<a[j-1]={16,...,48} v=12<a[j-1]={16,...,48} After fifth tur of for loop v=32<a[j-1]={16,...,48} After sixth tur of for loop v=54<a[j-1]=48 false! After seveth tur of for loop v=72<a[j-1]=54 false! After eighth tur of for loop Aralık.2010 Boraha Tümer, Ph.D. 13

14 Algorithm Aalysis of Isertio Sort Here, the ier loop has a stochastic coditio. Hece, we either have to work usig expected executio times or make a worst case aalysis. Barometer statemet (or piece of code): j=j-1 We fid how may times it is executed The outer loop agai turs -1 times. I the worst case, ier loop turs i-1 times at the i th tur of the outer loop. 21.Aralık.2010 Boraha Tümer, Ph.D. 14

15 21.Aralık.2010 Boraha Tümer, Ph.D. 15 Algorithm Aalysis of Isertio Sort cot d ) ( 2 2 1) ( 1 2 1) ( 1) ( O i i i j i = + = = = = =

16 Bubble Sort Pass through the array of elemets Exchage adjacet elemets, if ecessary Whe o exchages are required, the array is sorted. Make as may passes as the umber of elemets of the array 21.Aralık.2010 Boraha Tümer, Ph.D. 16

17 Algorithm of Bubble Sort void bubblesort() { it i, j, t; for (i=; i>=1; i--) for (j=2; j<=i; j++) do if (a[j-1] > a[j]) { t:=a[j-1]; a[j-1]:=a[j]; a[j]:=t; } } 21.Aralık.2010 Boraha Tümer, Ph.D. 17

18 Bubble Sort Example Iitial sequece a[j-1]>a[j] swap After first tur of outer for loop a[j-1]>a[j] swap After secod tur of outer for loop ier loop swaps outer loop swaps After third tur of outer for loop a[j-1]>a[j] swap After fourth tur of outer for loop No more swaps ecessary! Fial sequece a[j-1]>a[j] swap Keys i purple spots are those that have bee swapped to move towards their correct positio Aralık.2010 Boraha Tümer, Ph.D. 18

19 Algorithm Aalysis of Bubble Sort Barometer statemet (or piece of code): the coditio of if (a[j-1] > a[j]) We fid how may times it is executed The outer loop turs times. Ier loop turs i-1 times at the i th tur of the outer loop. i 1 = ( i 1) = i= 1 j= 2 i= 1 i= 1 i= 1 ( + 1) 2 2 = 2 i O( 2 2 ) 1= 21.Aralık.2010 Boraha Tümer, Ph.D. 19

20 Other (O(*lg())) Sortig Techiques We discuss three (iteral) sortig techiques Heapsort Mergesort Quicksort 21.Aralık.2010 Boraha Tümer, Ph.D. 20

21 Heapsort Idea: Delete_Mi operatio i a miimum heap removes the key i root. The hole at the root percolates dow where, i the array we keep the miimum heap, proper keys are moved left accordigly. The last cell i the array becomes available for storig aother key. If we perform the delete_mi operatio times i a -elemet mi heap ad place the i th key removed at the available cell A[-i], we will obtai a sorted sequece of the keys i the heap i descedig order. 21.Aralık.2010 Boraha Tümer, Ph.D. 21

22 Algorithm of Heapsort it heapsort(elmt_type *A) { // sorts keys i miheap A i descedig order... Elmt_Type x; for (i=1;i<=;i++) { x=deletemi(a); // O(lg) A[-i]=x; // O(1) } } 21.Aralık.2010 Boraha Tümer, Ph.D. 22

23 Heapsort Example Empty Empty Empty Aralık.2010 Boraha Tümer, Ph.D. 23

24 Heapsort Example Empty Empty Empty Aralık.2010 Boraha Tümer, Ph.D. 24

25 Heapsort Example Empty Empty Empty Shifts by are performed to restore the heap order property of the miimum heap. 21.Aralık.2010 Boraha Tümer, Ph.D. 25

26 Heapsort Example Empty Empty Empty Shifts by are performed to restore the heap order property of the miimum heap. 21.Aralık.2010 Boraha Tümer, Ph.D. 26

27 Heapsort Example Empty Empty Empty Shifts by are performed to restore the heap order property of the miimum heap. 21.Aralık.2010 Boraha Tümer, Ph.D. 27

28 Heapsort Example Empty Empty Empty Shifts by are performed to restore the heap order property of the miimum heap. 21.Aralık.2010 Boraha Tümer, Ph.D. 28

29 Heapsort Example Empty Empty Empty Shifts by are performed to restore the heap order property of the miimum heap. 21.Aralık.2010 Boraha Tümer, Ph.D. 29

30 Heapsort Example Empty Empty Empty Shifts by are performed to restore the heap order property of the miimum heap. 21.Aralık.2010 Boraha Tümer, Ph.D. 30

31 Heapsort Example Empty Empty Sequece i the array is sorted i descedig order! Empty Aralık.2010 Boraha Tümer, Ph.D. 31

32 Algorithm Aalysis of Heap Sort Every tur of the for loop: Oe delete_mi is performed, O(lg ); Key i the root is placed i last elemet of the curret queue, O(1). For loop is executed times for a heap with elemets Hece, ruig time of heap sort is O( lg). 21.Aralık.2010 Boraha Tümer, Ph.D. 32

33 Mergig two sorted lists Cosider two sorted lists L1, L2 ad a empty list R. The followig algorithm merges L1 ad L2 i R: a ad b poit to first elemet of L1 ad L2, respectively, c poits to the empty resultig list; while both lists have elemets left select as the ext elemet of resultig list mi(l1[a],l2[b]); advace the poiters or idices of the result list ad the iput list with the smaller elemet Apped rest of loger list (amog L1 ad L2) to R 21.Aralık.2010 Boraha Tümer, Ph.D. 33

34 Mergig two sorted lists: Algorithm struct odetype { it k; struct ode * ext; } typedef struct odetype odetype; typedef odetype * odeptrtype; it N,M; odeptrtype z;// z is a special header odeptrtype merge (odeptrtype a, odeptrtype b) { odeptrtype c; c=z; do if (a->k <=b->k) {c->ext=a; c=a; a=a- >ext;} else {c->ext=b; c=b; b=b->ext;} while (c->k < maxit); merge=z->ext; z->ext=z; } mai( ); { scaf ( %d %d, &M, &N z=malloc(odetype); z->k=maxit; z->ext=z /* Lists a ad b are costructed here */ merge (a,b); } 21.Aralık.2010 Boraha Tümer, Ph.D. 34

35 Mergig two sorted lists: Example L L c R z 21.Aralık.2010 Boraha Tümer, Ph.D. 35

36 Mergig two sorted lists: Example L1 a b L a<b c R 14 z 21.Aralık.2010 Boraha Tümer, Ph.D. 36

37 Mergig two sorted lists: Example L1 a b L a<b R c z Aralık.2010 Boraha Tümer, Ph.D. 37

38 Mergig two sorted lists: Example L1 a b L a<b c R z Aralık.2010 Boraha Tümer, Ph.D. 38

39 Mergig two sorted lists: Example L1 a b L a<b c R z 21.Aralık.2010 Boraha Tümer, Ph.D. 39

40 Mergig two sorted lists: Example L1 a b L a<b c R 14 z Aralık.2010 Boraha Tümer, Ph.D. 40

41 Mergig two sorted lists: Example L1 a b L a<b ull c R z 21.Aralık.2010 Boraha Tümer, Ph.D. 41

42 Mergig two sorted lists: Example L1 a b L a<b c R z 21.Aralık.2010 Boraha Tümer, Ph.D. 42

43 Mergesort: Idea This is a divide & coquer algorithm. Give the merge algorithm, we may sort a sequece of keys by dividig the sequece ito (divide sectio) 2 subsequeces each of /2 elemets, 4 subsequeces each of /4 elemets, 8 subsequeces each of /8 elemets,... subsequeces each of 1 elemet, recursively sort (usig mergesort) each cosecutive pair of sequeces, (coquer sectio) merge the two sorted subsequeces to obtai the sorted sequece (combie sectio) 21.Aralık.2010 Boraha Tümer, Ph.D. 43

44 Mergesort: Algorithm odeptrtype sort (odeptrtype c; it N) { odeptrtype a,b; it i; if (c->ext == z) retur c; else { a=c; for (i=1;i<n/2;i++;c=c->ext); b=c->ext; c->ext=z; retur merge(sort(a,n/2), sort(b,n-(n/2)) } } 21.Aralık.2010 Boraha Tümer, Ph.D. 44

45 Mergesort: Example z a sort z z b 21.Aralık.2010 Boraha Tümer, Ph.D. 45

46 Mergesort: Example a z a sort b z 37 z Aralık.2010 Boraha Tümer, Ph.D. 46

47 Mergesort: Example b z a sort b z 40 merge z 43 a z Aralık.2010 Boraha Tümer, Ph.D. 47

48 Mergesort: Example a b z 37 z merge a z Aralık.2010 Boraha Tümer, Ph.D. 48

49 Mergesort: Example z z b sort z a b Aralık.2010 Boraha Tümer, Ph.D. 49

50 Mergesort: Example z z a b sort sort z z z z a b a b 21.Aralık.2010 Boraha Tümer, Ph.D. 50

51 Mergesort: Example z z z z a merge b a merge b z z a b 21.Aralık.2010 Boraha Tümer, Ph.D. 51

52 Mergesort: Example z z a merge b z a 21.Aralık.2010 Boraha Tümer, Ph.D. 52

53 Mergesort: Example a z z a z b merge Aralık.2010 Boraha Tümer, Ph.D. 53

54 Algorithm Aalysis of Mergesort The maximum umber of comparisos per mergig of two sequeces for a total of elemets is -1= Θ(). How may partitios of how may elemets are merged? Mergig oce /2 keys with /2 keys = /2+/2-1 Mergig twice /4 keys with /4 keys = 2(/4+/4-1) Mergig 4 times /8 keys with /8 keys = 4(/8+/8-1)... Mergig /2 times oe key with aother key = /2. 21.Aralık.2010 Boraha Tümer, Ph.D. 54

55 21.Aralık.2010 Boraha Tümer, Ph.D. 55 Algorithm Aalysis of Mergesort ( ) t t t O O i i i i i lg ) ( 1) (2 lg ) ( 2 2 ) ( ) ( ) lg ( 1 lg lg 0 1 lg 0 lg lg lg Θ = = = = = = = Hece, ruig time of mergesort is Θ( lg).

56 Quicksort: Idea Aother popular divide & coquer sort algorithm. The idea here is that, at every call of a specific partitioig algorithm, a selected key (pivot) is placed at its correct positio, ad all keys less are moved to left while those greater tha that key are carried over to the right of the key. The origial partitioig algorithm used i quicksort is devised by C.A.R. Hoare. 21.Aralık.2010 Boraha Tümer, Ph.D. 56

57 Quicksort: Idea After partitioig, the origial sequece is divided ito three subsequeces: 1. Numbers to the left of pivot (still eeds sortig) 2. Pivot (correctly placed) 3. Numbers to the right of pivot (still eeds sortig) Recursively callig quicksort fuctio for the left ad right subsequeces, we sort the etire array. 21.Aralık.2010 Boraha Tümer, Ph.D. 57

58 Quicksort: Algorithm Quicksort(A,l,r) { //A is the array holdig the key sequece // l,r are the lowest ad highest idex values of // the key sequece i respective order. // m is the idex value of the pivot. if (l<r) m=partitio(a,l,r); Quicksort(A,l,m-1); Quicksort(A,m+1,r); } 21.Aralık.2010 Boraha Tümer, Ph.D. 58

59 Quicksort: Coceptual Example pivot Aralık.2010 Boraha Tümer, Ph.D. 59

60 pivot Quicksort: Coceptual Example Aralık.2010 Boraha Tümer, Ph.D. 60

61 Quicksort: Coceptual Example pivot 11 6 Empty! pivot Aralık.2010 Boraha Tümer, Ph.D. 61

62 pivot Quicksort: Coceptual Example Aralık.2010 Boraha Tümer, Ph.D. 62

63 Quicksort: Coceptual Example pivot pivot Aralık.2010 Boraha Tümer, Ph.D. 63

64 Quicksort: Coceptual Example pivot The right subsequece of the origial sequece is sorted similarly. 21.Aralık.2010 Boraha Tümer, Ph.D. 64

65 Partitioig*: Idea The idea here is to group keys so that all keys less tha pivot are at left of pivot, ad all greater keys are at the right of pivot, The pivot is the last key of the sequece. At ay time, the rest of the sequece cosists of three subsequeces: 1. the left subsequece holds keys less tha the pivot; 2. the middle subsequece holds keys greater tha the pivot; 3. the right subsequece holds keys ot yet sorted; Iitially, the left ad middle subsequeces are empty. Ay time a key i the right subsequece is processed (i.e., compared with pivot), it is purged from the right ad placed to the left or middle sequece. *mai referece for this algorithm is [1] 21.Aralık.2010 Boraha Tümer, Ph.D. 65

66 Partitioig: Algorithm Partitio(A,l,r) { x=a[r]; i=l-1; for (j=l;j<r; j++) { if (A[j]<=x) {i++; swap(a[i],a[j])} } swap(a[i+1],a[r]); retur i+1; } *mai referece for this algorithm is [1] 21.Aralık.2010 Boraha Tümer, Ph.D. 66

67 Partitioig: Example i l j Iitial Sequece r l i j After 1st tur of for loop r l i j After 2d tur of for loop r Aralık.2010 Boraha Tümer, Ph.D. 67

68 Partitioig: Example l i j After 3rd tur of for loop r l i j After 4th tur of for loop r l i After 5th tur of for loop j r Aralık.2010 Boraha Tümer, Ph.D. 68

69 l Partitioig: Example i After 6th tur of for loop j r l i After 7th tur of for loop j r l After 8th tur of for loop i j r Aralık.2010 Boraha Tümer, Ph.D. 69

70 l Partitioig: Example After 9th tur of for loop i j r l After 10th tur of for loop i j r l After 11th tur of for loop i j r Aralık.2010 Boraha Tümer, Ph.D. 70

71 l Partitioig: Example After 12th tur of for loop i j r l After 13th tur of for loop i j r l After 14th tur of for loop i j r Aralık.2010 Boraha Tümer, Ph.D. 71

72 Partitioig: Example l After 15th tur of for loop i j r l After 16th tur of for loop i j r l After 17th ad 18th tur of for loop i r j Aralık.2010 Boraha Tümer, Ph.D. 72

73 l Partitioig: Example After executio of retur statemet i r j Aralık.2010 Boraha Tümer, Ph.D. 73

74 Hoare s Partitioig: Idea The idea is to simply check the sequece startig at both eds to see that o keys less tha pivot should remai to right of pivot, ad o greater keys to left, If ay two such keys exist, they are swapped to have them at their correct sides. 21.Aralık.2010 Boraha Tümer, Ph.D. 74

75 Hoare s Partitioig: Idea To accomplish this, two poiters (e.g., i ad j) start at both eds of the sequece to sca through movig towards each other. Wheever a poiter locates a key i a icorrect positio (i.e., a greater key to left of pivot or a key less tha the pivot at its right), it stops movig. Whe both poiters stop, the keys they poit to are swapped ad they restart movig. Scaig termiates whe poiters pass crossig each other, ad the key is placed at the positio poited to by j. 21.Aralık.2010 Boraha Tümer, Ph.D. 75

76 Hoare s Partitioig: Algorithm Partitio_Hoare(A,l,r) { x=a[l]; i=l-1; j=r+1; while true do j-=1 while A[j]>x; do i+=1 while A[i]<x; if (i<j) swap A[i] A[j]; else { swap(a[j],a[l]) retur j; } } 21.Aralık.2010 Boraha Tümer, Ph.D. 76

77 Hoare s Partitioig: Example i l r j Iitial Sequece l i After 1st tur of while loop j r l After 2d tur of while loop j i r i>j, ot exchaged 21.Aralık.2010 Boraha Tümer, Ph.D. 77

78 Hoare s Partitioig: Example l At the ed of partitioig j i r keys < pivot Pivot correctly placed keys > pivot 21.Aralık.2010 Boraha Tümer, Ph.D. 78

79 Algorithm Aalysis for Quicksort Average case: Each pivot at positio m has a left ad a right subsequece with m-1 ad -m keys, respectively, to sort. Hece, the ruig time f() of quicksort for elemets ca be expressed as follows f()=f(m-1)+f(-m)+θ(). The term Θ() is the time that partitioig takes to place the key selected as pivot at its correct positio m. 21.Aralık.2010 Boraha Tümer, Ph.D. 79

80 21.Aralık.2010 Boraha Tümer, Ph.D. 80 Algorithm Aalysis for Quicksort m ad -m ca be ay umber betwee betwee 1 ad. To come up with a geeral solutio, we ca average f(m-1) ad f(-m): [ ] [ ] [ ] )) lg( ( )) lg( ( ) ( ) ( ) ( 2 ) ( ) ( ) ( 2 ) ( ) ( ) ( 1) ( 1 ) ( O O f m f E f E m f E f E m f m f E f E m m m = + = + Θ = + Θ = + Θ + = = = =

81 21.Aralık.2010 Boraha Tümer, Ph.D. 81 Algorithm Aalysis for Quicksort Best Case: The pivot is placed always i the middle. f()=2f(/2)+θ(); )) lg( ( )) lg( ( ) ( lg ) ( lg 2 ; 2 2 ) ( 0 2) ( : ; 2 1) ( 2 ) ( 2 ); ( 2) / ( 2 ) ( O O f c c f k k c c k f x CE c k f k f f f k k k k k = + = + = = = + = = + = = + Θ =

82 21.Aralık.2010 Boraha Tümer, Ph.D. 82 Algorithm Aalysis for Quicksort Worst Case: The sequece is sorted i the opposite order.. f()=f(-1)+θ(); ) ( ) (1 ) ( ; ) ( 0 1) ( : ; 1) ( ) ( ); ( 1) ( ) ( O O f c c c f x CE c f f f f = + + = + + = = + = + Θ =

83 Selectig the pivot... Pivot selectio method may essetially affect quicksort performace. Selectig the first (or the last) would work alright if the sequece is purely radomly built. If ot, the worst case is ot totally ulikely to occur. Selectig the pivot radomly works well. However, the radom umber geerator should geerate umbers sufficietly radomly. Furthermore, radom umber geeratio is a expesive process. Media-of-3 partitioig (media of a sequece is the /2 highest amog keys i a sequece) is to select as the pivot the secod largest amog the leftmost, rightmost ad middle keys i the sequece. 21.Aralık.2010 Boraha Tümer, Ph.D. 83

84 A O()-Expected-Time Selectio Algorithm Quicksort ca be modified to select the k th largest key i the sequece. 1. Select a pivot 2. Have it placed at its correct positio m 3. If (k<m) recursively call quicksort for the left subsequece oly 4. Else recursively call quicksort for the right subsequece oly. 21.Aralık.2010 Boraha Tümer, Ph.D. 84

85 Coceptual Example: O()-Expected- Time Selectio Pivot Problem: Fid third smallest key (i.e., key at cell 2)! Iitial Sequece placed at array cell 10 Sice 2 < 10, we cosider oly umbers less tha placed at array cell 8 Sice 2 < 8, we cosider oly umbers less tha placed at array cell 2 Sice 2=2, we stop. Third smallest key is Aralık.2010 Boraha Tümer, Ph.D. 85

86 Referece... [1] T.H. Corme, C.E. Leiserso, R.L. Rivest, C. Stei, Itroductio to Algorithms, 2d editio, MIT Press, 2003 [2] M.A. Weiss, Data Structures ad Algorithm Aalysis i C, 2d editio, Addiso-Wesley, Aralık.2010 Boraha Tümer, Ph.D. 86