Sorting in Linear Time. Data Structures and Algorithms Andrei Bulatov

Transcription

1 Sortig i Liear Time Data Structures ad Algorithms Adrei Bulatov

2 Algorithms Sortig i Liear Time 7-2 Compariso Sorts The oly test that all the algorithms we have cosidered so far is compariso The oly iformatio we obtai about a iput sequece a, a2, K, is give by ai < a j, ai a j, ai = a j, ai a j or a i > a j We will assume that all the iput umbers are differet. The the it suffices to use oly oe compariso ai a j a

3 Algorithms Sortig i Liear Time 7-3 The Decisio Tree Model Observe that for differet iput permutatios, eve if they differ oly i the order of elemets, our algorithm must perform differet actios Therefore, we view its work as recogizig a permutatio It is coveiet to represet this process as a decisio tree :2 2:3 :3,2,3 :3 2,,3 2:3,3,2 3,,2 2,3, 3,2,

4 Algorithms Sortig i Liear Time 7-4 Decisio Trees Let the iput sequece cotai elemets We assume they are umbers from to Therefore, the iput sequece is a permutatio π(), π(2),, π() Each iteral ode is labeled by i : j for some i ad j i the rage i,j Executio of the algorithm is a path from the root to a leaf At each iteral ode a compariso a is made i a j Depedig o the outcome either comparisos o the left or the right subtree Whe we come to a leaf, the algorithm recogizes a particular orderig

5 Algorithms Sortig i Liear Time 7-5 Lower Boud Theorem Ay compariso sort algorithm requires Ω(m) comparisos i the worst case Proof There are! permutatios of elemets. The algorithm should recogize all of them Therefore the decisio tree cotais k! leaves h The complete biary tree has 2 leaves where h is the height of the tree, the legth of the logest root-to-leaf path h! k 2 h log(!) = Ω( log ) Fially, ote that the height of the tree is ruig time i the worst case

6 Algorithms Sortig i Liear Time 7-6 Coutig Sort Suppose we are allowed to do more tha just comparisos We also assume that the iput umbers are i the rage 0 to k We give a algorithm that works i Θ() time provided k = O() The idea is: for each iput elemet x to cout how may elemets are less tha x, ad use this iformatio to place x to the right place i the output sequece We use A[..] the iput array B[..] the output array C[0..k] auxiliary storage Note that we do ot assume that all elemets are differet

7 Algorithms Sortig i Liear Time 7-7 Coutig Sort: Algorithm Coutig-Sort(A,B,k) for i=0 to k do set C[i]:=0 for j= to do set C[A[j]]:=C[A[j]]+ /* C[i] cotais the for i= to k do /* umber of elemets equal to i set C[i]:=C[i]+C[i-] /* C[i] cotais the for j= dowto do /* umber of elemts less tha or equal to i set B[C[A[j]]]:=A[j] set C[A[j]]:=C[A[j]]- edfor

8 Algorithms Sortig i Liear Time 7-8 Coutig Sort: Soudess ad Ruig Time Theorem Coutig Sort correctly sorts a sequece of elemets i O() time, provided k = O() Proof Ruig time: Each of the 4 for loops is iterated at most max{, k} times Soudess: Trace the cotet of arrays QED

9 Algorithms Sortig i Liear Time 7-9 Stable Sortig For the ext algorithm it is importat how Coutig Sort ad other sortig algorithms deal with equal elemets A sortig algorithm is called stable if it preserves the order of equal elemets i the iput sequece, that is, if a i = a j ad i < j i the iput setece, the output sequece Lemma Coutig Sort is stable Proof Suppose that A[i] = A[j] ad i < j. The i the last for loop we first output A[j] the A[i] goes first i the Moreover, betwee the outputs C[A[i]] = C[A[j]] is decremeted. Therefore A[i] is placed ito B with smaller idex tha A[j] ai

10 Algorithms Sortig i Liear Time 7-0 Radix Sort Puch card Radix sortig machie

11 Algorithms Sortig i Liear Time 7- Radix Sort: The Idea If the elemets of the iput sequece are d-digit itegers, we ca first sort the accordig the most sigificat digit, the the secod most sigificat digit, etc. This, however, requires a lot of copyig ad auxiliary storage Radix Sort: Sort i place accordig to the least sigificat digit But use a stable sortig algorithm!!

12 Algorithms Sortig i Liear Time 7-2 Radix Sort: Algorithm Radix-Sort(A,d) for i=0 to d do use a stable sortig algorithm to sort array A o digit i edfor Theorem Radix Sort correctly sorts a sequece of d-digit umbers i which each digit ca take up to k differet values i O(d(+k)) time.

13 Algorithms Sortig i Liear Time 7-3 Radix Sort: Aalysis Ruig time: RadixSort cosiders d digits i tur each pass takes O( + k) time (whe usig, say, Coutig Sort) Soudess: By iductio o the umber d of digits Base Case: d = Coutig Sort just sorts everythig Iductio Step: Suppose algorithm works correctly for d Radix sort o d-digit umbers is equivalet to d rus of radix sort o smaller d- umbers, followed by sortig o digit d. By Iductio Hypothesis Radix Sort sorts correctly o lower d digits

14 Algorithms Sortig i Liear Time 7-4 Radix Sort: Aalysis (ctd) Before the last sort o digit d, all the umbers are properly sorted accordigly their last d digits. Whe sort o digit d, cosider two elemets a ad b with d-th digits ad ad bd respectively () If a d < b d the the algorithm will put a before b, which is correct (2) If a d > b d the algorithm will put b before a, which is agai correct (3) If a d = b d the algorithm will leave a ad b i the same order as before, because it is stable. This order is agai correct, for the relative order of a ad b depeds i this case o the lower d digits. QED

15 Algorithms Sortig i Liear Time 7-5 Bucket Sort: The Idea Coutig Sort ad Radix Sort achieve sigificat speed up agaist compariso algorithms because they use certai assumptios about the iput umbers: They are small itegers, or itegers of bouded size. Bucket Sort uses a assumptio about the distributio of these umbers: They are take uiformly at radom from [0; ) The we: Split [0;) ito equal itervals (buckets) Put every iput elemet ito the correspodig bucket Sort each bucket Cocateate the buckets

16 Algorithms Sortig i Liear Time 7-6 Bucket Sort: Algorithm A[..] the iput array B[..] heads of buckets i-th bucket is orgaized as a list with a poiter B[i] to the top of the list Bucket-Sort(A,d) set :=legth(a) for i=0 to do isert A[i] ito list B[ A[i] ] edfor for i=0 to do sort list B[i] with isertio sort cocateate the lists B[],B[2],...,B[]

17 Algorithms Sortig i Liear Time 7-7 Bucket Sort: Soudess Theorem Bucket Sort correctly sorts a sequece of umbers from the iterval [0;) Proof. Obvious

18 Algorithms Sortig i Liear Time 7-8 Bucket Sort: Ruig Time Theorem The expected ruig time of Bucket Sort is O() Proof. Clearly, the first for loop takes Θ() time to complete Each iteratio of the secod for loop cotributes O is the umber of elemets i the i-th bucket. Therefore T + ( ) = Θ ( ) i= 0 O ( 2 i ) ( 2 i ) time where i

19 Algorithms Sortig i Liear Time 7-9 Bucket Sort: Ruig Time Proof (ctd). Take the expectatio of both sides E T ( )] = E Θ ( ) + i= [ We show that = Θ ) + E i= = Θ ( + ( ) i= 0 0 O ( 0 O ( O ( E [ 2 E[ i ] = 2 2 i ) 2 i ) i 2 ])

20 Algorithms Sortig i Liear Time 7-20 Bucket Sort: Ruig Time Proof (ctd). Defie idicator radom variables X = if ad oly if A[j] falls ito bucket i, otherwise = 0 ij X ij Thus i = X ij We get 2 j= 2 E [ ] = i E X ij i= 0 = E X ij X ik j= k = = 2 E Xij + Xij Xik i= j= k, k j = i= E[ X 2 ij ] + E[ X j= k, k j ij X ik ]

21 Algorithms Sortig i Liear Time 7-2 Bucket Sort: Ruig Time Proof (ctd). 2 The [ ] = + 0 ( ) = ad, sice X ad X are idepedet Fially E X ij ij ik E [ X X ] = E[ X ] E[ X ] = = ij E[ ik 2 i ] = ij j= = + ik 2 2 j= k, j k + ( ) 2 = 2 = +

22 Algorithms Sortig i Liear Time 7-22 Bucket Sort: Ruig Time Proof (ctd). For the ruig time we ow have + [ ( )] = Θ ( ) i= 0 = Θ( ) + i= 0 E T O ( E [ O(2 i 2 ]) ) = Θ( ) + O( ) = Θ( ) QED