Linear Time Complexity Sort Algorithm

Transcription

1 Liear Time Complexity Sort Algorithm M.R. Ghaeii Departmet of Computer Egieerig ad Iformatio Techology, Amirkabir Uiversity of Techology, Tehra, Ira M.R. Mirzababaei Departmet of Computer Egieerig ad Iformatio Techology, Amirkabir Uiversity of Techology, Tehra, Ira Abstract I The field of Computer Sciece ad Mathematics, sortig algorithms put elemets of a list i a certai order, ascedig or descedig. Sortig is perhaps the most widely studied problem i computer sciece ad is frequetly used as a bechmark of a systems performace. I this paper we preset a improved stable sort algorithm based o bucket sort algorithm that statistically ad i average does sortig operatio with liear time complexity O(). Our algorithm is 50% faster tha other compariso sortig algorithms e.g. Quick sort ad Merge sort. Keywords: Liear Time Complexity, o-compariso Sort Algorithm, Sigmoid Fuctio, Probability 1. Itroductio Sortig a geeric list of umbers is a well-studied problem that ca be efficietly solved by usig geeric algorithms, such as Quick Sort [1], ad Shell Sort []. However, a geeric algorithm may ot be the best choice whe the list which goig to be sorted has some iitial order. e.g., whe may elemets i the list are already sorted. A alterative i these cases is to use adaptive algorithms, which take the advatage of the partial order of the list to accelerate the sortig process [3]. A sortig algorithm is called adaptive if it sorts the sequeces that are close to fully sorted faster tha the radom sequeces. It sorts without kowig how far the list is from the sorted sequece [4]. A alterative to reduce the sortig time ad complexity of sortig algorithm is to chage the model that used to determie the key order. Most of the classic sortig algorithms works uder the compariso based model. They sort the list exclusively through pair wise compariso. However, there are also alterative sortig methods, which the cotet of their keys are used to obtai the positio without ay eed to compare them to each other. They ca obtai better results, because real machies allow may other operatios besides the compariso [5]. Examples of o-compariso methods are Radix Sort [6], ad Group Sort [7]. The umber of comparisos that a compariso sort algorithm requires icreases at least i proportio to log, where is the umber of elemets of list. Accordig to Kuths theoretical lower boud theorem for geeral sortig algorithms i compariso sortig algorithms miimum of the time complexity is O(log!) equals to O((log )) [6]: log(!) = log() log(e) + θ (log()) log() (1) Thus, the O((log )) boud is asymptotically tight. We atteded to the Importace of sortig issue ad ascedig humas requiremets to sortig data, to prepare that for processig, its so valuable to desig a kid of sortig algorithm that ca sort data i liear time complexity i average ad domiat case. Oe of the o-compariso based sortig algorithms is the bucket sort. Iput umbers to the bucket sort algorithm is cosidered to be betwee 0 ad 1. This rage divides to arbitrary umber of buckets, ad 544

2 supposed to have a suitable dispersio i our data. The algorithm works with liear time complexity, although, time complexity ca icrease to quadratic time complexity O(). I this algorithm we use a statistical perspective, accordig to reality ad the future usage of a algorithm to sort the data that we expect i real usage, ca be said that these data are reasoably uiform (ormal model ad dispersio) ad we have little data with very high dispersio ad a very irrelevat relatioship with each other. Also, whe we icrease the umber of sort list elemet, uiformity improves to suitable case. Thus, we respect the efficiecy of our sort algorithm i real ad huge usages.. Related Works Sortig is oe of the most widely studied problem i computer sciece ad is frequetly used as a bechmark of a systems performace [8]. We ca classify Sortig algorithm by some viewpoits, ad their classificatio ot similar; therefore, we metio some their specifics. We spot (x, y, z), refer to best time complexity (x), media time complexity (y), ad worst time complexity (z). -Quick Sort [1]: (log, log, ), Compariso, ofte ustable. -Efficiet Quick Sort [9]: (log, log, ), Compariso, ofte ustable. -Partitio Sort [10]: (log, log, (log ) ), Compariso. -Merge Sort [11]: (log, log, log), Compariso, stable. -Average Sort [1]: (log, log, log), Compariso, ustable. -Heap Sort [13]: (log, log, log), Compariso, ustable. -Modified Heap Sort [8]: (log, log, log), Compariso, ustable. -Isertio Sort [11]: (,, ), Compariso, stable. -Selectio Sort [11]: (,, ), Compariso, ustable. -Optimized Selectio Sort [14]: (,, ), Compariso, ustable. -Bubble Sort [15]: (,, ), Compariso, stable. -Freezig Sort [16]: (,, ), Compariso, stable. -Shell Sort []: (, (log ), (log )), Compariso, ustable. -Bucket Sort [11]: (,, ), o-compariso, stable. -Radix Sort [6]: (d( + k), d( + k), d( + k)), o-compariso, stable. Table 1, Algorithms Time Complexity [11] 545 Algorithm Worst-case ruig time Average-case ruig time Isertio Sort Ө() Ө() Merge Sort Ө(log) Ө(log) Heap Sort O(log) - Quick Sort Ө() Ө(log) Coutig Sort Ө(k+) Ө(k+) Radix Sort Ө(d(k+)) Ө(d(k+)) Bucket Sort Ө() Ө()

3 -Coutig Sort [11]: ( + k, + k, + k), o-compariso, stable. -Skewed Distributio Sort [17]: (,, ), o-compariso, stable. I this area, somebody worked i parallel sort algorithm that we ca metio some of these. -Parallel Merge Sort [18],[11]: O( /(log ) ) -Parallel Bucket Sort [19]: O(log ) -Parallel tree-sort [19]: O() Table, Processor Number & Ru Time Required by Parallel Sort Algorithms [19] Algorithm Processors Time Odd-eve traspositio O() Batchers bitoic O(log ) O(log) Stoes bitoic / O(log) Mesh-bitoic Muller-Preparata Hirschberg () O(log) 1+1/k Preparata (1) log Preparata () 1+1/k Ajtai et al ) O(log) Hirschberg (1) O( log O(klog) O(log) O(klog) O(log) 3. Liear Time Complexity Sort Algorithm I this algorithm we use a fuctio to help us i data classificatio. The fuctio ame is Sigmoid fuctio (Equatio ) y ( x) = e x () This fuctio is Ijective, Exteder ad derivable i all of real umber rage (R), ad we ca calculate y by x, ad x by y (Equatio 3). This fuctio maps all of real umbers ito rage of 0 ad 1, (0,1). But out of a symmetrical rage, fuctio output ted to 0 by beig x less tha miimum of that rage ad fuctio ted to 1 by beig x bigger tha maximum of that rage. We cosider aother form of sigmoid fuctio for our problem ad implemetatio. y ( x) = x ( x m ) 1+ e w 1 y x( y) = w l( )+m y (3) I formulas (Equatio 3), m refers to media poit, ad w refers to fuctio weight. Accordig to w we ca icrease or decrease fuctio acceptable rage.this fuctio is symmetrical, but our data may ot be symmetrical e.g. all of them are positive, therefore, we lose half of mappig area, so that, at first we fid data media ad spot that for m value, therefore, our diagram moves o x axis ad put ito our data media data.we must metio that we ca use average of data for m value too, but media value is more robust tha outlier data. If we use media value, we ca warraty that half of data are i half of 546

4 fuctio acceptable area ad aother half of data are i aother area. Figure 1, Sigmoid diagram with differet parameters y ' ( x) = e ( x m ) w ( x m ) w(1 + e w ) (4) We suppose to defie the fuctio acceptable area. The fuctio acceptable area is a area, i which derivatio of fuctio is bigger tha ε. x ( xmi, xmax ) : y' ( x) ε (5) Ad the are is i rage of (xmi, xmax). Figure, Derivatio of sigmoid fuctio As all of our parameters affect x, we ca solve the problem for mai sigmoid fuctio ad geeralize that for our problem. e x e x X =e x x =l( X ) y ' ( x) ε ε = ε εe x + (ε 1)e x + ε = 0 (6) x x (1 + e ) (1 + e ) εx + (ε 1) X + ε = 0 X = 547 (ε 1) ± (ε 1) 4ε (ε 1) ± 1 4ε X= ε ε

5 Fore example we assume ε =0.01 xmi = , xmax = If we wat to geeralize fuctio, we must set fuctio derivatio bigger tha εw but for reachig to our area, we igore w. (x equals to xmi or xmax). d x m (7) x' = β = β x = wβ + m w = w β 3.1 Algorithm At first we fid media value (based o Selectio Algorithm [0]), ext we fid efficiet data distace (d), ad by this value we ca calculate the acceptable area, this area is i rage of (-d,+d). Next, by d, we calculate w (Equatio 7), so that we ca calculate y values for our data ad sort y values by Bucket Sort Algorithm. If umber of data i a bucket get more tha a arbitrary costat (α), a Boolea variable is set to be True that shows we must call our sort for bucket data, but if Boolea variable is False, we sort buckets data i O(1). 3. Efficiet data distace (d) First we fid max1, max, mi1, mi, ad delete max1 ad mi1 form our data ad calculate ew the average, certaily media has the same value (media did t chage). Obviously our data are i rage of [mi, max], ow we wat to fid the best rage that the media is i its ceter (Equatio 8). d = (m mi ) + (max + mi m)c(average, m, mi, max, a, b) (8) 0 :x m a x m C ( x, m, mi, max, a, b) = :m a < x < m a+b b x m a+b 1 mi + max mi + max m< 0 m < max m a = b = mi + max mi + max b m m mi m 3.3 α Determiatio We ca set this value arbitrarily, but if we set a big value, algorithm efficiecy will be decreased, although time complexity wouldt chage. Must ot have ay relatio to, because if we have ay relatioship betwee ad, algorithm time complexity will be chaged. While Boolea value is False, we must check that the umber is ew or ot whe addig it to bucket, if its ew, icrease couter ad if couter is bigger tha α set Boolea value True. 3.4 Pseudo Code Sort(list, α, ε): Media FidMedia(list) Rage FidBaseRage(ε) (Mi1, Mi, Max,Max1) FidMi&Max(list) d FidD(list, media, Mi1, Mi, Max, Max1) W d/rage For i 0 : list.size tempid 548 Exp Re sult (list[i], media, w) * list.size

6 Bucket[tempId].add(list[i], α) For i 0 : list.size+1 If (Bucket[i].isSortAgai=True) the Sort(Bucket[i].list, α, ε) makesortarray() Rage FidBaseRage(ε): Rage l( (ε 1) 1 4ε ) ε (Mi1, Mi, Max,Max1) FidMi&Max(list) : Mi1 list[0] Mi list[0] Max1 list[0] Max list[0] For i=1 : list.size If (list[i]>max) the If (list[i]>max1) the Max1 list[i] Max list[i] If (list[i]<mi) the If (list[i]<mi1) the Mi1 list[i] Mi list[i] d FidD(list, media, Mi1, Mi, Max, Max1): Sum 0 Number 0 For i 0 : list.size If (Mi1 < list[i] < Max1) the Sum Sum + list[i] Number Number + 1 Average Sum/Number c C(Average, media, Mi, Max, a, b) d (media-mi)+(mi+max-media)*c c C(x, media, Mi, Max, a, b) : 549

7 ceter (Mi+Max)/ If (media-ceter 0) the b media Mi a b b Max media a 0 If (x media + a 0) the c 0 If (x media + a b < 0) the x media c b c 1 result ExpResult(umber, media, w): 1 result umber media 1+ e ( w ) Bucket: issortagai False add(elemet,α): if (!issortagai &&!list.haselemet(elemet)) DiffNum DiffNum + 1 If (DiffNum>α) the issortagai True list.add(elemet) makesortedarray: For i 0 : list.size+1 templist SortBucketList(Bucket[i].list) maiarray.add(templist) 4. Implemetatio ad Simulatio We implemet our algorithm by JAVA. The system properties are: Processor: Itel Core Duo E4600,.40GHz / Memory: *1GB DDR 800. We tried to make our iputs similar to real world data ad real usage. To decrease system errors, we tested each dataset 10 times, ad accepted the miimum; we ca use some optimizatio methods i our implemetatio, e.g.: Usig Lookup Table (LUT), or we sort bucket data while Boolea variable is false. We ca also do some optimizatio i algorithm, e.g.: Usig average of data (except mi1 ad max1) istead of media, therefore, d equals to (max mi)/. ad acceptable area equals to (max mi), we ca eve substitute sigmoid fuctio with combiatio of three liear fuctios. All of these optimizatios deped o usage ad situatio of problem. 550

8 As show i Figure 3, ad by a statistical view, our algorithm is more efficiet tha merge sort ad quick sort, ad its gradiet is less tha others. Although, our algorithm have liear time complexity, but those algorithms have O((log )) time complexity. 5. Compariso First we must calculate algorithm time complexity. Oe of the most importat algorithm features is its time complexity; because it will have heavy cost if the data grow up ad algorithm time complexity is t efficiet. Figure 3, Compariso result 5.1 Time complexity determiatio I this algorithm best case ad worst case do t eed to be assumed for below calculatios. 1- media calculatio: we ca fid data media by usig selectio algorithm i O() [0]. - calculatio y values ad determiatio bucket of umbers ad put umbers i their buckets. All of these acts are doe i O(), because we ca calculate exp(x) i O(1) ad we ca put umbers i their bucket i O(1) (we have α compariso i worst case i every additio but α does t have ay relatioship with ), so, we have doe all of them i O(). 3- mi, max, d ad acceptable area calculatio: we ca fid mi ad max i O(), ad calculate d ad acceptable area i O(1) because we do t eed ay traversal. All of them are doe i liear time complexity Best Case Best case of our algorithm is similar to best case of bucket sort, which by ormal or uiform distributio of umbers, Sortig is doe i O(). Number of bucket data is ot depeded to Worst Case If the data distributio is so irrelevat, ad our data is classified ito two classes i each step ad it is cotiued util the ed of sort, we must call our fuctio log times. I this case we ca prove that algorithm time complexity equals to O((log)) (by decisio tree or master theorem [11]). T () = T ( ) + θ () Master Theorem T () = θ ( log ) (9) 551

9 5.1.3 Average Case The iput distributio of worst case is so rare, ad statistically, we ca igore that data situatio ad accept liear time complexity for this algorithm i average case (you ca see average time complexity proof i appedix). I reality the data follows ormal or uiform models ad it is the iverse of worst-case data model. Ad radom dataset cat be a good dataset for sort algorithm compariso. Aother oticeable matter i this algorithm is idepedecy of data positios ad iput sequeces. Statistically, each iput sequeces ca occur ad some algorithms (e.g. Quick sort) are sesitive to iput sequeces. Differet iput sequeces has differet efficiecies, therefore, we have lower reliability i these algorithms. But our algorithm oly depeds o data ot sequeces of them; therefore, it is more reliable. Our algorithm is stable because we traverse iput ad store them respectively, therefore, sequece of data remais. Oe of the algorithm beefits is presetig a good data classificatio ad this algorithm is more tha a sortig algorithm ad we ca use this algorithm i other usages. Best case of our algorithm is similar to best case of bucket sort, which by ormal or uiform distributio of umbers, Sortig is doe i O(). Number of bucket data is ot depeded to. 6. Coclusio We see that statistically, we ca preset a liear sortig algorithm without ay limitatios i iput, ad this is a importat matter because by growig data size we eed lower time complexity algorithms to be able to process data i a reasoable time. Also i worst case situatio, we have O(log ) time complexity which is similar to average case of other compariso sort algorithms. This algorithm supports other abilities of sortig algorithms, e.g. stability. At last, we ca say that this algorithm has may advatages ad we ca use it istead of other sortig algorithms. Refereces 1. Hoare, C.A.R., (196). Quick Sort. The Computer Joural, vol. 5, o. 1, pp Shell, D.L., (195). A high speed sortig procedure. Commuicatios of ACM, vol., o. 7, pp. 30 3,. 3. Estivill-Castro, V., Wood, D., (199). A survey of adaptive sortig algorithms. ACM Computig Surveys, vol. 4, o. 4, pp Petersoo, O., Moffat, A., (1995). A framework for adaptive sortig. Discret Applied Mathematics, vol. 59, o. 1, pp Adersoo, A., Hagerup, T., Nilsso, S., Rama, R., (1998). Sortig i liear time. J.Computer ad Systems Sciece, vol. 57, o. 1, pp Kuth, D.E., (1973). The Art of Programmig-Sortig ad Searchig, Addiso Wesley, Readig, Mass. 7. Buretas, A., Solow, D., Agrawal, R., (1997). A aalysis ad implemetatio of a efficiet i-place bucket sort. Acta Iformatica, vol. 34, o. 9, pp Sharma, V., Sadhu, P.S., Sigh, S., Saii, B., (008). Aalysis of Modified Heap Sort Algorithm o Differet Eviromet. World Academy of Sciece, Egieerig ad Techology, vol Desai, M., Kapadiya, V., (011). Performace Study of Efficiet Quick Sort ad Other Sortig Algorithms for Repeated Data. Natioal Coferece o Recet Treds i Egieerig ad Techology. 10. Sigh, N.K., Pal, M., Chakraborty, S., (01). PARTITION SORT REVISITED. Iteratioal Joural o Computatioal Scieces ad Applicatios (IJCSA), vol., o. 1, pp Corme, T.H., Leiserso, C.E., Rivest, R.L., Stei, C., (009). Itroductio to Algorithms, Third 55

10 Editio, MIT Press ad McGraw-Hill. 1. Gurram, H.K., GovardhaaBabuKolli, (011). Average Sort. Iteratioal Joural of Experimetal Algorithms (IJEA), vol., o., pp Williams, J.W.J., (1964). HEAPSORT. Commuicatios of ACM, vol. 7, o. 4, pp Jadoo, S., Solehria, S. F.,Qayum, M., (011). Optimized Selectio Sort Algorithm is faster tha Isertio Sort Algorithm. Iteratioal Joural of Electrical ad Computer Scieces IJECS-IJENS, vol. 11 o., pp Astracha, O., (003). Bubble Sort: A Archaeological Algorithmic Aalysis. Duk Uiversity. 16. Kaur, S., Sodhi, T. S., Kumar, P., (01). Freezig Sort. Iteratioal Joural of Applied Iformatio Systems (IJAIS), vol., o. 4, pp de Moura, E.S., Navarro, G., Ziviai, N., (1999). Liear Time Sortig of Skewed Distributios. Strig Processig ad Iformatio Retrieval Symposium, pp Mawade, K.B., (010). Aalysis of Parallel Merge Sort Algorithm. Iteratioal Joural of Computer Applicatios, vol. 1, o. 19, pp Bitto, D., De Witt, D.J., D.K. Hsiao, Meo, J., (1984). Taxoomy of Parallel Sortig. Computig Surveys, vol. 16, o. 3, pp Blum, M., Floyd, R.W., Pratt, V., Rivest, R.L., Tarja, R.E., (1973). Time Bouds for Selectio. Joural of Computer ad System Scieces, vol. 7, o. 4, pp Appedix: Average Time Complexity Proof Let i be the radom variable deotig the umber of elemets placed i bucket B[i], so i time complexity equals to i =0 i =0 i =0 T () = θ () + T (i ) < θ () + O(i log(i )) < θ () + O(i ) (10) We ow aalyze the average case ruig time of our sort, by computig the expected value of the ruig time, where we take the expectatio over the iput distributio. Takig expectatios of both sides ad usig liearity of expectatio, we have E[T ()] = E[θ () + i= 01O(i )] =θ () + i= 01 E[O(i )] = θ () + i= 01O( E[i ]) We claim that E[i ] = 1 for i=0, 1,, -1. It is o surprise that each bucket i has the same value of E[i], sice each value i the iput array A is equally likely to fall i ay bucket. To prove our claim, we defie idicator radom variables Xij{A[j] falls i bucket i} for i=0, 1,, -1 ad j=0, 1,,. Thus i = j =i X ij. To compute E[i], we expad the square ad regroup terms: E[i ] = E[( j =1 X ij ) ] = E[ j =1 k =1 X ij X ik ] =E[ j =1 ( X ij ) + 1 j 1 k & k j X ij X ik ] = j =1 E[( X ij ) ] + 1 j 1 k &k j E[ X ij X ik ] Where the last lie follows by liearity of expectatio. We evaluate the two summatios separately. Idicator radom variable Xij is 1 with probability 1/ ad 0 otherwise, ad therefore E[ X ij ] = (1 1 ) = 1. Whe k j, the variables Xij ad Xik are idepedet, ad hece E[ X ij X ik ] = E[ X ij ].E[ X ik ] = (1 ).(1 ) = 1 553

11 so E[i ] = j=1 E[( X ij ) ] + 1 j 1 k &k j E[ X ij X ik ] = j= j 1 k &k j (1 ) =.(1 ) + ( 1).(1 ) = 1 + ( 1) = 1 Which proves our claim. At last, we coclude that the average case ruig time for bucket sort is (less tha) θ () +.O( 1 ) = θ () T () = θ () Eve if the iput is ot draw from a uiform distributio, our sort may still ru i liear time. As log as the iput has the property that the sum of the squares of the bucket sizes is liear i the total umber of elemets, θ ( ) + i= 01O ( E[ i ]) tells us that our sort will ru i liear time. Also, sice we call our algorithm for elemets placed i buckets, if we avoid worst case iput distributio, we reach to liear time complexity (elemets placed i buckets will have good distributio (maybe after some steps)). 554