6.851: Advanced Data Structures Spring Lecture 17 April 24

Transcription

1 6.851: Advaced Data Structures Sprig 2012 Prof. Erik Demaie Lecture 17 April 24 Scribes: David Bejami(2012), Li Fei(2012), Yuzhi Zheg(2012),Morteza Zadimoghaddam(2010), Aaro Berstei(2007) 1 Overview Up util ow, we have maily studied how to decrease the query time, ad the preprocessig time of our data structures. I this lecture, we will focus o maitaiig the data as compactly as possible. The for this sectio is to get very small space, that is ofte for static data structure. Goig to look a biary tries, usig biary alphabets. Aother way is to use bit strigs. It is easy to do liear space, it it is ot optimal. Our goal will be to get as close the iformatio theoretic optimum as possible. We will refer to this optimum as OPT. Note that most liear space data structures we have see are still far from the iformatio theoretic optimum because they typically use O() words of space, whereas OPT usually uses O() bits. This strict space limitatio makes it really hard to have dyamic data structures, so most space-efficiet data structures are static. Here are some possible goals we ca strive for: Implicit Data Structures Space = iformatio-theoretic-opt + O(1). Focusig o bits, ot words. The ideal is to use O() bits. The O(1) is there so that we ca roud up if OPT is fractioal. Most implicit data structures just store some permutatio of the data: that is all we ca really do. As some simple examples, we ca refer to Heap which is a Implicit Dyamic Data Structure, ad Sorted array which is static example of these data structures. Succict Data Structures Space = OPT + o(opt). I other words, the leadig costat is 1. This is the most commo type of space-efficiet Data Structures. Compact Data Structures Space = O(OPT). Note that some liear space data structures are ot actually compact because they use O(w OPT) bits. This saves at least a O(w) from the ormal data structures. For example, suffix tree has O() words space, but its iformatio theoretic lower boud is bits. O the other had, BST ca be see as a Compact Data Structure. Succict is the usual goal here. Implicit is very hard, ad compact is geerally to work towards a succict data structure. 1.1 mii-survey Implicit Dyamic Search Tree Static search trees ca be stored i a array with l per search. However, i order to iserts ad deletes makes the problem more tricky. There 1

2 is a old results that was doe i lg 2 usig poiters ad permutatio of the bits I 2003, Fraceschii ad Grossi [1] developed a implicitly dyamic search tree which supports isert, delete, ad predecessor i O(log()) time worst case, ad it is cache-oblivious. Succict Dictioary This is static, which has o iserts ad deletes. I a uiverse of size u, how may ways are there to have items. Use lg ( ) ( u lg ( u +O ) ( lg lg lg u) bits [2] or lg u ) +O( (lg lg ) 2 lg ) bits [6], ad support O(1) membership queries, same amout of time but with less space tha ormal dictioaries; u is the size of the uiverse from which the elemets are draw. Succict Biary Tries The umber of possible biary tries with odes is the th Catala umber, C = ( )/( u + 1) 4. Thus, OP T log(4 ) = 2. We ote that this ca be derived from a recursio formula based o the sizes of the left ad right subtrees of the root. I this lecture, we will show how to use 2 + o() bits of space. We will be able to fid the left child, the right child, ad the paret i O(1) time. We will also give some ituitio for how to aswer subtree-size queries i O(1) time. Subtree size is importat because it allows us to keep track of the rak of the ode we are at. Motivatio behid the research was to fit the Oxford dictioary oto a CD, where space was limited. Almost Succict k-ary trie The umber of such tries is C k = ( ) k+1 /(k+1) 2 (log(k)+log(e)). Thus, OPT = (log(k) + log(e)). The best kow data structures was developed by Beoit et al. [3]. It uses ( log(k) + log(e) ) + o() + O(loglog(k)) bits. This represetatio still supports the followig queries i O(1) time: fid child with label i, fid paret, ad fid subtree size. Succict Rooted Ordered Trees These are differet from tries because there ca be o abset childre. The umber of possible trees is C, so OPT = 2. A query ca ask us to fid the ith child of a ode, the paret of a ode, or the subtree size of a ode. Clark ad Muro [4] gave a succict data structure which uses 2 + o() space, ad aswers queries i costat time. Succict Permutatio I this data structure, we are give a permutatio π of items, ad the queries are of the form π k (i). Muro et. al. preset a data structure with costat query time ad space (1 + ɛ) log() + O(1) bits i [7]. They also obtai a succict data structure with log! + o() bits ad query time O(log / log log ). Compact Abelia groups Represet abelia group o items usig O(lg ) bits, ad represet a item i that list with lg bits. Graphs More complicated. We did ot go over ay details i class. Itegers Implicit -bit umber of itegers ca do icremet or decremet i O(lg ) bits reads ad O(1) bit writes. OPEN: O(1) word operatios. 2 Level Order Represetatio of Biary Tries As oted above, the iformatio theoretic optimum size for a biary trie is 2 bits, two bits per ode. To build a succict biary trie, we must represet the structure of the trie i 2 + o() bits. 2

3 We represet tries usig the level order represetatio. Visit the odes i level order (that is, level by level startig from the top, left-to-right withi a level), writig out two bits for each ode. Each bit describes a child of the ode: if it has a left child, the first bit is 1, otherwise 0. Likewise, the secod bit represets whether it has a right child. As a example, cosider the followig trie: We traverse the odes i order A, B, C, D, E, F, G. This results i the bit strig: A B C D E F G Exteral ode formulatio Equivaletly, we ca associate each bit with the child, as opposed to the paret. For every missig leaf, we add a exteral ode, represeted by i the above diagram. The origial odes are called iteral odes. We agai visit each ode i level order, addig 1 for iteral odes ad 0 for exteral odes. A tree with odes has + 1 missig leaves, so this results i bits. For the above example, this results i the followig bit strig: A B C D E F G Note this ew bit strig is idetical to the previous but for a extra 1 prepeded for the root, A. 2.1 Navigatig This represetatio allows us to compute left child, right child, ad paret i costat. It does ot, however, allow computig subtree size. We begi by provig the the followig theorem: Theorem 1. I the exteral ode formulatio, the left ad right childre of the ith iteral ode are at positios 2i ad 2i

4 Proof. We prove this by iductio o i. For i = 1, the root, this is clearly true. The first etry of the bit strig is for the root. The bits 2i = 2 ad 2i + 1 = 3 are the childre of the root. Now, for i > 1, by the iductive hypothesis, the childre of the i 1st iteral ode are at positios 2(i 1) = 2i 2 ad 2(i 1)+1 = 2i 1. We show that the childre of the ith iteral immediately follow the i 1st iteral ode s childre. Visually, there are two cases: either i 1 is o the same level as i or i is o the ext level. I the secod case, i 1 ad i are the last ad first iteral odes i their levels, respectively. Figure 1: i 1 ad i are o the same level. Figure 2: i 1 ad i are differet levels. Level orderig is preserved i childre, that is, A s childre precede B s childre i the level orderig if ad oly if A precedes B. All odes betwee the i 1st iteral ode ad the ith iteral ode are exteral odes with o childre, so the childre of the ith iteral ode immediately follow the childre of the i 1st iteral ode, at positios 2i ad 2i + 1. As a corollary, the paret of bit positio i is i/2. Note that we have two methods of coutig bits: we may either cout bit positios or rak oly by 1s. If we could traslate betwee the two i O(1) time, we could avigate the tree efficietly. 4

5 3 Rak ad Select Now we eed to establish a correspodece betwee positios i the -bit strig ad actual iteral ode umber i the biary tree. Say that we could support the followig operatios o a -bit strig i O(1) time, with o() extra space: rak(i) = umber of 1 s at or before positio i select(j) = positio of jth oe. This would give us the desired represetatio of the biary trie. The space requiremet would be 2 for the level-order represetatio, ad o() space for rak/select. Here is how we would support queries: left-child(i) = 2rak(i) right-child(i) = 2rak(i) + 1 paret(i) = select( i/2 ) Note that level-ordered trees do ot support queries such as subtree-size. This ca be doe i the balaced paretheses represetatio, however. 3.1 Rak This algorithm was developed by Jacobse, i 1989 [5]. It uses may of the same ideas as RMQ. The basic idea is that we use a costat umber of recursios util we get dow to sub-problems of size k = lg()/2. Note that there are oly 2 k = possible strigs of size k, so we will just store a lookup table for all possible bit strigs of size k. For each such strig we have k = O(log()) possible queries, ad it takes log(k) bits to store the solutio of each query (the rak of that elemet). Noetheless, this is still oly O(2 k k log k) = O( lg() lg lg()) = o() bits. Notice that a aive divisio of the etire -bit strig ito chuks of size 1 2 lg does ot work. because we eed to save lg relative idices, each takig up to lg bits, for a total of Θ() bits, which is too much. Thus, we eed to use a techique that uses idirectio twice. Step 1: We build a lookup table for bit strigs of legth 1 2 lg. As we argued before, this takes O( lg lg lg ) = o() bits of space. Step 2: Split the the -bit strig ito (lg 2 )-bit chuks. At the ed of each chuk, we store the cumulative rak so far. Each cumulative rak would take lg bits. Ad there are at most / lg 2 chuks. Thus, the total space required is O( lg ) = O( lg 2 lg ) bits Step 3: Now we eed to split each chuk ito ( 1 2 lg )-bit sub-chuks. The, at the ed of each sub-chuk, we store the cumulative rak withi each chuk. Sice each chuk oly has size lg 2, we oly eed lg lg bits for each idex. There are at most O(/ lg ) sub-chuks. Thus, the total space required is O( lg lg lg ) = o() bits. Notice how we saved space because the size of the cumulative rak withi each small chuk is less. 5

6 Figure 3: Divisio of bit strig ito chuks ad sub-chuks, as i the rak algorithm Step 4: To fid the total rak, we just do: Rak = rak of chuk + relative rak of sub-chuk withi chuk + relative rak of elemet withi sub-chuk (via lookup table). lg lg This algorithm rus i O(1) time, ad it oly takes O( lg ) bits. It is i fact possible to improve the space to O( i class. lg k ) bits for ay costat k. But this is ot covered 3.2 Select This algorithm was developed by Clark ad Muro i 1996 [4]. Select is similar to rak, although more complicated. This time, sice we are tryig to fid the positio of the ith oe, we will break our array up ito chuks with equal amouts of oes, as opposed to chuks of equal size. Step 1: First, we will pick every (lg lg lg )th 1 to be a special oe. We will store the idex of every special oe. Storig a idex takes lg bits, so this will take O( lg /(lg lg lg )) = O(/ lg lg ) = o(). The, give a query, we ca fid divide it by lg lg lg to teleport to the correct chuk cotaiig our desired result. Step 2: Now, we eed to restrict our attetio to a sigle chuk, which cotais lg lg lg 1 bit. Let r be the total umber of bits i a chuk. If r > (lg lg lg ) 2 bits: This is whe the 1 bits are sparse. Thus, we ca afford to store a array of idices of every 1 bit 6

7 Figure 4: Divisio of bit strig ito ueve chuks each cotaiig the same umber of 1 s, as i the select algorithm i this chuk. There are (lg lg lg ) 1 bits. Storig each would take up at most lg space (The size of the chuk is already polylog(), so lg bits is more tha eough). Ad there are at most such sparse chuks. So, i total, we use space: (lg lg lg ) 2 O( (lg lg lg ) lg ) = o( (lg lg lg ) 2 lg lg ) bits. If r < (lg lg lg ) 2 bits: We have reduced to a bit strig of legth r (lg lg lg ) 2. This is good, because aalogously to Rak, these chuks are small eough to be divided ito sub-chuks without requirig too much space for storig the sub-chuk s relative idices. Step 3: We basically repeat steps 1 ad 2 o all reduced bit strigs, ad further reduce them ito bit strigs of legth (lg lg ) O(1). Step 1 : With the reduced chuks of legth O(lg lg lg ) 2, we agai pick out special 1 s, ad divide it ito every (lg lg ) 2 th 1 bit. Sice withi each chuk, the relative idex oly takes O(lg lg ) bits, the total space required is: O( lg lg ) = O( (lg lg ) 2 lg lg ) bits. Step 2 : Withi groups of (lg lg ) 2 1 bits, say it has r bits total: if r (lg lg ) 4, the store relative idices of 1 bits. There are (lg lg ) 4 such groups, withi each there are (lg lg ) 2 1 bits, ad each relative idex takes lg lg bits to store. For a total of: O( (lg lg ) 2 lg lg ) = O( (lg lg ) 4 lg lg ) bits. if r < (lg lg ) 4, the r < 1 2 lg. The it is small eough for us to use step 4: Step 4: use lookup table for bit strig of legth 1 2 lg. Like i rak, the total space for the lookup table is at most: O( lg lg lg ). Thus, we agai have O(1) query time ad O( lg lg 7 ) bits space.

8 Agai, this result ca be improved to O(/ lg k ) bits for ay costat k, but we will ot show it here. 4 Subtree Sizes We have show a Succict biary trie which allows us to fid left childre, right childre, ad parets. But we would still like to fid sub-tree size i O(1) time. Level order represetatio does ot work for this, because level order gives o iformatio about depth. Thus, we will istead try to ecode our odes i depth first order. I order to do this, otice that there are C (catala umber) biary tries o odes. But there are also C rooted ordered trees o odes, ad there are C balaced paretheses strigs with paretheses. Moreover, we will describe a bijectio: biary tries rooted ordered trees balaced paretheses. This makes the problem much easier because we ca work with balaced paretheses, which have a atural bit ecodig: 1 for a ope paretheses, 0 for a closed oe. 4.1 The Bijectios Figure 5: A example biary trie with circled right spie. We will use the biary trie i Figure 5. Fidig the right spie of the trie, the recurse util every ode lives i a spie. To make this ito a rooted ordered tree, we ca thik of rotatig the trie 45 degrees couter-clockwise. Thus, the top three odes of the tree will be the right spie of the trie (A,C,F). To make the tree rooted, we will add a extra root *. Now, we recurse ito the left subtrees of A,C, ad F. For A, the right spie is just B,D,G. For C, the right spie is just E: C s oly left child. Figure 6 shows the resultig rooted ordered tree. There is a bijectio betwee the two represetatios. To go from rooted ordered trees to balaced paretheses strigs, we do a DFS (or Euler tour) of the ordered tree. We will the put a ope paretheses whe we first touch a ode, ad the a closed paretheses the secod time we touch it. Figure 7 cotais a paretheses represetatio of the ordered tree i Figure 2. 8

9 Figure 6: A Rooted Ordered Tree That Represets the Trie i Figure 5. ( ( ( ) ( ) ( ) ) ( ( ) ) ( ) ) * A B B C C D D A E F F G G E * Figure 7: A Balaced Paretheses Strig That Represets the Ordered Tree i Figure 6 Now, we will show how the queries are trasformed by this bijectio. For example, if we wat to fid the paret i our biary trie, what does this correspod to i the paretheses strig? The bold-face is what we have i the biary trie, ad uder that, we will describe the correspodig queries from the 2 bijectios. Biary Trie Rooted Ordered Tree Balaced Paretheses Node Node Left Pare[ ad matchig right] Left Child First Child Next char [if (, else oe] Right Child Next Siblig Char after matchig ) [if ( ] Paret Previous Siblig or Paret if prev char ), its mathcig ( ; if (, that ( Subtree size size(ode)+size(right sibligs) 1/2 distace to eclosig ) Could use rak ad select to fid the matchig parethesis for the balaced parethesis represetatio. Refereces [1] G. Fraseschii, R.Grossi Optimal Worst-case Operatios for Implicit Cache-Oblivious Search Trees, Proocedig of the 8th Iteratioal Workshop o Algorithms ad Data Structures (WADS), , 2003 [2] A.Brodik, I.Muro Membership i Costat Time ad Almost Miimum Space, Siam J. Computig, 28(5): , 1999 [3] D.Beoit, E.Demaie, I.Muro, R.Rama, V.Rama, S.Rao Represetig Trees of Higher Degree, Algorithmica 43(4): , 2005 [4] D.Clark, I.Muro Eifficet Suffix Trees o Secodary Storage, SODA, , [5] G.Jacobso Succict Static Data Structures, PHD.Thesis, Caregie Mello Uiversity,

10 [6] R. Pagh: Low Redudacy i Static Dictioaries with Costat Query Time, SIAM Joural of Computig 31(2): (2001). [7] J. Ia Muro, Rajeev Rama, Vekatesh Rama, ad Satti Sriivasa Rao: Succict Represetatios of Permutatios, ICALP (2003), LNCS 2719, pp