Enumeration of MSO Queries on Strings with Constant Delay and Logarithmic Updates

Transcription

1 Enumeration of MSO Queries on Strings with Constant Deay and Logarithmic Updates ABSTRACT Matthias Niewerth University of Bayreuth We consider the enumeration of MSO queries over strings under updates. For each MSO query we buid an index structure enjoying the foowing properties: The index structure can be constructed in inear time, it can be updated in ogarithmic time and it aows for constant deay time enumeration. This improves from the previous known index structures aowing for constant deay enumeration that woud need to be reconstructed from scratch, hence in inear time, in the presence of updates. We aow reabeing updates, insertion of individua abes and remova of individua abes. ACM Reference Format: Matthias Niewerth and Luc Segoufin Enumeration of MSO Queries on Strings with Constant Deay and Logarithmic Updates. In PODS 18: 35th ACM SIGMOD-SIGACT-SIGAI Symposium on Principes of Database Systems, June 10 15, 2018, Houston, TX, USA. ACM, New York, NY, USA, 13 pages. 1 INTRODUCTION Query evauation is a centra task in databases and a vast iterature is devoted to the study of its compexity. In particuar computing the whoe set of answers may be too demanding in terms of resources of the system as the set of answers may be much arger than the database itsef. There are many possibiities to overcome this probem: computing the best answers, returning the number of answers, samping, enumerating.... In this paper we view the query evauation probem as an enumeration probem consisting in generating the answers one by one with some reguarity. We try to achieve two goas. The first goa is to output the first soution as quicky as possibe. The second goa is to have a sma deay between any two consecutive soutions. An enumeration agorithm is then often divided into two parts: the preprocessing phase, before the first output, and the enumerating phase, when the soutions are generated. The time needed before outputting the first soution is caed the preprocessing time. During the preprocessing phase the system produces an index structure that can be used by the second phase. The second phase consists in Permission to make digita or hard copies of a or part of this work for persona or cassroom use is granted without fee provided that copies are not made or distributed for profit or commercia advantage and that copies bear this notice and the fu citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or repubish, to post on servers or to redistribute to ists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. PODS 18, June 10 15, 2018, Houston, TX, USA 2018 Copyright hed by the owner/author(s). Pubication rights icensed to the Association for Computing Machinery. ACM ISBN /18/06... $ Luc Segoufin INRIA and ENS Um navigating within the index structure as fast as possibe in order to compute the next soution from the current one. Another interesting point of view is to think of the preprocessing phase as a buider of a compact representation of the output whie the enumerating phase is a streaming decompression agorithm. When viewing the query evauation probem as an enumeration probem, the best we can hope for is a inear preprocessing time and a constant time deay. This of course cannot be aways achieved but has been obtained in many interesting scenarios, MSO queries evauated over structures of bounded treewidth [4, 16], FO queries over structures of bounded degree [11, 14], or over structures of ow degree [12], or over structures of bounded expansion [15], XPath queries over XML documents [8]... However a these resuts suffer from one imitation: if an update is made on the database the index structure buit during the preprocessing phase has to be reconstructed competey from scratch. This issue has been addressed in two cases: FO queries and structures of bounded degree [7] and hierarchica conjunctive queries and arbitrary finite reationa structures [6]. In these cases the index structure can be updated in constant time upon insertion and deetion of a tupe. In this paper we aso tacke this update issue and exhibit a new index that can be computed in inear time during the preprocessing phase, aows for constant-deay enumeration and can be updated in ogarithmic time. We consider MSO queries over strings (equivaenty, MSO queries over structures of bounded pathwidth). Our index structure can be updated in ogarithmic time upon the foowing updates: reabeing, insertion and remova of individua nodes. As mentioned above, we aready know index structures, computabe in inear time, that permit constant deay enumeration for MSO queries over trees (or equivaenty structures of bounded treewidth) [4, 16]. But it seems unikey that those index structures can be updated in ogarithmic time upon very simpe updates such as reabeing a node. This update issue has been considered in the Booean case for MSO queries over words by [5]. Given an MSO Booean query and an input word they construct in inear time an index structure such that, given the index structure, one can te in constant time whether the input words satisfies or not the MSO query and moreover, the index structure can be modified in ogarithmic time upon updates that can be either reabeing of a position of the word or adding/deeting a node at the beginning or end of the word. The same paper obtains simiar resuts over trees but with a ogarithmic square time for updating the index structure. This resut has been extended to non Booean queries in [18] but with a non constant deay. In that paper, given an MSO query and a word they compute in inear time an index structure that aows for enumeration with ogarithmic time deay and ogarithmic time

2 updates. For trees the compexity becomes ogarithmic square deay and update time. Our resut improves on that by providing an index structure that can be computed in inear time, aows for constant deay enumeration and ogarithmic time updates. For this we use a new index structure that is competey different from those of [5, 16, 18]. Ours is based on Krohn-Rhodes theory. This theory says that any reguar anguage (hence any Booean MSO query) can be decomposed into a cascade of simpe anguages that are either reset automaton or permutation automaton. In a reset automaton each input etter either induces the state of the automaton or does not change the state at a. In a permutation automaton each etter induces a permutation of the state of the automaton. The composition of these basic bocks is the wreath product [19]. Our index structure is constructed by induction on the cascade of reset and permutation automata used in the definition of the MSO query. For each bock we show how to compose it with the previous one whie maintaining the constant deay enumeration and ogarithmic time update property. To demonstrate our index structure, we wi use the foowing query. Exampe 1.1 (Running Exampe). Let Φ(x) be an MSO query that seects a positions x of a string v, such that in the prefix of v up to x the number of b s between the first position of abe a and x is even. It is uncear whether our proof method can be extended to trees, as there are ony preiminary resuts on Krohn-Rhodes ike decompositions for trees [9]. We give a sma discussion of these resuts in the concusion. Structure of the Paper After giving the basic definitions in Section 2, we reduce the genera probem to the specia case of unary MSO formuas in Section 3. In Section 4, we prove a restricted variant of our main theorem that ony deas with reabeing updates. This proof uses a data structure for string manipuation, whose impementation detais are provided in Section 5. We expain how our agorithm and data structure can be extended to hande insertion and remova updates in Section 6 and concude in Section 7. 2 PRELIMINARIES Strings We consider strings over a finite aphabet Σ. Let v Σ be some string over Σ, then v i denotes the abe of v at position i and v [i, j] denotes the substring starting at position i up to position j. For two strings v and w of the same ength n, the join v w is defined as (v 1,w 1 )... (v n,w n ). The joined string uses the aphabet Σ v Σ w, where Σ v and Σ w are the aphabets of v and w, respectivey. Automata A (deterministic, finite) automaton (DFA) A is a tupe (Q, Σ, δ,q 0, F ), where Q is a set of states, δ : Q Σ Q is a transition function, q 0 is the initia state, and F is the set of accepting states. By δ we denote the extension of δ to strings, i.e., δ (q,w) is the state that can be reached from q by reading w. We define the size of an automaton to be the number of its states. The anguage L(A) accepted by A is the set {w δ (q 0,w) F }. A reguar anguage is a anguage that can be recognized by a finite automaton. Let δ be the transition function of an automaton with Q as set of states. For each symbo σ, δ σ : Q Q denotes the induced transition function q δ (q, σ ). We ca an automaton a reset automaton, if for each symbo σ, δ σ is either the identity on Q or a constant function. We ca an automaton a permutation automaton, if for each symbo σ, δ σ is a permutation of Q. Each permutation automaton has an associated group G = (G,, 1) and an associated input homomorphism h : Σ G, such that G = Q and δ (q, σ ) = q h(σ ) for each q Q. The terms reset automaton and permutation automaton were introduced by [19]. Transducer A transducer T is a tupe (Q, Σ, Λ, δ, λ,q 0 ), where Q, Σ, δ, and q 0 are defined as in a DFA, Λ is a finite output aphabet and λ : Q Σ Λ is an output function. For a transducer T, we et [T ] be its input-output function, inductivey defined by [T ](ε) = ε and [T ](va) = [T ](v)λ(δ (q 0,v), a). If the transducer T is cear from the context we aow to write v to denote [T ](v). Given an automaton A = (Q, Σ, δ,q 0, F ), we denote by A T the transducer that uses the same transitions as the automaton and outputs at every position the read symbo and the current state, i.e., A T = (Q, Σ, Σ Q, δ, λ,q 0 ) with λ(q, σ ) = (σ,q). We denote a transducer as a reset transducer, if it resuts from a reset automaton. Likewise we use the term permutation transducer. Cascade Product Let A 1 = (Q 1, Σ, δ 1,q 1, F 1 ) and A 2 = (Q 2, Σ Q 1, δ 2,q 2, F 2 ) be automata. Then the cascade product A 1 A 2 is defined as the automaton (Q 1 Q 2, Σ, δ, (q 1,q 2 ), F 1 F 2 ), where δ ((p,q), a) = (δ 1 (p, a), δ 2 (q, (a,p))). The cascade product of n automata is defined as mutipication from the eft, i.e., A 1 A n def = (( (A1 A 2 ) ) ) A n 1 A n. It wi be convenient to view the cascade product A 1 A 2 as a composition of the associated transducers. Indeed it foows from the definitions that we can test whether a string v Σ is accepted by A 1 A 2 by reading the ast etter of [A T 2 ]([AT 1 ](v)): it has to be of the form (a,q 1,q 2 ) with δ 1 (q 1, a) F 1 and δ 2 (q 2, (a,q 1 )) F 2. We therefore denote by A T 1 AT 2 the corresponding composition of transducers. We note that the operation () T distributes over the cascade product, i.e., (A 1 A n ) T = A T 1 AT n.

3 start b,c a,c b start q a 0 q 1 q 2 b a,c Figure 1: Automaton A for Φ(x) from Exampe 1.1 b,c q ε a a,b,c q a Automaton A 1 Σ 2 \ (b,q a ) (b,q a ) start q 0 q 1 (b,q a ) Automaton A 2 Figure 2: Krohn-Rhodes decomposition of A Krohn-Rhodes Theorem Σ 2 \ (b,q a ) Theorem 2.1 ([17, 19]). Let A be a DFA. Then there exists a cascade A 1 A n of automata, such that L(A 1 A n ) = L(A); and each A i is either a reset automaton or a permutation automaton. The origina theorem by Krohn and Rhodes [17] was formuated over monoids and uses the wreath product instead of automata cascades. The version depicted above using cascades is by Maer [19] who aso shows that the construction of A 1,..., A n is effective. In terms of transducers Krohn-Rhodes Theorem impies that recognition by an automaton A can be decided by checking the ast etter of the output of a composition of transducers computed from A, each of the transducers being either a reset or permutation automaton. Exampe 2.1. The query Φ from Exampe 1.1 can be answered using the automaton A depicted in Figure 1 by running A on the input string and returning a nodes visited by A in state q 1. In Figure 2 is depicted the Krohn-Rhodes decomposition of A. The automaton A 1 = (Σ, Q 1, δ 1,q ε, {q a }) checks whether the prefix has an a, whie the automaton A 2 = (Σ Q 1, δ 2,q 0, {q 0 }) counts the number of b s moduo 2. Note, that A 2 ony counts b s after A 1 has read the first a, as it ony changes state on input symbo (b,q a ) and not on (b,q ε ). It is easy to verify that A = A 1 A 2. The automaton A 1 is a reset automaton, as the abes b and c are sef oops on a states whie the abes a is a reset abe that changes the state to q a (regardess of the previous state). The automaton A 2 is a permutation automaton, as the abe (b,q a ) permutes both states, whie a other abes perform the identity permutation. Queries and enumeration In this paper a query is a formua of monadic second order ogic (MSO) with ony first-order free variabes. The signature of MSO queries on strings uses as universe a positions in a string, a inear order < over the positions and for each symbo a, a unary reation R a that contains exacty those positions abeed a. In the foowing, we use the reations,, > and = that can be derived from < in the usua manner and we identify R a with a, i.e., we write a(x) to denote that position x is abeed a. It is we known that over words, sentences of MSO correspond to reguar anguages. In the non Booean case the answers of the query are the tupes of positions making the formua true. We say that a query q can be enumerated using inear preprocessing time and constant deay over a cass of databases (in this paper strings) if there exists an agorithm working in two consecutive phases given a database D in the cass: a preprocessing phase, computing in inear time 1 an index structure; an enumeration phase, outputting one by one and without repetitions the set of answers to q over D with a constant time deay between any two consecutive outputs. If moreover the intermediate index structure can be updated in ogarithmic time upon an update of the databases we say that the enumeration agorithm has ogarithmic time for updates. We consider the foowing updates: reabeing of a position, insertion and remova of individua positions. 3 REDUCTION TO THE UNARY CASE In this section we reduce the probem to the unary case. Given a procedure that can enumerate unary MSO queries in exicographic order with constant deay and ogarithmic updates after inear preprocessing time, we show how to extend it to arbitrary MSO queries. This is based on cassica composition emmas for MSO over words and a norma form for MSO due to Simon s factorization forests. We assume that from any unary MSO query ψ and any string v we can have, after a pre-processing time inear in v, a function next(ψ, x) that computes from any position x of v the smaest position y > x such that v = ψ (y) in constant time. Moreover this function can be maintained in ogarithmic time upon updates. We wi see in Section 4 how this can be achieved. Our goa in this section is to extend the unary case to arbitrary MSO queries. 3.1 Monomias: simpe binary MSO queries We first ift the unary case to simpe binary queries that are caed monomias. A monomia over a finite aphabet A is a reguar expression of the form a 0 A 1 a 1A 2 A ma m where the a i are in A and the A i are subset of A. They define binary queries over words v in A : the set of pairs of positions (x,y) such that the factor between x and y is in the anguage defined by the reguar expression. We wi use the foowing properties of monomias Proposition 3.1. Let Ψ = a 0 A 1 A ma m be a monomia query and (x 1,y 1 ), (x 2,y 2 ), and (x 3,y 3 ) be answers to Ψ on a string v. Then the foowing statements are true: (a) Monotonicity 1: y i = min{z v = Ψ(x i, z)} and x 1 x 2 impy y 1 y 2 (b) Monotonicity 2: y i = max{z v = Ψ(x i, z)} and x 1 x 2 impy y 1 y 2 (c) Continuity: x 1 = x 3 and y 1 y 2 y 3 impy v = Ψ(x 1,y 2 ) 1 We use the RAM mode of computation, which is standard when deaing with inear time.

4 Proof. We start with (a). Towards a contradiction, assume y 2 < y 1. Let z1 1,..., z1 m be positions attesting that v = Ψ(x 1,y 1 ), i.e., v z 1 = a i for i [0,m] and v i [z 1 i +1,zi+1 1 1] A i+1 for i [0,m 1]. Let anaogousy z1 2,..., z2 m be positions attesting v = Ψ(x 2,y 2 ). Let k be the smaest number such that z k 1 z2 k (exists because y 2 < y 1 ). As v [z 1 k 1 +1,z1 k 1] A k, we can concude that v [z 1 k 1 +1,z2 k 1] A k and therefore the positions z 1 1,..., z1 k 1, z2 k,..., z2 m are attesting v = Ψ(x 1,y 2 ). Hence y 1 y 2, a contradiction. We now prove (b). Towards a contradiction, assume y 2 < y 1. Let z1 1,..., z1 m be positions attesting that v = Ψ(x 1,y 1 ), i.e., v z 1 = a i for i [0,m] and v i [z 1 i +1,zi+1 1 1] A i+1 for i [0,m 1]. Let anaogousy z1 2,..., z2 m be positions attesting v = Ψ(x 2,y 2 ). Let k be the smaest number such that z k 1 z2 k (exists because y 2 < y 1 ). As v [z 1 k 1 +1,z1 k 1] A k, we can concude that v [z 1 k 1 +1,z2 k 1] A k and therefore the positions z 2 1,..., z2 k 1, z1 k,..., z1 m are attesting v = Ψ(x 2,y 1 ). Hence y 1 y 2, a contradiction. It remains to prove (c). We consider two cases depending on the reative position of x 1 and x 2. Assume x 1 x 2. If y 2 = y 3 we are done. So assume y 2 < y 3. The construction of the proof of (a) then gives a witness for v = Ψ(x 1,y 2 ) as desired. Assume now x 2 x 1. If y 2 = y 1 we are done. So assume y 1 < y 2. The construction of the proof of (b) then gives a witness for v = Ψ(x 1,y 2 ) as desired. Given a monomia query Ψ and a string v, ca bue a position of the set of the first projection of the answers to Ψ on v and red a position of the set of the second projection. Hence Ψ returns a subset of the pairs of bue-red nodes. Positions are naturay ordered by the order on v. The proposition says that given a bue node, the set of red nodes that forms a soution with it forms a segment among the red nodes (continuity). Moreover this segment shifts to the right when we move from one bue node to the next one to its right (monotonicity). This justifies the foowing unary MSO queries associated to any monomia Ψ: Ψ 1 (x) = y Ψ(x,y) Ψ 2 (y) = x Ψ(x,y) Ψ2 min (y) = x [ Ψ(x,y) y < y Ψ(x,y ) ] Ψ2 max (y) = x [ Ψ(x,y) y > y Ψ(x,y ) ] Ψ 1 (x) = y [ Ψ(x,y) ( y < y Ψ(x,y )) x < x [Ψ 1 (x ) y < y Ψ(x,y )] ] Ψ 1 (x) = y [ Ψ(x,y) ( y > y Ψ(x,y )) x < x [Ψ 1 (x ) Ψ(x,y)] ] The first two queries compute the bue and red nodes. The third and forth queries compute the set of red nodes that are minima (resp. maxima) for some bue node. The fifth and sixth query return Agorithm 1 Enumerate Monomia 1: function next(x,y,y min,y max ) 2: if y < y max then 3: y next(ψ 2,y) 4: ese if x < x max then 5: x next(ψ 1, x) 6: if Ψ1 (x) then y min next(ψ2 min,y min ) 7: if Ψ1 (x) max next(ψ2 max,y max ) 8: y y min 9: output(x, y) those bue nodes that induce a strict shift of their associated red segment. We now show how to enumerate monomia queries assuming we can enumerate their associated unary MSO queries. The exicographicay smaest soution can simpy be computed by taking the smaest bue node with its associated minima red node. These nodes can be obtained in constant time, as we assume that unary queries can be enumerated exicographicay. The enumeration of the remaining soutions is based on the function described in Agorithm 1. Given a soution (x,y) this function computes in constant time the next soution. The enumeration is then trivia. The agorithm assumes that we have precomputed a goba constant caed x max denoting the maxima bue node. It aso maintains two variabes y min and y max containing the minima and maxima y associated to the current soution x. We note that the cas to next in the Agorithm are enumerating the unary MSO formuas Ψ 1, Ψ 2, Ψ2 min, and Ψ2 max. They are not recursive cas. We first show that the agorithm is correct. If the test of ine 2 is true then it foows from Proposition 3.1(c), that the next soution is indeed computed by ine 3. Otherwise the next soution is the next bue node and the next minima associated red node. Line 5 computes the next bue node and ine 6 computes the associate minima red nodes. It is either the current one or the next one as shown by Proposition 3.1(a). It remains to updates vaue of y max and this is done by ine 7. It s correct because of Proposition 3.1(b). We now argue that it can be impemented to work in constant time. The ony difficuty are the tests Ψ1 and Ψ 1 in ines 6 and 7. These tests can be done in constant time by making use of the fact that we go through the bue nodes in increasing order, as we assume that enumeration of unary queries can be done exicographicay. We argue for Ψ1 (x), the tests for Ψ 1 can be eiminated in the same way. In order to perform the test Ψ1 (x) we add an extra variabe x that we initiaize with the second vaue 2 satisfying Ψ1. We then repace the test Ψ1 (x) with x = x. If the test is true then we repace x with next(ψ1, x ) before proceeding. If the test is not true, then x < x, and we are sure that Ψ1 is not true. Finay note that there is no extra preprocessing. The ony preprocessing is the one of the unary queries associated to Ψ. And those can be updated in ogarithmic time. This soves the monomia case. Notice that the enumeration is done in exicographica order. 2 The first vaue satisfying Ψ1 is the first bue node.

5 3.2 From monomias to arbitrary MSO queries. Of course monomias are definabe in MSO. It turns out that, moduo recooring, MSO queries are essentiay Booean combination of monomias. This observation is a simpe consequence of a resut by Cocombet and has been used in severa paces, for instance [16]. For the sake of competeness we sketch the proof beow. A detais are present in [16]. Adding unary MSO interpretation to monomias. We note that the enumeration procedure for monomias presented in Section 3.1 immediatey extends to monomias where each etter is interpreted as a unary MSO query. That is, a(x) shoud be understood as ϕ a (x) for some unary MSO query ϕ a, or in more detai, a monomia a 0 A 1 a 1A 2 A ma m shoud be understood as the formua that says that there are positions x 0 < x 1 < < x m, such that ϕ ai (x i ) hods for each i and every position y between x i 1 and x i satisfies ϕ a (y) for some a A i. Indeed none of the proofs in Section 3.1 reies on the fact that a is a etter but just on the fact that it is a property of the current position. From monomias to binary queries. This is a key step that foows from Cocombet s norma form of MSO over words [10]. This resut says that any binary MSO query is equivaent, over words, to a query of the form ȳ zθ where θ is a Booean combination of atoms of the form a(x) or x < y and unary MSO formuas. It is then easy to see that such formuas are union of monomias with unary MSO interpretations for etters [3, 21]. See [16] for detais. Each monomia of the union can be enumerated exicographicay with constant deay and ogarithmic updates from Section 3.1. Because the enumeration is exicographic, the whoe union can aso be enumerated with constant deay (fokore, see for instance [13]). The enumeration stores the ast seen resut for each monomia of the union and aways outputs the smaest of these resuts. From binary queries to arbitrary queries. This is a simpe consequence of the Composition Lemma: any MSO query over words is a union of conjunctions of binary queries. Each conjunct induces a inear order constraint among a variabes. Hence the union is strict, i.e. no answer can be an answer of two disjuncts, and each disjunct can be treated separatey. Furthermore we can assume that each disjunct invoves consecutive variabes in the inear order. We therefore have an induced inear order among the formuas of the conjunct that can be treated exicographicay one by one using the procedure described above. Atogether this shows that once the unary case is soved, any MSO query can be enumerated with constant deay and ogarithmic updates after a inear pre-processing phase. 4 ENUMERATION OF UNARY QUERIES In this section we consider ony reabeing updates and give an enumeration agorithm for unary MSO queries. We use a data structure aowing for constant deay enumeration and ogarithmic time updates. We ony give in this section a high eve description of the data structure with its main features. In Section 5 we provide the technica detais of its impementation. In Section 6 we wi see how to hande the remaining updates such as insertions and removas. It turns out that reabeing is the hardest case. Theorem 4.1. On strings, unary MSO queries can be enumerated using inear preprocessing time, constant deay and ogarithmic time for reabeing updates. As usua when deaing with MSO formua, the constants are non-eementary in the size of the formua. In order to get better constants one needs to use equivaent but more verbose query anguages. The rest of this section is devoted to the proof of Theorem 4.1. The first step is to decompose a unary MSO query into reguar anguages. This is based on the foowing fokore resut 3 : Lemma 4.2. To any unary MSO query Ψ(x) there exists a finite set I and Booean MSO formuas Φ i,eft and Φ i,right for i I such that for every word v of ength n and any position a of v we have ( ) v = Ψ(a) iff v[1,a] = Φ i,eft v [a,n] = Φ i,right. i I Because Booean MSO formuas define reguar anguages, we view the Φ i,eft and Φ i,right as reguar anguages. Because reguar anguages are cosed under Booean operations, we can further assume in Lemma 4.2 that the union is disjoint. We can therefore treat each component of the disjunct separatey as this wi not give any dupicate during the enumeration phase. In the foowing we assume that we work with a unary MSO query Ψ(x), given as a pair (Φ eft, Φ right ) of reguar anguages as in Lemma 4.2. In Subsection 4.1, we provide a data structure aowing for constant deay enumeration and ogarithmic time updates for the unary query returning those positions x such that initia segment up to x beongs to the reguar anguage Φ eft. The case of Φ right is mosty symmetric to the previous one and is briefy described in Section 4.2. The data structure representing Ψ(x) is essentiay a cross-product of the previous two structures. In Subsection 4.4 we sum up and give the proof of Theorem 4.1. For our exampe query Φ(x), Lemma 4.2 yieds exacty one pair of anguages, where the eft anguage is the anguage defined by the automaton A of Figure 1 and the right anguage accepts every string, as Φ(x) does not care which symbos occur to the right of x. 4.1 Data structure for Φ eft Let A T 1 AT m be the transducers decomposing the reguar anguage Φ eft as given by Theorem 2.1. By construction, each transducer is either a reset transducer or a permutation transducer. We assume that A i has Q i as set of states, δ i as transition function, Σ i as aphabet and, in the case of permutation transducer, G i as associated group and h i as associated homomorphism function Σ i G i. The genera idea is as foows. Given an input string v our data structure maintains its cascade images by the various transducers, i.e. A T 1 (v), AT 2 (AT 1 (v)) and so on. We view each string as a ayer, the bottom ayer being v, the bottom but one ayer being A T 1 (v) and the top ayer being Am T ( (A T 1 (v)) ). For each string in the data structure, we make sure we can enumerate with constant deay a positions having a abe from a fixed set of abes. Appying this to the top string and to the accepting states yieds the constant deay enumeration. Updates are propagated bottom-up. The reabeing 3 It can be proved using a simpe Ehrenfeucht-Fraïssé game argument.

6 update on S 1 : v i σ v [i, j] w [i, j] for every q Q : v q i (σ,q) v q [i, j] wq [i, j] v i (σ, project Q (v i )) v [i, j] w [i, j] v [i+1,k v i ] vpv i+1 [i+1,k v i ] v [i,k v i ] vpv i [i,k v i ] v [j+1,k v j+1 ] vpw j+1 [j+1,k v j+1 ] for every д G : v д i (σ, project G (v д )) v д [i, j] wд д 2 [i, j] v д [i+1,n] vд д 1 [i+1,n] v д [j+1,n] vд д 3 [j+1,n] where ki v def = first position of a reset abe in v after i pi v def = δ (project Q (v i 1 ),v i 1) def д 1 = h (v [1,i 1] ) h (σ ) h (v [1,i] ) 1 def д 2 = h (v [1,i 1] ) h (w [1,i 1] ) 1 д 3 def = h (v [1,i 1] ) h (w [i, j] ) h (v [1, j] ) 1 Tabe 1: How updates are propagated from ayer S 1 to ayer S. Top: Update in in a string v of S 1. Midde: Update if S is a reset ayer. Bottom: Update if S is a permutation ayer. of the bottom ayer may require various updates to the next ayer which in turns may require significant changes to the next ayer and so on. We cope with this probem as foows: we enrich the data structure with severa string at each ayer, intuitivey each added string contains a precomputation of one of the expected changes, and we aow for interna updates sighty more compicated than just reabeing. We now turn to the detais. To each m we associate the ayer set S defined by induction assuming S 0 contains exacty one string v input, which is the current input string: S ={v v S 1 } {v q v S 1,q Q } if A T is a reset transducer S ={v д v S 1,д G } if A T is a permutation transducer Here, for v = σ 1... σ n, v q is the string (σ 1,q)... (σ n,q) for each q Q and v д is the string v д = (σ 1,д 1 )... (σ n,д n ) with д 1 = д and д i+1 = δ (д i, α i ). We note that for permutation ayers, v = v 1, hence each ayer contains the image by A T of a strings of the previous ayer. Notice that a S can be easiy computed in inear preprocessing time. In our running exampe, we thus have three ayers, S 0, the owest one containing the input string, S 1, the midde one associated to the reset automaton A 1 from Figure 2, and S 2, the upper one, associated with the permutation automaton A 2 from Figure 2. We wi iustrate our agorithm with the string u = cbabcb. In this case, initiay S 0 contains the string u, S 1 initiay contains the three strings and u = (c,q ε )(b,q ε )(a,q ε )(b,q a )(c,q a )(b,q a ), u q = (c,q)(b,q)(a,q)(b,q)(c,q)(b,q) for q {q ε,q a }. We aow the foowing updates on the strings in S : Definition 4.3 (Update Operations). Here, v and w are strings from S, and i and j are positions. reabe: v i σ repace: v [i, j] w [i, j] The reabe operations changes the abe at position i to σ, whie the repace operation repaces the substring of v from position i to position j (incusive) with the substring that resides in w at the same position. In our running exampe we wi consider an update u 1 a that changes the first position of u to abe a. Furthermore, the data structure aows the foowing queries: Definition 4.4 (String qery operations). searchabe (v, i, Σ ), searchabe (v, i, Σ ) enumerate(v, Σ ) The searchabe operation returns the biggest (smaest) position in v with some abe σ Σ that is equa or arger (equa or smaer) than i. The enumerate operation enumerates a positions in v with some abe σ Σ. We show in Section 5, how these operations (except enumerate) can be impemented to take ony ogarithmic time. Furthermore, we wi show that enumerate works with constant deay. Obviousy, each ayer has to react to any of the updates isted in Definition 4.3 from the previous ayer. To ensure overa ogarithmic time updates, each ayer may do ony constanty many updates for each update of the previous eve. We show how this is done in the remaining part of this section. The propagation of the updates are summarized in Tabe 1. In the tabe we use project Q to denote the projection of some symbo to its component from Q, i.e., keeping ony the state from automaton A and disregarding a other information. The projection is extended from symbos to strings in the obvious way. Reset ayer. Assume A T is a reset transducer and that a reabe v i σ is performed on a string v of S 1. In order to update S we need to do the foowing: First we need to change the abe at position i of a strings of S that are derived from v. Second we need to make sure that the states of v are modified appropriatey. To do this et p be the state reached by A T on v at position i (which can be recovered using project Q (v i )) as in Tabe 1). Let q be the state δ (p, σ ) (denoted pi+1 v in Tabe 1). Notice that q is the new state of the transducer A T at position i + 1 after reading σ in state p. Because A T is a reset transducer this state remains unchanged unti a reset etter is read. Let j be this position (denoted ki v in

7 Tabe 1). Notice that j can be computed from our data structure using the query searchabe (v, i, Σ ) where Σ contains a reset etters of A T. Notice that because AT is a reset transducer, a positions of v before i and after j are unaffected by the update on v. Hence S is correcty updated by the sequence of updates depicted in Tabe 1: vi u (σ,u) for a u Q, v i (σ,q), and v [i+1, j] vq. This iustrates the need for the repace update [i+1, j] as the interva [i + 1, j] may be arge. The update u 1 a from our running exampe, yieds the reabe updates u q ε 1 (a,q ε ), u q a 1 (a,q a ), and u 1 (a,q ε ), and the repace update u [2,3] uq a, as position 3 is the next position after [2,3] 1 with the reset abe a. Permutation ayer. Assume now that A T is a permutation transducer and that a reabe v i σ is performed on a string v of S 1. In order to update S we need to do the foowing: First we need to change the abe at position i of a strings in S that are derived from v. Second we need to make sure that the states of each such strings are updated starting from position i + 1. This second operation can be done using the fact that G is a group and that we have aready computed v д for a д G. Let д σ = h (σ ) be the group eement associated to σ. Let д i be the group eement reached by A T at position i when running on v starting in the identity state. The new next state of A T at position i + 1 is now дд iд σ and the remaining states can be computed by resuming the computation from there. Notice now that дд i д σ is aso the state reached at position i + 1 by A T on v before the update when starting in state h = дд i д σ (д i д a ) 1. Hence a the remaining computation is aready present in v h. The updates of S can therefore be done using repace operations: v д [i+1,n] vдд iд σ (д i д a ) 1 for each д G. We note that a these repacements shoud be done in parae, effectivey permuting the suffixes of the strings. The operations can be seriaized using an additiona string v temp S that that aows for cycic exchanges. The reabe u 1 a from our running exampe has triggered (among others) the reabe update u 1 (a,q ε ) on S 1. This update triggers in particuar the reabe updates u,q 0 1 (a,q ε,q 0 ). As this shift the count of a from the second position, we obtain the new computation by reading the string u,q 1 from the second position: u,q 0 [2,n] u,q 1 [2,n]. In parae we perform a simiar change on u,q 1 The remaining cases are treated simiary and are depicted in Tabe 1 showing: Proposition 4.5. The updates given in Tabe 1 are correct. Notice that that one update of S 1 triggers a constant number of updates in S. The tota number of updates is thus independent from the ength of the strings but exponentia in the number of ayers. This concudes the description of our data structure for Φ eft. Assuming the data structure can impement the operations of Definition 4.4 in ogarithmic time, we have seen that the structure can be updated in ogarithmic time. Moreover the output string of the ast ayer associates to each position i of the current input string the state of the automata equivaent to Φ eft that is reached when running from the initia position of the string. 4.2 Data structure for Φ right For the Booean query Φ right, and its decomposition into transducers, we compute a simiar structure. Our goa is here to have at any position information that aows us to determine whether the suffix is in the anguage defined by Φ right. For this we mirror what we have done in the previous subsection. We consider the mirror anguage consisting of a strings w such that when read from right to eft the sequence beongs to Φ right. We then proceed as in the previous subsection except that a transducers given by Krohn-Rhodes theorem now read the string from right to eft. In particuar, when updating reset ayers, we have to search the next reset state to the eft using searchabe instead of searchabe. 4.3 Join Layer From Sections 4.1 and 4.2 we obtain two data structures S eft and S right that can be updated in ogarithmic time. Here, S eft and S right refer to the topmost ayer of each structure. We add in this section on top of them a new ayer S that combines both of them. If v and w are two strings of ength n we define v w as the string of ength n over the product aphabet where each position contains the abes of v and w at this position. We define S = {v w v S eft,w S right }. We note that S can be computed in time inear in n. We now describe how we react to an update in S eft or S right. By symmetry we ony consider the case where the update occurs in S eft. v i σ (v w) i (σ,w i ) v [i, j] w [i, j] (v w) [i, j] (w u) [i, j] It is straightforward to verify that the updates are correct wrt. the definition of S. 4.4 Putting Everything Together It is now straightforward to prove Theorem 4.1. Proof. Let Ψ(x) be a MSO unary query. By Lemma 4.2 we can assume without oss of generaity that Ψ(x) consists in a pair (Φ eft, Φ right ) where Φ eft and Φ right are reguar anguages and Ψ(x) returns a the positions i of a word such that the prefix is in Φ eft and the suffix is in Φ right. Given a stringv we buid from it the data structure of Sections 4.1 to 4.3. It foows from these sections that this structure can be buit in time inear in the size of v and can be updated in time ogarithmic in the size of v. It remains to show that it aows for constant deay enumeration. For this, by construction of the structure, each position i of the output string of the data structure contains the state of the automata for Φ eft when executed from the beginning of the string and the state of mirror of Φ right when executed from the end of the string. Let Σ X be the set of a abes that indicate that the position shoud be in the output of Φ(x). Using enumerate(v, Σ X ) we can enumerate a positions satisfying Φ(x), where v is the output string of the data structure.

8 v 1 v 2 v 3 v 7 v 8 w 1 w 2 w 3 w 4 w 5 w 6 w 7 w 8 Figure 3: Datastructure representing two strings v and w after the operation v [4,6] w [4,6]. 5 STRING MANIPULATION DATA STRUCTURE In this section we describe a data structure that can perform the operations used in the high-eve agorithm we presented in the previous section. We spit this description into two parts: First, we specify a basic data structure that supports a operations except enumeration. 4 Afterwards, we describe an advanced data structure that is an extension of the basic data structure and additionay supports constant deay enumeration. 5.1 Basic Data Structure In order to simpify the presentation we assume that the ength of each string is n = 2 k for some natura number k. We represent a string v by a fu binary tree t v of height k. The string is represented at the eaves. Each eaf node x has an aphabet abe σ (x); an integer pos(x) from [1, n] identifying its position in the string. Each inner node x has a abe ab(x) 2 Σ denoting the set of a abes used by the eaves beow x, two pointers (x) and (x) for the eft and right chid, respectivey, the interva [i, j] of the positions of the eaves beow x. A set S of strings is then represented by an acycic graph containing a tree t v for each string v S. We note that nodes of the graph can be part of severa strings. However, if a eaf node is part of two strings t v and t w, then it wi aways be in the same position pos(x) in v and w and v pos(x ) = w pos(x ) = σ (x). If some inner node x is part of two strings v and w, then it represents the same substring in both strings. In Figure 3, we depict the data structure S for two string v and w that share some eaf nodes and one inner node. This is exacty the data structure, our agorithm woud produce after the operation v [4,6] w [4,6] is appied to two strings that did not share any nodes before. Note that given a node x we can easiy compute the path from the root to x. Moreover it is possibe to test whether x is part of some tree t v in ogarithmic time ooking at its associated interva [i, j] and then foowing the corresponding path starting from the root of t v. 4 Actuay, the basic data structure aows for enumeration, but with ogarithmic deay, whie we need constant deay. Given a strings in S, the data structure is initiaized in inear time by computing a separate tree t v for each string v. We now expain how the structure evoves after each update operation and how each operation can be performed within the desired time constraints. In order to prove forma correctness, we make sure the data structure satisfies the foowing invariants that shoud hod for each string v S before and after each operation. In the foowing t v is the binary tree representing v and x 1,..., x n are the eaves of t v from eft to right. (inv1) v = σ (x 1 )... σ (x n ); (inv2) for each inner node x of t v, ab(x) contains exacty those symbos of Σ that are used by the eaves beow x; We make sure that the initiaization enforces the invariants (inv1) and (inv2). We now shorty describe how we can hande the read operations, before we describe how to hande the update operations. 5.2 Read Operations The operation searchabe (v, i, Σ ) is impemented as foows: Compute the path from the root of t v to x i, the eaf representing v i. If σ (x i ) Σ, return x i. Otherwise go up from x i unti visiting a node x, where x i is in the eft subtree and the right subtree contains some σ Σ, i.e. Σ ab( (x)). Find the eftmost eaf y in the subtree rooted at (x) such that σ (y) Σ using the abes of the inner nodes. The operation searchabe (v, i, Σ ) is fuy symmetric. The operation enumerate(v, Σ ) can be impemented with ogarithmic deay using searchabe. A constant deay impementation is described beow in Section Updates The operations changing the data structure wi a use the foowing three steps. Ony the detais of these steps (especiay the second step) are different. 1. Ensure that inner nodes whose chid pointers or abes change are ony used by one tree. 2. Do the actua update by changing the chid pointers of some inner nodes and possiby adding a new eaf node. 3. Update the abes of inner nodes 5. The first step ensures that for a unchanged strings, their tree structures wi not change. Therefore the invariants (inv1) and (inv2) are maintained for them. For this first step we introduce an operation isoate(v, x) that takes as argument a string v and a node x of t v. For each y in the path from x to the root of t v, if y aso beongs to t w for w v, then it creates a fresh copy of y and changes the pointers such that t v uses this new copy (a other trees are unchanged.) For a set X of nodes, isoate(v, X ) performs the operation isoate(v, x) for each x X. The second step reestabishes the invariant (inv1) for changed strings. The impementation detais differ for the different update operations. The third step reestabishes (inv2) for changed strings. It essentiay consist in abe changes. 5 We never change abes of eaf nodes.

9 We now describe the individua update operations and argue that each operation maintains a invariants. We start with the reabe operation v i σ. Let x i be the eaf representing v i in t v. We do the foowing operations: 1. isoate(v, parent(x i )) 2. Change the chid pointer in parent(x i ) to point to a new node y i abeed σ. 3. Update the set of used symbos in the ancestors of parent(x i ). It is easy to see that invariants (inv1) and (inv2) are maintained. The runtime is ogarithmic, as each step can be impemented in ogarithmic time. We continue with the repace operation v [i, j] w [i, j]. Let X [i, v j] be the set of a inner nodes x of t v such that the interva associated to x is incuded in [i, j] and is maxima with respect to this property: the interva associated to the parent of x is not incuded in [i, j]. It is straightforward to check that a nodes in X [i, v j] are either on the path from the root to v i or on the path from the root to v j. Hence X [i, v contains at most ogarithmicay many j] nodes of t v (X [i, v contains at most two nodes per depth eve) and j] can be computed in ogarithmic time. Let f : X [i, v j] X [i, w j] be the bijection between Xv [i, j] and X [i, w j] that connects nodes whose subtrees represent the same interva. The update consists of the foowing steps: 1. isoate(v, X [i, v j] ) 2. for each node x X [i, v, repace the pointer to x in parent(x) j] by a pointer to f (x). 3. Update the set of used symbos in the ancestors of the nodes in X [i, v j]. A the operations can be performed in ogarithmic time. Once we have computed the paths (and corresponding paths in a other strings of S to determine which nodes need to be isoated in Step 1), a required updates to inner nodes of t v can be done in constant time per node. Note that a affected nodes are in X [i, v or ancestors j] of those nodes, thus a affected nodes are on the paths from the root to v i and to v j. It is aso easy to check that the operations maintain invariants (inv1) and (inv2). This finishes the description of the basic data structure. 5.4 Advanced Data Structure The basic data structure is a we need for the ower ayers of our data structure. However for enumeration of S, we need one more operation: enumerate. We note that the ist of abes that shoud be enumerated is fixed and known in advance. Therefore, we provide an impementation that enumerates exacty those positions of a string v, that are abeed by some symbo from a fixed set Σ X. We write x v y to denote that the nodey shoud be enumerated after node x when enumerating v, i.e., x,y t v, σ (x), σ (y) Σ X, pos(x) < pos(y), and there is no z t v with σ (z) Σ X and pos(x) < pos(z) < pos(y). The basic idea of our impementation is that for each pair of eaf nodes x,y with x v y for some string v, we add a pointer from x to y. Using these pointers, the enumerate operation can be easiy impemented. The difficuty is to maintain those pointers in ogarithmic time. There are essentiay two probems. First, a eaf node can be part of severa trees and therefore requires severa pointers, we then need a way to choose which one we need to foow. Second, a repace operation v [i, j] w [i, j] can insert ineary many eaves from a tree t w into a tree t v. It is not possibe to update the pointers of a these eaves during an update. Therefore, somehow, the existing pointers of t w have to be used when enumerating the string v in the interva [i, j]. However, when eaving this interva, the enumerating agorithm shoud not foow pointers from t w any more, but has to continue in t v. Our idea to sove this probem is to give priorities to pointers. For instance after a v [i, j] w [i, j] operation, when enumerating v, the agorithm shoud by defaut foow the pointers of v and ony use pointers of w when those for v are not avaiabe, giving priority to the v-pointers. Hence during the repace operation v [i, j] w [i, j], we just add a v-pointer from the ast Σ X -node of v before position i to the first Σ X -node of w in the interva [i, j], and another v- pointer to the ast Σ X -node of w in the interva [i, j] pointing to the first Σ X -node of v after position j. This way, v can be correcty enumerated after the repace update. However, we sti need to refine our soution. Consider the sequence of updates v [i, j] w [i, j], v [i, j] u [i, j], v [i,k] w [i,k] with k > j. After the second repace, there is a danging v-pointer eft at the node x in position j of w that is not currenty used by v. This pointer wi confuse the enumeration agorithm after x is reinserted into v by the ast repace operation. Unfortunatey, we cannot simpy remove a danging pointers after a repace, because there can be many of them. Thus we need a way of invaidating od pointers, which eads us to the foowing definition, where each pointer carries some information about the pointers that shoud be foowed during the further enumeration. We associate to each eaf node x a set of pointers pointers(x). We denote by source(p) and target(p), the source and target of the pointer p. In particuar if p pointers(x) then its source is x. We associate to each pointer p a abe which consists of a set of pointers, denoted abe(p). The idea of the abe is as foows. By defaut, abe(p) contains exacty one pointer, which is the next pointer to use after outputting the target of p. After a repace operation v [i, j] w [i, j], abe(p) may contain two pointers p 1, p 2, where p 1 is the next pointer to foow (pointing to some node of w), and p 2 is some pointer pointing back to some node in v at the end of interva [i, j]. Once the enumeration reaches source(p 2 ) it shoud foows p 2 and no other pointers with that source. We add dummy nodes before and after each string that are abeed with some abe from Σ X and are never modified. We use these dummy nodes to avoid a case distinction regarding whether a node is the first or the ast one in a string carrying some abe from Σ X. The first node of each string v wi aways contain a pointer to the first rea Σ X -position of v. The ast rea Σ X -position of v aways points to the dummy node at the end. We use the notation p = x {p 1,...,p k } y to denote that pointer p points from node x to node y and has abe(p) = {p 1,...,p k }.

10 Agorithm 2 Enumerate String The interna state consists of x P = {p 1,...,p k } 1: function startenumerate(v) 2: x x v 3: P pointers(x v ) 4: function next 5: p p s.t. p P and source(p ) = x 6: x target(p) 7: P P \ {p} 8: P P {q abe(p) r P. q < r } 9: output(pos(x)) w the ast visited node a set of pointers x y {2} {3} {4} {5} {6} {} 1 {8} 7 2 {9, 3} 8 3 v {12, 8} 11 u {13} {10} 9 {14} 13 Figure 4: Exampe showing the use of pointers in the advanced data structure The abe is omitted if it is not reevant. We write p < r to denote pos(source(p)) < pos(source(r )). The advanced data structure is initiaized just as the basic structure. Afterwards, a pointer p x = x {p y } y is added for each pair of eaf nodes x,y that satisfy x v y for some v. Before expaining how we maintain those pointers, et s have a ook at the enumeration agorithm, Agorithm 2, to see the indented meaning of the abes. During the enumeration of v, whie visiting the node x, the agorithm maintain a set P, caed P (v, x) in the seque, that contains a set of pointers, a with different sources, that we must foow ater during the enumeration. In particuar P (x, v) aways contains a pointer p with source x that provides the next eement to output. When switching from x to y the program removes p from P (v, x) (ine 7) and adds a q abe(p) to P, if the source of q has a smaer position than the positions of a sources of nodes aready in P (v, x), enforcing a stack discipine in P (v, x) (ine 8). Later, we show that each P (v, x) and therefore the set P in Agorithm 2 has constant size. We et p(v, x) be the maxima entry of P (v, x), i.e., the pointer with the rightmost source node. Intuitivey, p(v, x) is the pointer that utimatey returns to the string v, even after many repace updates. In the extreme case, it just points to the dummy node at the end of v. Exampe 5.1. In Figure 4, we depicted the reevant pointers of three strings u, v, and w after the operations v [3,5] w [3,5] and u [2,4] v [2,4]. Each pointer has a pointer identifier beow and the set abe above the arrow For the node marked x in the figure, we obtain P (x,w) = {3}, P (x,v) = {3, 9}, and P (x,u) = {3, 12}. It becomes cear from the exampe, why we fiter the pointers in Line 8 of Agorithm 2. Otherwise, P (x,u) woud contain the pointer 6 {} 10 {} 14 Agorithm 3 Cean Pointers 1: function cean(x) 2: V {v x t v } 3: for v V do 4: y searchabe (v, pos(x) + 1, Σ X ) 5: p v x P (y,v ) y 6: q q s.t. max(abe(q )) = p(x,v) 7: in abe(q) repace p(x,v) by p v 8: pointers(x) {p v v V } 9, a pointer never reached by the enumeration, as the enumeration foows pointer 12, which occurs before pointer 9. The correctness of the enumeration foows from and depends on the foowing invariant: (inv3) For every two nodes x,y with x v y for some v there is a pointer x y in P (x,v). Before we expain the update agorithm, we introduce an important heper function. Agorithm 3 takes a eaf node x and changes the pointers in such a way, that each string v with x t v uses a different pointer towards the next node. This way, subsequent changes to some pointer of x wi affect ony one string, aowing updates without interfering with other strings. Furthermore, Agorithm 3 ensures that the number of pointers eaving x is bounded by S. Agorithm 3 reies on the foowing invariant: (inv4) For each x with σ (x) Σ X and each two strings v,w with x t v and x t w it hods that p(v, x) p(w, x). Agorithm 3 repacesp(v, x) in the pointer that introducedp(v, x) into P (v, x) by p v, which is the correct pointer to foow when at x. This repacement ensures that P (x,v) contains ony p v. Agorithm 3 sets abe(p v ) such that P (v,y) is not changed and the enumeration can safey resume from y. Note that it is not obvious at this point how to compute P (v, x) and max(abe(q)) = p(x,v) in constant time. We wi see how this can be achieved. Exampe 5.2. To understand Agorithm 3, we again ook at Figure 4 and study the changes introduced by the ca cean(x). The agorithm introduces a new pointer p v = x {9,4} y for v and repaces the 9 in abe(8) by p v. Simiary, it introduces new pointers p u and p w. The effect for the enumeration of v is, that p v is used instead of 3. The set P (v,y) is unchanged as the 9 is reintroduced by pointer p v. An update to v that requires the pointer eaving x to be changed to a different node can now be done without interfering with the strings u and w. We now present the update agorithms for reabe and repace updates. We use the notation p q to denote that the target node and abe set of pointer p is changed to those of pointer q. Agorithm 4 starts just ike in the basic data structure. Afterwards it adjusts the pointers as needed. In both cases (reabe and repace), it first searches the position x preceding the update and uses Agorithm 3 to ensure that each pointer is ony used by one string. If the updated part of the string (interva [i, j] with j = i in