2014 Haskell January Test Regular Expressions and Finite Automata

Transcription

1 0 Hskell Jnury Test Regulr Expressions nd Finite Automt This test comprises four prts nd the mximum mrk is 5. Prts I, II nd III re worth 3 of the 5 mrks vilble. The 0 Hskell Progrmming Prize will be wrded for the best ttempt(s) t Prt IV. Credit will be wrded throughout for clrity, conciseness, useful commenting nd the pproprite use of Hskell s vrious lnguge fetures nd predefined functions. WARNING: The exmples nd test cses here re not gurnteed to exercise ll spects of your code. You my therefore wish to define your own tests to complement the ones provided.

2 Introduction Regulr expressions, or regexes, re used to describe serch ptterns in text processing pplictions. Exmples of their use include Linux s grep filter utility, text editors like vim nd emcs nd lso progrmming lnguges like Ruby, Jv nd, of course, Hskell! This exercise ims to explore some of the lgorithms nd dt structures trditionlly ssocited with regulr expression implementtions. Regulr Expressions For the purposes of this exercise regulr expression is defined recursively to be either:. A terminl chrcter tht defines pttern tht mtches only tht chrcter in given input string. For exmple the expression defines pttern tht mtches only the string "".. A sequence of two juxtposed regulr expressions. For exmple, the regulr expression b comprises two terminls, nd b, in sequence nd this defines pttern tht will mtch only the string "b". Note tht prentheses cn optionlly be used to brcket subterms of regulr expression in the usul wy, nd tht, for exmple, (bc), (b)c nd bc ll represent the sme expression (i.e. sequencing is ssocitive). 3. An lterntive, written (e e ) where e nd e re regulr expressions. For exmple, the expression (b c) defines pttern tht mtches either the string "b" or the string "c". Note tht the opertor is ssocitive nd commuttive, so the expressions ( bb c), (c (bb )) nd (( c) bb) ll define the sme mtching pttern. In this exercise we will lwys enclose lterntive expressions in prentheses.. A repetition ( Kleene str ) of zero or more occurrences of given expression, e sy, written e. For exmple, b c defines pttern tht mtches the chrcter followed by zero or more occurrences of the chrcter b followed by single occurrence of the chrcter c. 5. A repetition (Kleene plus) of one or more occurrences of given expression, e, written e +. For exmple, b + c defines pttern tht mtches the string "bbbc", but not "c", s there must be t lest one b following the initil. 6. An optionl occurrence of given expression, e, written e?. For exmple, (b)?d + defines pttern tht mtches the strings "d", "dd", "bd", "bdd" nd so on. 7. The null expression, which mtches only the empty string (""). For exmple, the expression ( ) defines pttern tht mtches the strings "" nd "" only. A common convention is to denote null expressions explicitly by the specil symbol ɛ, s in ( ɛ) for exmple, nd this ide will feture lter on in Section 3. Some more exmples of regulr expressions nd mtching/non-mtching strings re shown in Tble. If regulr expression e defines pttern tht mtches string s then we sy tht e ccepts s. For exmple, the expression (b c) ccepts the strings "", "c", "bbbccb" etc., s shown in the tble. Nmed fter the Americn mthemticin Stephen Cole Kleene.

3 Regex Mtching string exmples Non-mtching string exmples (x y)( ) "x" "x" "y" "x3" x "x" "y" "x " " " "x " "x x" (b c) "" "d" "c" "c" "bbbccb" "cccb" ( ) "" "b" "" "" (b)?d + "d" "b" "bd" "bd" "bddd" "bbd" Tble : Some regulr expressions nd exmples of mtching/non-mtching strings. Simplifiction Rules The + nd? opertors re, in fct, unnecessry nd cn expressed in terms of the other opertors vi the identities: for ll expressions e.. Representtion e + ee e? (e ) Regulr expressions, s described bove, cn be implemented in Hskell vi the following dt type: dt RE = Null -- Null expression Term Chr -- Terminl Seq RE RE -- Sequence Alt RE RE -- Alterntion Rep RE -- Repetition (*) Plus RE -- Repitition (+) Opt RE -- Optionl expression (?) deriving (Eq, Show) Tble gives some exmples of regulr expressions whose representtions re defined in the templte file for testing purposes. You re now in position to nswer the questions in Prt I. You my wish to complete these before reding on. 3

4 Regex Representtion Templte vrible (x y)( ) Seq (Alt (Term x ) (Term y )) (Alt (Term ) (Term )) re x Seq (Term x ) (Rep (Term )) re (b c) Rep (Alt (Seq (Term ) (Term b )) (Term c )) re3 ( ) Seq (Alt (Term ) Null) (Term ) re (b)?d + Seq (Opt (Seq (Term ) (Term b ))) (Plus (Term d )) re5 Tble : Exmple representtions defined in the templte 3 Non-deterministic Automt A key property of regulr expressions is tht the ptterns they define cn be described by Non- Deterministic (Finite) Automton, or NDA, which is n exmple of lbelled stte trnsition system with finite number of sttes. An exmple NDA for the regulr expression ( b) c (refigure in the templte) is shown in Figure b c Figure : NDA for ( b) c An NDA hs unique strt stte (stte in Figure ) nd unique ccepting stte, which we will lso refer to s the terminl stte, which is shown s double circle (stte in Figure ). The sequence of non- trnsition lbels visited on pth through the NDA from the strt stte to the ccepting stte defines string tht is ccepted by, i.e. cn be mtched by, the regulr expression. There my be mny such pths, indeed possibly infinitely mny, nd these collectively determine the set of strings tht re ccepted by the regulr expression. A trnsition lbelled with the specil symbol corresponds to null regulr expression nd cn be trversed unconditionlly; indeed, if there re multiple -lbelled trnsitions out of stte then ny of them my led to successful mtch, so ech, in principle, must be explored non-deterministiclly. The choice of which pth to tke is clled non-deterministic becuse we don t know in dvnce which one (if ny) will led to successful mtch in the worst cse, we my hve to try them ll. The strings ccepted by the NDA of Figure re precisely those defined by the corresponding regulr expression ( b) c, e.g. "c" (vi the pth Recll from Section tht null expressions re often denoted explicitly by the symbol ɛ, hence the nme used here.

5 ), "bc" (pth ), "c" (pth ) nd so on. Turning things round, we cn sk whether given input string will be ccepted by n NDA by using the individul chrcters of tht string to fire the non- trnsitions in the NDA, beginning from the strt stte. If ll the chrcters of the input string hve been used up nd it is possible to rech the ccepting stte unconditionlly from the current stte then the input string is ccepted by the NDA. If not, or if we rech point where no trnsitions cn fire, then the string is rejected. For exmple, the strings "c", "bc", "c" etc. will ll be ccepted. However, "cc" will be rejected, becuse we will rech the terminl stte with the second c left unused. For the string "d" the will fire the trnsition from 7 to 8 but none of the successor trnsitions (sttes, 7 nd 8) re lbelled with d, so this will lso be rejected. Note tht in regulr expression cretes cycle in the corresponding NDA. For exmple, in Figure the in ( b) c is implemented vi the bckwrd trnsition from stte 6 to 5. Note lso tht the short-circuit trnsition from to 3 llows the NDA to ccept or b zero times, s required. Remrk: The NDAs shown here re not necessrily optiml, but re shown here s generted by the NDA construction lgorithm tht you will be implementing lter on. 3. Representtion An utomton cn be described using the following Hskell dt types: type Stte = Int dt Lbel = C Chr deriving (Eq, Ord, Show) type Trnsition = (Stte, Stte, Lbel) type Automton = (Stte, [Stte], [Trnsition]) Sttes re lbelled with unique integer identifiers. A trnsition comprises source stte, trget stte nd lbel, which is either (unconditionl trnsition) or of the form C c where c is chrcter. An utomton is 3-tuple comprising, in order:. The (unique) strt stte. The list of terminl sttes 3. The list of trnsitions Note: Although n NDA hs only one terminl stte, in generl n utomton cn hve mny, s we shll see in Prt IV. As n exmple, the NDA of Figure (ndfigure in the templte) will be represented by: (,[],[(,3,),(,5,),(3,,),(,,C c ),(5,7,), (5,9,),(6,3,),(6,5,),(7,8,C ),(8,6,), (9,0,C b ),(0,6,)]) nd tht of x (nd in the templte) by: 5

6 (,[],[(,3,C x ),(3,,),(,,),(,5,),(5,6,C \ ), (6,,),(6,5,)]) The order in which the sttes, terminls nd trnsitions re listed is not significnt, lthough we lwys disply ech of them sorted lexicogrphiclly. The NDAs for the exmples given in Tble re shown grphiclly in Tble 3. Note tht the NDA for (b)?d + corresponds to the simplified version of the expression in which the + nd? hve been removed, viz. (b )dd. Wht to do There re four prts to this test nd most of the mrks re for Prts I III. Prt IV is worth only two of the 5 mrks vilble nd is hrd, so you re dvised to ttempt it only when you hve completed Prts I III. The exmple expressions referred to bove, (re, re,...) nd their corresponding NDAs, (nd, nd,...), re included in the templte for testing purposes. There re lso some exmples of so-clled deterministic utomt (d, d,...) ech of which hs the sme structure s non-deterministic utomton, but with possibly more thn one terminl stte. These feture in Prt IV but re still useful elsewhere for testing purposes. A function showre :: RE -> String is lso defined in the templte for testing purposes; this produces compct textul representtion of given RE. For exmple, showre re5, where re5 is Seq (Opt (Seq (Term ) (Term b ))) (Plus (Term d )), produces "(b)?d+".. Prt I: Bsics. Define function: lookup :: Eq => -> [(, b)] -> b tht will look up given item in list of item/vlue pirs, delivering the corresponding vlue. A precondition is tht there is exctly one occurrence of the item in the list. [ mrk]. Define function simplify :: RE -> RE tht will remove ny + or? expressions using the simplifiction rules in Section.. For exmple, showre (simplify re5) should produce "(b )dd*".. Prt II: Functions on NDAs. Define the following three functions for indexing utomt: strtstte :: Automton -> Stte terminlsttes :: Automton -> [Stte] trnsitions :: Automton -> [Trnsition] [3 mrks] tht will return respectively the strt stte, the list of terminl sttes nd the list of trnsitions in given utomton. For exmple, terminlsttes nd should return []. Use these wherever you see fit in the questions tht follow. [ mrk] 6

7 Regex (x y)( ) NDA 5 7 x y x x 3 5 ' 6 (b c) c 8 0 b 6 ( ɛ) (b)?d b 6 3 d 3 d Tble 3: NDAs for the regulr expressions in Tble 7

8 . Define function isterminl :: Stte -> Automton -> Bool tht delivers true iff the given stte is terminl (ccepting) stte in the given utomton. For exmple, isterminl nd nd isterminl 3 d should both return True nd isterminl 5 nd should return Flse. [ mrk] 3. Define function trnsitionsfrom :: Stte -> Automton -> [Trnsition] tht returns the list of trnsitions emnting from given stte in given utomton. For exmple the ppliction trnsitionsfrom 5 ndfigure should return [(5,7,),(5,9,)] nd trnsitionsfrom 0 nd3 should return [(0,6,C b )]. If the stte doesn t pper in the tomton then the result should be []. [ mrks]. Define function lbels :: [Trnsition] -> [Lbel] tht returns the lbels in the given list of trnsitions, with no duplictes. Any trnsitions should be excluded from the result. For exmple, lbels [(,,)] should return [] nd lbels (trnsitions nd3) should return [C,C c,c b ]. Hint: use nub from Dt.List to remove ny duplictes. [ mrk] 5. Define function ccepts :: Automton -> String -> Bool tht returns True iff the given utomton ccepts the given input string using the so-clled bcktrcking pproch, which is described below. Note tht this is not the most efficient wy to solve the problem, but it is the simplest. Acceptnce testing using bcktrcking [6 mrks] The ide is to try to mtch ll possible pths from given stte ginst the given input string. If ny of them succeeds then the string is ccepted. To implement this, define helper function, ccepts :: Stte -> String -> Bool, tht tkes the current stte, s sy, (initilly the unique strt stte) nd the string we re trying to mtch (initilly the input string), nd does the following: If the stte is terminl stte nd the string is null then the string is ccepted, i.e. there re no unmtched chrcters left. Use the isterminl function defined erlier nd the null function from the prelude. Otherwise, you need to use the trnsitionsfrom function to get the list of trnsitions emnting from s nd then try ech of them looking for mtch; this is the non-deterministic bit. The mtch succeeds if ny of these pths leds to the string being ccepted; conversely, it fils if the string is rejected by every pth followed. To implement this prt of the solution it is suggested, lthough not required, tht you define nother helper function try :: String -> Trnsition -> Bool tht tries out one such trnsition using the following rules: If the trnsition is lbelled with n with trget stte t then you simply continue the process from stte t by invoking ccepts recursively; the string is unchnged. 8

9 If the trnsition is lbelled with chrcter, c sy, with trget stte t, then you need to exmine the input string (use pttern mtching!). If the string is empty then the mtch fils t this point nd we bort (return Flse). If it is non-empty then look t the item t the hed, c sy. If c == c then you continue by invoking ccepts on the til of the string strting with stte t; otherwise the mtch hs gin filed t this point nd you return Flse..3 Prt III: Constructing n NDA Define function mkenda :: RE -> Automton tht will generte n NDA from given regulr expression. There is simple lgorithm for doing this which you should try to follow exctly, in order to simplify testing nd mrking. We strt with the top-level function, which is provided in the templte thus: mkenda :: RE -> Automton mkenda re = (, [], sort trnsitions) where (trnsitions, k) = mke (simplify re) 3 Your job is to define the function mke, which hs type mke :: RE -> Int -> Int -> Int -> ([Trnsition], Int) In ddition to the expression we re converting, the function tkes three integers, m, n nd k sy, in tht order. The ide is to construct n NDA which strts in the stte numbered m nd ends in the stte numbered n. The k represents the next vilble identifier tht cn be used to number ny dditionl sttes. The top-level cll defines the strt stte of the finl NDA to be nd the end stte to be, but this is just convention NDAs in generl cn hve rbitrry strt nd terminl sttes. Becuse this is the top level, must therefore be the unique terminting stte for the whole NDA, hence the [] in the resulting 3-tuple. Becuse stte identifiers nd hve been used, the next vilble identifier is 3, hence the fourth prmeter to mke. The trnsitions returned by mke re sorted lexicogrphiclly, using the sort function imported from the module Dt.List, in order to help with testing nd mrking. To mke the NDA for given regulr expression you simply put together nodes nd trnsitions ccording to the rules defined in Figure. The dotted lines indicte where the NDAs for ny subexpressions should be plced; these sub-ndas will be built recursively. As n exmple, given n expression Seq r r you recursively mke the NDA for r with strt nd end sttes m nd k respectively, then do the sme for r with strt nd end sttes k+ nd n respectively. You then dd one dditionl trnsition lbelled between sttes k nd k+ nd you re done. You hve to be creful to mintin the next vilble stte identifier t ech point: mking the NDA for r my consume n rbitrry number of such identifiers, for exmple. However, it tells you which ones it used s prt of its return vlue, whose type is (([Trnsition], Int). The Int here is the next vilble identifier, hving built the entire sub-nda for re, s represented by the list of trnsitions. The ide is then to use this s the k prmeter when mking the NDA for r. Once you see how this works for sequences, very similr rules pply for the other two recursive cses. The bse cses re simple. The result in ech cse is single trnsition ppropritely lbelled nd the next vilble stte identifier is the one we strted with, s no dditionl nodes re required. 9

10 m n m C c n () Null nd terminl cses: nd C c m r goes here k k+ r goes here n (b) Sequence: Seq r r k r goes here k+ m k+ r goes here k+3 n (c) Alterntive: Alt r r r goes here m k k+ n (d) Repetition: Rep r Figure : Rules for constructing n NDA If you follow the stte numbering scheme exctly s described then you should be ble to reproduce the NDAs for ech exmple expression in the templte. For exmple mkenda re should yield, idelly exctly, if not the equivlent of, nd, i.e. (,[],[(,3,C x ),(3,,),(,,),(,5,),(5,6,C \ ), (6,,),(6,5,)]) 0

11 You cn test the bses cses seprtely, e.g. mkenda (Term x ) should yield the 3-tuple (,[],[(,,C x )]), or the equivlent. Note tht n exct reproduction isn t necessry to get the mrks, however; wht mtters is whether the NDA does the right thing. [8 mrks] 5 Prt IV: Deterministic Finite Automt The finl prt of this exercise invites you to remove the trnsitions in the NDAs from Prt III, generting Deterministic Automton, or DA, where the next trnsition, if one is possible, is uniquely determined by the trnsition lbels. To illustrte wht you need to chieve, Tble shows the deterministic equivlents of the NDAs in Tble 3. These DAs re defined in the templte nd cn be used for testing. There re severl wys of building DA from n NDA. The lgorithm below explins the generl ide, together with detils of prticulr pproch, superset construction, tht flls out elegntly in Hskell tht you my wish to follow. Feel free to try different pproch, however, if you think you hve better ide! However you do it, the objective is to define function mkeda :: Automton -> Automton tht converts n NDA into DA. Note tht the representtions of both NDAs nd DAs re the sme. The functions you defined erlier will thus work eqully well on both. Trnslting n NDA into DA The ide is for the sttes of the DA, the so-clled metsttes, to correspond to sets of sttes in the originl NDA. The first objective is to compute the list of metsttes nd trnsitions representing the DA, together with the root (strt) metstte. Then ll you need to do is mp metsttes into ordinry, integer-lbelled, sttes nd determine the set of terminl sttes there my be more thn one. The suggestion is tht you mke the list of new metsttes nd new trnsitions ccumulting prmeters of helper function. The lgorithm then proceeds s follows: We work with input sets of sttes from the originl NDA, represented s lists, beginning with the singleton set contining just the strt stte, e.g. [] if the strt stte is. Now compute the frontier of the NDA, which is the list of non- trnsitions tht cn be reched by following trnsitions s fr s possible, strting from ech stte in the input set. If the frontier reches the terminl stte then the specil phntom trnsition (t, t, ) should be included in the list returned (see below). In order to void the need to do explicit cycle detection we ll ssume precondition tht ny cycle in the NDA must include t lest one non- trnsition it will if the NDA hs been built s described erlier. For exmple, the frontier of the NDA for (b c), i.e. nd3, strting from input stte set [], is [(,,),(5,9,C ),(7,8,C c )]. In generl there my be severl sttes in the input set, so simply conctente the frontiers for ech of them together. The recommendtion is tht you define function getfrontier :: Stte -> Automton -> [Trnsition] for clculting frontiers s described. The type signture for this function is provided in the templte. The set of unique source sttes of ech trnsition in the frontier defines new metstte, which will be ssocited with stte in the DA being constructed. For exmple, the metstte for the frontier bove will be [,5,7]. Note the role of the phntom trnsition: this

12 metstte will (lter) be mrked s terminl stte becuse it contins - the unique terminl stte in this prticulr NDA. Hint: use nub nd sort imported from Dt.List to ensure tht the metstte identifiers re uniquely determined by the originl stte identifiers they contin. If the metstte hs been visited before then return immeditely s we ve reched stte tht we ve seen before: the root is the metstte nd the ccumulted lists of metsttes nd trnsitions re unchnged. Otherwise, group the frontier trnsitions by trnsition lbel there my be severl trnsitions with the sme lbel nd these need to be combined (see below). If the frontier contins the specil phntom trnsition then this should be removed t this point. In the exmple bove the grouped trnsitions will be [(C,[9]),(C c,[8])] s the lbels re unique. However, the frontier of the NDA for ( ) from stte contins two trnsitions, i.e. [(3,7,C ),(5,9,C )], so these will be grouped thus: [(C,[7,9])]. The recommendtion is tht you use the lbels function defined erlier to define function grouptrnsitions :: [Trnsition] -> [(Lbel, [Stte])] to group trnsitions s described, mking sure tht the sttes in the list ([Stte]) re unique. The type signture for this function is provided in the templte. Hints: use list comprehension nd notice tht the lbels function will conveniently remove the phntom trnsition utomticlly, if it is present, becuse it is n trnsition. You now dd the new metstte, m sy, to the ccumulted list of metsttes nd recursively visit ech trget node set, dding one new trnsition to the ccumulted trnsitions list fter ech recursive cll. For exmple, if the grouped trnsitions re [(C,[5,6,7]),(C c,[8,9])] you recurse with the stte sets (lists) [5,6,7] nd [8,9], mintining the list of new metsttes nd trnsitions s you go ( fold?!). If the first of these genertes the root r, nd trnsition nd metstte lists ts nd ms respectively, then you dd one new trnsition of the form (m, r, C ) to ts before recursing gin on the stte list [8,9] nd doing likewise. If you choose to follow the lgorithm s described you my like to use the following function types defined in the templte: type MetStte = [Stte] type MetTrnsition = (MetStte, MetStte, Lbel) mkeda :: Automton -> Automton -- Suggested helper function for mkeda... mkeda :: [Stte] -> [MetStte] -> [MetTrnsition] -> (MetStte, [MetStte], [MetTrnsition]) The second nd third prmeters of mkeda, nd similrly the result tuple, re the ccumulting prmeters suggested. Good luck! [ mrks]

13 Regex (x y)( ) DA x y 3 x ' x (b c) c b ( ɛ) 3 (b)?d + d 3 d b d Tble : DAs for the exmples in Tble 3