Finite-State Techniques for Speech Recognition

Transcription

1 Finite-Stte Techniques for Speech Recognition motivtion definitions finite-stte cceptor (FSA) finite-stte trnsducer (FST) deterministic FSA/FST weighted FSA/FST opertions closure, union, conctention intersection, composition epsilon removl, determiniztion, minimiztion on-the-fly implementtion FSTs in speech recognition:recognition cscde reserch systems within SLS impcted by FST frmework conclusion Automtic Speech Recognition Finite-Stte Techniques [Hetherington]

2 Motivtion mny speech recognition components/constrints re finite-stte lnguge models (e.g., n-grms, on-the-fly CFGs) lexicons phonologicl rules N-best lists word grphs recognition pths should use sme representtion nd lgorithms for ll consistency mke powerful lgorithms vilble t ll levels flexibility to combine or fctor in unforeseen wys AT&T [Pereir, Riley, Ljolje, Mohri, et l.] Automtic Speech Recognition Finite-Stte Techniques [Hetherington]

3 Finite-Stte Acceptor (FSA) b ε b 3 ccepts ( b) b definition: finite number of sttes one initil stte t lest one finl stte trnsition lbels: lbel from lphbet Σ must mtch input symbol ɛ consumes no input ccepts regulr lnguge Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 3

4 Finite-Stte Trnsducer (FST) b: :b ε:ε ε: 3 e.g., (b) (bb) definition, like FSA except: trnsition lbels: pirs of input:output lbels ɛ on input consumes no input ɛ on output produces no output reltes input sequences to output sequences (mybe mbiguous) FST with lbels x:x is n FSA Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 4

5 Finite-Stte Trnsducer (FST) finl sttes cn hve outputs, but we use ɛ trnsitions insted :b b :b ε:b trnsitions cn hve multiple lbels, but we split them up,b: c:b,c : b:b c:c Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 5

6 Weights /.6 /.4 b/.7 b/.3 /.5 trnsitions nd finl sttes cn hve weights (costs or scores) weight semirings (,,, ), prllel, series: x = x, x = x, x =, = (+,,, ) probbility (sum prllel, multiply series) / b/ /.5 b/.5 (min, +,, ) log probbility (best of prllel, sum series) /.4 b/. b/.3 / Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 6

7 Deterministic FSA or FST input sequence uniquely determines stte sequence no ɛ trnsitions t most one trnsition per lbel for ll sttes ε non-deterministic (NFA) deterministic (DFA) Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 7

8 Opertions constructive opertions: closure A nd A + union A B conctention AB complementtion A intersection A B composition A B (FSA only) (FSA only) (FST only, FSA ) identity opertions (optimiztion): epsilon removl determiniztion minimiztion Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 8

9 Closure: A +,A ε A + = b ε A = b ε A = b ε Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 9

10 Union: A B prllel combintion, e.g., ε b c d = ε ε c 3 ε b 5 d Automtic Speech Recognition Finite-Stte Techniques [Hetherington]

11 Conctention: AB seril combintion, e.g., b c d = b ε 3 4 ε c d Automtic Speech Recognition Finite-Stte Techniques [Hetherington]

12 FSA Intersection: A B output sttes ssocited with input stte pirs (, b) output stte is finl only if both nd b re finl trnsition with lbel x only if both nd b hve x trnsition weights combined with x x x y x (,) = (,) y x (x y) x yz = x y (,) x y z Automtic Speech Recognition Finite-Stte Techniques [Hetherington]

13 FST Composition: A B output sttes ssocited with input stte pirs (, b) output stte is finl only if both nd b re finl trnsition with lbel x:y only if hs x:α nd b hs α:y trnsition weights combined with IT:/ih/ IT:[ih] ε:/t/ ε:[tcl] /ih/:[ih] /t/:[t] ε:[t] /t/:[tcl] ε:[t] = (,) (,) (,) (,) ε:[t] (words phonemes) (phonemes phones) = (words phones) Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 3

14 ɛ FST Composition: ɛ Interction A output ɛ llows B to hold B input ɛ llows A to hold ε:b (,) ε:b = (,) ε:b :b (,) (,) multiple pths typiclly filtered (resulting in ded end sttes) Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 4

15 FST Composition: Prsing lnguge model from JSGF grmmr compiled into on-the-fly recursive trnsition network (RTN) trnsducer G: <top> = <forecst> <conditions>... ; <forecst> = [wht is the] forecst for <city> {FORECAST}; <city> = boston [msschusetts] {BOS} chicgo [illinois] {ORD}; wht is the forecst for boston G BOS FORECAST output tgs only <forecst> wht is the forecst for <city> boston </city> </forecst> brcketed prse Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 5

16 FST Composition Summry very powerful opertion cn implement other opertions: intersection ppliction of rules/trnsformtions instntition dictionry lookup prsing Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 6

17 Epsilon Removl (Identity) required for determiniztion compute ɛ-closure for ech stte:set of sttes rechble vi ɛ ε ε b c b b c c cn drmticlly increse number of trnsitions (copies) Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 7

18 FSA Determiniztion (Identity) subset construction output sttes ssocited with subsets of input sttes tret subset s superposition of its sttes worst cse is exponentil ( N ) loclly optiml:ech stte hs t most Σ trnsitions b b b weights:subsets of (stte, weight) weights might be delyed trnsition weight is subset weights worst cse is infinite (not common) Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 8

19 FST Determiniztion (Identity) subsets of (stte, output, weight) outputs nd weights might be delyed trnsition output is lest common prefix of subset outputs /t/:two /uw/:ε /t/:tea /iy/:ε 3 (,ε) /t/:ε (,TWO) (,TEA) /uw/:to /iy/:tea (3,ε) worst cse is infinite (not uncommon due to mbiguity) Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 9

20 FST Ambiguity input sequence mps to more thn one output (e.g., homophones) finite mbiguity (delyed to output sttes): :x b:ε :y b:ε 3 b:ε ε:x 3 ε:y Automtic Speech Recognition Finite-Stte Techniques [Hetherington]

21 FST Ambiguity cycles (e.g., closure) cn produce infinite mbiguity infinite mbiguity (cnnot be determinized): :x :y b:ε solution:our implementtion forces outputs t #, specil ɛ :x :y b:ε #:ε b:ε #:x #:y Automtic Speech Recognition Finite-Stte Techniques [Hetherington]

22 Minimiztion (Identity) miniml miniml number of sttes miniml deterministic with miniml number of sttes merge equivlent sttes, will not increse size cyclic O(N log N ), cyclic O(N ) /s/:boston 5 /t/:ε 8 /x/:ε /n/:ε 4 /b/:ε /o/:ε 3 /z/:bosnia 6 /n/:ε 9 /iy/:ε /x/:ε 5 /o/:austin /s/:ε 4 /t/:ε 7 /x/:ε /n/:ε 3 /z/:bosnia 5 /n/:ε 7 /iy/:ε 9 /x/:ε /b/:ε /o/:austin /o/:ε 3 /s/:ε /s/:boston 4 /t/:ε 6 /x/:ε 8 /n/:ε Automtic Speech Recognition Finite-Stte Techniques [Hetherington]

23 Exmple Lexicon ε:ε ε:ε 3 ε:ε 4 ε:ε 5 ε:ε 6 ε:ε 7 ε:ε 8 ε:ε 9 ε:ε ε:ε ε:ε ε:ε 3 ε:ε 4 ε:ε 5 ε:ε 6 ε:ε 7 ε:ε 8 z:zinc 9 z:zillions z:zigzg z:zest z:zeroth 3 z:zeros 4 z:zeroing 5 z:zeroes 6 z:zeroed 7 z:zero 8 z:zenith 9 z:zebrs 3 z:zebr 3 z:zelousness 3 z:zelously 33 z:zelous 34 z:zel n:ε g:ε n:ε 63 b:ε 64 b:ε c:ε 7 7 z:ε 7 t:ε t:ε t:ε g:ε h:ε n:ε 3 4 d:ε 5 h:ε 6 7 u:ε 8 u:ε 9 u:ε n:ε g:ε n:ε y:ε Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 3

24 Exmple Lexicon: ɛ Removed z:zinc z:zillions 3 z:zigzg 4 z:zest 5 z:zeroth 6 z:zeros 7 z:zeroing 8 z:zeroes 9 z:zeroed z:zero z:zenith z:zebrs 3 z:zebr 4 z:zelousness 5 z:zelously 6 z:zelous 7 z:zel n:ε g:ε n:ε 46 b:ε 47 b:ε c:ε 5 53 z:ε 89 t:ε t:ε t:ε g:ε 95 h:ε 77 n:ε d:ε 98 h:ε u:ε 79 u:ε 8 u:ε 8 n:ε g:ε n:ε y:ε Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 4

25 Exmple Lexicon: Determinized n:zinc 4 c:ε l:zillions n:ε 38 4 g:zigzg 6 z:ε 4 g:ε 3 z:ε n:zenith 7 5 t:ε ε:zero 48 h:ε 3 s:zest 8 t:ε 6 i:zeroing 3 n:ε 3 g:ε 39 3 b:ε 9 7 s:zeros 4 s:zeroes d:zeroed 33 8 t:zeroth 5 6 h:ε ε:zebr s:zebrs 5 9 ε:zel 49 8 u:ε ε:zelous n:zelousness l:zelously y:ε 45 lexicl tree shring t beginning of words:cn prune mny words t once Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 5

26 Exmple Lexicon: Minimized n:zinc 4 c:ε l:zillions g:zigzg 5 n:zelousness n:ε 7 z:ε 7 4 u:ε z:ε 3 ε:zel l:zelously ε:zelous 3 n:ε y:ε 9 g:ε 3 s:zest b:ε n:zenith t:ε t:ε 5 i:zeroing t:zeroth h:ε d:zeroed s:zeroes 3 s:zeros ε:zero 6 3 ε:zebr s:zebrs shring t the end of words Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 6

27 On-The-Fly Implementtion lzy evlution:generte only relevnt sttes/trnsitions enbles use of infinite-stte mchines (e.g., CFG) on-the-fly: composition, intersection union, conctention, closure ɛ removl, determiniztion not on-the-fly: trimming ded sttes minimiztion reverse Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 7

28 FSTs in Speech Recognition cscde of FSTs: (S A) (C } P {{ L G } ) R S:coustic segmenttion* A:ppliction of coustic models* C:context-dependent relbeling (e.g., diphones, triphones) P:phonologicl rules L:lexicon G:grmmr/lnguge model (e.g., n-grm, finite-stte, RTN) Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 8

29 FSTs in Speech Recognition in prctice: S A is coustic segmenttion with on-demnd model scoring C P L G:precomputed nd optimized or expnded on the fly composition S A with C P L G computed on demnd during forwrd Viterbi serch might use multiple psses, perhps with different G dvntges: forwrd serch sees single FST R = C P L G, doesn t need specil code for lnguge models, lexicl tree copying, etc... cn bevery fst esy to do cross-word context-dependent models Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 9

30 N -grm Lnguge Model (G): Bigrm in/. in/3.3 in ε/8. Boston/.8 Boston/4.3 ε/7. Boston in/8.7 * Nome/6.3 ε/3. Nome ech distinct word history hs its own stte direct trnsitions for ech existing n-grm ɛ trnsitions to bck-off stte (*), ɛ removl undesirble Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 3

31 P Phonologicl Rules (P) segmentl system needs to mtch explicit segments ordered rules of the form: {V SV} b {V l r w} => bcl [b] ; {m} b {} => [bcl] b ; {} b {} => bcl b ; {s} s {} => [s] ; {} s {} => s ; rule selection deterministic, rule replcement my be mbiguous compile rules into trnsducer P = P l P r P l pplied left-to-right P r pplied right-to-left Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 3

32 EM Trining of FST Weights FSA A x given set of exmples x strightforwrd ppliction of EM to trin P(x) our tools cn lso trin n RTN (CFG) FST T x:y given set of exmple pirs x : y strightforwrd ppliction of EM to trin T x,y P(x, y) [ T x y = T x,y det(t y ) ] P(x y) [Byes Rule] FST T x y within cscde S v x T x y U z given v : z x = v S y = U z trin T x y given x : y We hve used these techniques to trin P, L, nd (P L) Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 3

33 Conclusion introduced FSTs nd their bsic opertions use of FSTs throughout system dds consistency nd flexibility consistency enbles powerful lgorithms everywhere (write lgorithms once) flexibility enbles new nd unforeseen cpbilities (but enbles you to hng yourself too) SUMMIT (Jupiter) 5% fster when converted to FST frmework, yet much more flexible Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 33

34 References E. Roche nd Y. Schbes (eds.), Finite-Stte Lnguge Processing, MIT Press, Cmbridge, 997. M. Mohri, Finite-stte trnsducers in lnguge nd speech processing, in Computtionl Linguistics, vol. 3, 997. M. Mohri, M. Riley, D. Hindle, A. Ljolje, F. Pereir, Full expnsion of context-dependent networks in lrge vocbulry speech recognition, in Proc. ICASSP, Settle, Automtic Speech Recognition Finite-Stte Techniques [Hetherington] 34