Refactored Moses. Hieu Hoang MT Marathon Prague August 2013

Transcription

1 Refactored Moses Hieu Hoang MT Marathon Prague August 2013

2 What did you Refactor? Decoder Feature Func?on Framework moses.ini format Simplify class structure Deleted func?onality NOT decoding algorithms Tuning NOT tuning algorithm EMS (Experiment Management System)

3 Why did you Refactor? Feature Func?on Framework easier to implement new features use sparse features Simplify class structure easier to develop with Moses Delete func?onality easier to refactor code very linle dele?on

4 Why did you Refactor? Feature Func?on Framework easier to implement new features use sparse features Simplify class structure easier to develop with Moses Delete func?onality easier to refactor code very linle dele?on

5 Why did you Refactor? Feature Func?on Framework easier to implement new features use sparse features Simplify class structure easier to develop with Moses Delete func?onality easier to refactor code very linle dele?on

6 Language Model IN THE BEGINNING LanguageModel LanguageModelSingleFactor LanguageModelMul?Factor SRI IRST KEN

7 Language Model THEN. LM LMImplementa?on LMRefCount LMSingleFactor LMMul?Factor KEN LMPointerState SRI IRST

8 Language Model NOW LM LMImplementa?on KEN LMSingleFactor LMMul?Factor SRI IRST

9 Phrase Tables IN THE BEGINNING PhraseDic?onary Memory TreeAdaptor SCFG OnDisk

10 Phrase Tables THEN. PhraseDic?onary PDFeature Memory TreeAdaptor RuleTableTrie OnDisk Compact SCFG RuleTableUTrie

11 Phrase Tables NOW PhraseDic?onary TreeAdaptor RuleTableTrie OnDisk Compact Memory RuleTableUTrie

12 Why did you Refactor? Feature Func?on Framework easier to implement new features use sparse features Simplify class structure easier to develop with Moses Delete func?onality easier to refactor code very linle dele?on

13 Deleted 1. Transla?on Systems mul?ple engines in 1 decoding process replaced with alterna?ve weights func?on 2. Distributed Language Model send LM queries to different machines replace with mul?pass/asynchronous decoding? 3. Con?nue Par?al Transla?on start from non- empty hypothesis

14 Why did you Refactor? Feature Func?on Framework easier to implement new features use sparse features Simplify class structure easier to develop with Moses Delete func?onality easier to refactor code very linle dele?on

15 Adding new Feature Func?on THEN. Add entry to Parameter class Read parameter in Sta?cData add variable to hold new feature load feature Bespoke ini file: [new- feature- sec?on] file.name [weight- new- feature]

16 Adding new Feature Func?on Sta?cData Now. Simple framework to load feature ini file: [feature] WordPenalty Distor?on NEW- FEATURE file=path factor=0 [weight] WordPenalty0= Distor?on0= 0.14 NEW- FEATURE0= 0.54

17 MERT THEN. Only know about certain feature func?ons Hardcoded array = qw(d=weight- d lm=weight- l tm=weight- t w=weight- w g=weight- genera?on lex=weight- lex I=weight- i dlm=weight- dlm pp=weight- pp wt=weight- wt pb=weight- pb lex=weight- lex glm=weight- glm); - requires feature name and abbrevia?on

18 MERT THEN. Only know about certain feature func?ons Hardcoded array = qw(d=weight- d lm=weight- l tm=weight- t w=weight- w g=weight- genera?on lex=weight- lex I=weight- i dlm=weight- dlm pp=weight- pp wt=weight- wt pb=weight- pb lex=weight- lex glm=weight- glm); - requires feature name and abbrevia?on Deleted array Ask decoder for feature func?on abbrevia?ons deleted

19 Timeline of a Transla?on Rule File Load Memory Apply to input sentence Transla?on Op?on Hypothesis Search

20 Timeline of a Transla?on Rule File Load Source phrase Target phrase Memory Apply to input sentence Input sentence Input path Transla?on Op?on Search Transla?on context Segmenta?on Hypothesis

21 Timeline of a Transla?on Rule File Load Once Memory Apply to input sentence Per occurrence in sentence Transla?on Op?on Search Per hypothesis Hypothesis

22 File Feature Func?on API Loading Xà je suis X 1 I am X 1 Source phrase: je suis X Access to: 1 Target phrase: I am X 1 void Evaluate(source,!! target,! scores,!! estimated future scores)! Memory Feature func?ons that use this: Word Penalty Phrase penalty Language model (par?al)

23 Memory Feature Func?on API Apply to input sentence Input: je PRO suis VB 25 NUM ans NNS.. Input subphrase: je PRO suis VB 25 NUM Access to: Input scores: void Evaluate(input,!!! input path,! scores)! Transla?on Op?on Feature func?ons that uses this: Input feature Bag- of- word features.

24 Transla?on Op?on Feature Func?on API Hypothesis: Search enumerated tuples defines the search Access to: Current hypothesis Previous hypotheses Segmenta?on Hypothesis Stateless features: void Evaluate(hypo, scores)! void EvaluateChart(hypo, scores)!! Stateful features: state Evaluate(hypo,!!! previous state,!!! scores)! state EvaluateChart(hypo!!!! previous state,!!!! scores)!

25 Transla?on Op?on Hypothesis Feature Func?on API Decoding Feature func?ons that uses this: All stateful features Language models Distor?on model Lexicalized distor?on Some stateless features Global lexicalized model Word transla?on

26 Feature Func?on Loading: Apply to Input: Search: void Evaluate(source,!! target,! scores,!! estimated future scores)! void Evaluate(input,!!! input path,! scores)! Stateless features: void Evaluate(hypo, scores)! void EvaluateChart(hypo, scores)!! Stateful features: state Evaluate(hypo,!!! previous state,!!! scores)! state EvaluateChart(hypo!!!! previous state,!!!! scores)!

27 Strange Features func?ons (1) Language model implement 2 Evaluate() 1. Loading evaluate full n- grams es?mate future cost par?al n- grams 2. Decoding reprise de la session resump?on of the session evaluate overlapping n- grams

28 Strange Feature Func?ons (2) Phrase- tables Unknown Word Penalty Genera?on Model integral part of decoding process Uses no Evaluate() scores assign by decoder

29 Register a Feature Func?on Register in moses/ff/factory.cpp add entry MOSES_FNAME(ClassName);

30 Language Model Inherit from LanguageModelSingleFactor class LanguageModelSingleFactor :! {! LMResult GetValue(!!const std::vector<const Word*> &contextfactor,!state* finalstate = NULL) const = 0;! }!

31 Phrase Table Inherit from PhraseDic?onary Legacy API: class PhraseDictionary :...! {! public:! TargetPhraseCollection *GetTargetPhraseCollectionLEGACY(!!!Phrase &src) const;!! protected:! const TargetPhraseCollection!*GetTargetPhraseCollectionNonCacheLEGACY(Phrase &src);!! }!

32 Phrase Table New API: class PhraseDictionary :...! {! public:! void GetTargetPhraseCollectionBatch(InputPathList &);! }! InputPathList Input Sentence: Input Path List: je suis 25 ans. je je suis je suis 25 suis suis 25

33 Phrase- table Op?miza?on Input Sentence: je suis 25 ans. Input Path List: je je suis je suis 25 suis suis suis Phrase- table trie 0 je suis

34 Phrase- table Lalce Input Input Path List: einen einen wenbewerb einen wenbewerbs einen wenbewerbsbedingten einen wenbewerb bedingten einen wenbewerbs bedingten einen wenbewerb bedingten preissturz einen wenbewerbs bedingten preis einen wenbewerbsbedingten preissturz einen wenbewerbsbedingten preis.

35 New API: Ac?ve Parsing CKY+ Phrase Table Syntax Decoding class PhraseDictionary :...! {! virtual ChartRuleLookupManager *CreateRuleLookupManager(! const ChartParser &,! const ChartCellCollectionBase &) = 0;! }! class ChartRuleLookupManager! {! void GetChartRuleCollection(! const WordsRange &range,! ChartParserCallback &outcoll) = 0;! }!

36 Results Lalce decoding use all phrase- table implementa?ons Syntax model (nearly ) One in- memory phrase- table implementa?on One on- disk phrase- table Syntax models AND phrase- based model Zen s binary implementa?on phrase- based only deprecated to be removed

37 Pass regression tests tests have also changed BLEU unchanged subject to random variability Results Transla?on Quality # Description v th July # Description v th July En-es 1 pb De-en 1 pb hiero hiero (1) + CreateOnDiskPt (1) + POS de Es-en 1 pb (3) + POS en hiero (1) + CreateOnDiskPt (1) + CreateOnDiskPt (3) + CreateOnDiskPt 15.7 En-cs 1 pb (4) + CreateOnDiskPt hiero En-fr 1 pb truecase (1) + placeholders pb recase (1) + CreateOnDiskPt (2) + hiero (2) + Ken's incre algo (2) + POS en Cs-en 1 pb (2) + kbmira hiero (2) + pro (1) + placeholders (2) + CreateOnDiskPt (1) + CreateOnDiskPt Fr-en 1 pb truecase (2) + Ken's incre algo pb recased En-de 1 pb (2) + hiero hiero (2) + factors (1) + POS de (2) + kbmira (3) + POS en (2) + pro (1) + CreateOnDiskPt (2) + CreateOnDiskPt 22.46

38 Phrase- based Results Speed speed (sec) speed (sec) 1 v master + caching 1, % With compact pt: 3 v % 4 master (with caching) % Hierarchical Pop Limit Loading v 1.0 1,044 1,393 2,620 4, th Aug Reduction (excl load) -65% -26% -21% -18%

39 Comparison with the decoders