Index. caps method, 180 Character(s) base, 161 classes

Transcription

1 A Abjads, 160 Abstract syntax tree (AST), 3 with action objects, definition, 135 Action method for integers converts, S-Expressions, 171 Action objects ASTs, defined, 137.made attribute, 140 mathematical expression, proto tokens, 140 variables collection, Anchors ^ and $, 19 matches, 20 newline character, 19 zero-width assertion, Atom, 166 B Backreferences named captures, 52 prime number, 52 quoted string, 52 string, 52 string repetition operator, 53 Backtracking correctness, 70 first group, 66 frugal quantifier, 71 HTML tags, performance, possessive quantifier, 69 second character, 66, 68 Binary Linux packages, 8 but operator, 107 Bytes, 92, 161 C caps method, 180 Character(s) base, 161 classes Moritz Lenz 2017 M. Lenz, Parsing with Perl 6 Regexes and Grammars, 195

2 Character(s) (cont.) predefined, Unicode characters, 23 user-defined, 24 combining, 161 full-width, 163 general category, 164 precomposed, 161 Code point, 161 Combining characters, 161 Comb method, 38 Conjunction, 30 Control flow, 64 Control structures, 14 D Data formats address, 75 INI file, 76 invalid inputs, 78 JSON, 76 quoted strings (see Quoted strings) use anchors, 79 Declarative prefix, 72 Deterministic finite automaton (DFA) automaton for matching strings, 59, 61 lookup table, 58 Powerset construction, 61 die function, 149 Disjunctions, Docker-based installation, 8 9 E Eastern Arabic numerals, 160 End of line (eol), Error messages die function, 149 custom error method, 150 error location, FAILGOAL method, 156, 157 grammar, 157 high-water mark callframe, 153 dynamic variable, self.pos, 153 ws method, 153 match failures, 148 parse fail, 147 parser combinator, 156 F Factorial operator, 181 FAILGOAL method, 156, 157, 176 Finite state machine, 57 from product, 111 Frugal quantifier, 27, 71 Full-width characters, 163 G, H Glyphs defined, 162 font, 163 full-width characters,

3 Grammar(s) body, 99 defined, 1, inheritance MySQL, 100, 102 SQL, 100, 102 JSON, 104 match object, mathematical expression, 110 precedence, 112 from product, 111 proto regexes, , 108 regexes methods, role composition, subparse method, 99 Grapheme cluster (grapheme), Greedy quantifier, 27, 71 I, J Ideograms, 160 Infix operators, 174 K Kleene star, 4 L Lazy quantifier. See Frugal quantifier Left recursion, 113 Lexical analysis backtracking, 95 calculator, 94 Lexical scoping with dynamic variables, 128, 130 Ligature, 162 Lisp programming languages, 166 Literals meta characters vs., 18 strings, Longest token matching, declarative prefix, 72 Look-ahead assertion, 31 Look-behind assertions, 32 M Match object array, 48.made attribute, 136 named captures, 56 nest captures, 49 positional captures, 55 quantified captures, 49 Meta characters defined, 18 vs. literals, 18 Metasyntactic. See Meta characters Modifiers :exhaustive, 38 :global, 35 :ignorecase, 36 :ratchet,

4 Modifiers (cont.) runtime behavior of compiled regex, :sigspace, 37 N Named captures, 50 51, 55 Named regexes angle bracket syntax, 92 byte, 92 code objects, 92 cursor object, 92 syntactic forms, 93 Nondeterministic finite automata (NFA) code block {}, 72 empty block {}, 73 advantage and disadvantage, 64 automaton for matching strings, 63 declarative prefix, 72 defined, 62 epsilon transitions, 62 O Operator precedence parser (OPP), 113 advantages, 174 associativity, 181 caps method, 180 create extension role, 181 crux, 177 defined, 174 Grammar::ErrorReporting, 176 infix operators, 174 match object, opp-parse method, opp-reduce function, 178 Opp-parse method, P Parser combinator, Parse tree, 135 Parsing with grammars assembling, 116 Grammar::Tracer module Config::INI, 117 diagnostic tool, 117 keyval, output, TOP/eol, starting simple, 114, 116 symbol table, termlist, 126 Perl 6 regexes benefits, 5 control structures, 14 functions, classes, and methods, 15 PCRE, 5 strings, 13 variables, 12 Perl-Compatible Regular Expressions (PCRE), 5 Pointy block,

5 Positional captures, 47 48, 55 Possessive quantifier, 69 Postfix operators, 181 Powerset construction, 61 Precomposed characters, 161 Predictive parser, 109 Proto regexes but operator, 107 defined, 105 JSON grammar, 105 Pythonesque action objects handle_indentation method, 193 if-statement, leave_scope, 193 defined, 183 if-statement, 187 self.enter_scope() into method, 189 sentinel value, whitespace, 185 Q Quantifiers capture, 49 defined, 26 frugal, 27 greedy, 27 possessive, 69 RANGE, 27 separator, Quoted strings backslash, character class, 81 CSV file, 80 escape sequences, 82 naïve approach, 80 unbalanced input, 81 Quote forms, 34 R Rakudo Perl 6 binary linux packages, 8 docker-based installation, 8 9 running perl6, 9, 11 Rakudo OS Packages, 8 Rakudo Star, 8 zef, 7 Recursive descent parsing, 112 Regexes blocks, 44 code blocks, 45 defined, 1 searching, 1 2 variables, Regular expressions, 4 Regular language, 4 Role composition, S Secret Magic Sauce, 68 Sentinel value, S-Expressions 199

6 action method, 171, 173 atom, 166 defined, 165 grammar rules, identifiers, 167 Lisp programming languages, 166 nok, 169 proto token, 166 string contents, 167 test code, whitespace-delimited list of atoms, 169, 171 Smart-match operator, quote forms, 35 Split method, 39 State machines accepting, 58 DFA (see Deterministic finite automaton (DFA)) finite, 57 NFAs (see Nondeterministic finite automata (NFA)) Strings, 13 contents, 167 double-quoted, 43.match method, 34 replacement, Str.lines method, 151 Structured Query Language (SQL) inheritance, 100, 102 subst method, Syllabary, 160 Symbolic Expressions. See S-Expressions Symbol table defined, 124 elements, 125 explicitly call methods, 131 LEAVE phaser, 133 rule statement, 126 T Test Anything Protocol format, 85 Testing module failing test output, 85 for loops, 84 MAIN subroutine, 86, 88 named argument, 88 short summary, strings, Text processing tools, 4 Text segmentation, 160 Tokenization. See Lexical analysis TOP regex, 99 Top-down approach, 109 Two-step method, 94 U Unicode characters, 23 properties and categories, 25, uniprop method,

7 V Variables, 12, W, X, Y Web applications, 3 Whitespace INI parser, 121, 123 insertion of implicit, single blank, 97 SQL statements, 96 Writing systems alphabet, 159 consonant alphabets, 160 ideograms, 160 phonemes, 159 syllabary, 160 text segmentation, 160 ws method, 154 Z Zero-width assertion look-ahead, 31 look-behind,