RAQUEL Parser Code Design

Transcription

1 Parser Input A sequence of pointers-to-tokens. RAQUEL Parser Code Design Parser Output A tree of tokens linked together by pointers AND a sequence of error codes. Pre Conditions None. Post Conditions Length( Sequence( ErrorCodes ) ) = 0 AND { Input Tokens = { Tree Tokens Union { Input Parentheses Tokens AND { Tree Tokens Intersect { Input Parentheses Tokens = { OR Length( Sequence( ErrorCodes ) ) > 0 AND { Tree Tokens = { AND { Input Tokens = { Memory Organisation The RAQUEL Parser is a free-standing software module that is a component of the RAQUEL DBMS. It uses other software functions which are treated as subroutines of the Parser. The Parser and other functions with their relationships can be depicted as follows :- Parser Meta Parser Algebra Parser Assignment Parser Some of these functions also have their own subroutines. However all subroutines are selfcontained within the Parser. For efficiency, the parser s input token sequence and the parser s output token parse tree are held in the DBMS memory space, which is external to the parser module. Thus strictly speaking, the Parser and its relevant subroutines all manipulate the token sequence and its Page 1 of 8

2 transformation into a token tree as a side effect. Only the sequence of error codes is a direct functional output of the Parser. Errors are generally treated as exceptions and are passed up to the Parser function, which then manipulates them to produce an output sequence of error codes. Boundary Conditions Valid a RAQUEL statement may contain no parentheses. The Parser Handler will find nothing and the Parser will use the Meta Parser to parse the entire input token sequence. Valid the entire RAQUEL statement may be enclosed in parentheses. The Parser Handler will pass the entire input token sequence to the Meta Parser for parsing. Invalid the entire RAQUEL statement may consist of just a pair of matched parentheses. A pair of parentheses must enclose a RAQUEL expression of some kind. Data Descriptions The parser input is actually a sequence of pointers to tokens rather than a sequence of tokens. This is because : the output parse tree consists of tokens linked together by pointers, and it is easier to construct the tree by assigning the pointer values, unchanged, from the sequence to other tokens in the tree, thereby constructing the tree itself; the tokens vary significantly in size and structure, and therefore at the code level it would not be very practicable to have a sequence of tokens. The Algebra Parser and the Assignment Parser both have their own stacks. Each time one of these parsers is called, its stack is initiated. Were there to be a single stack used by both of these parsers, it could happen that tokens would be pushed on in the wrong sequence (owing to the left-to-right and right-to-left scans of the Algebra and Assignment Parsers respectively) and parsing would not be validly completed. Amending the input token sequence as parsing progresses to reflect the current situation ensures that parsing is executed in the correct order. For efficiency, short single-character codes are used in the stack rather than the longer codes used elsewhere to help describe the parsing process. The single-character codes are : Parentheses : ( and ). Relational data values : R and L (relvar and literal relvalue respectively). Operators : N, M, and D (niladic, monadic and dyadic respectively). Assignments : A and B (monadic and dyadic respectively). Expressions : E. Note that it is not necessary to distinguish between pseudo assignments and non-pseudo assignments where the stack is concerned, since the stack handling concerns pattern recognition, for which this is irrelevant. However pseudo assignments and non-pseudo assignments are taken into account by the grammar procedures, to ensure that the parse tree is built correctly. Page 2 of 8

3 Algebra Parser Pseudo Code The pseudo code showing the general purpose algorithm for the algebra parser is :- Index AlgebraParser( Begin, End ) { Create Stack. Index Begin. WHILE ( Index End ) { Push( Copy( TokenSequence(Index) ) on Stack IF Top_Of_Stack AlgebraProductions( RHS ) THEN Reduce( Top_Of_Stack, AlgebraProductions ) ELSE Index Index + 1 IF Whole_Of_Stack = Exp THEN RETURN( Begin - 1 ) If the sequence of tokens representing an algebra expression is a valid sequence, then the only token left on the stack after parsing is complete will be Exp. If some other token remains on the stack and/or there are additional tokens, then an error has occurred. The details of the error procedure are not specified here. The parser returns the token position immediately to the left of the algebra expression that has been parsed, because this cannot be a token that participates in an algebra expression. The pseudo code showing the reduction algorithm used by the parser is :- Reduce( Top_Of_Stack, GrammarProductions ) { Pop( Top_Of_Stack ) off Stack. Push( GrammarProductions( Top_Of_Stack(LHS) ) on Stack. Execute( (Top_Of_Stack( Procedure ) ). IF Top_of_Stack AlgebraProductions( RHS ) THEN Reduce( Top_Of_Stack, GrammarProductions ) Note that after reduction has occurred, recursion is used within the reduction procedure to handle any further patterns found on the stack. In executing a production rule s procedure, the input token sequence will be amended. As all grammar productions have an Exp token on their RHSs, an Exp token will replace one or more tokens in the input token sequence; a pointer held in the Exp token will point to the root of the sub parse tree created as a result. Page 3 of 8

4 Assignment Parser Pseudo Code The pseudo code showing the general purpose algorithm for the assignment parser is :- Index AssignmentParser( Index ) { Create Stack. IF Token( Index ) = Dyadic_Ass THEN Push( Copy( TokenSequence( Index+1 ) ) on Stack Push( Copy( TokenSequence( Index ) ) onto Stack IF Token( Index-1 ) 1 THEN Push( Copy( TokenSequence( Index-1 ) ) on Stack IF Token( Index - 1 ) = Monadic_Op THEN { Index Index - 1 IF ( Index - 1 ) 1 THEN Push(Copy(TokenSequence(Index-1)) on Stack IF Top_of_Stack AssignmentProductions( RHS ) THEN { Reduce( Top_of_Stack, AssignmentProductions ). RETURN( Index - 1 ) If the assignment token represents a dyadic assignment, then its right-hand assignment is first pushed onto the stack so as to permit a full dyadic assignment expression to be pushed onto the stack for potential recognition. If the token to the immediate left of the assignment represents a monadic operator, then the scan needs to be continued one more token to the left to allow for the monadic operator and its operand to together form the left-hand operarand of the assignment. The parsing would be simpler if the left-hand operand of an assignment always had to be an expression, as this option could then be ignored. The Assignment Parser uses the same reduction algorithm as the Algebra Parser, but with assignment not algebra grammar productions; its recursive nature enables it to handle assignments that have a monadic operator on their LHS as well as those assignments that don t. The parser returns the leftmost token position of the assignment expression that has been parsed, because this now represents a value that could be part of an algebra expression or the right-hand operand of a dyadic assignment. As with the Algebra Parser, executing an assignment production rule s procedure will amend the input token sequence in exactly the same way. Page 4 of 8

5 Meta Parser Pseudo Code The pseudo code showing the algorithm for the Meta Parser is :- MetaParser( Begin, End ) { IF End Begin THEN ErrorException( ) Index End WHILE ( Index Begin ) DO { IF Token( Index ) = Assignment THEN Index AssignmentParser( Index ) ELSE { Start TraceExpression( Begin, Index ). Index AlgebraParser( Start, Index ) RETURN Both AssignmentParser and AlgebraParser return an index pointing to the token at which the Meta Parser can continue its scan to the left. Having found the right-hand end of an algebra expression, it is necessary to trace it through to its left-hand end so that AlgebraParser can be called with the sequence of tokens that comprise the algebra expression. TraceExpression is the function that finds the lefthand end; the pseudo code showing its algorithm is :- Start TraceExpression( Begin, Index ) { Index Index - 1 WHILE ( Index Begin ) DO { IF Token( Index ) = Assignment THEN EXIT LOOP ELSE Index Index - 1 RETURN( Index + 1 ) Page 5 of 8

6 Parser Pseudo Code The pseudo code showing the algorithm for the main Parser program is :- ErrorSequence Parser( ) { ErrorSequence [ ]. IF Token(1) = tab THEN Delete( Token(1) ) IF Token( Last ) = THEN Delete( Token( Last ) ) Length LengthOf( TokenSequence ). Create NestLevel[ 1.. Length ]. MaxLevel 0. NestLevel Scan( TokenSequence ). IF MaxLevel > 0 THEN { FOR ( Level MaxLevel TO 1 BY -1 ) DO { Index 1. WHILE( Index Length ) DO { IF NestLevel( Index ) = MaxLevel THEN { Begin Index + 1. DO { Index Index + 1 UNTIL( NestLevel( Index ) < MaxLevel ). End Index - 1. MetaParser( Begin, End ). DeleteParentheses( Begin - 1 ). Reduction ( End - Begin ) + 2. ReduceNestLevel( Reduction, Begin-1 ). Index Index - Reduction + 1. Length Length - Reduction ELSE Index Index + 1 MaxLevel MaxLevel - 1 Page 6 of 8

7 IF NOT( LengthOf( TokenSequence ) = 1 AND TokenSequence( 1 ) = Exp ) THEN MetaParser( 1, Length ) Assign( Pointer_to_Parse_Tree ) The Parser first checks that the complete input token sequence has been passed to it, via checks on the first and last tokens, which it then deletes to avoid them being part of the parsing. It next scans the token sequence looking for parentheses and determining the level of nesting of parentheses. The latter is stored in the variable NestLevel which is a sequence having the same length as the token sequence but storing integers instead of tokens. The pseudocode of Scan is :- NestLevel Scan( TokenSequence ) { FOR ( Index 1 TO Length BY +1 ) DO { IF TokenSequence( Index ) = ( THEN Level Level + 1 ELSEIF TokenSequence( Index ) = ) THEN Level Level - 1 NestLevel( Index ) Level. IF Level > MaxLevel THEN MaxLevel The variable MaxLevel is global to Scan but internal to the Parser. Level DeleteParentheses removes the parentheses surrounding the expression which has just been parsed, and ReduceNestLevel removes the reduction number of integers from NestLevel starting at point Begin - 1 in the integer sequence in order to keep it synchronised with the reduction in length of the token sequence as a result of parsing. The working variables Index and Length also have to be amended to retain consistency. If the entire input sequence has not by then been reduced to a single Exp token, then it is parsed via the Meta Parser to complete the parsing. When parsing is complete the Exp token, which is the sole token that results from parsing valid input, can be dispensed with and the pointer to the root of the parse tree assigned from Exp to the parse tree variable. The above pseudo code does not attempt to outline how errors are dealt with. This is more appropriately left to the detailed design of the program code. Page 7 of 8

8 Identifying RAQUEL Tokens RAQUEL tokens do not actually contain the codes referred to above, but rather : a Data/Process Flag : this identifies whether the token represents data, an operator, an assignment, or a parenthesis. a Form Flag : this identifies individual possibilities within the above 4 categories. Between them, these two flags identify all the possible different kinds of token. This is relevant only at the program code level, and is mentioned here for completeness and for convenience when relating this documentation to documentation about the tokens. Page 8 of 8