Class: TokenStream

TokenStream(lexer)

new TokenStream(lexer)

A TokenStream represents a stream of tokens
Parameters:
Name Type Description
lexer Parser.Lexer The lexer to source the tokens from
Source:

Methods

checkNext(type, offset) → {boolean}

Check if the next token has a specific type
Parameters:
Name Type Description
type string The token type to check for
offset number The offset from the next token to the token to lookahead (optional, default: 0=next token)
Source:
Returns:
true if and only if the next token has the specified type.
Type
boolean

commit()

Remove the last rewind checkpoint
Source:

consume(type) → {Parser.Token}

Check the current token for the given token type, raising an error on mismatch, otherwise fetching the next token
Parameters:
Name Type Description
type string The type to check for in the current token
Source:
Throws:
The current token must match the given token type
Type
Parser.ParseError
Returns:
The (new) current token
Type
Parser.Token

enforceCompletion()

Throw an Parser.ParseError if there is text remaining. This is to be used to ensure complete parsing of an input.
Source:
Throws:
The end of the input must have been reached
Type
Parser.ParseError

lookahead(offset) → {Parser.Token}

Take a look at the current token without consuming it
Parameters:
Name Type Description
offset number The offset from the next token to the token to lookahead (optional, default: 0=next token)
Source:
Returns:
The lookahead token
Type
Parser.Token

pushCheckpoint()

Add a checkpoint for rewinding
Source:

rewind()

Rewind to the last rewind checkpoint
Source:

tryParsing(fn, defValue)

Perform a parsing action speculatively
Parameters:
Name Type Description
fn function The function object to call
defValue The default value to return if parsing fails
Source:
Returns:
The return value of the function or the given default value