scanner
Lua 5.1 lexical patterns

Version: 0.1.2
Generated: November 26, 2007

Description

This module exports several Lua lexical patterns, all implemented in LPeg. None of them have captures.

Dependencies

Example

The following example lists all the tokens in a Lua script:

local lpeg = require 'lpeg'
local scanner = require 'leg.scanner'

 -- this pattern captures all tokens, ignores spaces and comments,
 -- and matches with end of file, giving an error otherwise
patt = (lpeg.C(scanner.TOKEN) + scanner.SPACE + scanner.COMMENT)^0
    * (scanner.EOF + scanner.error'invalid character')
patt = lpeg.Ct(patt)

 -- opens the file passed as parameter and tries to match with patt
f = assert(io.open(arg[1]))

 -- a table storing all the tokens
ALL = patt:match(f:read'*a')
f:close()

 -- dumps all tokens on screen
for _, tk in ipairs(ALL) do
   print(tk)
end


Variables

ANY = LPeg pattern 
Matches any token, comment or space.
BANG = LPeg pattern 
Matches UNIX's shebang (e.g. #!/usr/bin/lua).
BOF = LPeg pattern 
Matches the beginning of a file.
COMMENT = LPeg pattern 
Matches any type of comment.
EOF = LPeg pattern 
Matches the end of a file.
IDENTIFIER = LPeg pattern 
Matches any Lua identifier.
IGNORED = LPeg pattern 
Matches everything ignored by the parser.
KEYWORD = LPeg pattern 
A pattern which matches any Lua keyword.
keywords = {} 

A table with Lua keyword-matching patterns, with the keywords in uppercase as keys.

Examples: keywords.WHILE, keywords['ELSEIF'].

NUMBER = LPeg pattern 
Matches any Lua number.
SPACE = LPeg pattern 
Matches any space character.
STRING = LPeg pattern 
Matches any Lua string.
SYMBOL = LPeg pattern 
A pattern which matches any Lua symbol.
symbols = {} 

A table with Lua symbol-matching patterns, with the symbols themselves as keys.

Examples: symbols['{'], symbols['+'].

TOKEN = LPeg pattern 
Matches any Lua identifier, keyword, symbol, number or string.

Functions

comment2text (comment) Strips all prefixing -- and enclosing --[=*[ from comment tokens.
error (msg) Returns a function which throws lexical errors.
string2text (str) Strips all enclosing ', ", and [=*[ from string tokens, and processes escape characters.
text2comment (text) Encloses the text with comment markers.
text2string (text) Transforms a text into a syntactically valid Lua string. Similar to string.format with the '%q' option, but inserting escape numbers and escape codes where applicable.


comment2text (comment)
    Strips all prefixing -- and enclosing --[=*[ from comment tokens.

    Parameters:

    • comment: the comment to strip.

    Returns:

    • the text without comment marking syntax.


error (msg)
    Returns a function which throws lexical errors.

    Parameters:

    • msg: the message to be concatenated to the error message.

    Returns:

    • A function built to be used as a LPeg pattern, which will throw an error when matched.

    Usage example:

    patt = intended_pattern^0 * (EOF + error'character after EOF')
    

    It may also be used as a normal function:

    function (subject, i)
     if bad_condition then
       error'bad condition'(subject, i)
     end
    end
    


string2text (str)
    Strips all enclosing ', ", and [=*[ from string tokens, and processes escape characters.

    Parameters:

    • str: the string to strip.

    Returns:

    • the text without string enclosers.


text2comment (text)
    Encloses the text with comment markers.

    Parameters:

    • text: the text to comment.

    Returns:

    • the text with comment marking syntax.


text2string (text)
    Transforms a text into a syntactically valid Lua string. Similar to string.format with the '%q' option, but inserting escape numbers and escape codes where applicable.

    Parameters

    • text: a string containing the text.

    Returns:

    • a string, similar to string.format with option '%q'.