Text.Lexer

Reexports

Definitions

toTokenMap : List (Lexer, k) -> TokenMap (Token k)

Totality: total
Visibility: export

any : Lexer

  Recognise any character.
  /./

Totality: total
Visibility: export

opt : Lexer -> Recognise False

  Recognise a lexer or recognise no input. This is not guaranteed
  to consume input.
  /`l`?/

Totality: total
Visibility: export

non : Lexer -> Lexer

  Recognise any character if the sub-lexer `l` fails.
  /(?!`l`)./

Totality: total
Visibility: export

choiceMap : Foldable t => (a -> Recognise c) -> t a -> Recognise c

  Produce recognisers by applying a function to elements of a container, and
  recognise the first match. Consumes input if the function produces consuming
  recognisers. Fails if the container is empty.

Totality: total
Visibility: export

choice : Foldable t => t (Recognise c) -> Recognise c

  Recognise the first matching recogniser in a container. Consumes input if
  recognisers in the list consume. Fails if the container is empty.

Totality: total
Visibility: export

concat : (xs : List (Recognise c)) -> Recognise (isCons xs && Delay c)

  Sequence a list of recognisers. Guaranteed to consume input if the list is
  non-empty and the recognisers consume.

Totality: total
Visibility: export

is : Char -> Lexer

  Recognise a specific character.
  /[`x`]/

Totality: total
Visibility: export

isNot : Char -> Lexer

  Recognise anything but the given character.
  /[\^`x`]/

Totality: total
Visibility: export

like : Char -> Lexer

  Recognise a specific character (case-insensitive).
  /[`x`]/i

Totality: total
Visibility: export

notLike : Char -> Lexer

  Recognise anything but the given character (case-insensitive).
  /[\^`x`]/i

Totality: total
Visibility: export

exact : String -> Lexer

  Recognise a specific string.
  Fails if the string is empty.
  /`str`/

Totality: total
Visibility: export

approx : String -> Lexer

  Recognise a specific string (case-insensitive).
  Fails if the string is empty.
  /`str`/i

Totality: total
Visibility: export

oneOf : String -> Lexer

  Recognise any of the characters in the given string.
  /[`chars`]/

Totality: total
Visibility: export

range : Char -> Char -> Lexer

  Recognise a character range. Also works in reverse!
  /[`start`-`end`]/

Totality: total
Visibility: export

some : Lexer -> Lexer

  Recognise a sequence of at least one sub-lexers
  /`l`+/

Totality: total
Visibility: export

many : Lexer -> Recognise False

  Recognise a sequence of at zero or more sub-lexers. This is not
  guaranteed to consume input
  /`l`\*/

Totality: total
Visibility: export

someUntil : Recognise c -> Lexer -> Lexer

  Repeat the sub-lexer `l` one or more times until the lexer
  `stopBefore` is encountered. `stopBefore` will not be consumed.
  /((?!`stopBefore`)`l`)\+/

Totality: total
Visibility: export

manyUntil : Recognise c -> Lexer -> Recognise False

  Repeat the sub-lexer `l` zero or more times until the lexer
  `stopBefore` is encountered. `stopBefore` will not be consumed.
  Not guaranteed to consume input.
  /((?!`stopBefore`)`l`)\*/

Totality: total
Visibility: export

manyThen : Recognise c -> Lexer -> Recognise c

  Repeat the sub-lexer `l` zero or more times until the lexer
  `stopAfter` is encountered, and consume it. Guaranteed to
  consume if `stopAfter` consumes.
  /`l`\*?`stopAfter`/

Totality: total
Visibility: export

count : (q : Quantity) -> Lexer -> Recognise (isSucc (min q))

  Recognise a sub-lexer repeated as specified by `q`. Fails if `q` has
  `min` and `max` in the wrong order. Consumes input unless `min q` is zero.
  /`l`{`q`}/

Totality: total
Visibility: export

digit : Lexer

  Recognise a single digit 0-9
  /[0-9]/

Totality: total
Visibility: export

digits : Lexer

  Recognise one or more digits
  /[0-9]+/

Totality: total
Visibility: export

binDigit : Lexer

  Recognise a single binary digit
  /[0-1]/

Totality: total
Visibility: export

binDigits : Lexer

  Recognise one or more binary digits
  /[0-1]+/

Totality: total
Visibility: export

hexDigit : Lexer

  Recognise a single hexidecimal digit
  /[0-9A-Fa-f]/

Totality: total
Visibility: export

hexDigits : Lexer

  Recognise one or more hexidecimal digits
  /[0-9A-Fa-f]+/

Totality: total
Visibility: export

octDigit : Lexer

  Recognise a single octal digit
  /[0-8]/

Totality: total
Visibility: export

octDigits : Lexer

  Recognise one or more octal digits
  /[0-8]+/

Totality: total
Visibility: export

alpha : Lexer

  Recognise a single alpha character
  /[A-Za-z]/

Totality: total
Visibility: export

alphas : Lexer

  Recognise one or more alpha characters
  /[A-Za-z]+/

Totality: total
Visibility: export

lower : Lexer

  Recognise a lowercase alpha character
  /[a-z]/

Totality: total
Visibility: export

lowers : Lexer

  Recognise one or more lowercase alpha characters
  /[a-z]+/

Totality: total
Visibility: export

upper : Lexer

  Recognise an uppercase alpha character
  /[A-Z]/

Totality: total
Visibility: export

uppers : Lexer

  Recognise one or more uppercase alpha characters
  /[A-Z]+/

Totality: total
Visibility: export

alphaNum : Lexer

  Recognise an alphanumeric character
  /[A-Za-z0-9]/

Totality: total
Visibility: export

alphaNums : Lexer

  Recognise one or more alphanumeric characters
  /[A-Za-z0-9]+/

Totality: total
Visibility: export

space : Lexer

  Recognise a single whitespace character
  /\\s/

Totality: total
Visibility: export

spaces : Lexer

  Recognise one or more whitespace characters
  /\\s+/

Totality: total
Visibility: export

newline : Lexer

  Recognise a single newline sequence. Understands CRLF, CR, and LF
  /\\r\\n|[\\r\\n]/

Totality: total
Visibility: export

newlines : Lexer

  Recognise one or more newline sequences. Understands CRLF, CR, and LF
  /(\\r\\n|[\\r\\n])+)/

Totality: total
Visibility: export

symbol : Lexer

  Recognise a single non-whitespace, non-alphanumeric character
  /[\^\\sA-Za-z0-9]/

Totality: total
Visibility: export

symbols : Lexer

  Recognise one or more non-whitespace, non-alphanumeric characters
  /[\^\\sA-Za-z0-9]+/

Totality: total
Visibility: export

control : Lexer

Totality: total
Visibility: export

controls : Lexer

  Recognise one or more control characters
  /[\\x00-\\x1f\\x7f-\\x9f]+/

Totality: total
Visibility: export

surround : Lexer -> Lexer -> Lexer -> Lexer

  Recognise zero or more occurrences of a sub-lexer between
  delimiting lexers
  /`start`(`l`)\*?`end`/

Totality: total
Visibility: export

quote : Lexer -> Lexer -> Lexer

  Recognise zero or more occurrences of a sub-lexer surrounded
  by the same quote lexer on both sides (useful for strings)
  /`q`(`l`)\*?`q`/

Totality: total
Visibility: export

escape : Lexer -> Lexer -> Lexer

  Recognise an escape sub-lexer (often '\\') followed by
  another sub-lexer
  /[`esc`]`l`/

Totality: total
Visibility: export

stringLit : Lexer

  Recognise a string literal, including escaped characters.
  (Note: doesn't yet handle escape sequences such as \123)
  /"(\\\\.|.)\*?"/

Totality: total
Visibility: export

charLit : Lexer

  Recognise a character literal, including escaped characters.
  (Note: doesn't yet handle escape sequences such as \123)
  /'(\\\\.|[\^'])'/

Totality: total
Visibility: export

intLit : Lexer

  Recognise an integer literal (possibly with a '-' prefix)
  /-?[0-9]+/

Totality: total
Visibility: export

binLit : Lexer

  Recognise a binary literal, prefixed by "0b"
  /0b[0-1]+/

Totality: total
Visibility: export

hexLit : Lexer

  Recognise a hexidecimal literal, prefixed by "0x" or "0X"
  /0[Xx][0-9A-Fa-f]+/

Totality: total
Visibility: export

octLit : Lexer

  Recognise an octal literal, prefixed by "0o"
  /0o[0-9A-Fa-f]+/

Totality: total
Visibility: export

digitsUnderscoredLit : Lexer

Totality: total
Visibility: export

binUnderscoredLit : Lexer

Totality: total
Visibility: export

hexUnderscoredLit : Lexer

Totality: total
Visibility: export

octUnderscoredLit : Lexer

Totality: total
Visibility: export

lineComment : Lexer -> Lexer

  Recognise `start`, then recognise all input until a newline is encountered,
  and consume the newline. Will succeed if end-of-input is encountered before
  a newline.
  /`start`[\^\\r\\n]+(\\r\\n|[\\r\\n])?/

Totality: total
Visibility: export

blockComment : Lexer -> Lexer -> Lexer

  Recognise all input between `start` and `end` lexers.
  Supports balanced nesting.
  
  For block comments that don't support nesting (such as C-style comments),
  use `surround`

Totality: total
Visibility: export