Allows lexers to be defined using regular expressions a la cl-ppcre. A lexer will convert (or tokenize) a text string into a list of tokens (consisting of a token-class and the token's image). For example:
(defun snip (s) (subseq s 1 (1- (length s)))) (defun un-squote (s) (regex-replace-all "''" (snip s) "'")) (defun un-dquote (s) (regex-replace-all "\"\"" (snip s) "\"")) (deflexer scan-csv (:priority-only t) ("[^\"',]+" value) ("'(?:[^']|'')*'" value un-squote) ("\"(?:[^\"]|\"\")*\"" value un-dquote) ("," comma))
See the distribution for a more complete example.
Mailing List
dso-lex@…. Manage your subscription at http://lists.fugue88.ws/listinfo/dso-lex.
Last modified 14 years ago
Last modified on 10/12/2008 02:49:04 AM