Stantler - ANTLR Clone in Common Lisp

This project isn't serious and isn't built for compatibility or performance. Just having fun making parser generators. :)

Objective

The goal is to make a self-hosting parser generator. ANTLR automatically generates a parser and lexer from grammar definitions. I think it would be cool to do the same thing in Common Lisp. Unlike Java, the original implementation language for ANTLR, I can do fun stuff like directly create a parser in memory from a text file.

Bootstrapping Strategy

Since there's a grammar definition for the ANTLR grammar itself, it's possible to bootstrap a parser by handwriting only the original tokenizer and lexer.

There are only a few things needed - A set of core matching rules (and, or, repeats, etc) - Tokenizer rules - Parser rules

Writing out all of the parser rules is a little annoying. It is a bit repetitive, but there are surprisingly easy ways to mitigate mistakes. It was much easier to tokenize and parse the entire input than I expected.

Once the input is parsed into a tree, the exciting part (for me) is being able to transform this tree into code. This is where Common Lisp shines.

The Unexpected

• The tokenizer inserts an EOF token at the end of input
• The token type (usually the rule that produced it) can be modified by the tokenizer
  • The ANLTR tokenizer contains a state machine!

Home > Q Science > QA Mathematics Computer science > Stantler - ANTLR Clone in Common Lisp