ANTLR Grammar line 1:6 mismatched input '<EOF>' expecting '.'

Andrei Ciobanu :

I am playing with antlr4 grammar files, and I wanted to write my own jsonpath grammar.

I've comeup with this:

grammar ObjectPath;

objectPath      : dnot;

dnot            : ROOT expr ('.' expr)
                | EOF
                ;

expr            : select #selectExpr
                | ID #idExpr
                ;

select          : ID '[]' #selectAll
                | ID '[' INT ']' #selectIndex
                | ID '[' INT (',' INT)* ']' #selectIndexes
                | ID '[' INT ':' INT ']' #selectRange
                | ID '[' INT ':]' #selectFrom
                | ID '[:' INT ']' #selectUntil
                | ID '[-' INT ':]' #selectLast
                | ID '[?(' query ')]' #selectQuery
                ;

query           : expr (AND|OR) expr # andOr
                | ALL # all
                | QPREF ID # prop
                | QPREF ID GT INT # gt
                | QPREF ID LT INT # lt
                | QPREF ID EQ INT # eq
                | QPREF ID GTE INT # gte
                | QPREF ID LTE INT # lte
                ;

/** Lexer **/
ROOT    : '$.' ;
QPREF   : '@.' ;
ID      : [a-zA-Z][a-zA-Z0-9]* ;
INT     : '0' | [1-9][0-9]* ;
AND     : '&&' ;
OR      : '||' ;
GT      : '>'  ;
LT      : '<'  ;
EQ      : '==' ;
GTE     : '>=' ;
LTE     : '<=' ;
ALL     : '*'  ;

After running this on a simple expression:

CharStream input = CharStreams.fromString("$.name");
ObjectPathLexer lexer = new ObjectPathLexer(input);
CommonTokenStream tokens = new CommonTokenStream(lexer);

ObjectPathParser parser = new ObjectPathParser(tokens);
ParseTree parseTree = parser.dnot();
ObjectPathDefaultVisitor visitor = ...
System.out.println(visitor.visit(parseTree));
System.out.println(parseTree.toStringTree(parser));

The output is ok, meaning that the "name" is actually retrieved from the json, but there's a warning I cannot explain:

line 1:6 mismatched input '<EOF>' expecting '.'

I've read that I need to explicitly have an EOF rule added to my starting one (dnot), but this doesn't seem to work.

Any idea what can I do ?

Bart Kiers :

Your input $.name cannot be parsed by your rule:

dnot            : ROOT expr ('.' expr)
                | EOF
                ;

$.name produces 2 tokens:

1. ROOT
2. ID

But your first alternative, ROOT expr ('.' expr), expects 2 expressions separated by a .. Perhaps you meant to make the second expr optional, like this:

dnot            : ROOT expr ('.' expr)*
                | EOF
                ;

And the EOF is generally added at the end of your start rule, to force the parser to consume all tokens. As you did it now, the parser successfully parsed ROOT expr, but then failed to parse further, and produces the warning you saw (expecting '.').

Since objectPath seems to be your start rule, I think this is what you want to do:

objectPath      : dnot EOF;

dnot            : ROOT expr ('.' expr)?
                ;

Also, tokens like these [], '[?(', etc look suspicious. I'm not really familiar with Object Path, but by glueing these chars to each other, input like this [ ] ([ and ] separated by a space) will not be matched by []. So if foo[ ] is valid, I'd write it like this instead:

select          : ID '[' ']' #selectAll
                | ...

and skip spaces in the lexer:

SPACES : [ \t\r\n]+ -> skip;