版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/c13232906050/article/details/82587223
FROM: Page 78, The Definitive ANTLR 4 Reference, 2nd Edition
| Token Category | Description and Examples |
|---|---|
| Punctuation | The easiest way to handle operators and punctuation is to directly reference them in parser rules. call : ID '(' exprList ')' ; Some programmers prefer to define token labels such as LP (left parenthesis) instead. call : ID LP exprList RP ; LP : '(' ; RP : ')' ; |
| Keywords | Keywords are reserved identifiers, and we can either reference them directly or define token types for them. returnStat : 'return' expr ';' |
| Identifiers | Identifiers look almost the same in every language, with some variation about what the first character can be and whether Unicode characters are allowed. ID : ID_LETTER (ID_LETTER | DIGIT)* ; // From C language fragment ID_LETTER : 'a'..'z'|'A'..'Z'|'_' ; fragment DIGIT : '0'..'9' ; |
| Numbers | These are definitions for integers and simple floating-point numbers. INT : DIGIT+ ; FLOAT : DIGIT+ '.' DIGIT* | '.' DIGIT+ ; |
| Strings | Match double-quoted strings. STRING : '"' ( ESC | . )*? '"' ; fragment ESC : '\\' [btnr"\\] ; // \b, \t, \n etc... |
| Comments | Match and discard comments. LINE_COMMENT : '//' .*? '\n' -> skip ; COMMENT : '/*' .*? '*/' -> skip ; |
| Whitespace | Match whitespace in the lexer and throw it out. WS : [ \t\n\r]+ -> skip ; |