Skip to content

fix: include token type in unexpected token error messages - #463

Merged
alecthomas merged 2 commits into
alecthomas:masterfrom
Pastalikek65:improve-unexpected-token-error
Aug 22, 2026
Merged

alecthomas merged 2 commits into
alecthomas:masterfrom
Pastalikek65:improve-unexpected-token-error

Conversation

@Pastalikek65

Copy link
Copy Markdown
Contributor

Unexpected token errors now report the symbolic name of the lexer's token type when one exists, e.g.:

unexpected token "group" of type Keyword (expected <ident>)

Token types without a symbolic name (such as literals) keep the previous format, e.g.:

unexpected token ")" (expected <ident>)

This is the format suggested by @alecthomas in #265 ("unexpected token "group" of type keyword").

Implementation notes:

  • parseContext now carries the lexer's symbol table (built once from the parser's lexer.Definition); every UnexpectedTokenError construction point (7 sites across parser.go/nodes.go) fills the new TokenType field.
  • UnexpectedTokenError.TokenType is ordered after the pre-existing exported fields, so external positional struct literals remain source-compatible.

Verification: go test ./... (root + _examples), golangci-lint run — no new issues vs. baseline.

Fixes #265.

Unexpected token errors now include the symbolic name of the lexer token type when one exists, e.g. "unexpected token \"group\" of type Keyword". Token types without a symbolic name (such as literals) are unchanged.

Fixes alecthomas#265.

@alecthomas alecthomas left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm. I think this should use the same <lower-symbol> syntax that we already use elsewhere for grammars.

@Pastalikek65

Copy link
Copy Markdown
Contributor Author

Thanks @alecthomas! I've updated the format to use the same <lower-symbol> syntax as elsewhere in the grammar (e.g. unexpected token "struct" of type <ident>), via tokenTypeName returning <lower>-wrapped symbols. All tests updated and passing.

@Pastalikek65

Copy link
Copy Markdown
Contributor Author

Updated the error format to use <lower-symbol> syntax (matching the rest of the grammar) in 9d6e022 — any chance you could take another look when convenient?

@alecthomas
alecthomas merged commit 2d62fe3 into alecthomas:master Aug 22, 2026
2 checks passed
@alecthomas

Copy link
Copy Markdown
Owner

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Include token type in unexpected token error message

2 participants