Skip to content

fix(ebnf): round-trip negation and lookahead assertions - #481

Open
MaxFreedomPollard wants to merge 1 commit into
alecthomas:masterfrom
MaxFreedomPollard:fix/ebnf-negation-roundtrip
Open

MaxFreedomPollard wants to merge 1 commit into
alecthomas:masterfrom
MaxFreedomPollard:fix/ebnf-negation-roundtrip

Conversation

@MaxFreedomPollard

Copy link
Copy Markdown

ebnf.ParseString parses the EBNF that Parser.String() emits, and the package's only test asserts that parsing and reprinting is the identity. Two forms do not survive that round trip.

Term.Negation is captured by the @("~")? tag on Term (ebnf/ebnf.go:31), but Term.String() (ebnf/ebnf.go:43) never emits it, so ~"a" prints back as "a". That flips the meaning of the production: the README describes ~<expr> as matching any token that is not the start of the expression. cmd/railroad/main.go:130 already reads Term.Negation, so Term.String() was the one consumer of the field that dropped it.

SubExpression.String() (ebnf/ebnf.go:92) prints (?="a"), but the printer it mirrors emits (?= "a") with a space after the assertion (ebnf.go:147 and ebnf.go:149), so lookahead groups do not round trip either.

Neither shows up in TestEBNF because the ebnf package's own grammar uses no negation and no lookahead group. Feeding back the string that TestEBNF_Other in the root package asserts makes both visible at once:

in:  Grammar = ((?= "good") <ident>) | ((?! "bad" | "worse") <ident>) | ~("anything" | "but") .
out: Grammar = ((?="good") <ident>) | ((?!"bad" | "worse") <ident>) | ("anything" | "but") .

Term.String() now prefixes ~ when Negation is set, and SubExpression.String() emits the space. The new table-driven test round-trips negated literals, tokens, productions, groups and repetitions, both lookahead assertions, and that TestEBNF_Other string verbatim. All 8 subtests fail on master and pass with the change.

Verified on go1.26.4 darwin/arm64 and golangci-lint 2.12.2 with the repo's config:

$ go test ./...
ok  github.com/alecthomas/participle/v2               0.263s
ok  github.com/alecthomas/participle/v2/ebnf          1.113s
ok  github.com/alecthomas/participle/v2/lexer         0.715s
ok  github.com/alecthomas/participle/v2/lexer/internal/conformance  3.898s

$ cd _examples && go test ./...   # 20 packages, all ok

$ golangci-lint run               # no output, exit 0

The ebnf package parses the EBNF that Parser.String() produces, and its
only test asserts that parsing then printing is the identity. Two forms
did not survive that round trip.

Term.Negation is captured by the `@("~")?` tag at ebnf/ebnf.go:31, but
Term.String() (ebnf/ebnf.go:43) never emitted it, so `~"a"` printed as
`"a"`. That inverts the meaning of the production: the README defines
`~<expr>` as matching any token that is not the start of the expression.
cmd/railroad/main.go:130 already reads Term.Negation, so Term.String()
was the one consumer of the field that dropped it.

SubExpression.String() (ebnf/ebnf.go:92) emitted `(?="a")`, while the
printer it is meant to mirror emits `(?= "a")` with a space after the
assertion (ebnf.go:147 and ebnf.go:149).

Neither showed up in TestEBNF because the ebnf package's own grammar uses
no negation and no lookahead group.

Term.String() now prefixes "~" when Negation is set, and
SubExpression.String() emits the space. The new table-driven test covers
both, including verbatim the Parser.String() output that TestEBNF_Other
asserts in the root package.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant