fix(query): emit fill_with SQL expressions verbatim

[owjs3901]

Summary

add_column.fill_with silently corrupted user-supplied SQL expressions. The migration reported success while writing wrong data ??or no data at all.

fill_with is a raw SQL expression slot: whatever the user wrote is spliced into the emitted UPDATE / INSERT ... SELECT via Expr::cust. But it was being run through convert_default_for_backend, which exists to normalise a column DEFAULT (a single literal or function call it is free to canonicalise). The two contracts are incompatible.

Reproduction

A scratch project with an add_column action carrying a backfill expression, driven through the real CLI (vespertide log). Identical command, identical input, HEAD (e8a091a) vs this branch:

Input (migrations/0002_backfill.vespertide.json):

{ "type": "add_column", "table": "subscription",
  "column": { "name": "metric", "nullable": false,
              "type": { "kind": "enum", "name": "billing_metric",
                        "values": ["MONTHLY_QUOTA", "SEAT"] } },
  "fill_with": "(CASE WHEN plan_key::text = 'API' THEN 'MONTHLY_QUOTA' ELSE 'SEAT' END)::billing_metric" }

Before

-- postgres
UPDATE "subscription" SET "metric" = (CASE WHEN plan_key::text = 'api' then 'monthly_quota' else 'seat' end)::billing_metric

-- mysql
UPDATE `subscription` SET `metric` = CAST((CASE WHEN plan_key AS CHAR)

-- sqlite
INSERT INTO "subscription_temp" (...) SELECT "id", "plan_key", "device_os", "device_family", (CASE WHEN plan_key AS "metric" FROM "subscription"

On PostgreSQL the three uppercase literals are lower-cased, so plan_key::text = 'api' never matches (silent no-op backfill) and 'monthly_quota' is not a valid billing_metric label, so the cast fails. On MySQL/SQLite the statement is truncated outright.

The second reported case pinpoints the cause ??WINDOWS sits before the first cast operator and survives, while ELSE / END after it do not:

-- postgres, before
UPDATE "subscription" SET "os_tags" = CASE WHEN device_os = 'win' THEN json_build_array('WINDOWS', device_family::text) else '[]'::json end

After

-- postgres
UPDATE "subscription" SET "metric" = (CASE WHEN plan_key::text = 'API' THEN 'MONTHLY_QUOTA' ELSE 'SEAT' END)::billing_metric
UPDATE "subscription" SET "os_tags" = CASE WHEN device_os = 'win' THEN json_build_array('WINDOWS', device_family::text) ELSE '[]'::json END

-- mysql
UPDATE `subscription` SET `metric` = (CASE WHEN plan_key::text = 'API' THEN 'MONTHLY_QUOTA' ELSE 'SEAT' END)::billing_metric

-- sqlite
INSERT INTO "subscription_temp" (...) SELECT ..., (CASE WHEN plan_key::text = 'API' THEN 'MONTHLY_QUOTA' ELSE 'SEAT' END)::billing_metric AS "metric" FROM "subscription"

Byte-for-byte on every backend.

Root cause

crates/vespertide-query/src/sql/add_column.rs fed fill_with to convert_default_for_backend at both fill sites ??L80 (SQLite temp-table rebuild) and L135 (the backfill UPDATE). crates/vespertide-query/src/sql/modify_column_nullable.rs L37 did the same.

Inside crates/vespertide-query/src/sql/helpers.rs:

• convert_default_for_backend (L227) ??parse_pg_type_cast (L265) ??convert_type_cast (L328).
• parse_pg_type_cast's non-quoted branch (L298) used split_once("::"), i.e. the first occurrence, and L300 applied to_lowercase() to everything that followed.
• convert_type_cast (L331) re-joined value + :: + the lower-cased tail on PostgreSQL, so the mangled body became the emitted SQL. On MySQL/SQLite the tail was discarded entirely (CAST(value AS TYPE) / bare value), which is where the truncation came from.

Both properties of the split were wrong: it picked the first operator instead of the last, and it was blind to :: occurring inside a single-quoted literal or inside parentheses.

What changed

1. fill_with is never rewritten ??new crates/vespertide-query/src/sql/fill_with.rs:

convert_fill_with_for_backend(fill, backend)

• PostgreSQL ??the dialect fill_with is authored in ??returns the value verbatim; convert_default_for_backend is not called at all.
• MySQL / SQLite only get a rewrite when the value is unambiguously atomic: a whole-string portable function spelling (NOW(), gen_random_uuid(), ??, or a single simple literal / identifier optionally carrying one trailing ::type cast. Whitespace, parentheses, commas, quotes or composite SQL keywords (CASE/WHEN/THEN/ELSE/END/?? ??verbatim passthrough.
• The keyword scan skips single-quoted content, so an enum label like 'not_started'::user_status is still recognised as a literal and still converts (it is data, not the NOT keyword).

Wired into add_column.rs (both fill sites) and modify_column_nullable.rs. Column DEFAULT handling is untouched ??convert_default_for_backend keeps its old behaviour for defaults.

2. parse_pg_type_cast fixed at the source (helpers.rs):

• New find_last_top_level_cast splits at the last top-level ::, skipping operators inside single-quoted literals ('' escape aware) and inside parentheses. An unterminated literal returns None rather than guessing. The scan is driven by bytes().enumerate() so the cursor is monotonic by construction — no hand-rolled index arithmetic that a mutant could stall into an infinite loop. Toggling in_quote on every ' handles the '' escape for free, so no separate escape branch (or quoted_literal_end helper) is needed.
• The two special-cased branches collapse into one path, so the quoted-prefix case ('[]'::json, 'it''s'::text, Unicode literals) is preserved ??all its existing tests and the two proptests still pass unchanged.
• Only the type name is lower-cased; the value is returned byte-for-byte.
• convert_default_for_backend now recurses through cast chains, so 'x'::text::json nests correctly (CAST(CAST('x' AS CHAR) AS JSON) on MySQL, 'x' on SQLite) instead of leaving a stray ::text behind ??required to avoid a regression from first-split to last-split.

3. Other backfill-carrying actions

• modify_column_default.backfill was already interpolated verbatim (modify_column_default.rs L85). No change needed; locked with a regression test so this defect cannot be copied onto that path.
• modify_column_type.fill_with is a BTreeMap<removed_value, replacement> of enum values emitted through Expr::val ??a value map, not a raw SQL slot ??so it never went through convert_default_for_backend and is out of scope here (it is the subject of the sibling quoting PR).

Deliberate behaviour changes

• PostgreSQL fill_with is now fully verbatim. fill_with: "NOW()" stays NOW() instead of becoming CURRENT_TIMESTAMP. Both are valid PostgreSQL and semantically identical; test_fill_with_now_converted_to_current_timestamp and its postgres_fill_now snapshot were updated. MySQL/SQLite still get CURRENT_TIMESTAMP, because NOW() is not a SQLite function and that rewrite is a whole-string match that cannot touch a fragment of a larger expression.
• MySQL/SQLite now emit a composite fill_with verbatim, PostgreSQL syntax included. If the user writes PG-only SQL and targets MySQL, the migration now fails loudly at parse time instead of silently writing wrong data ??strictly better than the previous truncated-but-executable statement.

Test evidence

cargo test --workspace --all-features   -> exit 0, 4402 passed, 0 failed, 3 documented #[ignore]
cargo clippy --workspace --all-targets --all-features -- -D warnings -> exit 0
cargo fmt --all --check                 -> exit 0
sh scripts/check-line-budget.sh         -> All tracked Rust files are within budget
cargo mutants --in-diff git.diff        -> 56 mutants: 53 caught, 3 unviable, 0 missed, 0 timeout

New tests:

• add_column.rs ??three verbatim-survival regression tests 횞 3 backends (9 snapshots), one per reported case:
  • fill_with_enum_cast_case_expression_is_verbatim ??text-cast column compared to uppercase API, returning MONTHLY_QUOTA / SEAT, wrapped in parens and cast to an enum type.
  • fill_with_json_array_case_expression_is_verbatim ??JSON array containing uppercase WINDOWS plus a text-cast column, with an ELSE branch casting '[]' to json.
  • fill_with_cast_operator_inside_quotes_is_verbatim ??comparison literal 'legacy::v1' contains a cast operator inside single quotes, followed by a trailing ::integer.
• fill_with.rs ??33 unit cases covering the verbatim rule, the PostgreSQL passthrough, cross-backend conversion of simple literals, atom classification, and the quote-aware keyword scan.
• sql/tests/helpers.rs ??parse_pg_type_cast last-top-level-split cases (quoted operator, cast chain, parenthesised cast, escaped quote), non-top-level rejection, empty-side rejection, and cast-chain conversion across all 3 backends.
• modify_column_default.rs ??backfill-expression verbatim regression 횞 3 backends.

Every pre-existing convert_default_for_backend / parse_pg_type_cast DEFAULT test is green and unmodified, including both Unicode proptests.

Scope

Confined to crates/vespertide-query/src/sql/. schemas/migration.schema.json and the MigrationAction enum are untouched, and modify_column_type is untouched, so this stays independently reviewable alongside the two sibling PRs (modify_column_type.fill_with double-quoting, and the data_migration action).

One release-process file is added: .changepacks/changepack_log_Qp7rvN2xKdLm9aBcT4wZs.json (Minor bump for vespertide-query), per the documented pre-merge changepack step. The random suffix keeps it conflict-free with the sibling PRs.

Minor rather than Patch because CI derives the cargo-semver-checks release-type only from Major/Minor — a Patch descriptor leaves it empty, so the gate falls back to the (still un-bumped) Cargo.toml version and trips over the public-API removals already queued from #181. On a 0.x crate Minor is also the correct tier for the PostgreSQL fill_with behaviour change described above.

Follow-up commit in this PR

The first CI run surfaced two problems, both fixed in abc0548:

• cargo-mutants flagged the byte scanner's i += 1 / i += 2 / quote + end cursor advances: mutating them to -= / *= stalls the scan into an infinite loop (5 TIMEOUTs, plus one genuine MISSED on i *= 2). Rather than adding a mutants.toml exclusion — the established route for this class, see E18 — the scanners were rewritten on top of bytes().enumerate(), which removes the whole non-termination mutant class and the MISSED one along with it. quoted_literal_end disappeared entirely; contains_composite_keyword lost its quote-skipping loop by dropping the short, label-like keywords (and, or, not, in, like) so a plain whole-word split keeps 'not_started'::user_status and 'in progress' convertible.
• A stray UTF-8 BOM had been written into add_column.rs, which made cargo-mutants skip that file wholesale (Diff content doesn't match source file). Removed; the file is now covered by the mutation gate.

Adjacent issue found, not fixed here

normalize_enum_default (helpers.rs) quotes a value when the column is an enum and needs_quoting returns true. needs_quoting bails on parentheses, so every expression in this PR is safe ??but a paren-less CASE ... END fill on an enum column would still be wrapped in quotes and stored as a literal string. That is a quoting bug, i.e. the sibling PR's theme, so it is flagged here rather than fixed, to keep the three PRs independent.

[github-actions]

Changepacks

vespertide@0.2.1 → 0.2.2 - crates/vespertide/Cargo.toml

Patch

• Auto-update: depends on 'vespertide-core' via a local workspace dependency

vespertide-cli@0.2.1 → 0.3.0 - crates/vespertide-cli/Cargo.toml

Minor

• Prisma ORM exporter 추가

vespertide-config@0.2.1 → 0.3.0 - crates/vespertide-config/Cargo.toml

Minor

• 성능 최적화 웨이브: 미사용 public API 정리(NameCase::is_*, EnumValues::variant_names/to_sql_values, sql::helpers 일부, find_primary_key_removals 시그니처)와 Orm::Prisma 변형 추가로 0.x 기준 breaking

vespertide-core@0.2.1 → 0.3.0 - crates/vespertide-core/Cargo.toml

Minor

• 성능 최적화 웨이브: 미사용 public API 정리(NameCase::is_*, EnumValues::variant_names/to_sql_values, sql::helpers 일부, find_primary_key_removals 시그니처)와 Orm::Prisma 변형 추가로 0.x 기준 breaking

vespertide-exporter@0.2.1 → 0.3.0 - crates/vespertide-exporter/Cargo.toml

Minor

• 성능 최적화 웨이브: 미사용 public API 정리(NameCase::is_*, EnumValues::variant_names/to_sql_values, sql::helpers 일부, find_primary_key_removals 시그니처)와 Orm::Prisma 변형 추가로 0.x 기준 breaking
• Prisma ORM exporter 추가

vespertide-loader@0.2.1 → 0.2.2 - crates/vespertide-loader/Cargo.toml

Patch

• Auto-update: depends on 'vespertide-config' via a local workspace dependency

vespertide-lsp@0.2.1 → 0.2.2 - crates/vespertide-lsp/Cargo.toml

Patch

• Auto-update: depends on 'vespertide-config' via a local workspace dependency

vespertide-macro@0.2.1 → 0.2.2 - crates/vespertide-macro/Cargo.toml

Patch

• Auto-update: depends on 'vespertide-config' via a local workspace dependency

vespertide-naming@0.2.1 → 0.3.0 - crates/vespertide-naming/Cargo.toml

Minor

• Prisma ORM exporter 추가

vespertide-planner@0.2.1 → 0.3.0 - crates/vespertide-planner/Cargo.toml

Minor

• 성능 최적화 웨이브: 미사용 public API 정리(NameCase::is_*, EnumValues::variant_names/to_sql_values, sql::helpers 일부, find_primary_key_removals 시그니처)와 Orm::Prisma 변형 추가로 0.x 기준 breaking

vespertide-query@0.2.1 → 0.3.0 - crates/vespertide-query/Cargo.toml

Minor

• add_column.fill_with가 사용자 SQL 표현식을 훼손하던 버그 수정: fill_with는 raw SQL 표현식 슬롯이므로 DEFAULT 정규화(convert_default_for_backend)를 태우지 않고 그대로 방출한다. parse_pg_type_cast도 첫 번째 :: 대신 따옴표/괄호를 건너뛴 마지막 top-level :: 에서 분리하도록 수정. 동작 변경: PostgreSQL은 fill_with를 항상 원문 그대로 방출하므로 NOW()가 더 이상 CURRENT_TIMESTAMP로 치환되지 않는다(MySQL/SQLite는 유지)
• 성능 최적화 웨이브: 미사용 public API 정리(NameCase::is_*, EnumValues::variant_names/to_sql_values, sql::helpers 일부, find_primary_key_removals 시그니처)와 Orm::Prisma 변형 추가로 0.x 기준 breaking