Skip to content

[codex] Optimize D1 full-field lookups#45

Merged
mandel59 merged 7 commits into
mainfrom
codex/optimize-d1-full-field-lookups
May 3, 2026
Merged

[codex] Optimize D1 full-field lookups#45
mandel59 merged 7 commits into
mainfrom
codex/optimize-d1-full-field-lookups

Conversation

@mandel59

@mandel59 mandel59 commented May 3, 2026

Copy link
Copy Markdown
Owner

Summary

  • Optimize full-field D1 lookups so the generated SQL can use indexes, and add the required D1 schema indexes.
  • Fix ids_similar to match the current IDS operators (, ⿿) while preserving legacy operator compatibility.
  • Promote the D1 Worker bindings to the 20260504-d1-indexes blue/green databases and clean up obsolete Cloudflare D1 databases.
  • Add shared backend API conformance coverage plus non-D1 regression tests for ids_similar.
  • Split database cache warmup into separate jobs for validation and release workflows so failed later steps do not force DB artifacts to be rebuilt every run.

Related Issues

Release Prep

  • Pending changeset bumps @mandel59/mojidata, @mandel59/mojidata-api-core, and @mandel59/mojidata-cli at patch level.
  • Version and changelog generation are intentionally left to the Release workflow after this feature PR is merged to main.

Validation

  • corepack yarn changeset status --since=main
  • corepack yarn ci:test
  • corepack yarn ci:pack
  • corepack yarn workspace @mandel59/mojidata-api test
  • corepack yarn workspace @mandel59/mojidata-api-sqljs test
  • corepack yarn workspace @mandel59/mojidata-api-better-sqlite3 test
  • corepack yarn workspace @mandel59/mojidata-api-node-sqlite test
  • corepack yarn workspace @mandel59/mojidata-api-sqlite-wasm test
  • corepack yarn workspace @mandel59/mojidata-api-d1 test
  • corepack yarn workspace @mandel59/mojidata-cli prepare
  • corepack yarn mojidata-api:d1:check
  • corepack yarn mojidata-api:d1:smoke -- --base-url https://mojidata-api-d1.mandel59.workers.dev
  • ruby -e "require 'yaml'; YAML.load_file('.github/workflows/validate.yml'); puts 'validate.yml OK'"
  • ruby -e "require 'yaml'; YAML.load_file('.github/workflows/release.yml'); puts 'release.yml OK'"
  • corepack yarn build:idsdb-stack
  • corepack yarn workspace @mandel59/idsdb-fts5 run prepare

mandel59 added 7 commits May 4, 2026 01:23
Rewrite mojidata full-field SQL predicates so D1 can use existing IVS, SVS, TGHB, MJIH, and KDPV indexes, and add missing IDS and MJIH phonetic indexes to generated databases.

Related: #41

Generated-by: Codex/GPT-5.4
Switch the standalone D1 Worker bindings to the 20260504-d1-indexes release databases after import and smoke validation.

Related: #41

Generated-by: Codex/GPT-5.4
Include current IDS mirror and rotation operators in ids_similar while retaining legacy aliases, and update SQL-shape tests for API core and D1.

Related: #42

Generated-by: Codex/GPT-5.4
Cover current IDS mirror and rotation operators across sql.js, better-sqlite3, node:sqlite, and sqlite-wasm API DB paths.

Related: #42

Generated-by: Codex/GPT-5.4
Introduce a shared HTTP API conformance helper and run it against the default Node, sql.js, better-sqlite3, node:sqlite, sqlite-wasm, and D1 app paths.

Related: #43

Generated-by: Codex/GPT-5.4
Add a dedicated prepare-db-cache job that materializes mojidata, idsdb, and idsdb-fts5 artifacts before validation so test failures do not prevent saving fresh DB caches.

Related: #44

Generated-by: Codex/GPT-5.4
Add the same database cache warmup job to the release workflow so release validation and publishing steps can reuse prepared mojidata and IDS artifacts even when later release steps fail.

Related: #44

Generated-by: Codex/GPT-5.4
@mandel59 mandel59 marked this pull request as ready for review May 3, 2026 17:30
@mandel59 mandel59 merged commit 37e81e0 into main May 3, 2026
2 checks passed
@github-actions github-actions Bot mentioned this pull request May 3, 2026
mandel59 added a commit that referenced this pull request May 3, 2026
Drop Linux optional dependency cache files that were generated by the release workflow from the Changesets release PR, keeping the release diff focused on version and changelog updates.

Related: #45

Generated-by: Codex/GPT-5.4
mandel59 added a commit that referenced this pull request May 3, 2026
Keep the Linux optional dependency archives generated by the release workflow because this repository tracks .yarn/cache for zero-install consistency across development and CI platforms.

Related: #45

Generated-by: Codex/GPT-5.4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant