feat: support array format for create_index columns#1008
Open
jamesvillarrubia wants to merge 1 commit intoxataio:mainfrom
Open
feat: support array format for create_index columns#1008jamesvillarrubia wants to merge 1 commit intoxataio:mainfrom
jamesvillarrubia wants to merge 1 commit intoxataio:mainfrom
Conversation
9499bfd to
f8302c6
Compare
f8302c6 to
3bf9888
Compare
afc6df7 to
a62ed66
Compare
a62ed66 to
85a523e
Compare
85a523e to
1b5cf6e
Compare
1b5cf6e to
b087cb0
Compare
Add support for array format in create_index.columns while maintaining
full backward compatibility with existing map format.
## Problem
Multi-column indexes need predictable column ordering for query planner
optimization. The map format (v0.10.0+) doesn't preserve order, causing
non-deterministic index creation where columns could appear in any order.
## Solution
Support both formats in a single 'columns' field:
Array format (preferred):
columns:
- name: user_id
- name: created_at
sort: DESC
Map format (legacy, still works):
columns:
user_id: {}
created_at:
sort: DESC
## Implementation
- Custom UnmarshalJSON/UnmarshalYAML (~160 lines) detect and convert formats
- OpCreateIndex stores columns as []IndexColumn internally
- Execution pipeline uses ordered array (no map conversion)
- YAML map format preserves key insertion order
- Deprecation warnings logged for multi-column/partial indexes using maps
## Backward Compatibility
All existing migrations work unchanged:
- Single-column map format: no warnings
- Multi-column map format: deprecation warning logged
- Array format: preferred for all cases
No breaking changes. Users migrate at their own pace.
b087cb0 to
887cb36
Compare
Collaborator
|
Hi @jamesvillarrubia, thanks for the PR. It looks as though the use of We have two choices for how to resolve #1001:
IMO it would be better to make a clean breaking change to the migration format now (and have the automated support for rewriting migration files via |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Multi-column indexes require predictable column ordering for optimal query planner performance. The current
OpCreateIndeximplementation stores columns in amap[string]IndexField, which has non-deterministic iteration order in Go. This causes index columns to be created in random order rather than the user-specified order.Example:
May generate
CREATE INDEX ... (deleted_at, product_id, is_active)instead of the intended order.Additionally, per-column settings (collation, sort order, nulls order, operator classes) were added in v0.10.0, but the map format makes it unclear which column should come first in multi-column indexes.
Solution
Support both array and map formats for the
columnsfield:Array Format (Recommended)
create_tableoperation formatMap Format (Backward Compatible)
Implementation
OpCreateIndex.Columnsinternally stores[]IndexColumn(array)UnmarshalJSON/UnmarshalYAMLmethods detect format and convert appropriatelyoneOfto validate both formatsDeprecation Strategy
Map format is deprecated only for:
Single-column indexes can continue using map format indefinitely.
Warning message:
Changes
types.goOpCreateIndex.Columnsfrom map to[]IndexColumnop_create_index_unmarshal.goop_create_index_format.godbactions.goschema.jsononeOfvalidation for both formatscreate_index.gocreate_index.mdxwith examplesTest Coverage
Unit Tests (
op_create_index_execution_test.go):Integration Tests (
op_create_index_test.go):pg_indexesto validate column orderFormat Tests (
op_create_index_format_test.go):Example
Testing
✅ All 50+ existing
create_indextests pass✅ New execution test catches column ordering bugs
✅ Integration test validates real PostgreSQL indexes
✅ Format tests verify both JSON and YAML parsing
✅ Backward compatibility with map format maintained
✅ No linter errors
Migration Path
Existing migrations continue to work unchanged. Users can adopt array format gradually:
Before:
After: