OSAC-1338: Create upgrade playbook and migration framework#461
Conversation
Add upgrade.yaml playbook and upgrade.sh script to handle migrations from previous Enclave versions to the current version. Framework structure: - upgrade.sh: Entry point script following sync.sh patterns with getValue(), step_done(), and timestamped logging - playbooks/upgrade.yaml: Main orchestration playbook that loads configuration and runs migrations - playbooks/tasks/migrations.yaml: Migration coordinator that conditionally includes migrations based on enabled_plugins - playbooks/tasks/migrations/foundation_plugin_catalog_source.yaml: Reusable migration task for foundation plugins Initial migrations: - ODF operator subscription: core catalog → redhat-operator-index-odf - LVMS operator subscription: core catalog → redhat-operator-index-lvms Background: Previous Enclave versions mirrored foundation plugin operators as part of the core catalog source. The plugin framework now creates dedicated catalog sources per plugin (introduced in #420, #456, #457). This migration ensures existing deployments are updated to use the new plugin-specific catalog sources. The migration is idempotent and only runs when the respective plugin is enabled in the deployment configuration. Assisted-by: Claude Code <noreply@anthropic.com>
|
Warning Review limit reached
More reviews will be available in 50 minutes and 19 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (3)
WalkthroughThis PR introduces a comprehensive enclave upgrade workflow combining Bash orchestration, Ansible playbook dispatch, plugin discovery, and operator subscription migration. The upgrade.sh entry point drives ansible-playbook execution with logged output, while migrations.yaml recursively discovers foundation plugins and patches their operator Subscriptions' catalog sources from legacy to mirror-specific values on disconnected deployments only. ChangesUpgrade Workflow and Subscription Migrations
Sequence Diagram(s)sequenceDiagram
participant User
participant upgrade.sh
participant config/global.yaml
participant playbooks/upgrade.yaml
User->>upgrade.sh: invoke
upgrade.sh->>config/global.yaml: getValue workingDir
config/global.yaml-->>upgrade.sh: workingDir path
upgrade.sh->>upgrade.sh: create timestamped log dir/file
upgrade.sh->>upgrade.sh: write start timestamp
upgrade.sh->>playbooks/upgrade.yaml: ansible-playbook with ANSIBLE_LOG_PATH
playbooks/upgrade.yaml-->>upgrade.sh: execution complete
upgrade.sh->>upgrade.sh: write "Done" completion timestamp
sequenceDiagram
participant MigrationTask as migrations.yaml
participant SubMigration as operator_catalog_source.yaml
participant KubeAPI as Kubernetes API
MigrationTask->>SubMigration: include task with operator_name,<br/>legacy_source, new_source
SubMigration->>KubeAPI: query Subscription by name/namespace
KubeAPI-->>SubMigration: Subscription resource or not found
alt Subscription found and source needs update
SubMigration->>KubeAPI: PATCH spec.source = new_source
KubeAPI-->>SubMigration: patch applied (retry until success)
SubMigration->>SubMigration: debug: migrated from legacy to new
else Already on new_source or source undefined
SubMigration->>SubMigration: debug: already migrated or skip
else Subscription not found
SubMigration->>SubMigration: debug: subscription not found
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 10 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (10 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The catalog source name is constructed as {catalog_mirror}-{plugin_name},
where catalog_mirror can be either mirror_rh_operator_catalog
(mirror-redhat-operators) or mirror_certified_rh_operator_catalog
(mirror-certified-operators) depending on the plugin's catalog type.
This matches the catalog source naming in deploy_plugin.yaml:
- catalog_mirror: mirror-redhat-operators (default)
- catalog_mirror: mirror-certified-operators (when plugin.catalog == certified)
Both ODF and LVMS use the default redhat catalog, so they use
mirror-redhat-operators as the catalog_mirror prefix.
Assisted-by: Claude Code <noreply@anthropic.com>
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@playbooks/tasks/migrations.yaml`:
- Line 18: The when condition "'odf' in enabled_plugins" can fail if
enabled_plugins is undefined; update the conditional(s) in
playbooks/tasks/migrations.yaml (the when expressions referencing
enabled_plugins) to safely handle a missing variable by using the default filter
(e.g., check membership against enabled_plugins | default([])) so the play
remains rerunnable for older or partial configs; apply the same change to the
other occurrence referenced in the comment.
In `@playbooks/tasks/migrations/foundation_plugin_catalog_source.yaml`:
- Around line 42-45: The current when clause uses
"r_plugin_subscription.resources[0].spec.source != catalog_mirror + "-" +
plugin_name" which causes the task to run for any non-matching/custom source;
change this to an equality check so the play only runs when the existing
subscription source exactly matches the legacy catalog (i.e. use
"r_plugin_subscription.resources[0].spec.source == catalog_mirror + "-" +
plugin_name"); make the same change for the other occurrence covering lines
46-56 so the patch is restricted to the legacy catalog source only.
In `@upgrade.sh`:
- Around line 11-12: The shell invocations in upgrade.sh are using unquoted
variable expansions (e.g. python -c '...'< $global_vars | jq -r $1 and similar
on lines mentioned), which can break on paths with spaces or glob characters;
update all occurrences to quote the expansions (use "$global_vars", and pass the
jq argument as "$1" or "${1}" where used, and similarly quote any other
variables on lines 16-17, 25 and 28-29) so file redirections and jq arguments
are protected from word-splitting and globbing.
- Around line 1-3: The script currently enables errexit and pipefail but is
missing nounset; update the top of upgrade.sh so it uses nounset (set -u) to
avoid silent failures from unset variables — replace the separate "set -o
pipefail" and duplicate "set -e" pattern with a single robust line like "set
-euo pipefail" after the shebang, ensuring you remove the redundant "set -e" so
the script enforces errexit, nounset and pipefail consistently.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 30116a0d-00a2-4f6f-80df-f610be885923
📒 Files selected for processing (4)
playbooks/tasks/migrations.yamlplaybooks/tasks/migrations/foundation_plugin_catalog_source.yamlplaybooks/upgrade.yamlupgrade.sh
1. upgrade.sh: Add set -u (nounset) and quote all variable expansions - Changed from 'set -e' + 'set -o pipefail' to 'set -euo pipefail' - Quoted all variable expansions to handle paths with spaces 2. migrations.yaml: Add default([]) to enabled_plugins checks - Prevents failure if enabled_plugins is undefined - Makes the playbook rerunnable for older/partial configs 3. foundation_plugin_catalog_source.yaml: Add safety checks - Verify spec.source is defined before comparison - Only migrate subscriptions from known core/legacy catalogs - Pattern matches: redhat-operators, cs-*, or catalog_mirror prefix - Prevents accidental overwrite of custom catalog sources Assisted-by: Claude Code <noreply@anthropic.com>
|
✨ Claude Code: Addressed CodeRabbit review comments in ffd3cd4 Fixed:
These changes make the upgrade framework more robust and prevent accidental overwrite of custom catalog sources. |
Foundation plugin catalog source migrations only apply to disconnected deployments where catalog sources are mirrored locally. Connected deployments use upstream Red Hat catalogs directly and don't need migration. Added disconnected check to both ODF and LVMS migration tasks. Assisted-by: Claude Code <noreply@anthropic.com>
The legacy catalog source is exactly catalog_mirror (e.g., "mirror-redhat-operators"), not a pattern. Changed from regex pattern matching to exact equality check. This ensures we only migrate subscriptions that are using the exact legacy core catalog source, not custom or other catalog sources. Assisted-by: Claude Code <noreply@anthropic.com>
|
✨ Claude Code: Additional updates in f552922 and 3c22ab4 Changes:
The migration now has proper guards:
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai please review again and approve, all your comments have been addressed. |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Fetch current state of all modified files from the PR
echo "=== upgrade.sh ==="
cat upgrade.sh
echo ""
echo "=== playbooks/upgrade.yaml ==="
cat playbooks/upgrade.yaml
echo ""
echo "=== playbooks/tasks/migrations.yaml ==="
cat playbooks/tasks/migrations.yaml
echo ""
echo "=== playbooks/tasks/migrations/foundation_plugin_catalog_source.yaml ==="
cat playbooks/tasks/migrations/foundation_plugin_catalog_source.yamlLength of output: 5935
Risk level: LOW — all patches are non-destructive and well-guarded.
One minor non-blocking nit: All previously flagged issues have been addressed. Approving the PR. [approve] |
Replace hardcoded ODF and LVMS migration blocks with dynamic discovery that automatically handles ALL foundation plugins. Changes: - Discover all foundation plugins using the same pattern as collect_core_plugin_operators.yaml - Loop over each foundation plugin's operators using subelements() - Automatically determine catalog_mirror based on plugin.catalog field: - redhat (default) → mirror_rh_operator_catalog - certified → mirror_certified_rh_operator_catalog - Skip operators with explicit operator.source defined Benefits: - Extensible: New foundation plugins automatically get migrated without code changes - Maintainable: Follows existing Enclave patterns for plugin discovery - Safe: Multiple guards (disconnected mode, enabled plugins, no custom source) - DRY: No hardcoded plugin names Example migration flow for ODF, LVMS, VAST-CSI: - odf/odf-operator: mirror-redhat-operators → mirror-redhat-operators-odf - lvms/lvms-operator: mirror-redhat-operators → mirror-redhat-operators-lvms - vast-csi/vast-csi-operator: mirror-certified-operators → mirror-certified-operators-vast-csi Assisted-by: Claude Code <noreply@anthropic.com>
|
✨ Claude Code: Generalized migration to handle ALL foundation plugins in d9a3337 Changes: Implementation:
Example flow (ODF, LVMS, VAST-CSI enabled):
Benefits:
|
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
playbooks/tasks/migrations/foundation_plugin_catalog_source.yaml (1)
51-61:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winDebug message logic incorrect for undefined or custom subscription sources (Risk: Minor – informational output only).
Impact: The else clause (lines 58-61) assumes migration succeeded, but migration is skipped when:
spec.sourceis undefined (line 47 guards migration, but line 59 accesses undefinedspec.source→ Jinja2 error)spec.sourceis a custom catalog (line 49:spec.source != catalog_mirror, migration skipped, but debug incorrectly reports "migrated from <custom> to ...")This misleads operators during troubleshooting and may error in edge cases.
🛡️ Proposed fix to handle all subscription source states
- name: Display {{ plugin_name | upper }} migration status ansible.builtin.debug: msg: >- {% if r_plugin_subscription.resources | length == 0 %} {{ plugin_name | upper }} subscription not found — no migration needed + {% elif r_plugin_subscription.resources[0].spec.source is not defined %} + {{ plugin_name | upper }} subscription spec.source not defined — skipped {% elif r_plugin_subscription.resources[0].spec.source == catalog_mirror + "-" + plugin_name %} {{ plugin_name | upper }} subscription already using plugin-specific catalog source — no migration needed + {% elif r_plugin_subscription.resources[0].spec.source == catalog_mirror %} + {{ plugin_name | upper }} subscription migrated from {{ catalog_mirror }} + to {{ catalog_mirror }}-{{ plugin_name }} {% else %} - {{ plugin_name | upper }} subscription migrated from {{ r_plugin_subscription.resources[0].spec.source }} - to {{ catalog_mirror }}-{{ plugin_name }} + {{ plugin_name | upper }} subscription uses custom source {{ r_plugin_subscription.resources[0].spec.source }} — skipped to preserve custom configuration {% endif %}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@playbooks/tasks/migrations/foundation_plugin_catalog_source.yaml` around lines 51 - 61, The debug block assumes r_plugin_subscription.resources[0].spec.source always exists and misreports custom sources; update the Jinja conditionals around r_plugin_subscription, r_plugin_subscription.resources[0].spec, and r_plugin_subscription.resources[0].spec.source to explicitly handle three cases: (1) no subscription resources -> keep "subscription not found — no migration needed" using r_plugin_subscription.resources | length == 0, (2) spec.source is undefined -> emit "subscription source undefined — migration skipped" referencing r_plugin_subscription.resources[0].spec, and (3) spec.source defined but not equal to catalog_mirror or plugin-specific -> emit "subscription uses custom source {{ r_plugin_subscription.resources[0].spec.source }} — migration skipped"; preserve the existing message only when spec.source equals catalog_mirror + "-" + plugin_name (already using plugin-specific catalog) or when migration actually occurred, using the symbols r_plugin_subscription, plugin_name, and catalog_mirror to locate and adjust the template logic.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@playbooks/tasks/migrations/foundation_plugin_catalog_source.yaml`:
- Around line 51-61: The debug block assumes
r_plugin_subscription.resources[0].spec.source always exists and misreports
custom sources; update the Jinja conditionals around r_plugin_subscription,
r_plugin_subscription.resources[0].spec, and
r_plugin_subscription.resources[0].spec.source to explicitly handle three cases:
(1) no subscription resources -> keep "subscription not found — no migration
needed" using r_plugin_subscription.resources | length == 0, (2) spec.source is
undefined -> emit "subscription source undefined — migration skipped"
referencing r_plugin_subscription.resources[0].spec, and (3) spec.source defined
but not equal to catalog_mirror or plugin-specific -> emit "subscription uses
custom source {{ r_plugin_subscription.resources[0].spec.source }} — migration
skipped"; preserve the existing message only when spec.source equals
catalog_mirror + "-" + plugin_name (already using plugin-specific catalog) or
when migration actually occurred, using the symbols r_plugin_subscription,
plugin_name, and catalog_mirror to locate and adjust the template logic.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: fd01a1e1-c249-40c0-aa8e-e8831ea9e8bc
📒 Files selected for processing (3)
playbooks/tasks/migrations.yamlplaybooks/tasks/migrations/foundation_plugin_catalog_source.yamlupgrade.sh
Catalog sources are named cs-{catalog_mirror}-{plugin_name}-v4-20, not
just {catalog_mirror}-{plugin_name}.
Legacy source: cs-mirror-redhat-operators-v4-20
New source: cs-mirror-redhat-operators-odf-v4-20
This matches the naming in deploy_plugin.yaml line 191:
default_operator_source: "cs-{{ catalog_mirror }}-v4-20"
Updated:
- New source pattern to include cs- prefix and -v4-20 suffix
- Legacy source check to exact match: cs-{catalog_mirror}-v4-20
- Documentation to reflect correct naming
Assisted-by: Claude Code <noreply@anthropic.com>
|
✨ Claude Code: Fixed catalog source naming in 397112b Correction: Catalog sources follow the pattern Examples:
Migration now correctly:
|
Rename foundation_plugin_catalog_source.yaml to operator_catalog_source.yaml
and remove all plugin-specific references to make it a truly generic task.
Changes:
1. Renamed file (git history preserved):
- foundation_plugin_catalog_source.yaml → operator_catalog_source.yaml
2. Made operator_catalog_source.yaml generic:
- Removed plugin-specific variable names (plugin_name, catalog_mirror)
- Added generic variables: legacy_source, new_source
- Changed variable names: r_plugin_* → r_operator_*
- Simplified header to reflect generic nature
3. Updated migrations.yaml to compute sources:
- Computes legacy_source and new_source before calling the task
- Passes pre-computed source strings to the generic task
- Keeps all plugin-specific logic (discovery, filtering, catalog
determination) in migrations.yaml
Benefits:
- Clear separation of concerns: plugin logic vs. generic patching
- operator_catalog_source.yaml is 100% reusable for any operator
subscription migration
- More explicit: legacy_source and new_source are clearer than
implicit string construction
- Better naming: file reflects what it does, not who calls it
Assisted-by: Claude Code <noreply@anthropic.com>
|
✨ Claude Code: Refactored migration task in f085edb Changes: What changed:
Benefits:
Git history preserved with |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@playbooks/tasks/migrations/operator_catalog_source.yaml`:
- Around line 46-56: The debug message currently prints "migrated" whenever
r_operator_subscription.resources[0].spec.source != new_source, which can
produce false positives when a patch was blocked; change the conditional logic
so the "migrated" message is only emitted if the subscription source actually
changed to new_source (i.e., current source == new_source) or if you have an
explicit successful-patch indicator (e.g., a result variable like
r_operator_patch.changed is true); add a final branch that reports "migration
not performed/patch blocked — current source is {{
r_operator_subscription.resources[0].spec.source }}" for cases where the source
is neither new_source nor migrated (for example custom sources or
legacy_source), referencing operator_name,
r_operator_subscription.resources[0].spec.source, new_source and the patch
result variable when present.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 33469899-22c7-4d1b-bb38-8ec574fb38a4
📒 Files selected for processing (2)
playbooks/tasks/migrations.yamlplaybooks/tasks/migrations/operator_catalog_source.yaml
Move foundation plugin catalog source migration logic from migrations.yaml into its own dedicated file for better organization. Changes: - Created foundation_plugin_catalog_sources.yaml with: - Plugin discovery (find, load, filter foundation plugins) - Catalog mirror determination (redhat vs certified) - Operator subscription migration loop - Simplified migrations.yaml to just: - Include foundation_plugin_catalog_sources.yaml Benefits: - migrations.yaml becomes a clean registry/index of all migrations - Each migration is self-contained in its own file - Easier to add new migrations (just create new file and include it) - Better organization as migrations grow File structure: playbooks/tasks/migrations/ ├── foundation_plugin_catalog_sources.yaml (specific migration) ├── operator_catalog_source.yaml (generic utility) └── (future migrations can be added here) Assisted-by: Claude Code <noreply@anthropic.com>
|
✨ Claude Code: Extracted migration to separate file in d4a18f6 Changes: New structure: migrations.yaml is now a clean registry: - name: Foundation plugin catalog source migration
ansible.builtin.include_tasks:
file: migrations/foundation_plugin_catalog_sources.yamlBenefits:
Perfect separation of concerns:
|
1. Move disconnected condition to migrations.yaml: - Apply condition at the include level in migrations.yaml - Remove redundant disconnected checks from foundation_plugin_catalog_sources.yaml - Cleaner control flow - condition in one place 2. Fix debug message logic in operator_catalog_source.yaml: - Only show "migrated" when r_operator_patch.changed is true - Add branch for custom catalog sources (not legacy_source) - Add final catch-all branch for other cases - Prevents false positives when patch was blocked Addresses CodeRabbit review comments. Assisted-by: Claude Code <noreply@anthropic.com>
|
✨ Claude Code: Addressed CodeRabbit review in 2df5bef Changes:
Debug message flow:
Addresses all outstanding CodeRabbit review comments. |
|
e2e connected will be fixed in #469 |
|
post merge reviews are welcome |
rporres
left a comment
There was a problem hiding this comment.
So this covers the need of amending operators catalog sources, that looks great.
I like a lot the idea of migrations.
I think migrations should be somehow bound to a from/destination version to avoid testing hell to confirm the need for them to be perfectly idempotent.
A few comments here and there, nothing very important. This is a great starting point
| set -euo pipefail | ||
|
|
||
| # Upgrade script for Red Hat Sovereign Enclave | ||
| # Run sync.sh BEFORE running this script to ensure content is synchronized |
There was a problem hiding this comment.
It would be great to have something that checks this...
| getValue(){ | ||
| python -c 'import sys, yaml, json; print(json.dumps(yaml.safe_load(sys.stdin)))' < "$global_vars" \ | ||
| | jq -r "$1" | ||
| } | ||
|
|
||
| step_done(){ | ||
| echo -e "\e[38;5;10m Done...\033[0m" | tee -a "${log}" | ||
| date | tee -a "${log}" | ||
| } |
There was a problem hiding this comment.
nit: can we avoid duplicating these from bootstrap.sh?
| # Background: Previous Enclave versions mirrored foundation plugin operators | ||
| # as part of the core catalog source (e.g., cs-mirror-redhat-operators-v4-20). | ||
| # The plugin framework now creates dedicated catalog sources per plugin (e.g., | ||
| # cs-mirror-redhat-operators-odf-v4-20 for ODF). |
There was a problem hiding this comment.
So this migration is to be applied if we're migrating from 0.1.0, right? It shouldn't be run at any other point although it looks idempotent, right?
Summary
This PR adds an upgrade framework to handle migrations from previous Enclave versions to the current version.
Changes
New files:
upgrade.sh- Entry point script followingsync.shpatternsplaybooks/upgrade.yaml- Main orchestration playbookplaybooks/tasks/migrations.yaml- Migration coordinatorplaybooks/tasks/migrations/foundation_plugin_catalog_source.yaml- Reusable migration taskInitial migrations:
redhat-operator-index-odfredhat-operator-index-lvmsBackground
Previous Enclave versions mirrored foundation plugin operators as part of the core catalog source. The plugin framework now creates dedicated catalog sources per plugin (introduced in #420, #456, #457). This migration ensures existing deployments are updated to use the new plugin-specific catalog sources.
Design
The upgrade framework is designed for extensibility:
upgrade.sh- Minimal wrapper that followssync.shpatterns:getValue()function to extract configstep_done()helper with color-coded output${workingDir}/logs/${DSTAMP}-upgradeplaybooks/upgrade.yaml- Clean orchestration:common/load-vars.yamltasks/migrations.yamlplaybooks/tasks/migrations.yaml- Migration registry:enabled_pluginsplaybooks/tasks/migrations/foundation_plugin_catalog_source.yaml- Reusable task:Testing
The migration is:
Usage
Future Work
Additional foundation plugin migrations can be added by:
include_tasksblock inplaybooks/tasks/migrations.yamlOther upgrade phases (operator upgrades, validation) can be added as additional playbooks imported from
upgrade.yaml.Links
🤖 Generated with Claude Code
Summary by CodeRabbit