Skip to content

Prevent resource restoration when inactivity detection is disabled - #1532

Open
jjblack604 wants to merge 1 commit into
gnocchixyz:masterfrom
jjblack604:fix/metric-inactive-resource-restoration
Open

jjblack604 wants to merge 1 commit into
gnocchixyz:masterfrom
jjblack604:fix/metric-inactive-resource-restoration

Conversation

@jjblack604

@jjblack604 jjblack604 commented Oct 19, 2026 •

Copy link
Copy Markdown

Summary

Prevent Gnocchi from automatically restoring ended resources when [metricd] metric_inactive_after = 0.

Ceilometer polling and notification processing are asynchronous, so measurements may arrive after a resource has been explicitly marked as ended. Currently, Gnocchi can interpret these measurements as evidence that the resource is active again and clear its ended_at timestamp.

When automatic inactivity detection is disabled, incoming measurements should not override explicit resource lifecycle updates.

Changes

  • Only allow automatic resource restoration when metric_inactive_after > 0.
  • Preserve existing restoration behaviour when inactivity detection is enabled.
  • Update existing resource recovery tests to explicitly enable inactivity detection.
  • Add regression coverage ensuring newer measurements do not restore ended resources when inactivity detection is disabled.

Testing

  • tox -e pep8 — PASS
  • git diff --check — PASS
  • Targeted storage metadata tests — 5/5 PASS
    • Resource restoration with inactivity detection enabled.
    • No restoration for older measurements.
    • No restoration when inactivity detection is disabled.
    • Existing metadata update and truncation tests.

Full py311-postgresql-file suite on Fedora 44 / PostgreSQL 18:

  • 1,116 passed
  • 52 skipped
  • 2 expected failures
  • 6 failures

The six failures appear unrelated to this change and involve PostgreSQL 18 foreign-key constraint handling during archive-policy and resource-type deletion.

Fixes #1527

Backport

This fix is also applicable to stable/4.7, as discussed in #1527.

@LukeRepko

Copy link
Copy Markdown

+1 from another production user. We run Gnocchi 4.7 for Rackspace Flex with metric_inactive_after unset, and we hit exactly what #1527 describes. Late Ceilometer polling samples clear ended_at on instances and other resources after Nova deletes them, so deleted resources show up as active again. We rely on Gnocchi for accurate resource lifecycle data, and these restores make it wrong. We'd very much like to see this merged and backported to stable/4.7.

One suggestion: this changes the behavior of a default configuration, so a release note would help operators who currently rely on restores with metric_inactive_after = 0.

Related: #1539 adds [storage] resource_restore_grace_period for the case this PR leaves open, deployments with metric_inactive_after > 0, where restoring stays on and late samples can still restore resources that were explicitly ended. The two don't conflict in the code. I'll rebase mine on top of this one if it lands first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Prevent resource restoration when metric_inactive_after is disabled

2 participants