Skip to content

replace celery State with lightweight uuid to name cache#21

Closed
tulioz wants to merge 1 commit intoevent-watcher-reconnectfrom
event-watcher-lightweight-cache
Closed

replace celery State with lightweight uuid to name cache#21
tulioz wants to merge 1 commit intoevent-watcher-reconnectfrom
event-watcher-lightweight-cache

Conversation

@tulioz
Copy link
Copy Markdown
Contributor

@tulioz tulioz commented Apr 7, 2026

celerymon only needs task name from the State object, this replaces the full celery State (10k Task objects with 30+ fields, Worker objects, WeakSets, taskheap, mutex) with a simple OrderedDict mapping uuid to task name

@aviator-app
Copy link
Copy Markdown

aviator-app bot commented Apr 7, 2026

Current Aviator status

Aviator will automatically update this comment as the status of the PR changes.
Comment /aviator refresh to force Aviator to re-examine your PR (or learn about other /aviator commands).

This PR was merged using Aviator.

Stack

  1. #20 add reconnection logic to event watcher thread (merged)
  2. 👉 #21 replace celery State with lightweight uuid to name cache 👈 (this pr)

See the real-time status of this PR on the Aviator webapp.
Use the Aviator Chrome Extension to see the status of your PR within GitHub.

@tulioz
Copy link
Copy Markdown
Contributor Author

tulioz commented Apr 7, 2026

/gemini review

@tulioz tulioz force-pushed the event-watcher-lightweight-cache branch from b114a2a to 9815901 Compare April 7, 2026 22:46
@tulioz tulioz force-pushed the event-watcher-reconnect branch from 0557af8 to 372b1eb Compare April 7, 2026 22:46
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces Celery's internal state tracking with a custom OrderedDict-based cache for task names in the EventWatcher class. Feedback suggests refactoring the hardcoded cache limit into a class-level constant and using the move_to_end method for more idiomatic cache management.

Comment thread celerymon/event_watcher.py Outdated
Comment thread celerymon/event_watcher.py
@tulioz tulioz force-pushed the event-watcher-lightweight-cache branch from 9815901 to 1e30667 Compare April 7, 2026 23:11
@tulioz tulioz force-pushed the event-watcher-reconnect branch from 372b1eb to 1bee059 Compare April 7, 2026 23:11
@tulioz
Copy link
Copy Markdown
Contributor Author

tulioz commented Apr 7, 2026

/gemini review

@tulioz tulioz force-pushed the event-watcher-lightweight-cache branch from 1e30667 to f248e8a Compare April 7, 2026 23:20
@tulioz tulioz force-pushed the event-watcher-reconnect branch from 1bee059 to d181f72 Compare April 7, 2026 23:20
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces the dependency on Celery's internal state management with a custom LRU cache using an OrderedDict to map task UUIDs to names. This change simplifies the EventWatcher initialization and introduces a configurable cache limit to manage memory usage. Review feedback suggests optimizing the cache update logic by using move_to_end to properly refresh the LRU position for all task events and ensuring that the retrieved task name is never null to maintain compatibility with Prometheus labels.

Comment thread celerymon/event_watcher.py
Comment thread celerymon/event_watcher.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants