Skip to content

feat(hermes): add Rocket.Chat alarms for incident escalation (opensre hermes watch) - #4643

Open
mdiniz97 wants to merge 5 commits into
Tracer-Cloud:mainfrom
mdiniz97:feat/rocketchat-hermes-alarms
Open

feat(hermes): add Rocket.Chat alarms for incident escalation (opensre hermes watch)#4643
mdiniz97 wants to merge 5 commits into
Tracer-Cloud:mainfrom
mdiniz97:feat/rocketchat-hermes-alarms

Conversation

@mdiniz97

@mdiniz97 mdiniz97 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Fixes #4215

Describe the changes you have made in this PR -

Adds Rocket.Chat as an alarm delivery provider for opensre hermes watch (live-tail Hermes error logs, classify incidents, escalate) — the Hermes counterpart to the watchdog Rocket.Chat alarms in #4217, reusing RocketChatAlarmDispatcher rather than reimplementing it.

  • integrations/hermes/sinks.pyTelegramSink.__init__ and make_telegram_sink hardcoded integrations.telegram.alarms.AlarmDispatcher as the dispatcher type, even though the sink only ever calls dispatch(name, message) -> bool. Added a local AlarmDispatcherPort Protocol (matching the existing tools.system.watch_dog.monitor.AlarmDispatcherPort pattern) so a RocketChatAlarmDispatcher satisfies the type without a mypy error. No behavior change — all 39 pre-existing tests pass unchanged.
  • surfaces/cli/commands/hermes.py--provider telegram|rocketchat on opensre hermes watch (default telegram), branching dispatcher construction the same way the watchdog CLI does; --chat-id help text and the startup echo line generalized to name both providers.
  • No message-formatting changes needed: unlike the watchdog's alarm body (Telegram HTML, fixed in feat(rocketchat): add Rocket.Chat alarms for the watchdog (/watch, opensre watchdog) #4217 after a real formatting bug), TelegramSink's incident message is already plain text — safe to send to either provider as-is.
  • Docs: docs/hermes.mdx gets the --provider flag, ROCKETCHAT_* env vars, and provider-neutral intro copy; docs/messaging/rocketchat.mdx gets a new "Hermes incident escalation" section. Also fixed a doc-accuracy issue found while editing the adjacent watchdog section: it claimed watchdog/Hermes alarms follow the same token-only rule as scheduled deliveries (cron/Sentry digest) — they don't; both accept a webhook-only configuration too, since neither carries a fixed --chat-id requirement the way cron/digest do. Also fixed a stale AGENTS.md repo-map line (drifted after feat(rocketchat): add Rocket.Chat alarms for the watchdog (/watch, opensre watchdog) #4217, still said "Telegram alarm dispatch" only).

Testingmake lint, make format-check, make typecheck, make check-layers-strict all pass. 190 tests across tests/hermes/ and tests/cli/test_hermes_command.py pass (10 new: provider flag help text, default-telegram dispatcher construction, rocketchat dispatcher construction, --chat-idchannel_override passthrough, invalid-provider rejection), zero regressions. Verified end-to-end against a real Rocket.Chat workspace: constructed the exact CLI code path (TelegramSink + RocketChatAlarmDispatcher from real load_credentials_from_env() output) and dispatched a synthetic HIGH incident — delivered without exception (screenshot below).

Demo/Screenshot for feature changes and bug fixes -

CleanShot 2026-08-01 at 14 30 57@2x

Code Understanding and AI Usage

Did you use AI assistance (ChatGPT, Claude, Copilot, etc.) to write any part of this code?

  • Yes, I used AI assistance (continue below)

If you used AI assistance:

  • I have reviewed every single line of the AI-generated code
  • I can explain the purpose and logic of each function/component I added
  • I have tested edge cases and understand how the code handles them
  • I have modified the AI output to follow this project's coding standards and conventions

Explain your implementation approach:

This closes out the alarm-delivery parity work started with the watchdog (#4214/#4217): both features hardcoded Telegram via the same AlarmDispatcher class, and #4217 already built the reusable RocketChatAlarmDispatcher + cooldown primitive specifically so this PR could stay small. The only real design decision here was how to make TelegramSink accept either dispatcher without weakening its typing — a local structural Protocol (mirroring an existing pattern already in the codebase) does that with a four-line change and zero behavior risk, rather than widening the concrete type or reaching for Any.

I checked whether the watchdog's HTML-formatting bug (caught during that PR's manual demo) applies here too before assuming it doesn't — it doesn't, because TelegramSink's message builder never used <b>/<code> tags in the first place, so Rocket.Chat renders the same plain-text body correctly with no formatter split needed.


Checklist before requesting a review

  • I have added proper PR title and linked to the issue
  • I have performed a self-review of my code
  • I can explain the purpose of every function, class, and logic block I added
  • I understand why my changes work and have tested them thoroughly
  • I have considered potential edge cases and how my code handles them
  • If it is a core feature, I have added thorough tests
  • My code follows the project's style guidelines and conventions

TelegramSink.__init__ and make_telegram_sink hardcoded
integrations.telegram.alarms.AlarmDispatcher as the dispatcher type,
even though the sink only ever calls dispatch(name, message) -> bool
on it. Add a local AlarmDispatcherPort Protocol (matching the existing
tools.system.watch_dog.monitor.AlarmDispatcherPort pattern) so a
RocketChatAlarmDispatcher can be passed in without a type error, with
no behavior change — all 39 existing tests pass unchanged.
--provider telegram|rocketchat (default telegram, unchanged behavior)
on hermes_watch, branching dispatcher construction the same way the
watchdog CLI does. --chat-id help text and the startup echo line
generalized to name both providers.
…laim

hermes.mdx: --provider flag, ROCKETCHAT_* env vars, intro/note mention
both providers. rocketchat.mdx: new 'Hermes incident escalation'
section; also corrects the watchdog section's inaccurate claim that
alarms follow the scheduled-delivery (cron/digest) rule of requiring
token credentials only — watchdog/Hermes alarms actually accept a
webhook-only configuration too, unlike cron/digest which need an
explicit --chat-id destination a webhook's fixed destination can't
honor.
Drifted after Tracer-Cloud#4217 added Rocket.Chat watchdog alarms — the repo map
still said 'Telegram alarm dispatch' only.
Copilot AI review requested due to automatic review settings August 1, 2026 17:31
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Greptile code review

This repo uses Greptile for automated review. Before merge, aim for Confidence Score: 5/5 with zero unresolved review threads — see CONTRIBUTING.md.

Run a review — add a PR comment with:

@greptile review

Give it ~5-10 minutes (sometimes longer) for results, then fix feedback and re-trigger until you reach Confidence Score: 5/5.

Optional: automate with the greploop skill.

@greptile-apps

greptile-apps Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds Rocket.Chat as a Hermes incident-delivery provider while preserving Telegram as the default.

  • Introduces a provider-neutral dispatcher protocol for the Hermes sink.
  • Adds provider selection and Rocket.Chat credential wiring to opensre hermes watch.
  • Adds CLI coverage for provider selection, dispatcher construction, destination overrides, and invalid values.
  • Updates Hermes and Rocket.Chat documentation with provider-neutral delivery guidance.

Confidence Score: 5/5

The PR appears safe to merge.

The previously reported provider-specific documentation wording has been corrected, and no blocking failure remains.

Important Files Changed

Filename Overview
integrations/hermes/sinks.py Replaces the Telegram-specific dispatcher annotation with a minimal structural dispatch protocol.
surfaces/cli/commands/hermes.py Adds provider selection and constructs the corresponding Telegram or Rocket.Chat dispatcher.
tests/cli/test_hermes_command.py Covers help output, default and Rocket.Chat dispatcher selection, destination forwarding, and invalid-provider rejection.
docs/hermes.mdx Documents both delivery providers and consistently applies provider-neutral wording to deduplication and investigation behavior.
docs/messaging/rocketchat.mdx Documents Rocket.Chat delivery for Hermes escalation and clarifies supported credential modes.

Reviews (2): Last reviewed commit: "docs(hermes): make correlator/investigat..." | Re-trigger Greptile

Comment thread docs/hermes.mdx

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Rocket.Chat as an alarm delivery provider for opensre hermes watch, aligning Hermes incident escalation with the watchdog’s provider model by reusing the existing RocketChatAlarmDispatcher and loosening Hermes sink typing via a local dispatcher Protocol.

Changes:

  • Add --provider telegram|rocketchat to opensre hermes watch and branch dispatcher construction accordingly.
  • Introduce a minimal AlarmDispatcherPort Protocol in Hermes sinks so both Telegram and Rocket.Chat dispatchers type-check without coupling to a concrete provider.
  • Update docs and CLI tests to cover the new provider flag and Rocket.Chat configuration.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/cli/test_hermes_command.py Adds CLI wiring tests for --provider, defaulting behavior, and --chat-id → Rocket.Chat channel override.
surfaces/cli/commands/hermes.py Adds --provider flag and constructs the correct dispatcher/credentials loader based on provider.
integrations/hermes/sinks.py Replaces concrete AlarmDispatcher typing with a local Protocol (AlarmDispatcherPort).
docs/messaging/rocketchat.mdx Documents Hermes incident escalation support for Rocket.Chat alongside watchdog alarms.
docs/hermes.mdx Updates Hermes docs to mention Rocket.Chat and the new --provider flag / env vars.
AGENTS.md Updates repo map wording for watchdog alarms to include Rocket.Chat provider support.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +163 to +167
Loads the selected provider's credentials from the environment,
constructs a :class:`HermesAgent` wired to a :class:`TelegramSink`
(delivering via Telegram or Rocket.Chat depending on ``--provider``),
then waits for ``SIGINT``/``SIGTERM`` before shutting the agent down
cleanly.
Comment thread docs/hermes.mdx Outdated
| `--chat-id ID` | Overrides the provider's default destination (`TELEGRAM_DEFAULT_CHAT_ID` or `ROCKETCHAT_DEFAULT_CHANNEL`) for this run |
| `--cooldown-seconds N` | Per-fingerprint cooldown before the same incident is sent again (default `300`) |
| `--from-start` | Replay the file from the beginning, then tail |
| `--investigate` / `--no-investigate` | Run an OpenSRE RCA for `HIGH` / `CRITICAL` incidents and append the summary to Telegram |
Review flagged that the command reference now covers both providers,
but the adjacent dedup and RCA-trigger sentences still said 'Telegram'
specifically, which would read as those features not applying to
Rocket.Chat deliveries.
Copilot AI review requested due to automatic review settings August 1, 2026 17:41
@mdiniz97

mdiniz97 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor Author

@greptile review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (3)

surfaces/cli/commands/hermes.py:46

  • The Click group docstring still says Hermes routes incidents to Telegram, but this command now supports Rocket.Chat via --provider. This string is used in opensre hermes --help, so it should be provider-neutral.
from integrations.telegram.alarms import AlarmDispatcher
from integrations.telegram.credentials import load_credentials_from_env
from tools.investigation.capability import run_investigation


@click.group(name="hermes", invoke_without_command=True)
@click.pass_context
def hermes_command(ctx: click.Context) -> None:
    """Live-tail Hermes logs and route detected incidents to Telegram."""

integrations/hermes/sinks.py:179

  • TelegramSink now accepts a provider-agnostic AlarmDispatcherPort, so it can deliver via Rocket.Chat too. However, the exception log line still says telegram sink failed, which will be misleading when --provider rocketchat is in use; similarly, nearby docstrings still describe the sink as Telegram-specific. Consider making the log message/docstrings provider-neutral (e.g., "Hermes incident sink").
    )

    def __init__(
        self,
        dispatcher: AlarmDispatcherPort,
        *,
        investigation_bridge: InvestigationBridge | None = None,

surfaces/cli/commands/hermes.py:78

  • The --from-start help text still hardcodes Telegram, but the watcher can now deliver via Rocket.Chat too. Update the wording to avoid provider-specific copy in --help.
@click.option(
    "--chat-id",
    "chat_id",
    type=str,
    default=None,
    help=(
        "Chat/channel to deliver incidents to. Overrides the provider's "
        "default (TELEGRAM_DEFAULT_CHAT_ID or ROCKETCHAT_DEFAULT_CHANNEL) "
        "when both are set."
    ),

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Rocket.Chat alarms for Hermes incident escalation (opensre hermes watch)

2 participants