Repository navigation
feat(storage): event subscribers act as the agent whose record the event names #7207
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
0d95879
feat(flows): run flow events under their owning agent scope
senamakel 2eacd36
feat(task_sources): fire connection fetches per storage scope
senamakel a5c78f6
refactor(flows): collapse owner map initializer onto one line
senamakel 08e6721
refactor(task_sources): flatten redundant block in connection fetch
senamakel e90d957
fix(notifications): store cron completion notifications under the job…
senamakel 6fa9ba0
test(storage): cover owner resolution for scoped jobs
senamakel 04e3793
refactor(core): use ? operator for early returns in owner lookups
senamakel 661cadf
docs(storage): document owner lookup and event subscriber scopes
senamakel 50bfb10
Merge remote-tracking branch 'origin/storage-scope-propagation' into …
senamakel 7e2792e
Merge upstream/main into storage-event-agents: one agent registry
senamakel 099446a
fix(storage): restrict agent context fallback in saas mode
senamakel ec48a69
fix(flows): revalidate cached flow owners and scope trigger config
senamakel 403445d
feat(cron): track job owner for completion notifications
senamakel a366d29
test(flows): cover agent-scoped trigger config and local live pass
senamakel b485fce
refactor(flows): extract owner resolution into a helper
senamakel e7fd961
test(flows): cover owner cache invalidation when a flow is removed
senamakel cb9e4ea
chore: files changed crates/openhuman-core/src/integrations/task_sour…
senamakel 3c6232a
fix(cron): visit agent jobs held in the storage backend
senamakel ac9d84a
Merge upstream/main (#7204 landed) into storage-event-agents
senamakel f1e69d5
fix(security): report device scope as unsearched on read error
senamakel 49b1759
Merge storage-scope-propagation follow-ups into storage-event-agents
senamakel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| //! Which agent a just-completed cron job belonged to. | ||
| //! | ||
| //! `CronJobCompleted` names only the job, and a one-shot job | ||
| //! (`delete_after_run`) is already gone from the store by the time a | ||
| //! subscriber sees the event, so its owner cannot be looked up there. The | ||
| //! scheduler publishes the event inside the job's own task, which carries the | ||
| //! agent's context, and notes the owner here first; the notification bridge | ||
| //! takes it back out (`desktop::notifications`). | ||
| //! | ||
| //! In-process only — the bus delivers locally — and bounded: if no subscriber | ||
| //! drains it (the bridge is not registered), it is cleared rather than grown. | ||
|
|
||
| use std::collections::HashMap; | ||
| use std::sync::{LazyLock, Mutex}; | ||
|
|
||
| /// Notes kept before the map is cleared outright. | ||
| const CAPACITY: usize = 1024; | ||
|
|
||
| /// Job id → the agent it ran as. | ||
| static OWNERS: LazyLock<Mutex<HashMap<String, String>>> = LazyLock::new(Default::default); | ||
|
|
||
| /// Notes that `job_id` just completed as the current context's agent, when | ||
| /// it ran as one. Call just before publishing `CronJobCompleted`. | ||
| pub fn note(job_id: &str) { | ||
| let Some(agent) = crate::core::runtime::CoreContext::current() | ||
| .and_then(|context| context.session_agent().map(str::to_string)) | ||
| else { | ||
| return; | ||
| }; | ||
| let mut owners = OWNERS | ||
| .lock() | ||
| .unwrap_or_else(std::sync::PoisonError::into_inner); | ||
| if owners.len() >= CAPACITY { | ||
| owners.clear(); | ||
| } | ||
| owners.insert(job_id.to_string(), agent); | ||
| } | ||
|
|
||
| /// The agent `job_id` completed as, if one was noted; removes the note. | ||
| pub fn take(job_id: &str) -> Option<String> { | ||
| OWNERS | ||
| .lock() | ||
| .unwrap_or_else(std::sync::PoisonError::into_inner) | ||
| .remove(job_id) | ||
| } | ||
|
|
||
| #[cfg(test)] | ||
| #[path = "completion_owner_tests.rs"] | ||
| mod tests; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| use super::*; | ||
| use crate::core::runtime::{ContextOverlay, CoreContext, DomainSet}; | ||
|
|
||
| #[tokio::test] | ||
| async fn a_job_completed_as_an_agent_is_noted_once() { | ||
| let agent = CoreContext::for_test(DomainSet::full(), None).derive_with( | ||
| ContextOverlay::new( | ||
| crate::config::Config::default(), | ||
| DomainSet::full(), | ||
| Default::default(), | ||
| ) | ||
| .session_agent("completion-owner-agent"), | ||
| ); | ||
| CoreContext::scope(agent, async { note("completion-owner-job") }).await; | ||
| assert_eq!( | ||
| take("completion-owner-job").as_deref(), | ||
| Some("completion-owner-agent") | ||
| ); | ||
| assert_eq!(take("completion-owner-job"), None, "taken once"); | ||
| } | ||
|
|
||
| #[test] | ||
| fn a_job_completed_without_an_agent_is_not_noted() { | ||
| note("completion-owner-local"); | ||
| assert_eq!(take("completion-owner-local"), None); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,6 +15,7 @@ | |
| mod tests; | ||
|
|
||
| mod dedup_commit; | ||
| mod owner; | ||
| mod run_digest; | ||
| mod trigger; | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| //! Which agent a flow belongs to, for the subscribers that act on a flow | ||
| //! named only by id in an event (`FlowScheduleTick`, `FlowRunFinished`). | ||
| //! | ||
| //! A flow saved from inside an agent's context lives in that agent's storage | ||
| //! scope (`crate::storage`). The subscribers run outside any agent, so they | ||
| //! look the flow up here and handle the event as its owner | ||
| //! (`crate::storage::agents::within_agent`); without a storage backend every | ||
| //! flow is `local` and this is a no-op. | ||
|
|
||
| use std::collections::HashMap; | ||
| use std::sync::{LazyLock, Mutex}; | ||
|
|
||
| use crate::config::Config; | ||
|
|
||
| /// Flows whose owner this process has resolved (`None` = `local`). | ||
| static OWNERS: LazyLock<Mutex<HashMap<String, Option<String>>>> = LazyLock::new(Default::default); | ||
|
senamakel marked this conversation as resolved.
|
||
|
|
||
| /// The agent `flow_id` belongs to: `None` for `local`, or when no scope has | ||
| /// the flow (the handler then reports the flow as unknown, as before). | ||
| pub(super) async fn flow_owner(config: &Config, flow_id: &str) -> Option<String> { | ||
| // No backend: every record is `local`. | ||
| crate::storage::installed()?; | ||
| resolve(config, flow_id).await | ||
| } | ||
|
|
||
| /// [`flow_owner`]'s cache and lookup, across whatever scopes exist. | ||
| async fn resolve(config: &Config, flow_id: &str) -> Option<String> { | ||
| let cached = OWNERS | ||
| .lock() | ||
| .unwrap_or_else(std::sync::PoisonError::into_inner) | ||
| .get(flow_id) | ||
| .cloned(); | ||
| if let Some(owner) = cached { | ||
| // Still there? A removed flow's id may be reused, or the flow moved: | ||
| // re-check the cached scope and resolve again when it lost the flow. | ||
| let still_there = crate::storage::agents::within_agent(owner.as_deref(), async { | ||
| matches!(crate::flows::store::get_flow(config, flow_id), Ok(Some(_))) | ||
| }) | ||
| .await; | ||
| if still_there { | ||
| return owner; | ||
| } | ||
| forget(flow_id); | ||
| } | ||
| let found = crate::storage::agents::find_owner("flow owner", || async { | ||
| matches!(crate::flows::store::get_flow(config, flow_id), Ok(Some(_))) | ||
|
senamakel marked this conversation as resolved.
|
||
| }) | ||
| .await; | ||
| let owner = found?; | ||
|
senamakel marked this conversation as resolved.
|
||
| tracing::debug!( | ||
| target: "flows", | ||
| %flow_id, | ||
| agent = owner.as_deref().unwrap_or("local"), | ||
| "[flows] resolved the flow's owner" | ||
| ); | ||
| OWNERS | ||
|
senamakel marked this conversation as resolved.
|
||
| .lock() | ||
| .unwrap_or_else(std::sync::PoisonError::into_inner) | ||
| .insert(flow_id.to_string(), owner.clone()); | ||
| owner | ||
| } | ||
|
|
||
| /// Forgets a cached owner (a removed flow's id may be reused). | ||
|
senamakel marked this conversation as resolved.
senamakel marked this conversation as resolved.
|
||
| pub(super) fn forget(flow_id: &str) { | ||
| OWNERS | ||
| .lock() | ||
| .unwrap_or_else(std::sync::PoisonError::into_inner) | ||
| .remove(flow_id); | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| #[cfg(test)] | ||
| #[path = "owner_tests.rs"] | ||
| mod tests; | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.