Skip to content

feat: add --transport-dir file-backed mode to run-local-flow - #48

Merged
Exohayvan merged 6 commits into
mainfrom
ai/next-step-20260629-234551
Jun 30, 2026
Merged

Exohayvan merged 6 commits into
mainfrom
ai/next-step-20260629-234551

Conversation

@Exohayvan

Copy link
Copy Markdown
Contributor

Objective

Implement the next-step transport path for local flow execution so run-local-flow can process workers from per-node file-backed inboxes instead of always replaying a shared dispatch log.

Why it matters: this moves the command from a single-log replay model toward node-addressed local transport semantics using existing local transport primitives, without adding real networking.

What changed

  • Added a new optional --transport-dir flag to run-local-flow in CLI argument parsing.
  • Extended run_local_flow(...) with a transport mode:
    • writes dispatch as before,
    • materializes addressed dispatch messages into per-node inbox files using the existing local transport helper,
    • routes available workers through inbox-based processing via process-local-inbox using transport_dir.
  • Kept default behavior unchanged when --transport-dir is omitted.
  • Added transport metadata to successful run-local-flow output only in transport mode: transport_dir, transport_inbox_count, and deterministic transport_inbox_paths.
  • Added and updated command-level tests to lock in behavior:
    • default output has no transport fields,
    • transport mode produces inbox files for available nodes only,
    • offline nodes are not processed and do not get inbox/worker logs,
    • transport-backed run is audit-compatible and idempotent on rerun.
  • Added README documentation for a single run-local-flow --transport-dir usage example and explicitly clarified this as local file-backed transport only.

Validation

  • Ran PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m unittest discover -s tests -p test_cli.py -v and ... -p test_cli_transport.py -v; both passed.
  • Ran full suite command PYTHONTONTWRITEBYTECODE=1 PYTHONPATH=src python -m unittest discover -s tests -v in branch state; all tests passed (250 tests, exit code 0).
  • Validation coverage comes from executable test runs in this branch.

Safety notes

  • No force push used.
  • No hard reset used.
  • Local automation paths are not published in this PR body.

@Exohayvan

Copy link
Copy Markdown
Contributor Author

Automated review did not approve this PR

Automated Hermes/GPT-5.5 review decision for PR #48: bad.

Review notes

Ad-hoc verification completed.

Temporary script:

  • Created: [local temp path redacted]
  • Used OS-safe tempfile with hermes-verify- prefix.
  • Cleaned up successfully:
    CLEANED_UP=[local temp path redacted]

What it verified:

  • Created a temporary manifest with two available nodes and one echo job.
  • Ran run-local-flow with --transport-dir.
  • Confirmed the available node without an assignment gets an empty inbox instead of causing a missing inbox failure.
  • Confirmed the resulting local flow artifacts pass audit-local-flow.

Ad-hoc verification output:
{"audit_ok": true, "behavior": "run-local-flow --transport-dir creates empty inbox for available node without assignment", "empty_inbox_messages": 0, "ok": true, "processed_assignment_count": 1, "processed_node_ids": ["local-node-a", "local-node-b"], "transport_inbox_count": 2, "verification": "ad-hoc"}

Exit code:
0

This is focused ad-hoc verification of the changed behavior, not a claim that the full canonical test suite is freshly green.

Safety notes

  • Local review log path intentionally redacted.
  • No force push used by the automation.
  • No hard reset used by the automation.

@Exohayvan

Copy link
Copy Markdown
Contributor Author

Automated review passed, but merge command failed

Automated Hermes/GPT-5.5 review decision for PR #48: good-but-merge-failed.

Review notes

Summary:
Reviewed PR #48 on branch ai/next-step-20260629-234551.

Graphify orientation:
Used graphify before broad inspection because graphify-out/graph.json exists:

  • graphify query "What changed around local flow transport and CLI commands in this AetherMesh Core PR?"
  • graphify explain "run-local-flow"
  • graphify path "run-local-flow" "process-local-inbox"

Implementation direction:

  • The PR adds an opt-in --transport-dir path to run-local-flow.
  • It preserves the existing default run-local-flow behavior when --transport-dir is omitted.
  • With --transport-dir, dispatch still writes the normal dispatch log, then materializes addressed assignment messages into per-node local transport inboxes and processes workers from those inboxes.
  • This is aligned with the project’s near-term direction: small, local-only, runnable transport seam without claiming real networking or distributed coordination.
  • README wording correctly frames it as file-backed local transport only.
  • Tests cover default shape preservation, transport-backed flow, idempotent reruns, audit compatibility, offline-node behavior, and transport CLI behavior.

Issues:

  • No blocking issues found.
  • I did not make changes.

Validation:
Canonical validation from README:
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m unittest discover -s tests -v

Result:

  • Ran 250 tests
  • OK

Additional functional smoke test:
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m aethermesh_core.cli run-local-flow --manifest examples/local-batch.json --output-dir "$tmpdir/flow" --transport-dir "$tmpdir/transport"
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m aethermesh_core.cli audit-local-flow --output-dir "$tmpdir/flow"

Result:

  • run-local-flow completed successfully
  • transport_inbox_count: 2
  • processed_assignment_count: 5
  • total_contribution_units: 5
  • audit-local-flow returned ok: true
  • audit counts: dispatch_messages=7, flow_messages=17, receipts=5, ledger_records=5, credited_receipt_units=5

Working tree:

  • git status --short returned clean.

DECISION: good

Safety notes

  • Local review log path intentionally redacted.
  • No force push used by the automation.
  • No hard reset used by the automation.

@Exohayvan

Copy link
Copy Markdown
Contributor Author

Automated review did not approve this PR

Automated Hermes/GPT-5.5 review decision for PR #48: bad.

Review notes

Ad-hoc verification completed.

Temporary script created:
[local temp path redacted]

Ad-hoc behavior verified:

  • Created a temporary manifest with 3 available nodes and 1 job.
  • Ran run-local-flow --transport-dir.
  • Confirmed:
    • exit code was 0
    • all 3 available nodes were processed
    • only 1 assignment was processed
    • transport_inbox_count was 3
    • assigned node inbox had 1 message
    • unassigned available nodes had empty inbox files
    • audit-local-flow succeeded on the generated artifacts

Output:
AD_HOC_VERIFY_OK pr48 transport-dir creates empty inboxes for unassigned available nodes

Cleanup:
CLEANUP_OK [local temp path redacted]

This is targeted ad-hoc verification of the changed behavior, not a fresh full-suite green claim.

Safety notes

  • Local review log path intentionally redacted.
  • No force push used by the automation.
  • No hard reset used by the automation.

@Exohayvan

Copy link
Copy Markdown
Contributor Author

Automated review passed, but merge command failed

Automated Hermes/GPT-5.5 review decision for PR #48: good-but-merge-failed.

Review notes

Summary:
Reviewed PR #48 on the checked-out branch in:
[local path redacted]

Used Graphify first because graphify-out/graph.json exists:

  • graphify query "PR 48 changes in CLI transport local flow AetherMesh Core"
    • Oriented around cli.py, scheduler.py, messages.py, message_log.py, ledger.py, simulation.py, node_service.py, dispatch.py, validation.py.
  • graphify explain "run-local-flow transport materialize-local-inboxes process-local-inbox"
    • No exact combined node match.
  • graphify path "run_local_flow" "materialize_local_inboxes"
    • Found: run_local_flow() <--calls-- main() --calls--> materialize_local_inboxes()

Implementation direction:

  • This PR adds an optional --transport-dir path to run-local-flow.
  • The direction is aligned with the project’s near-term goals: local, deterministic, file-backed transport before real networking.
  • The implementation keeps the existing default run-local-flow behavior unchanged when --transport-dir is omitted.
  • With --transport-dir, it materializes dispatch messages into per-node inboxes, then processes workers through the same process-local-inbox path used by the existing transport seam.
  • Tests cover the new run-local-flow transport path, rerun/idempotency behavior, audit compatibility, offline-node exclusion, and preservation of the old output shape.

Issues:

  • No blocking issues found.
  • Minor note only: graphify-out generated files make the PR large/noisy, but the repo explicitly tracks graphify-out and AGENTS.md says the graph belongs there, so I do not consider this a merge blocker.

Changes made:

  • None. The PR appears good as-is.

Validation:
Ran canonical project validation from README.md:

Command:
TMPDIR="$PWD/.tmp-validation" PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m unittest discover -s tests -v

Result:

  • Ran 250 tests
  • OK
  • Included new transport-flow tests:
    • test_run_local_flow_uses_transport_inboxes_and_audits ... ok
    • test_materialize_local_inboxes_and_process_transport_inbox ... ok
    • test_transport_inbox_failure_does_not_overwrite_artifacts ... ok
    • test_process_local_inbox_requires_one_input_source ... ok

Final repo status after validation:

  • Clean working tree.

DECISION: good

Safety notes

  • Local review log path intentionally redacted.
  • No force push used by the automation.
  • No hard reset used by the automation.

@Exohayvan

Copy link
Copy Markdown
Contributor Author

Automated review passed, but merge command failed

Automated Hermes/GPT-5.5 review decision for PR #48: good-but-merge-failed.

Review notes

Summary:
PR #48 is good. The implementation is small, focused, and aligned with AetherMesh Core’s current direction: extending the local-only flow so it can optionally route worker processing through file-backed per-node transport inboxes without introducing real networking, rewards, dashboards, or broad abstractions.

Graphify orientation used:

  • graphify query "How does local inbox materialization and process-local-inbox work in this repository?"
    • Oriented around process_local_inbox, LocalNodeService, LocalMessageBus, load_local_inbox, materialize_local_inboxes, message log replay, ledger persistence, and validation.
  • graphify explain "process-local-inbox"
    • No exact node match for the hyphenated CLI command.
  • graphify path "materialize-local-inboxes" "process-local-inbox"
    • Found a short path through load_message_log_messages() to process_local_inbox().

Implementation review:

  • Adds --transport-dir to run-local-flow.
  • Keeps the existing default run-local-flow behavior unchanged when --transport-dir is omitted.
  • Uses the existing local transport seam (materialize_local_inboxes, load_local_inbox, local_inbox_path) instead of duplicating transport logic.
  • Preserves idempotency through existing per-node state handling.
  • Keeps offline nodes visible in the flow summary but does not process them as workers.
  • Adds useful tests for:
    • Transport-backed full flow.
    • Audit compatibility.
    • Rerun/idempotency behavior.
    • Default output shape when transport is not used.
  • README update is accurate and appropriately scoped.

Issues:

  • No blocking issues found.
  • Minor note: run-local-flow --transport-dir can still partially write the dispatch artifact before a later transport write failure, but this is consistent with existing side-effectful flow behavior and not a reason to block this PR.

Changes made:

  • None. The PR did not require fixes.

Validation:

  • Canonical project validation from README:

    • Command:
      PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m unittest discover -s tests -v
    • Result:
      Ran 250 tests in 0.191s
      OK
  • Feature smoke test:

    • Command:
      PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m aethermesh_core.cli run-local-flow --manifest examples/local-batch.json --output-dir [local temp path redacted] --transport-dir [local temp path redacted]
      followed by:
      PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m aethermesh_core.cli audit-local-flow --output-dir [local temp path redacted]
    • Result:
      Audit returned:
      {"ok": true, ... "counts": {"dispatch_messages": 7, "flow_messages": 17, "ledger_records": 5, "receipts": 5, "total_contribution_units": 5, "credited_receipt_units": 5}}

DECISION: good

Safety notes

  • Local review log path intentionally redacted.
  • No force push used by the automation.
  • No hard reset used by the automation.

@Exohayvan

Copy link
Copy Markdown
Contributor Author

Automated review passed, but merge command failed

Automated Hermes/GPT-5.5 review decision for PR #48: good-but-merge-failed.

Review notes

Summary:

  • Reviewed PR feat: add --transport-dir file-backed mode to run-local-flow #48 on branch ai/next-step-20260629-234551.
  • Used Graphify first because graphify-out/graph.json exists:
    • graphify query "PR #48 implementation direction and changed code in AetherMesh Core"
    • graphify explain "contribution tracking"
    • graphify path "work" "contribution"
  • Inspected the PR diff against origin/main.
  • The PR adds an opt-in --transport-dir path to run-local-flow, routing the existing dispatch-plus-worker flow through file-backed per-node local transport inboxes.
  • This direction is aligned with AetherMesh Core’s near-term goal: strengthen local node/message/transport behavior without jumping into real networking, production peer discovery, rewards, or dashboard work.
  • The implementation is small and focused:
    • CLI option added to run-local-flow.
    • Dispatch log is still written under the flow output directory.
    • Transport inboxes are materialized when opted in.
    • Worker processing then consumes each node’s local transport inbox.
    • Existing no-transport behavior remains unchanged.
    • README documents the new opt-in flow.
    • Tests cover normal transport flow, idempotent reruns, offline nodes, and audit compatibility.

Issues:

  • No blocking issues found.
  • I did not make changes.
  • Minor note: generated graphify-out/* changes are large relative to the code diff, but this repository explicitly keeps Graphify artifacts and AGENTS.md says the automation loop normally refreshes them, so this is acceptable here.

Validation:

  • Canonical README unit test command:

    • PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m unittest discover -s tests -v
    • Result: Ran 250 tests in 0.194s / OK
  • Manual transport flow smoke test:

    • PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m aethermesh_core.cli run-local-flow --manifest examples/local-batch.json --output-dir "$tmpdir/flow" --transport-dir "$tmpdir/transport"
    • PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=src python -m aethermesh_core.cli audit-local-flow --output-dir "$tmpdir/flow"
    • Result summary:
      • processed_assignment_count: 5
      • transport_inbox_count: 2
      • total_contribution_units: 5
      • receipt_count: 5
      • audit ok: true
      • audit counts: dispatch_messages 7, flow_messages 17, ledger_records 5, receipts 5, credited_receipt_units 5, total_contribution_units 5

DECISION: good

Safety notes

  • Local review log path intentionally redacted.
  • No force push used by the automation.
  • No hard reset used by the automation.

@Exohayvan
Exohayvan force-pushed the ai/next-step-20260629-234551 branch from ffd5b78 to cebafa2 Compare June 30, 2026 15:52
@Exohayvan
Exohayvan merged commit 926ab3f into main Jun 30, 2026
22 checks passed
@Exohayvan
Exohayvan deleted the ai/next-step-20260629-234551 branch June 30, 2026 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant