Skip to content

fix(linux): avoid blocking proc reads for WSL agents - #2179

Open
dark2momo wants to merge 3 commits into
herdrdev:masterfrom
dark2momo:codex/fix-process-probe-hang
Open

fix(linux): avoid blocking proc reads for WSL agents#2179
dark2momo wants to merge 3 commits into
herdrdev:masterfrom
dark2momo:codex/fix-process-probe-hang

Conversation

@dark2momo

@dark2momo dark2momo commented Aug 1, 2026

Copy link
Copy Markdown

Context

This was encountered on WSL2 while Herdr was managing several Codex panes. After one of the Codex processes became stuck in an uninterruptible or exiting state, Herdr commands repeatedly failed with:

herdr: protocol error: I/O error: Resource temporarily unavailable (os error 11)

The failure occurred three times in one day. Restarting the affected pane did not reliably clear the stuck process; terminating the WSL distribution was the recovery used in each case.

Inspection showed affected Codex threads blocked in kernel process-exit and virtual-memory paths. Herdr already identified the foreground executable as codex from /proc/<pid>/stat, so reading its remote-memory-backed cmdline or environ was unnecessary and exposed Herdr to the same stall.

This PR does not attempt to recover the stuck agent or fix the underlying WSL kernel behavior. It keeps Herdr's process discovery responsive when an already identifiable agent enters that state, while preserving argv inspection for wrapper processes.

Environment

  • Windows 11
  • WSL 2.7.10.0
  • Ubuntu 22.04
  • Linux 6.18.33.2-microsoft-standard-WSL2
  • Codex CLI 0.145.0

Summary

  • avoid reading /proc/<pid>/cmdline for known agent executables on WSL, where procfs can block in access_remote_vm while a multithreaded process exits
  • skip remote-memory-backed proc reads for uninterruptible and dead process states
  • preserve argv inspection for wrappers such as node, and apply the same guard to agent environment hints
  • cover the WSL fast path and unsafe process states with unit tests

Validation

  • just test-one remote_memory
  • just test-one foreground_job_on_wsl
  • just test-one proc_stat_parsing
  • just lint
  • just check through formatting, Clippy, 3,149 Rust tests, integration assets, plugin marketplace tests, and Windows target Clippy

The final maintenance-script step of just check could not run locally because this machine provides Python 3.10, while scripts/agent_detection_manifest_check.py imports the Python 3.11+ standard-library tomllib module. The other maintenance tests in that invocation completed before the missing-module error; CI provides the repository's supported environment.

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 09e458fa-3843-4f5e-b028-6fba8acd6440

📥 Commits

Reviewing files that changed from the base of the PR and between 4127643 and 16aa669.

📒 Files selected for processing (1)
  • src/platform/linux.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/platform/linux.rs

📝 Walkthrough

Walkthrough

Linux process metadata now includes process state. Foreground-group discovery and process-agent lookup guard command-line reads for unsafe states and known WSL agents. WSL detection is cached, and tests cover state parsing and wrapper handling.

Changes

Linux process discovery

Layer / File(s) Summary
Process metadata and read safety
src/platform/linux.rs
Process metadata now includes process state. Cached WSL detection and safety checks suppress reads for uninterruptible, zombie, dead, and known agent processes.
Foreground group discovery integration
src/platform/linux.rs
Foreground lookup uses shared group processing and child-group fallback. Unsafe processes skip command-line reads while wrappers remain inspectable.
Process-state and command-line validation
src/platform/linux.rs
Tests cover process-state parsing, updated process fixtures, unsafe states, WSL agents, and readable wrapper processes.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ForegroundJobLookup
  participant ProcMetadataParser
  participant CmdlineReader
  ForegroundJobLookup->>ProcMetadataParser: Read process-group metadata
  ProcMetadataParser-->>ForegroundJobLookup: Return PID, group, command name, and state
  ForegroundJobLookup->>CmdlineReader: Read command arguments when safety checks pass
  CmdlineReader-->>ForegroundJobLookup: Return command arguments or no result
Loading

Possibly related PRs

  • herdrdev/herdr#2042: Both changes update foreground process-group discovery and fallback logic.
  • herdrdev/herdr#2052: Both changes update foreground-process discovery and /proc metadata parsing.
  • herdrdev/herdr#2269: Both changes update foreground process-group discovery, /proc parsing, and fallback handling.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main Linux and WSL process-read fix.
Description check ✅ Passed The description directly explains the WSL procfs issue, the implemented safeguards, and the validation performed.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@kangal-bot kangal-bot added the ai-review Trigger automated AI reviews for pull requests admitted by the PR gate label Aug 1, 2026
@greptile-apps

greptile-apps Bot commented Aug 1, 2026

Copy link
Copy Markdown

Greptile Summary

This PR makes Linux foreground-agent discovery avoid potentially blocking remote-memory-backed procfs reads on WSL and for unsafe process states.

  • Caches WSL detection for subsequent process probes.
  • Parses process state alongside process group and command name.
  • Uses command-name detection before deciding whether to read process argv or environment.
  • Adds tests covering WSL agent handling, wrapper inspection, and unsafe process states.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
src/platform/linux.rs Adds process-state-aware and WSL-aware guards around procfs command-line and environment reads, with focused unit coverage.

Reviews (3): Last reviewed commit: "Merge branch 'master' into codex/fix-pro..." | Re-trigger Greptile

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@ogulcancelik

Copy link
Copy Markdown
Collaborator

Thanks for the detailed investigation. We cannot reproduce this WSL-specific failure locally. Before merging, could you capture the blocked Herdr thread's kernel stack or wchan, showing the /proc/<pid>/cmdline or environ read, and confirm that the patched build remains responsive under the same workload? The EAGAIN client error alone does not establish that procfs probing is the cause.

@dark2momo

Copy link
Copy Markdown
Author

Thanks. I could not reproduce the original end-to-end server/client failure
under the multi-agent workload, so I do not want to overstate the result.

I was able to reproduce the exact blocking path in Herdr's Linux platform code
using a real WSL2 process stuck in D-state:

  • target process: state=D, wchan=__vma_start_write
  • unpatched Herdr thread: state=D, wchan=__access_remote_vm
  • bounded trace:
openat(..., "/proc/254810/environ", O_RDONLY|O_CLOEXEC) = 3</proc/254810/environ>
read(3</proc/254810/environ>, <unfinished ...>) = ?

The patched revision, tested against the same PID through the same
process_agent_hint() path, completed in 0.09 seconds and did not open that
process's cmdline or environ. I could not read the kernel stack without
elevated permissions.

I also ran the patched server through five 12-agent stress cycles with no
command failures or responsiveness loss. This validates the underlying procfs
blocking condition and the guard introduced by the patch, but it does not
reproduce the original complete EAGAIN failure chain. Please let me know
whether this evidence is sufficient or whether you would prefer the PR remain
unmerged until the full workload reproduces.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-review Trigger automated AI reviews for pull requests admitted by the PR gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants