Skip to content

fix(pipeline): run compound command stages concurrently - #1242

Open
hartsock wants to merge 1 commit into
reubeno:mainfrom
hartsock:fix/compound-pipeline-stage-deadlock
Open

fix(pipeline): run compound command stages concurrently#1242
hartsock wants to merge 1 commit into
reubeno:mainfrom
hartsock:fix/compound-pipeline-stage-deadlock

Conversation

@hartsock

@hartsock hartsock commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Found during practical testing of compound commands.

A compound command used as a non-final pipeline stage runs inline to completion instead of being spawned, so the stage that would drain its output is never started. Once it writes more than one pipe buffer (64 KiB on Linux/macOS) it blocks forever.

# completes (62 KiB)
i=0; while [ $i -lt 2000 ]; do printf '%s\n' 0123456789012345678901234567890; i=$((i+1)); done | cat
# hangs (65 KiB)
i=0; while [ $i -lt 2100 ]; do printf '%s\n' 0123456789012345678901234567890; i=$((i+1)); done | cat

bash completes both. The common shape is ... | while read x; do ...; done | sort | head.

Spawn the command onto a blocking task when it owns its shell — the same shape execute_via_builtin_in_owned_shell already uses for builtin stages. The parent-shell case (single-command pipeline, or the last stage under lastpipe) still runs inline, preserving its side effects.

The owned-shell path already executed against Box::new(shell.clone()), so stage side effects were discarded before this change as well; only when the stage runs changes, not which shell it mutates.

Function and ExtendedTest share the inline shape but write nothing to stdout, so they cannot fill a pipe — left unchanged.

Adds one compat case (128 KiB through a while stage). Compat suite: 1773 passed / 0 failed, against 1772 / 0 on main — exactly the added case, known-failures unchanged at 380.

Tested in our downstream fork.

@hartsock

Copy link
Copy Markdown
Contributor Author

Found alongside #1184 and #1244 during practical testing of brush embedded as a shell library.

@github-actions

github-actions Bot commented Jul 25, 2026

Copy link
Copy Markdown

Performance Benchmark Report

Benchmark name Baseline (μs) Test/PR (μs) Delta (μs) Delta %
clone_shell_object 17.66 μs 17.48 μs -0.18 μs 🟢 -1.03%
eval_arithmetic 0.15 μs 0.15 μs -0.00 μs ⚪ Unchanged
expand_one_string 1.69 μs 1.62 μs -0.07 μs ⚪ Unchanged
for_loop 32.18 μs 31.68 μs -0.50 μs ⚪ Unchanged
full_peg_complex 57.08 μs 56.80 μs -0.28 μs ⚪ Unchanged
full_peg_for_loop 6.17 μs 6.27 μs 0.09 μs 🟠 +1.52%
full_peg_nested_expansions 16.11 μs 16.17 μs 0.07 μs ⚪ Unchanged
full_peg_pipeline 4.24 μs 4.26 μs 0.02 μs ⚪ Unchanged
full_peg_simple 1.78 μs 1.80 μs 0.02 μs ⚪ Unchanged
function_call 3.46 μs 3.42 μs -0.03 μs ⚪ Unchanged
instantiate_shell 54.74 μs 55.07 μs 0.34 μs ⚪ Unchanged
instantiate_shell_with_init_scripts 27053.55 μs 26861.39 μs -192.16 μs ⚪ Unchanged
parse_peg_bash_completion 2111.45 μs 2108.34 μs -3.12 μs ⚪ Unchanged
parse_peg_complex 20.71 μs 20.82 μs 0.11 μs ⚪ Unchanged
parse_peg_for_loop 2.01 μs 2.04 μs 0.04 μs 🟠 +1.84%
parse_peg_pipeline 2.05 μs 2.11 μs 0.06 μs 🟠 +2.83%
parse_peg_simple 1.07 μs 1.08 μs 0.01 μs ⚪ Unchanged
run_echo_builtin_command 16.73 μs 16.64 μs -0.10 μs ⚪ Unchanged
tokenize_sample_script 3.42 μs 3.35 μs -0.06 μs 🟢 -1.79%

Code Coverage Report: Only Changed Files listed

Package Base Coverage New Coverage Difference
Overall Coverage 🟢 27.23% 🟢 27.23% ⚪ 0%

Minimum allowed coverage is 20%, this run produced 27.23%
Maximum allowed coverage difference is -5%, this run produced 0%
3.2% |
| brush-test-harness/src/execution.rs | 🟢 80.2% | 🟢 83.17% | 🟢 2.97% |
| brush-test-harness/src/reporting.rs | 🔴 29.93% | 🔴 46.48% | 🟢 16.55% |
| Overall Coverage | 🟢 75.49% | 🟢 75.76% | 🟢 0.27% |

Minimum allowed coverage is 70%, this run produced 75.76%
Maximum allowed coverage difference is -5%, this run produced 0.27%

Test Summary: bash-completion test suite

Outcome Count Percentage
✅ Pass 1582 75.01
❗️ Error 17 0.81
❌ Fail 156 7.40
⏩ Skip 339 16.07
❎ Expected Fail 13 0.62
✔️ Unexpected Pass 2 0.09
📊 Total 2109 100.00

@hartsock
hartsock force-pushed the fix/compound-pipeline-stage-deadlock branch 2 times, most recently from bd50bd8 to 0934b05 Compare July 27, 2026 03:24
A compound command used as a non-final pipeline stage ran inline to
completion instead of being spawned. Stages are set up in order, so the
downstream stage did not exist yet and nothing drained the pipe; once
the compound command wrote more than one pipe buffer (64 KiB on Linux
and macOS) it blocked forever.

Spawn the command onto a blocking task when it owns its shell, the same
way builtins already run as owned-shell pipeline stages. The
parent-shell case (single-command pipeline, or the last stage under
lastpipe) still runs inline, preserving its side effects.

Reproduced with `while ...; done | cat` emitting 32-byte lines: 2000
iterations (62 KiB) completed; 2100 (65 KiB) hung.

Assisted-by: Claude Code:claude-opus-4-8
@hartsock
hartsock force-pushed the fix/compound-pipeline-stage-deadlock branch from 0934b05 to 91cc76a Compare July 31, 2026 03:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant