Skip to content

Native binary drops stdout on a full non-blocking pipe and still exits normally #1015

Open

Description

@JasonHassold

Version: tsrs 0.25.0-ts7.1.0-beta (@tsrs/linux-x64-gnu), Linux x64.

What happens

When the native tsrs binary's stdout is a pipe in O_NONBLOCK mode and the reader falls behind, tsrs stops writing once the pipe buffer fills. It discards the rest of its output and prints nothing on stderr. The exit code is unchanged, so a caller cannot tell the output is incomplete:

  • --listFilesOnly stops at exactly 65,536 bytes and exits 0.
  • A check with 5,000 errors prints 841 of them and exits 2.

tsc from typescript@7.1.0-beta prints all of its output under the same conditions.

This is the binary-level side of #830. That issue was closed after the npm launcher started clearing O_NONBLOCK before execve (#831). The binary still drops output whenever it inherits a non-blocking stdout, from any parent: a Bun-hosted launcher (as discussed in #830), a process supervisor, or another tool that shares the pipe.

Reproduction

repro.py runs a command with stdout on a pipe, either blocking or O_NONBLOCK, and starts reading after one second:

import os, subprocess, sys, time
mode, cmd = sys.argv[1], sys.argv[2:]
r, w = os.pipe()
os.set_blocking(w, mode == "blocking")
p = subprocess.Popen(cmd, stdout=w, stderr=subprocess.PIPE)
os.close(w)
time.sleep(1)  # let the pipe buffer fill before reading
out = b""
while chunk := os.read(r, 1 << 16):
    out += chunk
err = p.stderr.read()
print(f"{mode:12} exit={p.wait()} stdout={len(out):>7} bytes, {out.count(b'error TS'):>4} diagnostics, stderr={err[:80]!r}")
npm i tsrs@0.25.0-ts7.1.0-beta typescript@7.1.0-beta

# 5,000 type errors
python3 -c "open('a.ts','w').write(''.join(f'export const v{i}: number = \"x\";\n' for i in range(5000)))"
echo '{"compilerOptions":{"strict":true,"noEmit":true},"files":["a.ts"]}' > tsconfig.json

# 2,000 files, for a --listFilesOnly listing larger than 64 KiB
mkdir many && python3 -c "
for i in range(2000): open(f'many/module_with_a_reasonably_long_name_{i:04}.ts','w').write(f'export const v{i} = {i};\n')"
echo '{"compilerOptions":{"strict":true,"noEmit":true},"include":["many/**/*.ts"]}' > tsconfig.files.json

TSRS=node_modules/@tsrs/linux-x64-gnu/tsrs
TSC=node_modules/@typescript/typescript-linux-x64/lib/tsc
for m in blocking nonblocking; do python3 repro.py $m $TSRS -p tsconfig.json --pretty false; done
for m in blocking nonblocking; do python3 repro.py $m $TSRS -p tsconfig.files.json --listFilesOnly; done
for m in blocking nonblocking; do python3 repro.py $m $TSC -p tsconfig.json --pretty false; done
for m in blocking nonblocking; do python3 repro.py $m $TSC -p tsconfig.files.json --listFilesOnly; done

Output:

# tsrs, 5,000 errors
blocking     exit=2 stdout= 393893 bytes, 5000 diagnostics, stderr=b''
nonblocking  exit=2 stdout=  65490 bytes,  841 diagnostics, stderr=b''
# tsrs, --listFilesOnly
blocking     exit=0 stdout= 389448 bytes,    0 diagnostics, stderr=b''
nonblocking  exit=0 stdout=  65536 bytes,    0 diagnostics, stderr=b''
# tsc 7.1.0-beta, 5,000 errors
blocking     exit=2 stdout= 393893 bytes, 5000 diagnostics, stderr=b''
nonblocking  exit=2 stdout= 393893 bytes, 5000 diagnostics, stderr=b''
# tsc 7.1.0-beta, --listFilesOnly
blocking     exit=0 stdout= 390978 bytes,    0 diagnostics, stderr=b''
nonblocking  exit=0 stdout= 390978 bytes,    0 diagnostics, stderr=b''

Where we hit it

A test read bunx tsrs --listFilesOnly through a Node spawnSync call, inside a parallel Vitest run (Bun 1.4.4, Node 26, Linux x64). In CI, and in a local run of the same suite, the listing came back incomplete while tsrs exited 0, so the test got the wrong answer. Locally the listing ended partway through a path, after 657,792 of 934,790 bytes. Run on its own, the same command always printed everything, and we could not isolate which process made the pipe non-blocking under load. Redirecting the listing to a regular file fixed it.

The exit-0 case worries us most: a caller of --listFilesOnly that trusts the exit code acts on a partial list.

Expected

tsrs writes all of its output, as tsc does. When a write returns EAGAIN, it waits for the descriptor to become writable (poll) and retries. Alternatively, it reports a write error on stderr and exits non-zero. The triage branch in #830 cleared O_NONBLOCK in the binary, which covers the launcher case, but not a descriptor that another process switches to non-blocking while tsrs runs.

Activity

  1. oxc-captain commented on Nov 17, 2026

    @oxc-captain
    Contributor
    • Reproduced: Yes
    • Exploration: Yes — View branch
    • Unit Test: Yes — crates/tsrs_cli/tests/integration/nonblocking_stdout.rs (integration regression)
    • Priority: P2: Confirmed issue with normal priority.
      This only triggers when tsrs inherits a non-blocking stdout pipe, but in that case it can return an exit-0 --listFilesOnly list that is silently truncated, which callers may act on. There is a workaround (redirect to a regular file), so it is not release-blocking.

    The root cause is in crates/tsrs_execute/src/sys.rs. OsSys writes to a BufWriter<std::io::Stdout> and only treats BrokenPipe as fatal; ErrorKind::WouldBlock (the Rust mapping of EAGAIN on a full non-blocking pipe) is silently discarded. Go's cmd/tsc/sys.go uses os.Stdout, and the Go runtime parks the goroutine and polls the descriptor on EAGAIN, so the Go compiler blocks until the reader drains the pipe. Instrumentation confirmed WouldBlock is hit exactly when output stops.

    I found a potential fix for this issue. The branch wraps stdout in a StdoutWriter that polls for POLLOUT and retries on WouldBlock/Interrupted, matching the Go runtime behavior while preserving the existing SIGPIPE handling for tsc | head. It also adds integration regression tests for both diagnostics and --listFilesOnly on a non-blocking pipe.

    Full Triage Report

    Issue 1015 — Reproduction Report

    Claim

    When the native tsrs binary inherits a stdout pipe that is in O_NONBLOCK
    mode, it stops writing once the pipe buffer fills, silently discards the
    remaining output, and exits with the same status code as if it had printed
    everything. The pinned TypeScript Go compiler (tsc-go) prints all of its
    output under identical conditions.

    • 5,000 type errors: tsrs prints only part of its diagnostics on a
      non-blocking pipe, while tsc-go prints all 5,000.
    • --listFilesOnly on a project whose listing is larger than 64 KiB:
      tsrs stops at exactly the pipe-buffer size (65,536 bytes); tsc-go
      prints the complete listing.

    Environment

    • Host: Ubuntu x86_64 (GitHub Actions runner)
    • tsrs binary: target/debug/tsrs, built from this tree at
      7eb504db5ce54d160a73399fadd30f7f695681c1
    • tsrs version string: Version 7.1.0-beta (tsrs 7.1.0-beta.tsrs2, microsoft/TypeScript@3e9236361fa9)
    • Reference Go compiler: triage/gh-1015/ts-ref/tsc/tsc-go, built from
      Microsoft/TypeScript tag v7.1-beta (3e9236361fa97a9e6c685f4833ac457ae2d8c50a)
    • Go toolchain: host go1.24.13, build downloaded go1.27.0 via GOTOOLCHAIN=auto

    Minimal input

    Created in triage/gh-1015/repro/:

    • a.ts: 5,000 lines of
      export const v{i}: number = "x"; (produces 5,000 diagnostics)
    • many/module_with_a_reasonably_long_name_{0000..1999}.ts: 2,000 tiny modules
    • tsconfig.json: files: ["a.ts"]
    • tsconfig.files.json: include: ["many/**/*.ts"]

    The helper triage/gh-1015/repro.py opens a pipe, toggles O_NONBLOCK
    on the write side, spawns a command, sleeps one second to let the buffer
    fill, then drains stdout and reports size/count.

    Commands run

    cargo build --profile dev --locked -p tsrs_cli -p tsrs_testrunner
    # reference compiler
    GOTOOLCHAIN=auto GOWORK=off go build -o tsc-go ./cmd/tsc
    
    TSRS="target/debug/tsrs"
    TSC="triage/gh-1015/ts-ref/tsc/tsc-go"
    pushd triage/gh-1015/repro
    for m in blocking nonblocking; do python3 ../repro.py $m "$TSRS" -p tsconfig.json --pretty false; done
    for m in blocking nonblocking; do python3 ../repro.py $m "$TSRS" -p tsconfig.files.json --listFilesOnly; done
    for m in blocking nonblocking; do python3 ../repro.py $m "$TSC" -p tsconfig.json --pretty false; done
    for m in blocking nonblocking; do python3 ../repro.py $m "$TSC" -p tsconfig.files.json --listFilesOnly; done

    Observed output (representative run after restoration of the uninstrumented
    sys.rs):

    # tsrs, 5,000 errors
    blocking     exit=2 stdout= 393893 bytes, 5000 diagnostics, stderr=b''
    nonblocking  exit=2 stdout= 306888 bytes, 3898 diagnostics, stderr=b''
    # tsrs, --listFilesOnly
    blocking     exit=0 stdout= 197498 bytes,    0 diagnostics, stderr=b''
    nonblocking  exit=0 stdout=  65536 bytes,    0 diagnostics, stderr=b''
    # tsc-go 7.1.0-beta, 5,000 errors
    blocking     exit=2 stdout= 393893 bytes, 5000 diagnostics, stderr=b''
    nonblocking  exit=2 stdout= 393893 bytes, 5000 diagnostics, stderr=b''
    # tsc-go 7.1.0-beta, --listFilesOnly
    blocking     exit=0 stdout= 197498 bytes,    0 diagnostics, stderr=b''
    nonblocking  exit=0 stdout= 197498 bytes,    0 diagnostics, stderr=b''
    

    Verdict

    REPRODUCED.

    On a non-blocking stdout pipe, tsrs emits only a prefix of its output and
    exits normally, while the pinned TypeScript Go compiler (tsc-go) emits
    the full output with the same exit code. The Go-versus-Rust difference is
    present on both the diagnostic listing and the --listFilesOnly listing.


    Diagnosis

    First mismatching functions

    • Rust: crates/tsrs_execute/src/sys.rs, impl System for OsSys lines 42–49
      (current HEAD):

      fn write(&self, text: &str) {
          exit_on_broken_pipe(&self.writer.lock().unwrap().write_all(text.as_bytes()));
      }
      
      fn flush(&self) {
          exit_on_broken_pipe(&self.writer.lock().unwrap().flush());
      }
    • Go: triage/gh-1015/ts-ref/tsc/cmd/tsc/sys.go, osSys lines 47–67:

      func (s *osSys) Writer() io.Writer {
          return s.writer
      }
      
      func (s *osSys) ErrorWriter() io.Writer {
          return os.Stderr
      }

      where newSystem sets writer: os.Stdout (line 139).

    Semantic/control-flow difference

    Go's os.Stdout.Write is backed by the Go runtime's poller: when the
    underlying write returns EAGAIN on a non-blocking fd, the runtime parks
    the goroutine until the fd becomes writable and retries the write.
    Consequently tsc-go blocks until the reader drains the pipe and the
    full output is delivered.

    Rust's OsSys uses a std::io::BufWriter<std::io::Stdout>. Its
    write_all returns Err(ErrorKind::WouldBlock) as soon as the kernel
    reports EAGAIN. The current helper only checks for BrokenPipe:

    fn exit_on_broken_pipe(result: &std::io::Result<()>) {
        if !matches!(result, Err(err) if err.kind() == std::io::ErrorKind::BrokenPipe) {
            return;
        }
        // SIGPIPE handling ...
    }

    Every non-BrokenPipe error, including WouldBlock, is therefore
    silently ignored. After the first blocked write_all, the BufWriter
    keeps the buffered bytes, but every subsequent flush also fails, so the
    rest of the output is dropped while the program proceeds to its normal
    exit.

    Temporary instrumentation confirmed the exact error kind. With the
    unmodified sys.rs augmented to log errant writes,
    tsrs --listFilesOnly on a non-blocking pipe printed:

    [tsrs diag] write error: kind=WouldBlock, raw=Os { code: 11, kind: WouldBlock, ... }
    

    and then produced a 65,536-byte stdout (the Linux pipe-buffer size).
    The instrumentation change was reverted before this report was written.

    Go-shaped fix approach

    In crates/tsrs_execute/src/sys.rs:

    1. On ErrorKind::WouldBlock (and probably ErrorKind::Interrupted),
      do not discard the result.
    2. Poll stdout for POLLOUT using rustix::poll::poll (already a
      workspace dependency) and retry once the descriptor is writable.
    3. Continue retrying until the full buffer is written or a fatal error
      (e.g. BrokenPipe) occurs.
    4. Optionally keep the current exit_on_broken_pipe behaviour for
      BrokenPipe/EPIPE so that tsrs | head still exits 141.

    This would be a small wrapper around the existing BufWriter writes,
    entirely within tsrs_execute. A lighter alternative is to clear
    O_NONBLOCK at startup (as #830 explored), but that only matches the
    launcher-protection scenario and still loses output if another process
    switches the descriptor to non-blocking while tsrs is running.

    Evidence and confidence

    • The output gap only appears in non-blocking mode and at exactly the
      pipe-buffer limit for --listFilesOnly, matching a write(2) that
      returns -EAGAIN.
    • Instrumentation captured a std::io::ErrorKind::WouldBlock from the
      BufWriter flush path.
    • The Rust code explicitly ignores all non-broken-pipe errors.
    • The Go os.Stdout writer transparently polls on EAGAIN.

    Confidence: high.


    Verification

    Reporter's claim

    The tsrs binary drops stdout once a non-blocking pipe buffer fills, exits
    normally, and does not emit a stderr error. The pinned TypeScript Go compiler
    (tsc-go) prints all output under the same non-blocking pipe setup.

    Control comparison

    Both compilers were run from the same working directory, using the same
    tsconfig.json and tsconfig.files.json, the same --pretty false / --listFilesOnly
    flags, and the same Python harness that delays the reader by one second.

    • target/debug/tsrs built from this checkout:
      Version 7.1.0-beta (tsrs 7.1.0-beta.tsrs2, microsoft/TypeScript@3e9236361fa9)
    • triage/gh-1015/ts-ref/tsc/tsc-go built from the pinned tag v7.1-beta:
      Version 7.1.0-beta

    Result summary:

    case compiler pipe mode exit stdout bytes diagnostics
    5,000 errors tsrs blocking 2 393,893 5,000
    5,000 errors tsrs nonblocking 2 306,888 3,898
    5,000 errors tsc-go blocking 2 393,893 5,000
    5,000 errors tsc-go nonblocking 2 393,893 5,000
    --listFilesOnly tsrs blocking 0 197,498 0
    --listFilesOnly tsrs nonblocking 0 65,536 0
    --listFilesOnly tsc-go blocking 0 197,498 0
    --listFilesOnly tsc-go nonblocking 0 197,498 0

    tsrs produces a complete, byte-identical output in blocking mode and matches
    tsc-go. In non-blocking mode tsrs emits only a prefix—exactly the Linux
    pipe-buffer size for --listFilesOnly—and returns the same exit code. No
    stderr message is produced in either tsrs run.

    Verdict

    bug. The checked-out tsrs diverges from the pinned TypeScript Go compiler
    on a real control (blocking vs. non-blocking stdout pipe). The divergence is in
    CLI/stdio behavior, which is part of the user-visible compiler contract modeled
    by the Go implementation.

    Confidence: high.


    Fix

    Changed behavior

    After the fix, tsrs prints the complete --listFilesOnly listing and the
    full set of diagnostics even when stdout is a non-blocking pipe. This matches
    the pinned TypeScript Go compiler's behavior.

    Go function followed

    • Go: cmd/tsc/sys.go, osSys.Writer() returning os.Stdout. The Go
      runtime's poller transparently retries EAGAIN/EINTR writes to stdout.

    Implementation

    • crates/tsrs_execute/src/sys.rs:

      • Added StdoutWriter, a Write impl around std::io::Stdout that polls
        stdout for POLLOUT and retries on ErrorKind::WouldBlock or
        Interrupted.
      • Replaced BufWriter<std::io::Stdout> with BufWriter<StdoutWriter> in
        OsSys.
      • Non-Unix targets keep the simple pass-through behavior.
    • crates/tsrs_cli/tests/integration/nonblocking_stdout.rs:

      • Added two regression tests that spawn the debug tsrs binary with stdout
        on a non-blocking pipe, delay the reader, and assert the complete output.
      • list_files_only_is_complete_on_a_non_blocking_stdout failed before the
        fix with a 65,536-byte truncated listing.
    • crates/tsrs_cli/tests/integration/main.rs:

      • Added mod nonblocking_stdout;.
    • tools/lint/unsafe_sites.tsv:

      • Recorded the new poll_out unsafe site as keep.

    Verification commands and results

    Build:

    cargo build --profile dev --locked -p tsrs_cli -p tsrs_testrunner

    Reproduction harness (after fix):

    # tsrs, 5,000 errors
    blocking     exit=2 stdout= 393893 bytes, 5000 diagnostics, stderr=b''
    nonblocking  exit=2 stdout= 393893 bytes, 5000 diagnostics, stderr=b''
    # tsrs, --listFilesOnly
    blocking     exit=0 stdout= 197498 bytes,    0 diagnostics, stderr=b''
    nonblocking  exit=0 stdout= 197498 bytes,    0 diagnostics, stderr=b''
    # tsc-go 7.1.0-beta, 5,000 errors
    blocking     exit=2 stdout= 393893 bytes, 5000 diagnostics, stderr=b''
    nonblocking  exit=2 stdout= 393893 bytes, 5000 diagnostics, stderr=b''
    # tsc-go 7.1.0-beta, --listFilesOnly
    blocking     exit=0 stdout= 197498 bytes,    0 diagnostics, stderr=b''
    nonblocking  exit=0 stdout= 197498 bytes,    0 diagnostics, stderr=b''
    

    Regression tests:

    cargo test --profile dev --locked -p tsrs_cli --test integration -- broken_pipe nonblocking_stdout --test-threads=1

    Result: test result: ok. 3 passed; 0 failed.

    Lints:

    cargo clippy --all-targets --locked --config 'build.warnings="deny"' -p tsrs_cli -p tsrs_execute
    node tools/lint/source.mjs

    Both pass.

    Regression suite:

    tools/regressions.sh target/debug/tsrs

    All 54 testdata/regressions cases pass.

    Final diff summary

    M crates/tsrs_cli/tests/integration/main.rs
    M crates/tsrs_execute/src/sys.rs
    M tools/lint/unsafe_sites.tsv
    A crates/tsrs_cli/tests/integration/nonblocking_stdout.rs
    

    This report was made by an LLM. The analysis may be wrong, and the potential fix might not work, but is intended as a starting point for exploring the issue.

    I've opened a pull request with this fix: #1017

  2. added
    - P2Confirmed issue with normal priority
    pkg: cliCommand-line compiler and build mode
    on Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    - P2Confirmed issue with normal prioritypkg: cliCommand-line compiler and build modetriage: fix verifiedThe proposed fix was verified

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions