Skip to content

JcodeClient.close() can leave the daemon process running (servers.json registration race) #818

Description

@robottwo

JcodeClient.close() can leave the daemon process running (servers.json registration race)

Summary

close() on an instance from JcodeClient.launch() (TypeScript SDK,
@1jehuang/jcode-sdk) resolves successfully, but the underlying
jcode ... serve daemon process can be left running indefinitely,
reparented to init. It does not appear as a zombie and is not hung — a
plain kill -TERM <pid> sent directly to it terminates it immediately.
The problem is that close() never sends it a signal in the first place.

Under concurrent launch()+close() calls (the pattern you'd use to run
several agent instances in parallel), I measured this happening in
roughly 1 out of every 8 instances on the machine I tested on. It also
reproduces reliably as a single, isolated race when instrumented directly
(see "Root cause" below) — the frequency above is this specific
hardware/load, not a claim about the general rate.

Environment

  • jcode: v0.67.1 (88a19f38e)
  • @1jehuang/jcode-sdk: 1.1.0 (installed via npm install @1jehuang/jcode-sdk, which pulled @1jehuang/jcode-linux-x64@1.1.0 as the platform binary)
  • Node: v22.22.2
  • OS/kernel: Linux 6.18.5-fc-v18 x86_64 (a constrained single-vCPU sandboxed VM — see caveat below)
  • No real provider credentials involved; ANTHROPIC_API_KEY was set to a dummy string purely so the daemon's provider-init check at startup didn't refuse to boot. No network calls to a model provider occur before close() in the repro.

Root cause (found by reading sdk/typescript/src/launch.ts)

close() calls stopInstanceDaemon(), which calls readDaemonPidSync()
to find the daemon's PID:

function readDaemonPidSync(jcodeHome: string, runtimeDir: string): number | undefined {
  let raw: string;
  try {
    raw = fs.readFileSync(path.join(jcodeHome, "servers.json"), "utf8");
  } catch {
    return undefined;
  }
  ...
}

and if it can't find an entry, stopInstanceDaemon() returns immediately
without sending any signal:

async function stopInstanceDaemon(...): Promise<void> {
  const pid = readDaemonPidSync(jcodeHome, runtimeDir);
  if (pid === undefined) return;   // <-- silently gives up here
  ...
}

servers.json is written by the daemon itself, as part of its own
startup sequence — apparently on a separate timeline from when the API
socket becomes connectable, which is what launch() actually waits on
before resolving. I confirmed this directly:

jcodeHome: /tmp/jcode-sdk-instance-bmnmWX
servers.json exists immediately after launch()? false      <-- launch() has resolved, file isn't there yet

Waiting 3 seconds before calling close() let the same daemon shut down
cleanly (servers.json existed and had the correct pid/socket
entry by then). So this is a startup-ordering race: launch()
resolving does not imply servers.json has been written
, but
close()'s cleanup depends entirely on that file.

I also confirmed the daemon isn't the thing misbehaving — sending it a
direct SIGTERM after the leak occurred killed it instantly. close()
simply never signals it.

Reproduction

Minimal script (attached, repro.mjs):

import { JcodeClient } from "@1jehuang/jcode-sdk";

const client = await JcodeClient.launch({
  workingDir: process.cwd(),
  inheritLogins: false,
  env: { ANTHROPIC_API_KEY: "sk-ant-repro-000000000000000000000000000" },
});

await client.close();
// check `ps` afterward for a lingering `jcode ... serve` process

This single-shot repro didn't reproduce reliably in my environment
(0/8 in a clean sequential foreground loop) — the race window is narrow
under low load. It reproduces far more reliably:

  1. Directly, by instrumentation — reading servers.json state right
    after launch() resolves shows it missing, every time I checked, in this
    environment. This is the actual bug; whether it manifests as a full
    leak depends on additional scheduling luck.
  2. Under concurrency — firing several launch() + close() pairs at
    once (Promise.all, attached as concurrent_repro.mjs) leaked 1 of 8
    daemons in one run. This is the pattern most likely to matter in
    practice: an application launching a batch of parallel agent instances.
daemons before: 0
daemons after 8 concurrent launch+close pairs: 1 [ '<pid>' ]
leaked: 1 / 8

Expected behavior

After close() resolves, no daemon process associated with that
instance should remain running, regardless of how soon after launch()
close() is called.

Suggested directions for a fix

I'm not familiar enough with the daemon's Rust-side bootstrap sequence to
know which of these fits best, but a few options:

  1. Have the daemon write its servers.json entry before it opens the
    API socket, so "socket is connectable" implies "registry entry
    exists." This seems like the cleanest fix if there's no ordering
    dependency the other way around.
  2. Have stopInstanceDaemon() retry/poll for the servers.json entry
    for a short grace window (e.g. up to a second or two) instead of a
    single synchronous read that gives up immediately.
  3. Have the bridge process (which is the one launch() actually spawns
    and holds a handle to) track the daemon's PID itself as a fallback,
    rather than relying solely on a file the daemon writes independently.

Secondary observation (possibly related, less rigorously isolated)

In one run, a sequential_repro.mjs variant (8x launch()+close() in
a loop, awaited one at a time in the same process) took long enough that
an outer timeout 60 killed the Node process mid-run. After that forced
kill, both a leaked daemon and several un-removed
jcode-sdk-instance-* temp directories were left behind. This may just
be an artifact of killing Node before its exit handler could run (the
README says close() on ephemeral homes can legitimately take up to
cleanupTimeoutMs, default 30s), rather than a distinct bug — flagging
it in case it's useful, but I haven't isolated it the way I did the main
issue above.

Caveat about my environment

This sandbox's PID 1 is a firecracker-init process rather than a
standard init/systemd, and it's a single-vCPU box, both of which are
somewhat unusual and could plausibly affect scheduling/timing. I
couldn't test on a standard host/VM/Docker container to confirm the race
reproduces there too. That said, the root cause I found by reading the
source (launch() resolving before servers.json is guaranteed to
exist) doesn't look environment-specific — it's a genuine ordering gap
between two independent operations, so I'd expect it to reproduce
elsewhere too, just perhaps less often on faster/less loaded hardware,
and potentially more often on a busy CI runner or a host launching many
instances concurrently, which is exactly the "launch many agents at
once" use case the SDK seems designed for.

Happy to run more diagnostics if it'd help track this down further.


Repro scripts

repro.mjs
// Minimal repro: JcodeClient.close() can leak the underlying daemon process.
//
// Root cause (see bug report): close() -> stopInstanceDaemon() reads the
// daemon's pid from `${jcodeHome}/servers.json`. That file is written by the
// daemon itself as part of its own startup, on a separate timeline from the
// API socket becoming connectable (which is what launch() actually waits
// on). If close() runs before the daemon has written its servers.json entry,
// readDaemonPidSync() returns undefined, stopInstanceDaemon() returns
// immediately without sending any signal, and the daemon is orphaned
// (reparented to init) with nothing left tracking it.
//
// Usage:
//   npm install @1jehuang/jcode-sdk
//   node repro.mjs
//
// Expected: no `jcode ... serve` process remains after close() resolves.
// Actual: one is left running indefinitely.

import { JcodeClient } from "@1jehuang/jcode-sdk";
import { execSync } from "node:child_process";

function jcodeServeProcesses() {
  try {
    return execSync("ps -eo pid,ppid,stat,cmd | grep '[j]code.*serve'").toString().trim();
  } catch {
    return "(none)";
  }
}

console.log("jcode serve processes before:\n" + jcodeServeProcesses());

const client = await JcodeClient.launch({
  workingDir: process.cwd(),
  inheritLogins: false,
  // Any non-empty value; no real network call happens before close().
  env: { ANTHROPIC_API_KEY: "sk-ant-repro-000000000000000000000000000" },
});

console.log("\nlaunch() resolved. jcode serve processes:\n" + jcodeServeProcesses());

await client.close();
console.log("\nclose() resolved.");

await new Promise((r) => setTimeout(r, 2000));
console.log("\njcode serve processes 2s after close() resolved:\n" + jcodeServeProcesses());
console.log(
  "\nIf a process is listed above, close() failed to reap it -- this is the bug.",
);
concurrent_repro.mjs
import { JcodeClient } from "@1jehuang/jcode-sdk";
import { countJcodeDaemons } from "./count_daemons.mjs";

const N = parseInt(process.argv[2] || "8", 10);

async function trial(i) {
  const client = await JcodeClient.launch({
    workingDir: process.cwd(),
    inheritLogins: false,
    env: { ANTHROPIC_API_KEY: `sk-ant-repro-${i}-000000000000000000000` },
  });
  await client.close();
  return i;
}

const before = countJcodeDaemons().count;
console.log("daemons before:", before);

await Promise.all(Array.from({ length: N }, (_, i) => trial(i)));

await new Promise((r) => setTimeout(r, 2000));
const after = countJcodeDaemons();
console.log(`daemons after ${N} concurrent launch+close pairs:`, after.count, after.pids);
console.log(`leaked: ${after.count - before} / ${N}`);
count_daemons.mjs
import fs from "node:fs";

export function countJcodeDaemons() {
  let count = 0;
  const pids = [];
  for (const entry of fs.readdirSync("/proc")) {
    if (!/^\d+$/.test(entry)) continue;
    try {
      const exe = fs.readlinkSync(`/proc/${entry}/exe`);
      if (exe.endsWith("jcode-linux-x86_64.bin")) {
        // Confirm it's actually running the "serve" subcommand, via cmdline
        // (NUL-separated, so no risk of matching shell wrapper text).
        const cmdline = fs.readFileSync(`/proc/${entry}/cmdline`, "utf8").split("\0");
        if (cmdline.includes("serve")) {
          count++;
          pids.push(entry);
        }
      }
    } catch {
      // Process exited between readdir and read, or no permission.
    }
  }
  return { count, pids };
}

if (import.meta.url === `file://${process.argv[1]}`) {
  console.log(countJcodeDaemons());
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    autonomous: likelyProbably hands-off: clearly worth fixing, agent can do it, minor judgment needed.bugSomething isn't workingtriage: fixed-pending-releaseFixed in code/committed; will close automatically on next releasetriage: reproducibleClear repro + clear fix path

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions