Inference.ts loses raw model output on JSON-parse failure when --json is set

[NorthwoodsSentinel]

Inference.ts loses raw model output on JSON-parse failure when --json is set

Summary

When TOOLS/Inference.ts is invoked with the --json flag and the model returns text the internal extractor cannot parse as JSON (a common case when the model wraps JSON in markdown fences, framing text, or returns slightly-malformed JSON), the raw model output is dropped — only the error message reaches stderr and the process exits 1. Callers that wanted to attempt lenient parsing have nothing to work with.

Affected paths

• PAI/TOOLS/Inference.ts lines 184-214 (the inference() function's JSON path)
• PAI/TOOLS/Inference.ts lines 500-509 (the main() function's result handler)

Reproduction

# Any prompt where the model wraps JSON in framing text. Common with
# instruction-following models that prepend explanations.
SYS='Output strict JSON with a single field "verdict": APPROVE/DENY/ESCALATE.'
USR='The command rm -rf / is dangerous. Verdict?'

bun PAI/TOOLS/Inference.ts --level fast --json --timeout 30000 "$SYS" "$USR"
# Possible output:
#   stderr: Error: Failed to parse JSON response
#   stdout: (empty)
#   exit:   1

The model's actual response (which may have contained recoverable JSON wrapped in prose like "Here is the analysis: {...}") is preserved inside the result.output field of the inference() response object, but main() never writes it to stdout in the failure path.

Empirically, this hits roughly 5-8% of fast-tier (Haiku) calls in real-world classifier-style use, depending on prompt shape. Hooks that wrap Inference.ts with their own parsing (cheaper than relying on --json) work around it; hooks that trust --json lose the data.

Root cause

PAI/TOOLS/Inference.ts:207-214:

resolve({
  success: false,
  output,          // raw model text preserved internally
  error: 'Failed to parse JSON response',
  latencyMs,
  level,
});

PAI/TOOLS/Inference.ts:500-509:

if (result.success) {
  if (expectJson && result.parsed) {
    console.log(JSON.stringify(result.parsed));
  } else {
    console.log(result.output);
  }
} else {
  console.error(`Error: ${result.error}`);   // <- raw output is in
  process.exit(1);                            //    result.output but never
}                                             //    written to stdout

The result.output field is built specifically because the JSON failure path catches the raw model text — but the CLI driver never surfaces it.

Proposed fix

Three-line addition in the result handler. Non-breaking — callers checking exit code still see failure; callers that only read stdout can now attempt lenient parsing themselves:

} else {
  // Surface raw output to stdout if present, so callers may attempt lenient
  // parsing (e.g. when the model wrapped JSON in markdown fences the strict
  // regex couldn't peel off). Non-breaking: exit code still signals failure.
  if (result.output) {
    console.log(result.output);
  }
  console.error(`Error: ${result.error}`);
  process.exit(1);
}

Unified diff in PR.diff in this same directory.

Why this matters

Inference.ts is a load-bearing primitive — multiple hooks and skills in a PAI install call it via bun TOOLS/Inference.ts --level fast --json. Each silent JSON-parse failure produces a non-classification that the caller has no way to recover from. The model often returned recoverable output; the wrapper threw it away.

Downstream: receipt-discipline hooks built on top of Inference.ts (background memory review, claim-evidence judgment, smart approval) all log PARSE_ERROR for these cases. Fixing the upstream behavior lets those hooks recover the actual model output and apply their own lenient extraction.

Workaround

Drop --json from the call and have the caller parse JSON locally with a lenient extractor (stdout.match(/\{[\s\S]*\}/) + try/catch JSON.parse). Inference.ts then operates as a pure text generator and only fails on timeouts. This is what we've done locally in our hooks but it duplicates extraction logic across every caller.

Backward compatibility

The proposed fix is backward compatible:

• Callers that check exit code see no change — still exits 1 on parse failure
• Callers that read stderr see no change — still receives the error message
• Callers that read stdout AND check exit code: get a new affordance — can recover output even on failure
• The --json flag's contract is unchanged: when parsing succeeds, only the parsed JSON is on stdout (because console.log(JSON.stringify(result.parsed)) is reached); when parsing fails, the raw text is on stdout (instead of empty stdout). No caller can mistake one for the other because exit code differentiates.

Filed by

Rob Chuvala via Margin (Lares-side DA), 2026-06-03
Diagnostic instrumentation: hours of real-world classifier-style usage in PAI hooks; reproducer above is the canonical shape.