Skip to content

fix(repair): recover relaxed-JSON object args and drop null optionals - #59

Merged
senamakel merged 1 commit into
mainfrom
fix-tool-call-errors
Oct 10, 2026
Merged

senamakel merged 1 commit into
mainfrom
fix-tool-call-errors

Conversation

@senamakel

@senamakel senamakel commented Oct 10, 2026 •

Copy link
Copy Markdown
Member

Summary

Production Langfuse traces (OpenHuman, Oct 3–10) show use_skill failing schema validation in about 20% of its errors. Models send the nested args object as a string that is not strict JSON, or send null for an optional property whose schema doesn't allow null. The argument repair pass only tried strict JSON for a string in an object-typed slot and kept the null, so the call was rejected.

  • coerce_value: for an object-typed property given a string, fall back to recover_object(strip_code_fence(..)) when strict parsing fails.
  • coerce_to_schema: drop null for an optional property whose declared type has no null. A null in a required property is left for the validator to report.

Tests

  • an_object_typed_string_that_is_relaxed_json_is_recovered (failed before the fix)
  • null_for_an_optional_non_nullable_property_is_dropped (failed before the fix)
  • null_for_a_required_property_is_kept

cargo test -p tinytools-agent: 395/395 pass. clippy -D warnings and fmt are clean.

Consumed by tinyhumansai/tinyagents#365 and tinyhumansai/openhuman (tool-call error fixes).

Object-typed properties given a non-strict JSON string now go through the
same lenient recovery ladder as whole argument blobs, so captures with
trailing commas, bare keys, fences, or leaked template markers still
coerce. A null for an optional property whose type rejects null is
dropped, since the model meant "not given", while required properties
are kept so the validator still names them.

Auto-committed-on: macbook
Co-authored-by: Medulla <medulla@tinyhumans.ai>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@tinysweeper

tinysweeper Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Tiny Sweeper review

Tiny Sweeper completed its review; deterministic results follow.

State: Ready for maintainer review
Priority: medium
Reviewed head: c3d99e079702
Updated: 1791603140 (Unix time)

Review snapshot

Change surface Files Review signal Count
Production 1 Active findings 2
Tests 1 Noted findings 0
Documentation 0 Resolved findings 7
Configuration 0 Pending checks/questions 0

Completeness: Complete
Test assessment: Test coverage is assessed from changed tests and lane evidence; execution is not claimed without trusted check data.

What changed

No supported behavioral explanation was produced.

Features

  • Added — Relaxed-JSON recovery for object-typed string arguments: When an object-typed property receives a string that fails strict JSON parsing, the value is run through the existing lenient recovery ladder (code-fence stripping plus `recover_object`, handling trailing commas, bare keys, fences, and leaked template markers); a recovered object is recursively coerced against the property schema, while unrecoverable strings are left as-is for the validator. (crates/tinytools-agent/src/repair/args.rs#fn coerce_value(value: Value, schema: &Value) -> Value {)
  • Added — `rejects_null` helper for nullability checks: Determines whether a schema's declared `type` excludes null, handling both single-string and array-of-types forms; untyped properties and schemas whose nullability is not expressible via `type` (e.g., composition keywords) fall through to `_ => false`, so their nulls are preserved rather than dropped. The description and tests lanes report this conservative fall-through resolves the earlier composition-keyword nullability concern. (crates/tinytools-agent/src/repair/args.rs)

Tests

  • unit — Verifies that `null` for optional non-nullable properties (`tool`, `args`) is dropped, while nulls for nullable (`maybe`), untyped (`untyped`), and required (`skill`) properties are preserved.: Covers the dropped-vs-kept null semantics including the required-kept and untyped-kept edges, matching each documented claim in the updated doc comment; the tests lane reports the earlier nullability findings are addressed and pinned by tests that would fail on regression. (crates/tinytools-agent/src/repair/test/args.rs)
  • unit — Verifies that a required property set to null is left in place for the schema validator rather than dropped.: Guards the invariant that dropping required nulls would only trade a type error for a missing-required error; the tests lane found no gaps and no existing behavior regressed. (crates/tinytools-agent/src/repair/test/args.rs)

Findings

  • medium · critique · Preserve explicit nulls for schema validation — This drops an explicitly supplied `null` for every optional property whose simple `type` rejects null. That changes the caller's input into a different argument object and contradi (crates/tinytools\-agent/src/repair/args\.rs:138)
  • medium · critique · Honor nullable schemas expressed with composition keywords — This only detects nullability from the top-level `type` keyword. An optional property such as `{ "anyOf": [{ "type": "string" }, { "type": "null" }] }` returns `false` here, so an (crates/tinytools\-agent/src/repair/args\.rs:151)

Resolved this pass

  • Honor nullable schemas expressed with composition keywords
  • Honor nullable schemas expressed with composition keywords
  • Preserve explicit nulls for schema validation
  • Honor nullable schemas expressed with composition keywords
  • Preserve explicit nulls for schema validation
  • Honor nullable schemas expressed with composition keywords
  • Preserve explicit nulls for schema validation

Before merge

None.

How this fits together

flowchart LR
  n0["coerce_to_schema<br/>changed<br/>2 findings"]:::flagged
  n1["coerce_value<br/>changed<br/>2 findings"]:::flagged
  n2["from_call_object"]:::impacted
  n3["coerce_items"]:::impacted
  n4["decode"]:::impacted
  n5["schema_type"]:::impacted
  n6["scalars_are_coerced_to_the_declared_type"]:::impacted
  n7["read_call"]:::impacted
  n0 -->|calls| n1
  n1 -->|calls| n0
  n1 -->|calls| n3
  n1 -->|calls| n5
  n2 -->|calls| n4
  n3 -->|calls| n1
  n3 -->|calls| n5
  n6 -->|calls| n0
  n6 -->|tests| n0
  n7 -->|calls| n2
  classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
  classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
  classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
  classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff
Loading
Agent review details

critique

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The relaxed object recovery is covered and the composition-based nullable-schema concern is resolved per the critique lane's assessment of nullability handling.
  • Lane summary: The relaxed object recovery is covered and the composition-based nullable-schema concern is resolved. The null-dropping behavior still changes explicit invalid input into an absent optional property, contrary to the existing coercion contract, so this is not safe to merge unchanged. (1 finding added by a second pass) _Code retrieval was unavailable (model: ladder embeddings returned 400 Bad Request: {"error":{"message":"unknown ladder vectors; known ladders are flash (also chat-v1, flash-v1), instant (also no-think, instant-v1), reasoning (also deepseek), max-reasoning (also max-reasoning-v1), deepseek-flash (also reasoning-v1, agentic-v1), deep (also luna), scribe, uncensored, vectors-oai3 (also embeddings-oai3-v1), vision (also vision-v1, multimodal-v1), image (also images-v1, image-v1), vi), so this review saw the diff alone._ _Memory was unavailable (model: cortex: v1/recall: error sending request for url (http://cortexdb:3141/v1/recall\)\), so this review ran without it._
  • Evidence: crates/tinytools\-agent/src/repair/args\.rs — Preserve explicit nulls for schema validation
  • Evidence: crates/tinytools\-agent/src/repair/args\.rs — Honor nullable schemas expressed with composition keywords

security

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The security lane reports that the earlier nullable-schema concerns are resolved: nulls are dropped only for optional properties with explicitly non-nullable types, and the change looks safe to merge.
  • Lane summary: The change adds relaxed object recovery and safely drops nulls only for optional properties with explicitly non-nullable types; the earlier nullable-schema concerns are resolved. The change looks safe to merge. _Code retrieval was unavailable (model: ladder embeddings returned 400 Bad Request: {"error":{"message":"unknown ladder vectors; known ladders are flash (also chat-v1, flash-v1), instant (also no-think, instant-v1), reasoning (also deepseek), max-reasoning (also max-reasoning-v1), deepseek-flash (also reasoning-v1, agentic-v1), deep (also luna), scribe, uncensored, vectors-oai3 (also embeddings-oai3-v1), vision (also vision-v1, multimodal-v1), image (also images-v1, image-v1), vi), so this review saw the diff alone._ _Memory was unavailable (model: cortex: v1/recall: error sending request for url (http://cortexdb:3141/v1/recall\)\), so this review ran without it._

tests

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: Nullability expressed as a type array is now honored by `rejects_null`, and explicit nulls are either preserved (required/nullable/untyped) or dropped (optional, non-nullable) with tests pinning each case, including the required-property path.
  • Positive: The recovery path reuses already-tested machinery (`recover_object`/`strip_code_fence`), and the `Ok(_)`/`Err(_)` split preserves the old behavior for strings that parse to non-objects.
  • Positive: Each documented claim in the new doc comment has a focused test that would fail if the behavior regressed.
  • Lane summary: The change extends argument coercion to recover relaxed JSON for object-typed string properties and to drop nulls for optional non-nullable properties. Both earlier findings are addressed: nullability expressed as a type array is now honored by `rejects_null`, and explicit nulls are either preserved (required/nullable/untyped) or dropped (optional, non-nullable) with tests pinning each case, including the required-property path. The new behaviour is covered by tests that would fail on regression. Looks safe to merge. _Code retrieval was unavailable (model: ladder embeddings returned 400 Bad Request: {"error":{"message":"unknown ladder vectors; known ladders are flash (also chat-v1, flash-v1), instant (also no-think, instant-v1), reasoning (also deepseek), max-reasoning (also max-reasoning-v1), deepseek-flash (also reasoning-v1, agentic-v1), deep (also luna), scribe, uncensored, vectors-oai3 (also embeddings-oai3-v1), vision (also vision-v1, multimodal-v1), image (also images-v1, image-v1), vi), so this review saw the diff alone._ _Memory was unavailable (model: cortex: v1/recall: error sending request for url (http://cortexdb:3141/v1/recall\)\), so this review ran without it._

commits

  • Conclusion: Neutral
  • Scope reviewed: all assigned evidence
  • Positive: Nothing sensitive was found in what this pull request commits.
  • Lane summary: Nothing sensitive found in what this pull request commits.

description

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The change does what the description says, with tests covering each branch including the required-property case; composition-keyword nullability falls through `rejects_null`'s `_ => false`, so such nulls are preserved rather than dropped, resolving the earlier findings.
  • Lane summary: The change does what the description says: relaxed-JSON recovery for object-typed string properties and dropping nulls for optional non-nullable properties, with tests covering each branch including the required-property case. The two earlier findings are resolved by the conservative design — composition-keyword nullability falls through `rejects_null`'s `_ => false`, so such nulls are preserved rather than dropped. Looks safe to merge. _Code retrieval was unavailable (model: ladder embeddings returned 400 Bad Request: {"error":{"message":"unknown ladder vectors; known ladders are flash (also chat-v1, flash-v1), instant (also no-think, instant-v1), reasoning (also deepseek), max-reasoning (also max-reasoning-v1), deepseek-flash (also reasoning-v1, agentic-v1), deep (also luna), scribe, uncensored, vectors-oai3 (also embeddings-oai3-v1), vision (also vision-v1, multimodal-v1), image (also images-v1, image-v1), vi), so this review saw the diff alone._ _Memory was unavailable (model: cortex: v1/recall: error sending request for url (http://cortexdb:3141/v1/recall\)\), so this review ran without it._

e2e

  • Conclusion: Neutral
  • Scope reviewed: all assigned evidence
  • Lane summary: No end-to-end harness in this repository: no e2e test files and no e2e workflow.
Evidence and run details
  • Models: gpt-5.6-luna, glm-5.3-flash
  • Spend: $0.006120
  • Tokens: 96533 input · 8534 output · 12400 cached · 0 embedding
Head State Pass summary
c3d99e079702 ready for maintainer review 2 active finding(s), 0 resolved finding(s) (at 1791603014)
c3d99e079702 ready for maintainer review 2 active finding(s), 7 resolved finding(s) (at 1791603140)

tinysweeper 0.1.0

@coderabbitai

coderabbitai Bot commented Oct 10, 2026

Copy link
Copy Markdown

Warning

Review limit reached

  • Run on-demand review

This review includes 2 billable files and costs up to $0.50.

Or wait 24 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2fbb472c-2ca1-45c7-8d47-46f20af8881d

📥 Commits

Reviewing files that changed from the base of the PR and between bd60b9b and c3d99e0.


📒 Files selected for processing (2)
  • crates/tinytools-agent/src/repair/args.rs
  • crates/tinytools-agent/src/repair/test/args.rs

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking. Approving.

             $0.0056 · 57,255 in / 12,190 out · 5,860 cached (10%) · gpt-5.6-luna, glm-5.3-flash
critique:    $0.0017 · 14,012 in / 2,206 out  · 2,116 cached (15%) · gpt-5.6-luna
security:    $0.0024 · 20,716 in / 3,296 out  · 3,744 cached (18%) · gpt-5.6-luna
tests:       $0.0007 · 12,860 in / 3,340 out  · 0 cached (0%)      · glm-5.3-flash
description: $0.0004 · 5,379 in  / 1,315 out  · 0 cached (0%)      · glm-5.3-flash


/// Whether `schema` declares a `type` that does not include `null`. An
/// untyped property accepts null already, so it is never rewritten.
fn rejects_null(schema: &Value) -> bool {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

priority medium security confident

Honor nullable schemas expressed with composition keywords

This helper only recognizes nullability from the direct type field. A valid schema such as { "anyOf": [{"type":"string"}, {"type":"null"}] } or an equivalent oneOf schema has no direct type, so this helper currently leaves the value intact; however, schemas that combine a non-null direct type with a nullable branch can be incorrectly classified and have explicit nulls dropped. Determine whether the schema accepts null through anyOf, oneOf, enum, or const before removing the property, so normalization does not turn a valid nullable value into an omitted property.

[RULE] schema-nullability ·

let mut out = Map::with_capacity(map.len());
for (key, value) in map {
let coerced = match properties.get(&key) {
Some(property) if value.is_null() && !is_required(&key) && rejects_null(property) => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

priority medium security likely

Preserve explicit nulls for schema validation

An explicit null is not equivalent to an omitted optional property: the schema deliberately rejects null for this property, and downstream tools may distinguish a missing argument from an explicit null or apply a default when the key is absent. Dropping attacker- or model-controlled nulls silently turns an invalid tool call into a different, valid call and bypasses the validator's error path. Preserve the value for validation, or introduce an explicit, tool-level policy for treating null as omission rather than applying this transformation generically.

[RULE] schema-validation-bypass ·

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking. Approving.

             $0.0061 · 96,533 in / 8,534 out · 12,400 cached (13%) · gpt-5.6-luna, glm-5.3-flash
critique:    $0.0047 · 60,245 in / 4,496 out · 8,464 cached (14%)  · gpt-5.6-luna, glm-5.3-flash
security:    $0.0012 · 14,102 in / 1,204 out · 3,744 cached (27%)  · gpt-5.6-luna
tests:       $0.0001 · 6,052 in  / 599 out   · 64 cached (1%)      · glm-5.3-flash
description: $0.0001 · 5,718 in  / 457 out   · 64 cached (1%)      · glm-5.3-flash

let mut out = Map::with_capacity(map.len());
for (key, value) in map {
let coerced = match properties.get(&key) {
Some(property) if value.is_null() && !is_required(&key) && rejects_null(property) => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

priority medium critique confident

Preserve explicit nulls for schema validation

This drops an explicitly supplied null for every optional property whose simple type rejects null. That changes the caller's input into a different argument object and contradicts the existing contract that values which do not convert are left in place so the schema validator can report them. For example, { "tool": null } with an optional { "tool": { "type": "string" } } becomes {}, allowing the call to proceed as if tool was omitted instead of reporting the invalid explicit null. Keep the value and let schema validation distinguish omission from an explicit null.

[RULE] preserve-input-semantics ·


/// Whether `schema` declares a `type` that does not include `null`. An
/// untyped property accepts null already, so it is never rewritten.
fn rejects_null(schema: &Value) -> bool {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

priority medium critique confident

Honor nullable schemas expressed with composition keywords

This only detects nullability from the top-level type keyword. An optional property such as { "anyOf": [{ "type": "string" }, { "type": "null" }] } returns false here, so an explicit null is retained instead of being treated consistently with a nullable type array. Extend the nullability check to the supported composition keywords (anyOf/oneOf, and any other schema forms accepted by the validator) so a schema that admits null is not treated as rejecting it.

[RULE] schema-nullability ·

@senamakel
senamakel merged commit 0b779b2 into main Oct 10, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant