Skip to content

Add validated advisory tool recovery decisions - #61

Closed
senamakel wants to merge 6 commits into
mainfrom
jev-tool-recovery
Closed

senamakel wants to merge 6 commits into
mainfrom
jev-tool-recovery

Conversation

@senamakel

@senamakel senamakel commented Oct 10, 2026 •

Copy link
Copy Markdown
Member

Tool recovery needs a different decision contract from tool selection: whether an admitted next attempt can recover, which failure class the evidence supports, and whether a concrete correction or authorized alternate is useful. This adds that contract to tinytools-jev without changing the existing tool-search evaluator.

The new transport-neutral adviser builds bounded Choice/Noul/optional Score questions, validates complete typed answer batches, and abstains on trusted terminal facts, uncertain/applied writes, malformed answers, contradictions, low confidence and unavailable evaluators. Alternate questions require explicit wrong-tool advice or a calibrated repeated blocker and include none. The API never executes tools, grants permission or modifies host accounting. Hosts supply scrubbed observations and candidates and retain transport, cancellation, the total deadline, policy and fallback.

Part of tinyhumansai/openhuman#7341. This is the upstream library dependency; OpenHuman transport/configuration and host-driver wiring are separate. Existing argument normalization and unknown-tool suggestions are reused; broad coercion, automatic retries and rollout/default changes are outside this library change.

Validation: workspace format, clippy (--all-targets --all-features -- -D warnings), build (--all-targets --all-features), tests (--all-features) and rustdoc (-D warnings) passed. Recovery production-file line coverage is 97.50–100% after validator hardening. The Jev suite passes 49 unit tests, one public contract integration test and one doctest. Regression tests confirmed duplicate-option, empty-batch and zero-recoverability-threshold failures before their fixes. Added fake-evaluator tests and a public contract test; the original 20 Jev ranking tests pass unchanged. The initial public recovery test failed with the absent API before implementation.

Co-authored-by: Medulla <medulla@tinyhumans.ai>
@coderabbitai

coderabbitai Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 15790060-2542-457a-a07a-c7ed266933b1







📥 Commits

Reviewing files that changed from the base of the PR and between 360721b and 6a0bb43.








📒 Files selected for processing (13)
  • README.md
  • crates/tinytools-jev/README.md
  • crates/tinytools-jev/src/lib.rs
  • crates/tinytools-jev/src/recovery/README.md
  • crates/tinytools-jev/src/recovery/mod.rs
  • crates/tinytools-jev/src/recovery/mod_tests.rs
  • crates/tinytools-jev/src/recovery/questions.rs
  • crates/tinytools-jev/src/recovery/types.rs
  • crates/tinytools-jev/src/recovery/validation.rs
  • crates/tinytools-jev/src/recovery/validation_tests.rs
  • crates/tinytools-jev/tests/recovery_contract.rs
  • docs/plans/tool-recovery-advice.md
  • docs/specs/tool-recovery-advice.md







Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.









📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough
📝 Walkthrough

Walkthrough

The change adds a public, provider-neutral recovery module to tinytools-jev. It defines typed observations, requests, evaluator decisions, validation rules, and an adviser that returns advice or abstains. Documentation and tests describe and exercise the contract.

Changes

Recovery advice

Layer / File(s) Summary
Recovery model and request contract
crates/tinytools-jev/src/recovery/types.rs, crates/tinytools-jev/src/recovery/questions.rs, crates/tinytools-jev/src/lib.rs, crates/tinytools-jev/tests/recovery_contract.rs, crates/tinytools-jev/src/recovery/mod_tests.rs, crates/tinytools-jev/src/recovery/README.md, crates/tinytools-jev/README.md, README.md, docs/specs/tool-recovery-advice.md, docs/plans/tool-recovery-advice.md
The public API defines recovery classes, observations, typed questions and answers, evaluator inputs, thresholds, and advice. Request construction adds classification and recoverability questions, with optional correction and alternate questions. Documentation describes input limits, abstention, and host responsibilities.
Decision validation and advice interpretation
crates/tinytools-jev/src/recovery/validation.rs, crates/tinytools-jev/src/recovery/validation_tests.rs, crates/tinytools-jev/src/recovery/mod_tests.rs, crates/tinytools-jev/src/recovery/README.md
Validation checks observations, thresholds, answer types, IDs, and probability distributions. Advice interpretation applies confidence, recoverability, classification, correction, and alternate-selection rules. Tests cover invalid inputs and decisions, abstentions, and alternate-advice conditions.
Adviser evaluation flow
crates/tinytools-jev/src/recovery/mod.rs, crates/tinytools-jev/src/recovery/mod_tests.rs, crates/tinytools-jev/src/recovery/validation_tests.rs, crates/tinytools-jev/src/recovery/README.md
The adviser skips evaluation for specified trusted facts and effects, evaluates eligible requests once, and maps evaluator errors to abstention. Tests cover evaluator calls, advice outcomes, and cancellation of a pending evaluation.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Host
  participant RecoveryAdviser
  participant RecoveryRequest
  participant RecoveryEvaluator
  participant RecoveryDecision
  Host->>RecoveryAdviser: provide observation
  RecoveryAdviser->>RecoveryRequest: build validated request
  RecoveryAdviser->>RecoveryEvaluator: evaluate request
  RecoveryEvaluator-->>RecoveryAdviser: return decision or error
  RecoveryAdviser->>RecoveryDecision: validate and interpret decision
  RecoveryDecision-->>RecoveryAdviser: return advice or abstention
Loading






















Merge Risk: ⚪ Minimal · up to 6a0bb

No actionable recovery-advice issue remains identified; the change is mergeable after normal checks.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 6a0bb

The new API is opt-in and cannot execute tools or grant permission. Risk is limited, but safe integration depends on the host scrubbing observations, authorizing candidates, reconciling uncertain effects and enforcing cancellation and deadlines.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The added library-level exposure is observation and candidate data reaching a host-provided evaluator, followed by advisory output reaching the host. Recovery itself acquires no execution, credential or authorization mechanism. Any downstream tenant, asset or service exposure depends on host integration that is not established here.

Trust Boundaries and Controls

  • observed — Failure text and descriptions remain untrusted evidence. Validation limits prose to 2048 bytes per field, candidate keys to 128 bytes and candidates to 16, rejecting duplicate or reserved keys. These bounds neither scrub secrets nor authorize candidates; both obligations remain with the host before evaluator disclosure.
  • observed — Evaluator answers must match the complete requested ID, type and option sets and contain finite normalized distributions. Interpretation rejects unknown or tied classifications and contradictory prerequisite advice. Accepted alternates remain within supplied options and require the appropriate eligibility and recoverability checks; they cannot override trusted denial facts.

Resilience and Maintainability Implications

  • observed — The adviser has no internal timeout or retry. Hosts must enforce the remaining total deadline, cancellation, stable budgets and failed-invocation accounting. The evaluator contract requires dropped futures not to initiate further provider work; compliance by external implementations was not verified.











Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 70.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 50 functions across 8 files. (5 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title is concise and clearly describes the main change: adding validated advisory tool recovery decisions.

Full details: Docstring Coverage

Explanation

Docstring coverage is 70.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 50 functions across 8 files. (5 skipped: 5 unsupported.)


  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR













  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

I’m a rabbit with a question to send,
I ask what the failure might mean.
A typed little answer comes back,
Or abstention keeps choices in check.
No retry hops out on its own,
I nibble a carrot and leave control home.

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-10T18:43:19.063556Z 6a0bb43 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a621c8ae14

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/tinytools-jev/src/recovery/validation.rs Outdated
@tinysweeper

tinysweeper Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Tiny Sweeper review

⚠️ Review failed for 70b6582f1c87. forge: GitHub: API rate limit exceeded for installation ID 152184043. If you reach out to GitHub Support for help, please include the request ID 9EF2:258071:615004:14373E7:6ACA8EC3 and timestamp 2026-10-10 19:15:15 UTC. For more on scraping GitHub and how it may affect your rights, please review our Terms of Service (https://docs\.github\.com/en/site\-policy/github\-terms/github\-terms\-of\-service\)
Documentation URL: https://docs\.github\.com/en/rest/using\-the\-rest\-api/getting\-started\-with\-the\-rest\-api\#rate\-limiting

Last completed report

Tiny Sweeper review

Tiny Sweeper completed its review; deterministic results follow.

State: Changes requested
Priority: critical
Reviewed head: 760beb4e2e32
Updated: 1791659084 (Unix time)

Review snapshot

Change surface Files Review signal Count
Production 5 Active findings 7
Tests 4 Noted findings 0
Documentation 5 Resolved findings 328
Configuration 0 Pending checks/questions 0

Completeness: Complete
Test assessment: Test coverage is assessed from changed tests and lane evidence; execution is not claimed without trusted check data.

What changed

A new additive `recovery` module is added to tinytools-jev, providing bounded host observations, typed independent questions, strict decision validation and a conservative adviser, plus README/spec/plan documentation and an integration contract test. No removals or modifications to existing behavior were reported.

Features

  • Added — Bounded recovery observations and typed questions: Hosts supply scrubbed phase, effect facts, counts, argument shape and diagnostics; RecoveryRequest::new produces stable independent Choice/Noul questions with an optional Score question when a concrete correction is evidenced, and rejects oversized/empty/reserved inputs without truncation. (crates/tinytools-jev/src/recovery/types.rs, crates/tinytools-jev/src/recovery/questions.rs, crates/tinytools-jev/src/recovery/validation.rs)
  • Added — RecoveryAdviser with trusted-fact short-circuits and threshold validation: RecoveryAdviser::new validates versioned calibrated thresholds; RecoveryAdviser::advise validates observations, abstains on trusted success/denial/cancellation/terminal/auth/permission facts and applied/uncertain writes before evaluation, clears an unmet RepeatedBlocker reason, and abstains on evaluator failure without interpreting trusted host facts. (crates/tinytools-jev/src/recovery/mod.rs, crates/tinytools-jev/src/recovery/validation.rs)
  • Added — Strict decision validation and conservative interpretation: RecoveryDecision::validate checks answer IDs/types, exact option sets, duplicate question IDs and choice keys, finite bounded probabilities/confidence, three-position Score rubrics before indexing, normalized distributions (sum within 1e-6) and nonzero attempts; interpretation abstains on unknown classes, ties, low confidence, prerequisite-class contradictions and unqualified alternate/correction advice. (crates/tinytools-jev/src/recovery/validation.rs, crates/tinytools-jev/src/recovery/types.rs)
  • Added — Alternate candidate gating and correction scoring: Alternate questions are enabled only by explicit prior wrong-tool advice or a counted repeated blocker; accepted alternates require recoverability plus wrong-tool classification or a counted blocker; correction scores are ordered rubric expectations (0..=2), never success probabilities. (crates/tinytools-jev/src/recovery/mod.rs, crates/tinytools-jev/src/recovery/validation.rs, crates/tinytools-jev/src/recovery/questions.rs)
  • Added — Documentation of the bounded advisory contract: READMEs and spec/plan documents describe host responsibilities, byte/count limits, abstention reasons, the existing-result boundary and the fact that the API never executes or authorizes retries. (README.md, crates/tinytools-jev/README.md, crates/tinytools-jev/src/recovery/README.md, docs/specs/tool-recovery-advice.md, docs/plans/tool-recovery-advice.md)

Tests

  • unit — Trusted facts (success, policy/approval denial, expiry, cancellation, terminal fault, authentication, permission) and applied/uncertain writes skip evaluation without calling the evaluator.: Adequate; asserts the evaluator was never invoked. (crates/tinytools-jev/src/recovery/mod_tests.rs)
  • unit — Malformed answers (missing/extra IDs, wrong variants, NaN/infinite/out-of-range probabilities and confidence, unknown/missing option keys, zero attempts) abstain as InvalidDecision.: Adequate. (crates/tinytools-jev/src/recovery/mod_tests.rs)
  • unit — Low class confidence, unknown class, tied class probabilities and evaluator failure abstain with the corresponding reasons.: Adequate. (crates/tinytools-jev/src/recovery/mod_tests.rs)
  • unit — Oversized, empty, whitespace, control-character, reserved (none) and duplicate candidate keys, candidate-count and fact-count limits, and oversized/empty prose fields are rejected.: Adequate, including byte-limit boundaries. (crates/tinytools-jev/src/recovery/mod_tests.rs, crates/tinytools-jev/src/recovery/validation_tests.rs)
  • unit — Prerequisite-class contradictions, wrong alternate answers, low recoverability (abstaining as LowConfidence) and invalid optional Score/alternate answers abstain.: Adequate; previously flagged weaknesses (duplicate question-ID/choice-key tests and low-recoverability abstention reason) are now addressed with dedicated tests. (crates/tinytools-jev/src/recovery/validation_tests.rs, crates/tinytools-jev/src/recovery/mod_tests.rs)
  • unit — Threshold validation (nonfinite/out-of-range values, empty version, zero repeated blocker) and observation boundaries are explicit.: Adequate. (crates/tinytools-jev/src/recovery/validation_tests.rs)
  • unit — Correction Score is an ordered expectation weighted across all rubric positions, with rubric length enforced before indexing.: Adequate; earlier rubric-length finding is resolved with score_rubric_must_have_exactly_three_positions and short/long score answer tests. (crates/tinytools-jev/src/recovery/validation_tests.rs)
  • contract — Recovery questions remain distinct from tool selection in the public API surface.: Adequate; integration test asserts a two-question batch and stable class keys. (crates/tinytools-jev/tests/recovery_contract.rs)

Findings

  • medium · critique · Build an alternate-bearing request for the blocker gate test — This request is built from an input with `repeated_failures = 1`, while the shared threshold is 2. The request therefore does not contain an alternate question, so `choose(&mut ans (crates/tinytools\-jev/src/recovery/validation\_tests\.rs:493)
  • medium · critique · Assert the questions are actually distinct and well-formed — Checking only the vector length allows two duplicate, incorrectly typed, or otherwise invalid questions to pass, so this test does not establish that recovery questions are distinc (crates/tinytools\-jev/tests/recovery\_contract\.rs:12)
  • critical · security · Add the declared recovery module — This declares a public `recovery` module, but the reviewed tree contains no `crates/tinytools-jev/src/recovery.rs` or corresponding module directory. The crate therefore cannot com (crates/tinytools\-jev/src/lib\.rs:19)
  • medium · security · Exercise the blocker gate for every recovery class — This regression test checks the repeated-blocker gate only for `transient` and `wrong_tool`. The request and earlier tests define additional classifications such as `authentication (crates/tinytools\-jev/src/recovery/validation\_tests\.rs:460)
  • medium · security · Build score probabilities from the requested rubric — The fixture always emits exactly three score probabilities, regardless of the request's rubric. Any test that uses an empty, shortened, or extended rubric receives a decision with (crates/tinytools\-jev/src/recovery/test\_support\.rs:49)
  • medium · security · Preserve duplicate question IDs in malformed decisions — Collecting question answers into a `BTreeMap` silently overwrites an earlier answer when a request contains duplicate question IDs. The resulting fixture is no longer the malformed (crates/tinytools\-jev/src/recovery/test\_support\.rs:54)
  • medium · security · Preserve duplicate choice keys in malformed fixtures — The probabilities are collected into a map, so duplicate option keys are collapsed before the decision reaches validation. This prevents tests from preserving and checking a duplic (crates/tinytools\-jev/src/recovery/test\_support\.rs:35)

Resolved this pass

  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • Treat low recoverability as low confidence for corrections
  • Keep the answer populated when testing empty rubrics
  • Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Validate the score rubric before indexing probabilities
  • Enforce the blocker threshold without a class exception
  • Apply the repeated-blocker threshold to every recovery class
  • Test duplicate question IDs without an unknown answer
  • Test duplicate choice keys without an unknown probability key
  • Enforce the repeated-blocker threshold for every class
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Convert the boolean choice explicitly to a float
  • Treat low recoverability as low confidence for corrections
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Remove the wrong-tool exception from blocker thresholds
  • Exercise the blocker gate for every recovery class
  • Preserve duplicate question IDs in malformed decisions
  • Preserve duplicate choice keys in malformed fixtures
  • Build score probabilities from the requested rubric
  • Require the blocker threshold for every alternate reason
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • Treat low recoverability as low confidence for corrections
  • Keep the answer populated when testing empty rubrics
  • Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Validate the score rubric before indexing probabilities
  • Enforce the blocker threshold without a class exception
  • Apply the repeated-blocker threshold to every recovery class
  • Test duplicate question IDs without an unknown answer
  • Test duplicate choice keys without an unknown probability key
  • Enforce the repeated-blocker threshold for every class
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Convert the boolean choice explicitly to a float
  • Treat low recoverability as low confidence for corrections
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Remove the wrong-tool exception from blocker thresholds
  • Exercise the blocker gate for every recovery class
  • Preserve duplicate question IDs in malformed decisions
  • Preserve duplicate choice keys in malformed fixtures
  • Build score probabilities from the requested rubric
  • Require the blocker threshold for every alternate reason
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • Treat low recoverability as low confidence for corrections
  • Keep the answer populated when testing empty rubrics
  • Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Enforce the blocker threshold without a class exception
  • Apply the repeated-blocker threshold to every recovery class
  • Test duplicate question IDs without an unknown answer
  • Test duplicate choice keys without an unknown probability key
  • Enforce the repeated-blocker threshold for every class
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Convert the boolean choice explicitly to a float
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Remove the wrong-tool exception from blocker thresholds
  • Exercise the blocker gate for every recovery class
  • Preserve duplicate question IDs in malformed decisions
  • Preserve duplicate choice keys in malformed fixtures
  • Build score probabilities from the requested rubric
  • Require the blocker threshold for every alternate reason
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • Build score probabilities from the requested rubric
  • Keep the answer populated when testing empty rubrics
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • Treat low recoverability as low confidence for corrections
  • Keep the answer populated when testing empty rubrics
  • Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Validate the score rubric before indexing probabilities
  • Test duplicate question IDs without an unknown answer
  • Test duplicate choice keys without an unknown probability key
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Critical — Convert the boolean choice explicitly to a float
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Exercise the blocker gate for every recovery class
  • Preserve duplicate question IDs in malformed decisions
  • Preserve duplicate choice keys in malformed fixtures
  • Build score probabilities from the requested rubric
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • Treat low recoverability as low confidence for corrections
  • Keep the answer populated when testing empty rubrics
  • Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Validate the score rubric before indexing probabilities
  • Enforce the blocker threshold without a class exception
  • Apply the repeated-blocker threshold to every recovery class
  • Test duplicate question IDs without an unknown answer
  • Test duplicate choice keys without an unknown probability key
  • Enforce the repeated-blocker threshold for every class
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Critical — Convert the boolean choice explicitly to a float
  • Treat low recoverability as low confidence for corrections
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Remove the wrong-tool exception from blocker thresholds
  • Exercise the blocker gate for every recovery class
  • Preserve duplicate question IDs in malformed decisions
  • Preserve duplicate choice keys in malformed fixtures
  • Build score probabilities from the requested rubric
  • Require the blocker threshold for every alternate reason
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • Treat low recoverability as low confidence for corrections
  • Keep the answer populated when testing empty rubrics
  • Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Validate the score rubric before indexing probabilities
  • Enforce the blocker threshold without a class exception
  • Apply the repeated-blocker threshold to every recovery class
  • Test duplicate question IDs without an unknown answer
  • Test duplicate choice keys without an unknown probability key
  • Enforce the repeated-blocker threshold for every class
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Convert the boolean choice explicitly to a float
  • Treat low recoverability as low confidence for corrections
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Remove the wrong-tool exception from blocker thresholds
  • Exercise the blocker gate for every recovery class
  • Preserve duplicate question IDs in malformed decisions
  • Preserve duplicate choice keys in malformed fixtures
  • Build score probabilities from the requested rubric
  • Require the blocker threshold for every alternate reason
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • Treat low recoverability as low confidence for corrections
  • Keep the answer populated when testing empty rubrics
  • Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Validate the score rubric before indexing probabilities
  • Enforce the blocker threshold without a class exception
  • Apply the repeated-blocker threshold to every recovery class
  • Test duplicate question IDs without an unknown answer
  • Test duplicate choice keys without an unknown probability key
  • Enforce the repeated-blocker threshold for every class
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Convert the boolean choice explicitly to a float
  • Treat low recoverability as low confidence for corrections
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Remove the wrong-tool exception from blocker thresholds
  • Exercise the blocker gate for every recovery class
  • Preserve duplicate question IDs in malformed decisions
  • Preserve duplicate choice keys in malformed fixtures
  • Require the blocker threshold for every alternate reason
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • Treat low recoverability as low confidence for corrections
  • Keep the answer populated when testing empty rubrics
  • Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Enforce the blocker threshold without a class exception
  • Apply the repeated-blocker threshold to every recovery class
  • Test duplicate question IDs without an unknown answer
  • Test duplicate choice keys without an unknown probability key
  • Enforce the repeated-blocker threshold for every class
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Convert the boolean choice explicitly to a float
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Remove the wrong-tool exception from blocker thresholds
  • Exercise the blocker gate for every recovery class
  • Preserve duplicate question IDs in malformed decisions
  • Preserve duplicate choice keys in malformed fixtures
  • Build score probabilities from the requested rubric
  • Require the blocker threshold for every alternate reason
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • Treat low recoverability as low confidence for corrections
  • Keep the answer populated when testing empty rubrics
  • Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Enforce the blocker threshold without a class exception
  • Apply the repeated-blocker threshold to every recovery class
  • Test duplicate question IDs without an unknown answer
  • Test duplicate choice keys without an unknown probability key
  • Enforce the repeated-blocker threshold for every class
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Convert the boolean choice explicitly to a float
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Remove the wrong-tool exception from blocker thresholds
  • Exercise the blocker gate for every recovery class
  • Preserve duplicate question IDs in malformed decisions
  • Preserve duplicate choice keys in malformed fixtures
  • Build score probabilities from the requested rubric
  • Require the blocker threshold for every alternate reason
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • Treat low recoverability as low confidence for corrections
  • Keep the answer populated when testing empty rubrics
  • Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Enforce the blocker threshold without a class exception
  • Apply the repeated-blocker threshold to every recovery class
  • Test duplicate question IDs without an unknown answer
  • Test duplicate choice keys without an unknown probability key
  • Enforce the repeated-blocker threshold for every class
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Convert the boolean choice explicitly to a float
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Remove the wrong-tool exception from blocker thresholds
  • Exercise the blocker gate for every recovery class
  • Preserve duplicate question IDs in malformed decisions
  • Preserve duplicate choice keys in malformed fixtures
  • Build score probabilities from the requested rubric
  • Require the blocker threshold for every alternate reason
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • medium — Treat low recoverability as low confidence for corrections
  • medium — Keep the answer populated when testing empty rubrics
  • medium — Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Validate the score rubric before indexing probabilities
  • Enforce the blocker threshold without a class exception
  • Apply the repeated-blocker threshold to every recovery class
  • Enforce the repeated-blocker threshold for every class
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Remove the wrong-tool exception from blocker thresholds
  • Exercise the blocker gate for every recovery class
  • Require the blocker threshold for every alternate reason
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Treat low recoverability as low confidence for corrections
  • Keep the answer populated when testing empty rubrics
  • Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Enforce the blocker threshold without a class exception
  • Apply the repeated-blocker threshold to every recovery class
  • Test duplicate question IDs without an unknown answer
  • Test duplicate choice keys without an unknown probability key
  • Enforce the repeated-blocker threshold for every class
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Remove the wrong-tool exception from blocker thresholds
  • Exercise the blocker gate for every recovery class
  • Preserve duplicate question IDs in malformed decisions
  • Preserve duplicate choice keys in malformed fixtures
  • Build score probabilities from the requested rubric
  • Require the blocker threshold for every alternate reason
  • Require the three-entry rubric before indexing probabilities
  • Validate the score rubric before indexing probabilities
  • Validate rubric length before indexing score probabilities
  • Treat low recoverability as low confidence for corrections
  • Keep the answer populated when testing empty rubrics
  • Treat low recoverability as low confidence
  • Require a three-category rubric before indexing scores
  • Validate the score rubric before indexing probabilities
  • Enforce the blocker threshold without a class exception
  • Apply the repeated-blocker threshold to every recovery class
  • Test duplicate question IDs without an unknown answer
  • Test duplicate choice keys without an unknown probability key
  • Enforce the repeated-blocker threshold for every class
  • Abstain for low recoverability instead of rejecting the decision
  • Exercise the repeated-blocker gate through an alternate answer
  • Test the repeated-blocker gate through an alternate answer
  • Abstain when recoverability is below the advice threshold
  • Exercise repeated-blocker gating for every recovery class
  • Test the repeated-blocker threshold boundary
  • Remove the wrong-tool exception from blocker thresholds
  • Exercise the blocker gate for every recovery class
  • Preserve duplicate question IDs in malformed decisions
  • Preserve duplicate choice keys in malformed fixtures
  • Build score probabilities from the requested rubric
  • Require the blocker threshold for every alternate reason

Before merge

  • Address Add the declared recovery module (crates/tinytools\-jev/src/lib\.rs).

How this fits together

flowchart LR
  n0["rank_detailed"]:::impacted
  n1["ToolRanker"]:::impacted
  n2["shortlist"]:::impacted
  n3["RankCandidate"]:::impacted
  n4["build_request"]:::impacted
  n5["decode"]:::impacted
  n0 -->|calls| n2
  n0 -->|uses| n3
  n0 -->|calls| n4
  n0 -->|calls| n5
  n1 -->|uses| n3
  n2 -->|uses| n3
  n4 -->|uses| n3
  n5 -->|uses| n3
  classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
  classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
  classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
  classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff
Loading
Agent review details

critique

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Lane summary: Reviewed 14 files; 4 findings. (1 already reported on an earlier push) _Code retrieval was unavailable (model: ladder embeddings returned 400 Bad Request: {"error":{"message":"unknown ladder vectors; known ladders are flash (also chat-v1, flash-v1), instant (also no-think, instant-v1), reasoning (also deepseek), max-reasoning (also max-reasoning-v1), deepseek-flash (also reasoning-v1, agentic-v1), deep (also luna), scribe, uncensored, vectors-oai3 (also embeddings-oai3-v1), vision (also vision-v1, multimodal-v1), image (also images-v1, image-v1), vi), so this review saw the diff alone._ _Memory was unavailable (model: cortex: v1/recall: error sending request for url (http://cortexdb:3141/v1/recall\)\), so this review ran without it._
  • Evidence: crates/tinytools\-jev/src/recovery/validation\_tests\.rs — Build an alternate-bearing request for the blocker gate test
  • Evidence: crates/tinytools\-jev/tests/recovery\_contract\.rs — Assert the questions are actually distinct and well-formed

security

  • Conclusion: Failure
  • Scope reviewed: all assigned evidence
  • Positive: The module consistently treats observation text as untrusted data, requires host-side scrubbing, excludes raw argument values and secrets, and never grants authorization or execution.
  • Positive: Input bounds reject rather than truncate oversized data, limiting injection surface without silently changing meaning.
  • Lane summary: Reviewed 9 files; 8 findings. 5 files were not security-reviewed: README.md (prose or tabular data), crates/tinytools-jev/README.md (prose or tabular data), crates/tinytools-jev/src/recovery/README.md (prose or tabular data), docs/plans/tool-recovery-advice.md (prose or tabular data), docs/specs/tool-recovery-advice.md (prose or tabular data). (1 already reported on an earlier push) _Code retrieval was unavailable (model: ladder embeddings returned 400 Bad Request: {"error":{"message":"unknown ladder vectors; known ladders are flash (also chat-v1, flash-v1), instant (also no-think, instant-v1), reasoning (also deepseek), max-reasoning (also max-reasoning-v1), deepseek-flash (also reasoning-v1, agentic-v1), deep (also luna), scribe, uncensored, vectors-oai3 (also embeddings-oai3-v1), vision (also vision-v1, multimodal-v1), image (also images-v1, image-v1), vi), so this review saw the diff alone._ _Memory was unavailable (model: cortex: v1/recall: error sending request for url (http://cortexdb:3141/v1/recall\)\), so this review ran without it._
  • Evidence: crates/tinytools\-jev/src/lib\.rs — Add the declared recovery module
  • Evidence: crates/tinytools\-jev/src/recovery/validation\_tests\.rs — Exercise the blocker gate for every recovery class
  • Evidence: crates/tinytools\-jev/src/recovery/test\_support\.rs — Build score probabilities from the requested rubric
  • Evidence: crates/tinytools\-jev/src/recovery/test\_support\.rs — Preserve duplicate question IDs in malformed decisions
  • Evidence: crates/tinytools\-jev/src/recovery/test\_support\.rs — Preserve duplicate choice keys in malformed fixtures

tests

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: Tests pin conservative abstention semantics, threshold boundaries and future cancellation with assertions that can genuinely fail.
  • Positive: Previously raised blocking findings (rubric-length validation, duplicate IDs/keys, low-recoverability abstention reason, repeated-blocker gating) are each covered by new tests in this revision, per the tests lane.
  • Lane summary: The recovery module implements a bounded, advisory decision contract with strict validation, and the previously raised concerns — score rubric length validation, low-recoverability abstention for corrections and alternates, repeated-blocker gating across all classes, and duplicate ID/key coverage — are all fixed and pinned by tests in this diff. The change looks sound and safe to merge; nothing new blocks it. (2 earlier finding(s) still open) _Code retrieval was unavailable (model: ladder embeddings returned 400 Bad Request: {"error":{"message":"unknown ladder vectors; known ladders are flash (also chat-v1, flash-v1), instant (also no-think, instant-v1), reasoning (also deepseek), max-reasoning (also max-reasoning-v1), deepseek-flash (also reasoning-v1, agentic-v1), deep (also luna), scribe, uncensored, vectors-oai3 (also embeddings-oai3-v1), vision (also vision-v1, multimodal-v1), image (also images-v1, image-v1), vi), so this review saw the diff alone._ _Memory was unavailable (model: cortex: v1/recall: error sending request for url (http://cortexdb:3141/v1/recall\)\), so this review ran without it._

commits

  • Conclusion: Neutral
  • Scope reviewed: all assigned evidence
  • Lane summary: Nothing sensitive found in what this pull request commits.

description

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The description accurately matches the diff: additive advisory module with bounded typed questions, strict validation, abstention semantics and no execution/authorization capability.
  • Lane summary: The advisory recovery contract in tinytools-jev matches its description: bounded observations, typed questions, strict validation, calibrated thresholds and conservative abstentions are all implemented, and every earlier concern — rubric length before indexing, low-recoverability abstention as LowConfidence, duplicate question IDs/choice keys, and the repeated-blocker threshold applying to every class including wrong-tool — is now fixed and covered by tests. The change looks sound and ready to merge. (1 earlier finding(s) still open) _Code retrieval was unavailable (model: ladder embeddings returned 400 Bad Request: {"error":{"message":"unknown ladder vectors; known ladders are flash (also chat-v1, flash-v1), instant (also no-think, instant-v1), reasoning (also deepseek), max-reasoning (also max-reasoning-v1), deepseek-flash (also reasoning-v1, agentic-v1), deep (also luna), scribe, uncensored, vectors-oai3 (also embeddings-oai3-v1), vision (also vision-v1, multimodal-v1), image (also images-v1, image-v1), vi), so this review saw the diff alone._ _Memory was unavailable (model: cortex: v1/recall: error sending request for url (http://cortexdb:3141/v1/recall\)\), so this review ran without it._

e2e

  • Conclusion: Neutral
  • Scope reviewed: all assigned evidence
  • Lane summary: No end-to-end harness in this repository: no e2e test files and no e2e workflow.
Evidence and run details
  • Models: gpt-5.6-luna, glm-5.3-flash
  • Spend: $0.058323
  • Tokens: 734167 input · 66102 output · 126360 cached · 0 embedding
  • Continuity: summary cache chain restarted at the storage ceiling.
Head State Pass summary
f99abbf3445c changes requested 22 active finding(s), 37 resolved finding(s) (at 1791657079)
6a0bb436f8d8 ready for maintainer review 1 active finding(s), 105 resolved finding(s) (at 1791657854)
760beb4e2e32 changes requested 6 active finding(s), 140 resolved finding(s) (at 1791658532)
760beb4e2e32 ready for maintainer review 7 active finding(s), 287 resolved finding(s) (at 1791658768)
760beb4e2e32 changes requested 7 active finding(s), 328 resolved finding(s) (at 1791659084)

tinysweeper 0.1.0

Co-authored-by: Medulla <medulla@tinyhumans.ai>

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: 2 lane(s) blocking, worst finding is high.

Fix or reply to the findings below and push. The next review clears this automatically once they are gone — you should not need to dismiss anything by hand.

             $0.0293 · 415,345 in / 35,480 out · 52,525 cached (13%) · gpt-5.6-luna, glm-5.3-flash
critique:    $0.0148 · 195,979 in / 18,574 out · 32,596 cached (17%) · gpt-5.6-luna, glm-5.3-flash
security:    $0.0138 · 145,269 in / 12,413 out · 19,673 cached (14%) · gpt-5.6-luna
tests:       $0.0002 · 24,105 in  / 1,067 out  · 64 cached (0%)      · glm-5.3-flash
description: $0.0002 · 23,723 in  / 1,300 out  · 64 cached (0%)      · glm-5.3-flash

Comment thread crates/tinytools-jev/src/recovery/validation.rs Outdated
Comment thread crates/tinytools-jev/src/recovery/validation.rs Outdated
Comment thread crates/tinytools-jev/src/recovery/validation_tests.rs
Comment thread crates/tinytools-jev/src/recovery/validation_tests.rs
Comment thread crates/tinytools-jev/src/recovery/validation.rs Outdated
Comment thread crates/tinytools-jev/src/recovery/validation.rs
Comment thread crates/tinytools-jev/src/recovery/validation_tests.rs
Comment thread crates/tinytools-jev/src/recovery/validation_tests.rs
Comment thread crates/tinytools-jev/src/recovery/validation_tests.rs
Comment thread crates/tinytools-jev/src/recovery/validation.rs Outdated
senamakel and others added 2 commits October 10, 2026 21:38
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Co-authored-by: Medulla <medulla@tinyhumans.ai>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6a0bb436f8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/tinytools-jev/src/recovery/mod_tests.rs Outdated

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previously-blocking findings are resolved. Clearing the changes request.

             $0.0194 · 286,431 in / 21,193 out · 32,146 cached (11%) · gpt-5.6-luna, glm-5.3-flash
critique:    $0.0103 · 120,477 in / 10,016 out · 16,554 cached (14%) · gpt-5.6-luna, glm-5.3-flash
security:    $0.0084 · 88,150 in  / 7,699 out  · 12,520 cached (14%) · gpt-5.6-luna
tests:       $0.0002 · 25,539 in  / 851 out    · 1,536 cached (6%)   · glm-5.3-flash
description: $0.0002 · 25,157 in  / 676 out    · 1,408 cached (6%)   · glm-5.3-flash

Comment thread crates/tinytools-jev/src/recovery/validation_tests.rs
Co-authored-by: Medulla <medulla@tinyhumans.ai>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: 1 lane(s) blocking, worst finding is critical.

Fix or reply to the findings below and push. The next review clears this automatically once they are gone — you should not need to dismiss anything by hand.

             $0.0369 · 504,425 in / 36,967 out · 56,764 cached (11%) · gpt-5.6-luna, glm-5.3-flash
critique:    $0.0163 · 209,061 in / 17,512 out · 26,751 cached (13%) · gpt-5.6-luna, glm-5.3-flash
security:    $0.0200 · 216,119 in / 16,487 out · 26,941 cached (12%) · gpt-5.6-luna
tests:       $0.0002 · 25,964 in  / 812 out    · 1,536 cached (6%)   · glm-5.3-flash
description: $0.0002 · 25,569 in  / 592 out    · 1,408 cached (6%)   · glm-5.3-flash

Comment thread crates/tinytools-jev/src/recovery/mod.rs
Comment thread crates/tinytools-jev/src/recovery/mod.rs
Comment thread crates/tinytools-jev/src/recovery/test_support.rs Outdated
Comment thread crates/tinytools-jev/src/recovery/mod.rs
Comment thread crates/tinytools-jev/src/recovery/validation_tests.rs
Comment thread crates/tinytools-jev/src/recovery/mod.rs

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previously-blocking findings are resolved. Clearing the changes request.

             $0.0649 · 833,846 in / 64,613 out · 115,988 cached (14%) · gpt-5.6-luna, glm-5.3-flash
critique:    $0.0387 · 465,540 in / 41,739 out · 75,425 cached (16%)  · gpt-5.6-luna, glm-5.3-flash
security:    $0.0255 · 288,863 in / 19,313 out · 37,491 cached (13%)  · gpt-5.6-luna
tests:       $0.0002 · 26,001 in  / 1,231 out  · 1,536 cached (6%)    · glm-5.3-flash
description: $0.0002 · 25,606 in  / 682 out    · 1,408 cached (5%)    · glm-5.3-flash

Comment thread crates/tinytools-jev/src/recovery/mod.rs
Comment thread crates/tinytools-jev/src/recovery/types.rs
Comment thread crates/tinytools-jev/src/recovery/types.rs
Comment thread crates/tinytools-jev/src/recovery/validation_tests.rs
Comment thread crates/tinytools-jev/src/recovery/test_support.rs
Comment thread crates/tinytools-jev/src/recovery/test_support.rs
Comment thread crates/tinytools-jev/src/recovery/test_support.rs

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: 1 lane(s) blocking, worst finding is critical.

Fix or reply to the findings below and push. The next review clears this automatically once they are gone — you should not need to dismiss anything by hand.

             $0.0583 · 734,167 in / 66,102 out · 126,360 cached (17%) · gpt-5.6-luna, glm-5.3-flash
critique:    $0.0333 · 394,793 in / 39,174 out · 78,557 cached (20%)  · gpt-5.6-luna, glm-5.3-flash
security:    $0.0244 · 259,718 in / 23,407 out · 44,283 cached (17%)  · gpt-5.6-luna
tests:       $0.0002 · 26,075 in  / 1,124 out  · 1,856 cached (7%)    · glm-5.3-flash
description: $0.0002 · 25,680 in  / 860 out    · 1,536 cached (6%)    · glm-5.3-flash

Comment thread crates/tinytools-jev/src/recovery/validation_tests.rs
Comment thread crates/tinytools-jev/tests/recovery_contract.rs
Comment thread crates/tinytools-jev/src/lib.rs
Comment thread crates/tinytools-jev/src/recovery/validation_tests.rs
Comment thread crates/tinytools-jev/src/recovery/test_support.rs
Comment thread crates/tinytools-jev/src/recovery/test_support.rs
Comment thread crates/tinytools-jev/src/recovery/test_support.rs
Co-authored-by: Medulla <medulla@tinyhumans.ai>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@senamakel
senamakel marked this pull request as draft October 10, 2026 20:07
@senamakel

Copy link
Copy Markdown
Member Author

Closing this Jev-assisted tool-recovery implementation at the maintainer's request, together with tinyhumansai/openhuman#7341 and its implementation PRs: tinyhumansai/openhuman#7345, tinyhumansai/tinyagents#375, and #61.

The investigation did not establish an incremental recovery or task-completion benefit from this adviser for the observed workload. The maintainer's recent local sessions used native structured tool calls. Observed failures primarily involved command execution, missing runtime state, unavailable services, and a declared JSON Schema anyOf requirement that local validation did not enforce. Recognized schema failures also take the existing deterministic recovery path before this adviser. These findings call for focused validation and state-handling changes rather than proceeding with this decision-model integration.

This is a decision not to pursue this implementation; it does not claim that decision-assisted recovery can never be useful. The existing deterministic safeguards and independently merged compatibility fixes remain in place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant