Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,3 +159,6 @@ file.
## License

GPL-3.0-only. See [`LICENSE`](LICENSE).

Recovery decisions are additive in `tinytools-jev::recovery`; see its
[bounded advisory contract](crates/tinytools-jev/src/recovery/README.md).
13 changes: 13 additions & 0 deletions crates/tinytools-jev/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,16 @@ the evaluator decides:
`MAX_CANDIDATES` by the retriever. The family stage sets
`JevRequest::instructions` so the evaluator asks "which group" rather than
"which tool"; `JevRanking::families` reports what it chose.

## Recovery advice

The additive `recovery` module supplies a distinct bounded tool-failure contract:
`RecoveryObservation`, independent typed `RecoveryRequest` questions,
`RecoveryEvaluator`, strict `RecoveryDecision::validate` and `RecoveryAdviser`.
Its Noul asks about recovery without external prerequisites rather than whether a
request needs a tool. This API never authorizes or executes a retry. Hosts supply
scrubbed facts, explicit calibrated thresholds and filtered alternate candidates,
and retain deadlines, cancellation, ledgers and fallback.

See [recovery design and constraints](src/recovery/README.md) and the
[specification](../../docs/specs/tool-recovery-advice.md).
17 changes: 16 additions & 1 deletion crates/tinytools-jev/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,22 @@
//! Dependency-free Jev-backed tool ranking.
//!
//! A host supplies [`JevEvaluator`], retaining ownership of transport,
//! authentication, retry, and deadline policy.
//! authentication, retry, and deadline policy. The additive [`recovery`] module
//! supplies distinct advisory failure decisions through a [`recovery::RecoveryEvaluator`].
//! It owns no transport, authorization or execution loop.
//!
//! ```
//! use tinytools_jev::recovery::{RecoveryObservation, RecoveryPhase, RecoveryRequest};
//! let observation = RecoveryObservation::new(
//! RecoveryPhase::Execution, "Lookup documents", "Temporary service outage",
//! );
//! let request = RecoveryRequest::new(observation)?;
//! assert_eq!(request.questions.len(), 2);
//! # Ok::<(), tinytools::RankError>(())
//! ```

/// Provider-neutral bounded advice for unresolved tool failures.
pub mod recovery;
Comment thread
senamakel marked this conversation as resolved.

mod family;
#[cfg(test)]
Expand Down
57 changes: 57 additions & 0 deletions crates/tinytools-jev/src/recovery/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Recovery advice

`RecoveryObservation` carries a host-normalized phase, effect facts, counts,
argument shape and explicitly scrubbed diagnostic. `RecoveryRequest::new` builds
stable independent Choice/Noul questions; a concrete correction enables Score.
`RecoveryEvaluator` is an async transport-neutral port. `RecoveryAdviser` validates
its result and returns separate advice or an explicit abstention reason.

The nine classes are transient, rate_limited, authentication, permission,
wrong_arguments, wrong_tool, unavailable, unsupported and unknown. Unknown/ties
abstain. Recoverability asks whether a policy-permitted next attempt can recover
using evidenced corrections or transient conditions without an external
prerequisite. Score is a three-category ordered feasibility distribution; its
expectation (0..=2) is **not** a probability of success. Confidence is distribution
concentration and is **not** accuracy. Hosts must calibrate explicit versioned
thresholds, including the counted repeated-blocker threshold. Recoverability must
be finite and in `(0, 1]`; class and advice concentration thresholds accept finite
values in `[0, 1]`. A zero recoverability threshold would admit corrective advice
with no prospect of recovery and reject valid external-prerequisite classifications.

Candidate descriptions alone do not enable alternate evaluation. The host supplies
`RecoveryAlternateReason::WrongToolAdvice` after prior advice or `RepeatedBlocker`
after its stable ledger reaches the configured threshold. A host can make a second
bounded batch after wrong-tool advice within the **same total deadline**. Alternate
options include `none`; every other key must be supplied by the host, authorized,
known in the session, executable through the existing bridge, and exclude the failed
operation and refused services. An accepted alternate requires recoverability and
wrong-tool classification or a counted repeated blocker. It never executes a tool.

Each prose field is at most 2048 bytes; each candidate key is at most 128 bytes;
there are at most 16 candidates and 8 trusted facts. Oversized/empty required fields
are rejected without truncation. These limits do not scrub data: the host must
minimize identifying/private information, remove secrets and URL queries, and send
argument names/types instead of raw values. Text remains untrusted evidence.

The complete typed answer batch is validated for IDs, variants, finite bounded
probabilities/confidence, exact option sets and normalized distributions (sum within
1e-6). Provider failures, malformed or contradictory advice and insufficient
confidence abstain. Trusted success, denial, expired approval, cancellation,
terminal faults, authentication/permission facts and applied/uncertain writes skip
the evaluator. An unknown effect never proves that a retry is safe.

No transport, credentials, retry loop, deadline scheduler, ledger, prompt mutation,
normalizer or authorization mechanism lives here. Hosts retain all admission checks,
remaining-time/cancellation limits, stable operation/scope budgets, exact-repeat
guards, one accounting observation per failed invocation and keyword fallback.
Advice does not alter the original tool result or durable transcript.

## Existing result boundary

`tinytools::ToolResult` already distinguishes reported `is_error` from an execution
`Err`, carries optional `ToolErrorKind::{Retry,Failed}`, and round-trips producer
facts through host-only `metadata`. Its model rendering excludes metadata, but
serialization retains it. A retry hint is not effect safety, and parsed arbitrary
JSON (`ok:false`) is not globally trusted. Adapter-owned facts must be normalized by
the host; this contract adds no second failure envelope. Metadata also needs explicit
scrubbing before a decision service sees it.
70 changes: 70 additions & 0 deletions crates/tinytools-jev/src/recovery/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
//! Typed, bounded recovery advice. Hosts retain authorization, execution and accounting.
mod questions;
mod types;
mod validation;
use std::sync::Arc;
use tinytools::RankError;
pub use types::*;

/// Advises on unresolved failures using a host-provided decision evaluator.
#[derive(Debug, Clone)]
pub struct RecoveryAdviser {
evaluator: Arc<dyn RecoveryEvaluator>,
thresholds: RecoveryThresholds,
}
impl RecoveryAdviser {
/// Constructs an adviser with explicitly selected calibration thresholds.
///
/// # Errors
/// Rejects empty/oversized policy version and nonfinite/out-of-range thresholds.
pub fn new(
evaluator: Arc<dyn RecoveryEvaluator>,
thresholds: RecoveryThresholds,
) -> Result<Self, RankError> {
validation::validate_thresholds(&thresholds)?;
Ok(Self {
evaluator,
thresholds,
})
}
/// Evaluates once, or abstains without interpreting trusted host facts.
///
/// This future has no internal timeout/retry. The host must impose its remaining
/// deadline and cancellation and count the failed invocation regardless of advice.
///
/// # Errors
/// Rejects observations outside the bounded input contract before evaluation.
pub async fn advise(
&self,
observation: RecoveryObservation,
) -> Result<RecoveryAdvice, RankError> {
validation::validate_observation(&observation)?;
if !observation.facts.is_empty()
|| matches!(
observation.effect,
RecoveryEffect::UncertainWrite | RecoveryEffect::Applied
)
{
return Ok(RecoveryAdvice::Abstained(RecoveryAbstention::TrustedFact));
}
let mut observation = observation;
if observation.alternate_reason == Some(RecoveryAlternateReason::RepeatedBlocker)
Comment thread
senamakel marked this conversation as resolved.
Comment thread
senamakel marked this conversation as resolved.
&& observation.repeated_failures < self.thresholds.repeated_blocker
{
observation.alternate_reason = None;
}
let request = RecoveryRequest::new(observation)?;
Comment thread
senamakel marked this conversation as resolved.
Comment thread
senamakel marked this conversation as resolved.
let Ok(decision) = self.evaluator.evaluate(&request).await else {
return Ok(RecoveryAdvice::Abstained(
RecoveryAbstention::EvaluatorFailure,
));
};
Ok(validation::advice(&request, decision, &self.thresholds))
Comment thread
senamakel marked this conversation as resolved.
}
}
#[cfg(test)]
mod test_support;

#[cfg(test)]
#[path = "mod_tests.rs"]
mod tests;
Loading
Loading