[AI Foundry] Simplify rubric scoring and optional weights

[Sydney Lister (slister1001)]

Data Plane API Specification Update

Summary

Adds a focused ASSERT safety extension to the ordinary rubric simplification contract, whose current published changes are incorporated here.

• Shared baseline: rubric-level Boolean/Likert mode, optional configured weights, flat mode-specific dimension evidence, and approved-dimension generation controls.
• Return selected or inherited scoring, pass_threshold, and approved dimensions on the generation-job response.
• Add optional generation category with quality/safety intent. Safety requires a prompt objective plus agent/trace application context, resolves Boolean judgments, and rejects conflicting rating settings.
• Add optional Dimension.permissible, required by safety validation with no default. Safety criteria omit weights.
• Report impermissible_behavior_violated and permissible_behavior_violated separately, with independent summary identities and denominators. No combined safety score or new execution endpoint.
• Safety unavailable scores are null and unavailable verdicts are omitted. Ordinary rubric result properties remain flat at properties.mode, properties.decision, and properties.dimension_results.
• Preserve historical preview wire shapes through the existing versioning decorators. New fields are versioned to v1.

The incremental safety API surface is small. The full diff also includes the shared simplification models and generated OpenAPI JSON/YAML.

API Info

• API Spec engagement record: pending.
• Target branch: feature/foundry-release.
• Existing API version: v1; no API version is added.
• Review scope: private preview.
• Project: specification/ai-foundry/data-plane/Foundry.
• Source files: src/evaluators/models.tsp and src/openai/evaluations/models.tsp.

No SDK package publication, service deployment, feature enablement, or criterion-level mixed scoring is introduced.

Validation and implementation alignment

Historical validation includes TypeSpec compilation, OpenAPI generation, formatting, type/version/default assertions, and comparison of older preview wire shapes excluding documentation.

October 19 source comparison confirms that the published contract includes the current shared flat-result and generation-job response additions, plus the intended safety category, permissibility, and two-metric semantics. The ASSERT service implementation is being refreshed in a separate PR stack. This is not an exact-head, full-stack serialization or persisted-E2E sign-off.

Final service-stack alignment, including nullable safety result availability versus ordinary rubric results, current-revision CI, API/codeowner approval, and engagement/release-plan linkage remain pre-merge requirements. Keep this PR in draft for API review.

[github-actions]

Next Steps to Merge

✅ All automated merging requirements have been met! To get your PR merged, see aka.ms/azsdk/specreview/merge.

Comment generated by summarize-checks workflow run.

[Sydney Lister (slister1001)]

Gerardo Lecaros (@glecaros), could you review this focused ASSERT API extension, or route it to the appropriate API reviewer? We'd like to get it checked in this week, ideally by Friday, October 23.

The incremental API surface is small: safety generation category, criterion permissibility, and two named violation metrics with independent summaries. It reuses existing operations and incorporates the rubric simplification contract, including flat dimension evidence and generation-job controls. Most changed lines are shared models and generated OpenAPI.

This is a draft API review request, not a merge-readiness claim. Please particularly review safety's nullable unavailable scores and omitted verdicts alongside the ordinary rubric contract. Final alignment of the refreshed service stack remains a pre-merge check.