Repository navigation
Description
🧩 Name of the Activity
Validate the supplementary SDK naming review in azure-typespec-assessment against the existing PR assessment corpus.
🎯 Goal of this Activity
Follow up on the assessment integration feedback in #17121. The reviewer offered to help with integration and report-quality/performance validation through an assigned issue. The authoring and assessment integration is being retained in that PR.
Establish the quality and end-to-end performance impact before treating the new naming checks as validated across real PRs. Isolated naming eval duration must not be presented as average PR assessment duration or incremental overhead.
🌟 Champion Scenarios
- Confirmed C# ARM changes receive scoped naming recommendations with release-compatibility evidence and unchanged wire/other-language names.
- Unsupported SDK languages are reported as
not-covered, without borrowing C# rules. - Missing target, service-type, or release evidence is surfaced rather than guessed.
- Assessment remains read-only and never invokes SDK customization or generation for the supplementary naming pass.
🔧 How is this Activity Done Today?
The assessment report site currently lists 13 PRs (the review initially mentioned 12): Azure/azure-rest-api-specs#42435, Azure/azure-rest-api-specs#42853, Azure/azure-rest-api-specs#43308, Azure/azure-rest-api-specs#43718, Azure/azure-rest-api-specs#43745, Azure/azure-rest-api-specs#44200, Azure/azure-rest-api-specs#44454, Azure/azure-rest-api-specs#44742, Azure/azure-rest-api-specs#44882, Azure/azure-rest-api-specs#44988, Azure/azure-rest-api-specs#45162, Azure/azure-rest-api-specs#45348, and Azure/azure-rest-api-specs#45536.
#17121 keeps supplementary naming findings separate from assessment.json / assessment.html, whose existing schema is unchanged. It documents a sample supplementary report and adds hermetic coverage for supported, unsupported, and missing-context reporting. Those cases do not establish real-corpus report quality or full-assessment performance.
🛠️ Tools Needed to Support This Activity
Use the assessment skill's coordinator and guarded finalization workflow with pinned PR heads and target baselines. Record the model, skill revision, cache/dependency state, and environment for each run.
Acceptance criteria:
- Run the agreed corpus with and without naming integration using comparable settings; retain both main reports and supplementary naming output.
- Review recommendation accuracy, coverage, exceptions, compatibility blockers, and false positives.
- Record total and phase timings, plus failures or unavailable evidence, per PR. Report paired deltas and mean/median/range; distinguish cold setup costs from the naming step.
- Confirm that assessment does not modify source or invoke customization/generation during naming judgment.
- Link the results and remaining limitations back to Add shared TypeSpec naming skill for authoring #17121; do not claim a regression-free result until the comparison is complete.
Activity
haolingdong-msft commentedon Oct 16, 2026
dup #17297