Skip to content

test: add AILF tasks and report - #3294

Merged
colepeters merged 4 commits into
v5-alphafrom
sdk-3058/ailf-tasks
Nov 3, 2026
Merged

colepeters merged 4 commits into
v5-alphafrom
sdk-3058/ailf-tasks

Conversation

@colepeters

@colepeters colepeters commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Description

This PR moves our Sanity UI 5 AILF tasks from the AILF studio into the repo for better tracking/iteration along with the components. A copy of the report from an initial run with these tasks is also included.

Once we land v5 in the main branch, we can turn this into a GitHub workflow that we can trigger from the repo, or PRs. In the mean time, it needs to be run from the repo root in a terminal with npx @sanity/ailf@latest run --remote. A --debug flag can be added if you want to test things out.

What to review

Are the tasks and config as expected?

Testing

Ran locally to verify (which generated the report included in this PR).


Note

Low Risk
Adds evaluation config, tasks, and a report snapshot only; no changes to @sanity/ui runtime or published packages.

Overview
Adds the AI Literacy Framework (AILF) setup under .ailf/ so Sanity UI v5 can be evaluated from this repo instead of only from the AILF studio. ailf.config.ts points at production docs, assigns ownership to the SDK team, and loads tasks from the repo.

The bulk of the change is literacy task definitions (.ailf/tasks/*.yaml) and matching canonical reference solutions for migration and API pitfalls—v3→v5 props, tone/level vocabulary, icon subpaths, polyfills, accessibility patterns, and similar. A snapshot of the latest remote run is committed in .ailf/results/latest/ (report shows 81/100 on sanity-ui-v5, 516 tests).

Supporting tweaks: .env.example documents AILF_API_KEY for npx @sanity/ailf run --remote, .gitignore ignores .env, and oxfmt/oxlint skip .ailf/** so generated YAML/TSX does not get formatted or linted like app code.

Reviewed by Cursor Bugbot for commit 5965ee2. Bugbot is set up for automated code reviews on this repo. Configure here.

@vercel

vercel Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

4 Skipped Deployments
Project Deployment Actions Updated
sanity-ui-storybook Ignored Ignored Nov 3, 2026 8:31pm UTC
sanity-ui-workshop Ignored Ignored Nov 3, 2026 8:31pm UTC
ui-docs Ignored Ignored Nov 3, 2026 8:31pm UTC
ui-studio Ignored Ignored Nov 3, 2026 8:31pm UTC

Request Review

@changeset-bot

changeset-bot Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 5965ee2

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@somerandomdude somerandomdude left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as a good first pass. We'll no doubt be refining these in future iterations. Thanks again!

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6ad7a30. Configure here.

Comment thread .ailf/tasks/link.yaml Resolved
Comment thread .ailf/tasks/icon.yaml
- type: contains
value: '@sanity/icons/Search'
- type: contains
value: 'icon={SearchIcon}'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Icon task rejects valid element form

Medium Severity

The Icon wrapper rubric accepts both icon={SearchIcon} and icon={<SearchIcon />}, but the contains assertion requires only the component form. A correct answer using the documented element form fails the string check.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 6ad7a30. Configure here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll address this later

@colepeters
colepeters merged commit 90aeb8a into v5-alpha Nov 3, 2026
15 of 16 checks passed
@colepeters
colepeters deleted the sdk-3058/ailf-tasks branch November 3, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants