Skip to content

SceneDataTransformer: support system and URL provided transformations - #1614

Open
dprokop wants to merge 32 commits into
mainfrom
panel-system-transformations
Open

SceneDataTransformer: support system and URL provided transformations#1614
dprokop wants to merge 32 commits into
mainfrom
panel-system-transformations

Conversation

@dprokop

@dprokop dprokop commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Adds support for system transformations in scenes.

System transformations are currently:

  • Provided by SceneDataTransformer parent (e.g. VizPanel, gated with applyPluginTransformations state flag)
  • Not saved in state.transformations
    • read with getResolvedSystemTransformations
  • Discovered on SceneDataTransformer activation
  • Prepended before or appended after user transformations
  • Tagged with origin (i.e. plugin) and position (prepend, append)
  • Noop when system transformation provider throw

Also fixes variable interpolation dropping the operator function from object-form custom transformers.

🤖 Generated with Claude Code

📦 Published PR as canary version: 9.0.0--canary.1614.34603085399.0

✨ Test out this PR locally via:

npm install @grafana/scenes@9.0.0--canary.1614.34603085399.0
npm install scenes-app@9.0.0--canary.1614.34603085399.0
npm install @grafana/scenes-react@9.0.0--canary.1614.34603085399.0
# or 
yarn add @grafana/scenes@9.0.0--canary.1614.34603085399.0
yarn add scenes-app@9.0.0--canary.1614.34603085399.0
yarn add @grafana/scenes-react@9.0.0--canary.1614.34603085399.0

…rmer

Adds SceneDataTransformer.setSystemTransformations so runtime transformations can
be combined with the user configured ones without replacing them. Entries are
tagged with an origin ('system' for panel provided, 'url' for url provided) and a
position, are replaced per origin so different providers do not clobber each
other, and can be filtered out via the exported isSystemTransformation guard
before persisting or editing. Also fixes variable interpolation dropping the
operator function from object form custom transformers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gtk-grafana gtk-grafana added release Create a release when this pr is merged minor Increment the minor version when merged labels Aug 18, 2026
gtk-grafana added a commit to grafana/grafana that referenced this pull request Aug 18, 2026
Adds `PanelPlugin.setDataTransformations(supplier)`, so a panel can declare the
transformations it needs in order to render its data — extracting fields out of a JSON
column, reshaping long frames into wide ones — without giving up field overrides or the
user's own transformation pipeline.

Prepended transformations run before every user transformation and before field
overrides, so the fields they produce are matchable by overrides and targetable by the
user; appended ones run after all of them. The supplier is handed the query result
frames, so it can branch on frame shape or `meta`, and it is evaluated inside the
pipeline rather than pushed from a panel render, which keeps it free of a
render -> setState -> data -> render loop.

They are carried in `SceneDataTransformer.transformations` tagged `origin: 'system'`
and filtered out of editors and save models by the `isSystemTransformation` guard, so
they are never persisted, never editable, and dashboard JSON is unchanged. The
transformations tab shows them as read-only rows badged `System`, and the editors replay
the prepended stage so each row is configured against the field shape it will actually
receive.

Gated behind the `grafana.panelPluginTransformations` feature toggle. No existing panel
changes behavior.

Requires @grafana/scenes 8.15.0 (grafana/scenes#1614).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread packages/scenes/src/querying/SceneDataTransformer.ts Outdated
…ransform state on every call to setSystemTransformations
Comment thread packages/scenes/src/querying/SceneDataTransformer.ts Outdated
@gtk-grafana

gtk-grafana commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

@dprokop I made a few minor changes while working on the grafana implementation and PoCs

  1. Fix: custom transforms calling setState on every call of setSystemTransformations: 78eb437
  2. Fix: useDataTransformer wiping out system transforms (note adds new public setUserTransformations method): fdb7af0
  3. Test: add coverage for interpolation 10c9c43
  4. Chore: setSystemTransformations types d177c34
  5. Fix: bug introduced in setUserTransformations refactor 3074fbb + regression test
  6. Address question from Dom asking if we should remove url and rename system to plugin 00e5b54
  7. Regression with null transformations in v1 dashboards: 456fa65 : SceneDataTransformer: support system and URL provided transformations #1614 (comment)

Other then that everything looks good, marking as ready for review

@gtk-grafana
gtk-grafana marked this pull request as ready for review August 18, 2026 19:24
@gtk-grafana
gtk-grafana requested a review from a team as a code owner August 18, 2026 19:24
@gtk-grafana
gtk-grafana requested review from Sergej-Vlasov and harisrozajac and removed request for a team August 18, 2026 19:24
Comment thread packages/scenes/src/querying/SceneDataTransformer.ts
Comment thread packages/scenes/src/core/types.ts Outdated
gtk-grafana added a commit to grafana/grafana that referenced this pull request Aug 20, 2026
@gtk-grafana gtk-grafana added major Increment the major version when merged and removed minor Increment the minor version when merged labels Aug 25, 2026
gtk-grafana added a commit to grafana/grafana that referenced this pull request Aug 27, 2026
Comment thread packages/scenes/src/querying/SceneDataTransformer.ts Outdated
Comment thread packages/scenes/src/querying/SceneDataTransformer.ts Outdated
…ead pull system transformation provider from parent of SceneDataTransformer, e.g. VizPanel

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/scenes/src/components/VizPanel/VizPanel.tsx Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/scenes/src/querying/SceneDataTransformer.ts
@gtk-grafana gtk-grafana self-assigned this Sep 8, 2026
@gtk-grafana

Copy link
Copy Markdown
Contributor

Rewritten to check for parents of SceneDataTransformer that implement SystemTransformationsProvider. PTAL!

@torkelo torkelo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good except the duplicate plugin loading in VizPanel and some I think faulty comment changes

Comment thread packages/scenes/src/components/VizPanel/VizPanel.tsx Outdated
Comment thread packages/scenes/src/querying/SceneDataTransformer.ts
Comment thread packages/scenes/src/querying/SceneQueryRunner.ts Outdated
Comment thread packages/scenes/src/variables/adhoc/AdHocFiltersVariable.tsx Outdated
Comment thread packages/scenes/src/variables/DrilldownDependenciesManager.ts Outdated
Comment thread packages/scenes/src/querying/SceneQueryRunner.test.ts Outdated
Comment thread packages/scenes/src/components/VizPanel/VizPanel.tsx Outdated
Comment thread packages/scenes/src/components/VizPanel/VizPanel.tsx Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/scenes/src/querying/SceneDataTransformer.ts

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/scenes/src/querying/SceneDataTransformer.ts Outdated
@gtk-grafana
gtk-grafana requested a review from torkelo September 11, 2026 13:15

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7170dbf. Configure here.

Comment thread packages/scenes/src/components/VizPanel/VizPanel.tsx
Comment thread packages/scenes/src/components/VizPanel/VizPanel.tsx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

major Increment the major version when merged release Create a release when this pr is merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants