Skip to content

fix(dagster-openlineage): emit asset run-start without a placeholder output - #345

Open
dan-newton-sparta wants to merge 1 commit into
dagster-io:mainfrom
dan-newton-sparta:feat/openlineage-run-start-no-output
Open

dan-newton-sparta wants to merge 1 commit into
dagster-io:mainfrom
dan-newton-sparta:feat/openlineage-run-start-no-output

Conversation

@dan-newton-sparta

@dan-newton-sparta dan-newton-sparta commented Sep 9, 2026 •

Copy link
Copy Markdown

ASSET_MATERIALIZATION_PLANNED emitted a placeholder output dataset with no inputs and no facets. This change emits it as a pure run-start; the COMPLETE event still carries the full inputs, outputs, and facets.

Summary & Motivation

A planned/START event has no materialization metadata yet — no inputs, no schema, no column lineage. For lineage backends that build edges from every event this can produce an output-only edge with no input side. Emitting START as a pure run-start signal keeps the run-start semantics while deferring all dataset/facet information to COMPLETE, where it is actually known.

How I Tested These Changes

  • Updated the adapter and end-to-end tests: START now asserts no inputs/outputs; COMPLETE still carries the output.
  • uv run pytest green; ruff check, ruff format --check, and ty check . clean.
  • Tested locally end to end: ingested the emitted OpenLineage events into OpenMetadata and used it to visualise the resulting lineage.

Changelog

Added a ### Changed entry under ## [Unreleased] in CHANGELOG.md.

Related changes

Part of a set of small improvements to Dagster → OpenLineage. The lineage-fidelity changes are each independent and mergeable on their own; the ownership pair is stacked (#354 builds on #353).

Lineage fidelity:

Ownership:

…output

A planned/START event has no materialization metadata yet — no inputs, no
schema or column facets. It previously emitted a bare output dataset anyway,
which makes backends that build lineage edges from every event create a
spurious output-only edge with no input side. Emit START as a pure run-start
signal; the COMPLETE event still carries the inputs, outputs, and facets.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant