Re-anchor custom queued deadline references when a Dag run is cleared

[rjgoyln]

Summary

A custom deadline reference registered as DeadlineReference.TYPES.DAGRUN_QUEUED is never re-anchored when its Dag run is cleared — the built-in DAGRUN_QUEUED_AT deadline moves to the new queued_at, the custom one keeps pointing at the old one.

The registered timing only ever existed in the in-memory DeadlineReference.TYPES tuples of the process that ran the Dag file, so the clear path could recognise a queued-anchored deadline only by matching the built-in class name, which no custom class satisfies.

• Record the registered timing on the reference class and store it on the alert row.
• Select queued-anchored deadlines on the stored timing, keeping the class-name match as a fallback.
• Compare a stored reference only on the fields it has, and refresh a matched row in place.

The serializer adds the field rather than serialize_reference(), because custom references are expected to override that method and an override would drop it.

Compatibility

An alert serialized before this change keeps its id and gains the timing on the next parse rather than being replaced, which would strand the deadlines of running Dag runs.

closes: #71747

---

Was generative AI tooling used to co-author this PR?

• Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

[ferruzzi]

Cross-linking, since this overlaps two other open PRs on the same code and I do not think the three authors are aware of each other.

#70714 (opened 30 July) changes deadline references to receive the in-memory DagRun instead of dag_id and run_id, and deprecates required_kwargs. It overlaps this PR across a bunch of files. Worth a look before you rebase, particularly _evaluate_with's signature, which it narrows from **kwargs to a keyword-only dagrun.

On the interval workaround: once #70370 lands, storing numeric intervals in the tests to stay clear of the JSON bug is no longer necessary, and storing serialize(interval) instead would then hit the production write path. That legacy-float shortcut is what hid the TypeError in the existing test in the
first place, so it is worth removing rather than inheriting once the fix is in.

[ferruzzi]

#71802 also edits serialization/definitions/deadline.py, so it conflicts here too.

When you rebase on top of #70714, the required_kwargs: set[str] = set() line your diff has above the new evaluation_timing is deleted by that PR, and the deletion is importnt. Its replacement treats a present required_kwargs as an opt-in to the deprecated path, so keeping that line silently routes every reference including the built-ins through it rather than failing loudly. Just be careful when you do a rebase or merge conflict.

[rjgoyln]

Cross-linking, since this overlaps two other open PRs on the same code and I do not think the three authors are aware of each other.

#70714 (opened 30 July) changes deadline references to receive the in-memory DagRun instead of dag_id and run_id, and deprecates required_kwargs. It overlaps this PR across a bunch of files. Worth a look before you rebase, particularly _evaluate_with's signature, which it narrows from **kwargs to a keyword-only dagrun.

On the interval workaround: once #70370 lands, storing numeric intervals in the tests to stay clear of the JSON bug is no longer necessary, and storing serialize(interval) instead would then hit the production write path. That legacy-float shortcut is what hid the TypeError in the existing test in the first place, so it is worth removing rather than inheriting once the fix is in.

Thanks for the cross-links! I’d only accounted for #70370.

For #70714, I’ll keep the evaluation_timing deletions when rebasing and follow the keyword-only dagrun signature.

For the interval tests, agreed — I’ll switch them to serialize(interval) once #70370 lands and remove the note from the description.

#71802 noted as well. It should be a textual conflict in the interval handling rather than a behavioral one.

[ferruzzi]

It's going to be a bit of a mess untangling them all as we merge them, but we'll sort it out.

[rjgoyln]

Thanks for the review — all three are fair.

I've addressed the reference matching issue: matching now ignores fields the stored row lacks, and a matched row is refreshed in place so evaluation_timing is populated without changing the ID.

For recomputing from the user's reference and the missing _evaluate_with coverage, I haven't made those changes yet. They depend on #70714's new interface, where the reference takes an in-memory DagRun and _evaluate_with takes dagrun as a keyword-only argument. Since #70714 is still open, I'll follow it once that interface is available — happy to rebase onto your branch and do it here instead if you prefer.

[ferruzzi]

I've been working through a long chain of PRs which are all editing the same section of code, with the goal of merging them in the order that will minimize conflicts and this one is up next. If you can get it rebased and green, I'll merge it. This is not going to be a simple "click to fast-forward" rebase, but I tried to make it as painless as could. Be careful not to lose the dagrun being explicitly passed into _evaluate_with and the rest should flow pretty easily from there.

[rjgoyln]

Thanks for the review! I've rebased onto main and updated the test to distinguish reference re-anchoring from the built-in formula. Intervals now use serialize(interval).