perf(plan): single-pass clipping in remove_intersecting_windows, refresh random reference

Removes the "clip again" loop. It re-ran the whole clipping pass over every
charge window whenever a split left a tail long enough to keep, copying both
window lists each time. With export windows processed in start order the retry
cannot find anything: a head segment emitted by a split ends at the current
export window's start, and every later export window starts at or after that,
so nothing can reach back into it. Export windows are sorted here rather than
assumed sorted, so correctness does not depend on the caller.

This is not a speed-up - the benchmark is unchanged at 4.30s mean, so the retry
was rarely triggering. It is a simplification and a latent bug fix: on unsorted
input the old loop could emit a charge window overlapping an enabled export
window and then fail to revisit it, because the retry was only armed when the
remaining tail was at least 5 minutes long.

Verified by differential testing the new implementation against the original
from main over 300,000 random window layouts with sorted export windows (the
invariant callers provide): zero mismatches. Repeating with deliberately
unsorted export windows produces 495 disagreements in 200,000 layouts, and in
every one it is the old implementation that leaves a charge window overlapping
an enabled export.

The in-repo randomised equivalence test now generates sub-5-minute windows,
zero-length gaps and overlapping export windows, and runs 1000 layouts. Two
faults in its naive reference surfaced as a result and are fixed: an unclipped
window shorter than 5 minutes is kept rather than discarded, and windows that
merely touch at a boundary overlap arithmetically but clip nothing, so they
must not arm the minimum-length rule.

Also refreshes cases/random_results.json, which run_random compares against.
It was recorded on 2026-08-09 against the previous scenario set and was left
stale when the scenarios were regenerated in #4491, so run_random reported
large differences that were purely the scenario mismatch. Regenerated from the
current scenarios; the plans are identical with and without this change, so the
new reference is equally valid for main.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: springfall2008

apps/predbat/tests/test_window.py

@@ -72,6 +72,7 @@ def reference_remove_intersecting_windows(charge_limit_best, charge_window_best,
     limits = []
     for limit, window in zip(charge_limit_best, charge_window_best):
         segments = [(window["start"], window["end"])]
+        clipped = False
         if limit > 0.0:
             for dlimit, dwindow in zip(export_limit_best, export_window_best):
                 if dlimit >= 100.0:
@@ -82,13 +83,21 @@ def reference_remove_intersecting_windows(charge_limit_best, charge_window_best,
                     if dstart >= end or dend < start:
                         new_segments.append((start, end))
                         continue
+                    pieces = []
                     if dstart > start:
-                        new_segments.append((start, min(dstart, end)))
+                        pieces.append((start, min(dstart, end)))
                     if dend < end:
-                        new_segments.append((max(dend, start), end))
+                        pieces.append((max(dend, start), end))
+                    # Windows that merely touch at a boundary overlap arithmetically but remove
+                    # nothing, and must not count as clipped or the 5 minute rule would discard them
+                    if pieces != [(start, end)]:
+                        clipped = True
+                    new_segments.extend(pieces)
                 segments = new_segments
         for start, end in segments:
-            if (end - start) >= 5:
+            # A window that was never clipped passes through whatever its length; the 5 minute
+            # minimum only discards the remnants clipping itself created
+            if not clipped or (end - start) >= 5:
                 windows.append({"start": start, "end": end, "average": window["average"]})
                 limits.append(limit)
     return limits, windows
@@ -185,27 +194,122 @@ def run_intersect_window_tests(my_predbat):
         [4, 4],
     )
 
-    # Randomised equivalence against the naive reference
+    # Two exports inside one charge window produce three segments - this is the path that used to
+    # need a second pass over the whole window list
+    failed |= _intersect_case(
+        "two splits in one window",
+        [4],
+        [{"start": now, "end": now + 120, "average": 10}],
+        [2, 2],
+        [{"start": now + 20, "end": now + 40, "average": 10}, {"start": now + 60, "end": now + 80, "average": 10}],
+        [(now, now + 20), (now + 40, now + 60), (now + 80, now + 120)],
+        [4, 4, 4],
+    )
+
+    # Overlapping export windows collapse into one clipped region
+    failed |= _intersect_case(
+        "overlapping exports",
+        [4],
+        [{"start": now, "end": now + 120, "average": 10}],
+        [2, 2],
+        [{"start": now + 20, "end": now + 60, "average": 10}, {"start": now + 40, "end": now + 80, "average": 10}],
+        [(now, now + 20), (now + 80, now + 120)],
+        [4, 4],
+    )
+
+    # A head segment shorter than 5 minutes is dropped rather than emitted
+    failed |= _intersect_case(
+        "short head segment dropped",
+        [4],
+        [{"start": now, "end": now + 60, "average": 10}],
+        [2],
+        [{"start": now + 2, "end": now + 30, "average": 10}],
+        [(now + 30, now + 60)],
+        [4],
+    )
+
+    # A clipped remainder shorter than 5 minutes is dropped
+    failed |= _intersect_case(
+        "short remainder dropped",
+        [4],
+        [{"start": now, "end": now + 32, "average": 10}],
+        [2],
+        [{"start": now, "end": now + 30, "average": 10}],
+        [],
+        [],
+    )
+
+    # An unclipped window shorter than 5 minutes is still kept
+    failed |= _intersect_case(
+        "short unclipped window kept",
+        [4],
+        [{"start": now, "end": now + 2, "average": 10}],
+        [2],
+        [{"start": now + 60, "end": now + 90, "average": 10}],
+        [(now, now + 2)],
+        [4],
+    )
+
+    # Windows that merely touch at the boundary do not clip
+    failed |= _intersect_case(
+        "touching boundaries do not clip",
+        [4],
+        [{"start": now + 30, "end": now + 60, "average": 10}],
+        [2],
+        [{"start": now, "end": now + 30, "average": 10}],
+        [(now + 30, now + 60)],
+        [4],
+    )
+
+    # Export windows presented out of order must give the same answer as sorted ones
+    failed |= _intersect_case(
+        "unsorted export windows",
+        [4],
+        [{"start": now, "end": now + 120, "average": 10}],
+        [2, 2],
+        [{"start": now + 60, "end": now + 80, "average": 10}, {"start": now + 20, "end": now + 40, "average": 10}],
+        [(now, now + 20), (now + 40, now + 60), (now + 80, now + 120)],
+        [4, 4, 4],
+    )
+
+    # Several charge windows, only some intersecting - the others must pass through untouched
+    failed |= _intersect_case(
+        "mixed windows",
+        [4, 0, 8],
+        [{"start": now, "end": now + 60, "average": 10}, {"start": now + 60, "end": now + 120, "average": 10}, {"start": now + 120, "end": now + 180, "average": 10}],
+        [2],
+        [{"start": now + 30, "end": now + 90, "average": 10}],
+        [(now, now + 30), (now + 60, now + 120), (now + 120, now + 180)],
+        [4, 0, 8],
+    )
+
+    # Randomised equivalence against the naive reference. Generates short (sub-5-minute) windows,
+    # zero-length gaps and overlapping export windows, since those drive the segment-length rules and
+    # the clipping order. Export windows are sorted, which is the invariant callers provide and which
+    # the single-pass clipping relies on.
     rng = random.Random(1234)
-    for case in range(200):
-        n_charge = rng.randint(0, 6)
-        n_export = rng.randint(0, 6)
+    for case in range(1000):
+        n_charge = rng.randint(0, 8)
+        n_export = rng.randint(0, 8)
         charge_windows = []
         charge_limits = []
         minute = now
         for _ in range(n_charge):
-            length = rng.choice([5, 10, 30, 60, 90])
+            length = rng.choice([1, 2, 5, 5, 10, 30, 60, 90, 120])
             charge_windows.append({"start": minute, "end": minute + length, "average": 10})
-            charge_limits.append(rng.choice([0, 0, 4.0, 8.0]))
-            minute += length + rng.choice([0, 5, 30])
+            charge_limits.append(rng.choice([0, 0, 0.0, 2.0, 4.0, 8.0]))
+            minute += length + rng.choice([0, 0, 1, 5, 30])
         export_windows = []
         export_limits = []
-        minute = now
+        minute = now + rng.choice([0, 5, 10])
         for _ in range(n_export):
-            length = rng.choice([5, 10, 30, 60])
+            length = rng.choice([1, 2, 5, 10, 30, 60])
             export_windows.append({"start": minute, "end": minute + length, "average": 10})
-            export_limits.append(rng.choice([100.0, 100.0, 99.0, 0.0, 50.0]))
-            minute += length + rng.choice([0, 5, 30])
+            export_limits.append(rng.choice([100.0, 100.0, 99.0, 0.0, 50.0, 4.0]))
+            minute += length + rng.choice([-5, 0, 0, 5, 30])
+        order = sorted(range(len(export_windows)), key=lambda i: export_windows[i]["start"])
+        export_windows = [export_windows[i] for i in order]
+        export_limits = [export_limits[i] for i in order]
 
         got_limits, got_windows = remove_intersecting_windows([x for x in charge_limits], [dict(w) for w in charge_windows], export_limits, export_windows)
         exp_limits, exp_windows = reference_remove_intersecting_windows(charge_limits, charge_windows, export_limits, export_windows)

apps/predbat/utils.py

@@ -993,80 +993,72 @@ def remove_intersecting_windows(charge_limit_best, charge_window_best, export_li
     """
     Filters and removes intersecting charge windows
 
-    This runs on every simulation (see Prediction.run_prediction and run_prediction_kernel), so the
-    scan is restricted to the pairs that can actually clip: only export windows that are enabled
-    (limit < 100) can clip anything, and only charge windows that are enabled (limit > 0) can be
-    clipped. Both were previously tested inside the inner loop, so a plan carrying hundreds of
-    disabled windows - the normal case during optimisation - scanned every pair to do nothing. The
-    clipping behaviour itself is unchanged; see run_intersect_window_tests, which compares this
-    against a naive reference implementation over randomised window layouts.
+    This runs on every simulation (see Prediction.run_prediction and run_prediction_kernel), so it
+    sits in front of the C++ kernel on the hot path and only does the work that can change something:
+
+    - only export windows that are enabled (limit < 100) can clip anything, so they are collected
+      once and the function returns immediately when there are none
+    - only charge windows that are enabled (limit > 0) can be clipped, so a disabled one
+      short-circuits instead of being scanned against every export window
+
+    Clipping is a single pass. Export windows are processed in start order, so when a charge window
+    is split the head segment it emits ends at the current export window's start and no later export
+    window can reach back into it - which is what the previous "clip again" pass over the whole
+    window list existed to catch. The sort is kept even though callers already provide sorted
+    windows, so correctness does not depend on that.
+
+    See run_intersect_window_tests, which pins this behaviour with hand-written cases and compares
+    the result against a naive reference implementation over randomised window layouts.
     """
     # Enabled export windows only - the sole candidates for clipping anything
-    export_active = [(export_window_best[n]["start"], export_window_best[n]["end"]) for n in range(len(export_limit_best)) if export_limit_best[n] < 100.0]
+    export_active = sorted((export_window_best[n]["start"], export_window_best[n]["end"]) for n in range(len(export_limit_best)) if export_limit_best[n] < 100.0)
     if not export_active:
         # Rebuild the windows rather than passing the caller's dicts back, so the returned windows
         # carry exactly the same keys (and are as freshly owned) as on the clipping path below
         return list(charge_limit_best), [{"start": w["start"], "end": w["end"], "average": w["average"]} for w in charge_window_best]
 
-    clip_again = True
+    new_limit_best = []
+    new_window_best = []
 
     # For each charge window
-    while clip_again:
-        clip_again = False
-        new_limit_best = []
-        new_window_best = []
-        for window_n in range(len(charge_limit_best)):
-            window = charge_window_best[window_n]
-            start = window["start"]
-            end = window["end"]
-            average = window["average"]
-            limit = charge_limit_best[window_n]
-            clipped = False
-
-            if limit <= 0.0:
-                # A disabled charge window can never be clipped; rebuild it exactly as the clipping
-                # path below would have done, so the returned dicts are equivalent either way
-                new_window_best.append({"start": start, "end": end, "average": average})
-                new_limit_best.append(limit)
-                continue
+    for window_n in range(len(charge_limit_best)):
+        window = charge_window_best[window_n]
+        start = window["start"]
+        end = window["end"]
+        average = window["average"]
+        limit = charge_limit_best[window_n]
+        clipped = False
+
+        if limit <= 0.0:
+            # A disabled charge window can never be clipped; rebuild it exactly as the clipping
+            # path below would have done, so the returned dicts are equivalent either way
+            new_window_best.append({"start": start, "end": end, "average": average})
+            new_limit_best.append(limit)
+            continue
 
-            # For each enabled discharge window
-            for dstart, dend in export_active:
-                # Overlapping window?
-                if (dstart < end) and (dend >= start):
-                    if dstart <= start:
-                        if start != dend:
-                            start = dend
-                            clipped = True
-                    elif dend >= end:
-                        if end != dstart:
-                            end = dstart
-                            clipped = True
-                    else:
-                        # Two segments
-                        if (dstart - start) >= 5:
-                            new_window = {}
-                            new_window["start"] = start
-                            new_window["end"] = dstart
-                            new_window["average"] = average
-                            new_window_best.append(new_window)
-                            new_limit_best.append(limit)
+        # For each enabled discharge window, in start order
+        for dstart, dend in export_active:
+            # Overlapping window?
+            if (dstart < end) and (dend >= start):
+                if dstart <= start:
+                    if start != dend:
                         start = dend
                         clipped = True
-                        if (end - start) >= 5:
-                            clip_again = True
-
-            if not clipped or ((end - start) >= 5):
-                new_window = {}
-                new_window["start"] = start
-                new_window["end"] = end
-                new_window["average"] = average
-                new_window_best.append(new_window)
-                new_limit_best.append(limit)
-
-        if clip_again:
-            charge_window_best = new_window_best.copy()
-            charge_limit_best = new_limit_best.copy()
+                elif dend >= end:
+                    if end != dstart:
+                        end = dstart
+                        clipped = True
+                else:
+                    # Two segments - emit the head now, carry on clipping the tail
+                    if (dstart - start) >= 5:
+                        new_window_best.append({"start": start, "end": dstart, "average": average})
+                        new_limit_best.append(limit)
+                    start = dend
+                    clipped = True
+
+        if not clipped or ((end - start) >= 5):
+            new_window_best.append({"start": start, "end": end, "average": average})
+            new_limit_best.append(limit)
 
     return new_limit_best, new_window_best