Skip to content

LiveSessionDriverTest setup waits on a wall clock, so the file fails as a group under load #978

Description

@DROOdotFOO

Raxol.Harness.LiveSessionDriverTest has no real barrier between starting a driver and driving it. Its setup helper waits on a wall clock instead, so under load the whole file fails together.

Symptom

Nine or more tests fail at once, all identically, all inside the shared setup:

1) test 7. malformed events are rejected at the boundary and marked ...
   test/harness/live_session_driver_test.exs:660
   Assertion failed, no matching message after 2000ms
   The process mailbox is empty.
   code: assert_receive {:subscribed, forwarder_pid}
   stacktrace:
     test/harness/live_session_driver_test.exs:317: ...new_driver/2

They fail as a group because they share new_driver/2, so one bad moment takes the file.

Mechanism

LiveSessionDriver.start_link/1 does not synchronize at all:

def start_link(opts) do
  pid = spawn_link(__MODULE__, :run, [opts])
  {:ok, pid}
end

It returns {:ok, pid} before the spawned process has run a single instruction. Subscription happens later, inside the loop, when FakeLane.subscribe/1 sends {:subscribed, self()} to the test pid (live_session_driver_test.exs:58).

So the test's only barrier is:

assert_receive {:subscribed, forwarder_pid}, 2_000   # :317

That 2000 asserts nothing about behaviour. It is a setup barrier wearing a latency assertion's clothes, and what it actually measures is whether the scheduler got round to the driver process within two seconds.

The file is async: true with 17 tests, so with max_cases: 16 nearly all of them stand up a driver, a forwarder, a fake session and a render cadence concurrently. That is enough to starve the barrier on a busy machine. The file has ten assert_receive calls in total, so setup is the most common casualty rather than the only one.

Evidence

Same machine, same seed, back to back, running only this file:

tree --seed 0 results
master 17, 17, 17
master (unseeded) 17, 16
a feature branch 16, 14

Wall clock on the same file swings between ~40s and ~125s, and the slow runs are the ones that fail. Boot budget swings with it (811ms vs 3361ms on two master runs).

Two things worth separating:

  • It fails on master on its own, so this is not owned by any branch in flight.
  • CI does not currently catch it. The file carries no @moduletag, and the CI filter is --exclude integration --exclude property --exclude slow --exclude skip_on_ci, so CI runs it and passes on an uncontended runner. This only shows up where something else is competing for the machine.

Amplifying factor: #975 fixes a Code.ensure_loaded/1 call on the per-character charset path that costs ~0.5ms per translated character and serializes every test on the code server. Any ANSI-heavy suite is far more likely to trip a wall-clock barrier while that is unfixed. That makes this worse; it is not the cause, since the barrier is unsound either way.

Suggested fix

The barrier should be a handshake, not a duration.

  1. Make start_link/1 return only once the driver is ready. Spawn, then wait for a ready ack from the spawned process before returning {:ok, pid}. start_link returning then is the synchronization, and the setup helper can drop its assert_receive entirely. This is the fix that removes the class rather than widening the window.
  2. If the driver must stay fire-and-forget, give the helper a barrier with no wall clock in it, e.g. a monitored receive with :infinity plus ExUnit's own test timeout as the real backstop. A test that hangs is a better failure than nine that lie about which assertion broke.
  3. Raising 2_000 is the cheap patch. It reduces the frequency and keeps the fragility.

Whichever is chosen, the other nine assert_receive calls in this file deserve the same read: which of them are asserting a latency the driver actually promises, and which are just waiting for something to happen?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions