Raxol.Harness.LiveSessionDriverTest has no real barrier between starting a driver and driving it. Its setup helper waits on a wall clock instead, so under load the whole file fails together.
Symptom
Nine or more tests fail at once, all identically, all inside the shared setup:
1) test 7. malformed events are rejected at the boundary and marked ...
test/harness/live_session_driver_test.exs:660
Assertion failed, no matching message after 2000ms
The process mailbox is empty.
code: assert_receive {:subscribed, forwarder_pid}
stacktrace:
test/harness/live_session_driver_test.exs:317: ...new_driver/2
They fail as a group because they share new_driver/2, so one bad moment takes the file.
Mechanism
LiveSessionDriver.start_link/1 does not synchronize at all:
def start_link(opts) do
pid = spawn_link(__MODULE__, :run, [opts])
{:ok, pid}
end
It returns {:ok, pid} before the spawned process has run a single instruction. Subscription happens later, inside the loop, when FakeLane.subscribe/1 sends {:subscribed, self()} to the test pid (live_session_driver_test.exs:58).
So the test's only barrier is:
assert_receive {:subscribed, forwarder_pid}, 2_000 # :317
That 2000 asserts nothing about behaviour. It is a setup barrier wearing a latency assertion's clothes, and what it actually measures is whether the scheduler got round to the driver process within two seconds.
The file is async: true with 17 tests, so with max_cases: 16 nearly all of them stand up a driver, a forwarder, a fake session and a render cadence concurrently. That is enough to starve the barrier on a busy machine. The file has ten assert_receive calls in total, so setup is the most common casualty rather than the only one.
Evidence
Same machine, same seed, back to back, running only this file:
| tree |
--seed 0 results |
master |
17, 17, 17 |
master (unseeded) |
17, 16 |
| a feature branch |
16, 14 |
Wall clock on the same file swings between ~40s and ~125s, and the slow runs are the ones that fail. Boot budget swings with it (811ms vs 3361ms on two master runs).
Two things worth separating:
- It fails on
master on its own, so this is not owned by any branch in flight.
- CI does not currently catch it. The file carries no
@moduletag, and the CI filter is --exclude integration --exclude property --exclude slow --exclude skip_on_ci, so CI runs it and passes on an uncontended runner. This only shows up where something else is competing for the machine.
Amplifying factor: #975 fixes a Code.ensure_loaded/1 call on the per-character charset path that costs ~0.5ms per translated character and serializes every test on the code server. Any ANSI-heavy suite is far more likely to trip a wall-clock barrier while that is unfixed. That makes this worse; it is not the cause, since the barrier is unsound either way.
Suggested fix
The barrier should be a handshake, not a duration.
- Make
start_link/1 return only once the driver is ready. Spawn, then wait for a ready ack from the spawned process before returning {:ok, pid}. start_link returning then is the synchronization, and the setup helper can drop its assert_receive entirely. This is the fix that removes the class rather than widening the window.
- If the driver must stay fire-and-forget, give the helper a barrier with no wall clock in it, e.g. a monitored receive with
:infinity plus ExUnit's own test timeout as the real backstop. A test that hangs is a better failure than nine that lie about which assertion broke.
- Raising
2_000 is the cheap patch. It reduces the frequency and keeps the fragility.
Whichever is chosen, the other nine assert_receive calls in this file deserve the same read: which of them are asserting a latency the driver actually promises, and which are just waiting for something to happen?
Raxol.Harness.LiveSessionDriverTesthas no real barrier between starting a driver and driving it. Its setup helper waits on a wall clock instead, so under load the whole file fails together.Symptom
Nine or more tests fail at once, all identically, all inside the shared setup:
They fail as a group because they share
new_driver/2, so one bad moment takes the file.Mechanism
LiveSessionDriver.start_link/1does not synchronize at all:It returns
{:ok, pid}before the spawned process has run a single instruction. Subscription happens later, inside the loop, whenFakeLane.subscribe/1sends{:subscribed, self()}to the test pid (live_session_driver_test.exs:58).So the test's only barrier is:
That 2000 asserts nothing about behaviour. It is a setup barrier wearing a latency assertion's clothes, and what it actually measures is whether the scheduler got round to the driver process within two seconds.
The file is
async: truewith 17 tests, so withmax_cases: 16nearly all of them stand up a driver, a forwarder, a fake session and a render cadence concurrently. That is enough to starve the barrier on a busy machine. The file has tenassert_receivecalls in total, so setup is the most common casualty rather than the only one.Evidence
Same machine, same seed, back to back, running only this file:
--seed 0resultsmastermaster(unseeded)Wall clock on the same file swings between ~40s and ~125s, and the slow runs are the ones that fail. Boot budget swings with it (811ms vs 3361ms on two
masterruns).Two things worth separating:
masteron its own, so this is not owned by any branch in flight.@moduletag, and the CI filter is--exclude integration --exclude property --exclude slow --exclude skip_on_ci, so CI runs it and passes on an uncontended runner. This only shows up where something else is competing for the machine.Amplifying factor: #975 fixes a
Code.ensure_loaded/1call on the per-character charset path that costs ~0.5ms per translated character and serializes every test on the code server. Any ANSI-heavy suite is far more likely to trip a wall-clock barrier while that is unfixed. That makes this worse; it is not the cause, since the barrier is unsound either way.Suggested fix
The barrier should be a handshake, not a duration.
start_link/1return only once the driver is ready. Spawn, then wait for a ready ack from the spawned process before returning{:ok, pid}.start_linkreturning then is the synchronization, and the setup helper can drop itsassert_receiveentirely. This is the fix that removes the class rather than widening the window.:infinityplus ExUnit's own test timeout as the real backstop. A test that hangs is a better failure than nine that lie about which assertion broke.2_000is the cheap patch. It reduces the frequency and keeps the fragility.Whichever is chosen, the other nine
assert_receivecalls in this file deserve the same read: which of them are asserting a latency the driver actually promises, and which are just waiting for something to happen?