python-integration-tests deadlocks: this release adds confirm screens the pinned tests never ack

[BitHighlander]

python-integration-tests has been carried through this release as "a pre-existing CI blocker". It is now diagnosed rather than merely observed.

It is not a slow suite — it is two files that stall

Reproduced locally against the pinned emulator image, at both 440619703 (pre-release-work base) and head:

file	base 4406197	head
test_msg_eos_signtx.py	19 tests pass, then stalls	same
test_msg_thorchain_signtx.py	5 tests pass, then stalls	same

Each was killed at a 60s local timeout. Everything else in the suite completes in well under a second per file. So the 30-minute CI wall clock is consumed by these two, and the job is killed rather than failing.

Why that matters more than a red check

scripts/emulator/python-keepkey-tests.sh runs Phase 1 (screenshot capture, || true) and then Phase 2 — the documented actual gate. Because the job is killed inside Phase 2, Phase 2 never completes on this branch, generate-test-report.py never validates against SECTIONS, and every test file ordered after the stall is simply unmeasured.

A hang is worse than a failure here: it produces no JUnit XML for the affected range, so absence of a failure report is not evidence of passing.

Measurement trap for whoever picks this up

Do not measure by running the suite sequentially against one emulator. A killed hang leaves the device wedged, and every subsequent file reports spurious collection errors and failures. I initially recorded five collection errors and four failures this way; re-running each file against a fresh emulator showed most of them clean. Use one emulator per file.

Suggested fix direction

Bound the stall rather than the job: add a per-test timeout (pytest-timeout) so a stuck test fails its own test instead of consuming the whole budget, and give the two files a real diagnosis separately. Related prior context: THORChain tests have historically depended on network hosts that are now dead.

[BitHighlander]

Correcting this issue, including its title. I filed it as a "pre-existing CI blocker". That is wrong — this release introduced it.

The controlled experiment

Two tests hang: test_eos_signtx_updateauth and test_thorchain_sign_tx (both hang in isolation, so it is not test-order contamination).

firmware	test files	result
1af2ffe7de (pre-release base)	base pin fabd6c618	both pass (0.28s, 0.30s)
release/7.14.2 head	same base test files	both hang

Holding the test files constant and changing only the firmware reproduces it. Earlier I compared head against 440619703 — which already contained most of this release's work — and wrongly concluded "pre-existing".

Cause, from this release's own commit messages

This is not a defect. It is the release working as designed, with tests that have not caught up.

354315c65 (#415) states it outright:

Any memo with fields past the labelled ones now gets one confirm screen per extra field, titled "Additional memo field". A normal ShapeShift swap memo carries :<affiliate>:<bps>, so a THORChain swap gains 2 screens on all three paths.

424294ccc (exact-byte disclosure) paginates host-supplied signed payloads, adding pages for long EOS updateauth permission data.

The pinned suite acks a fixed number of ButtonRequests. Extra screens exhaust that sequence, the device blocks waiting for a ButtonAck that never arrives, and the test blocks waiting for a response. Deadlock — which is why it presents as a 30-minute timeout rather than a failure.

The affiliate-fee bytes those extra screens disclose were previously signed invisibly. Losing the screens would be the regression; the hang is the tests being out of date.

What the fix actually is — two parts, different owners

1. Update the pinned tests to acknowledge the new screens. This belongs with the deps/python-keepkey repin already blocked on test(evm): require chain_id, and stop asserting pre-EIP-155 signatures keepkey/python-keepkey#215. It is test work, not firmware work, and it must not be "fixed" by removing disclosure.

2. Add a per-test timeout to the harness (pytest-timeout, or an explicit per-test alarm) so a blocked test fails itself in seconds instead of consuming the entire 30-minute job budget and taking Phase 2 down with it. This is the part that makes the failure legible, and it is worth doing regardless of (1).

Both are still required before this branch can produce a green run. What changes is the diagnosis: this is not infrastructure rot to be worked around, it is a known, intentional behaviour change with a stale test suite and a failure mode that hides it.

Related

The silent-skip half of this — a killed job taking the whole downstream graph with it and showing green ticks on whatever finished — is closed separately by the aggregate gate in #474.

[BitHighlander]

The full inventory, now that #477 makes it visible

With a per-test timeout the suite completes in 133.59s: 22 failed, 362 passed, 47 skipped, 66KB of JUnit XML. Previously: a 30-minute job kill and no XML at all.

Exact failure messages, not categories I inferred:

count	message	what it is
9	Transaction signing disabled by policy	TON SignTx behind AdvancedMode — documented BREAKING change
5	Chain Id out of bounds	#445 working as designed; fixed by keepkey/python-keepkey#215
3	Enable AdvancedMode to blind-sign	TRON SignTx — documented BREAKING change
2	Failed: Timeout >60.0s	the deadlocks: test_eos_signtx_updateauth, test_thorchain_sign_tx
1	Arbitrary contract data signing disabled by policy	0x transformERC20 — see below
1	Enable AdvancedMode to blind-sign typed hashes	EIP-712 precomputed hash — documented
1	Structured EIP-712 disabled pending canonical display hardening	deliberately disabled for this release

Correction to my earlier comment: I reported "1 CallException still to be triaged individually". That was an artifact of my own keyword bucketing, not a real unexplained failure. Every one of the 22 is accounted for above.

The one worth naming separately

test__sign_transformERC20 fails with Arbitrary contract data signing disabled by policy even though python-keepkey master's copy of that test enables AdvancedMode. The pinned commit 81e581f does not — it has no apply_policy call at all. Controlled: the pinned file passes against 1af2ffe7de and fails against head, so #468 is what moved it (its 1480-byte payload now falls through to the AdvancedMode-gated raw path).

Same bucket as the rest — intentional change, stale pinned expectation — and master already accounts for it.

Consequence for the repin

All 22 are stale-expectation failures. None indicates a firmware defect. That makes the deps/python-keepkey repin the single action that resolves the majority: the chain_id five are fixed in #215, and test__sign_transformERC20 is already fixed on master. The remainder are policy-gate opt-ins and the two button-flow sequences, which are mechanical.

Worth stating plainly for whoever does that work: the fix is to teach the tests the new screens and policies, never to remove disclosure to make them pass. The extra THORChain screens disclose affiliate-fee bytes that were previously signed invisibly.