test: update 7.14.2 presign authentication ordering

[BitHighlander]

7.14.2 release tests

Current exact head: 36bc792368800aa6e976e5333bd43654522fa2e3.

This PR:

• uses semantic firmware-version gates for the 7.14.2 protocol changes;
• proves cancellation and invalid recovery failures end the active workflow;
• adds OLED A/B checks for Solana, TON, Ethereum network assets, and other signed display fields;
• proves chain 257 cannot borrow chain-1 ERC-20 metadata in standard or OutputAddressType_TRANSFER review;
• pins the exact Unknown token value OLED frame for chain-257 transfer, approve, and OutputAddressType_TRANSFER calls to 0xeeee..eeee;
• registers all chain-ID and native pseudo-address regressions in the report's mandatory OLED screenshot set;
• rejects empty or omitted Osmosis signing values before review;
• vendors transaction fixtures and denies explorer/API access during tests;
• makes the release report reconcile every test and retain every required OLED frame.

Exact-head verification in progress

• Firmware #458 pins this SHA at firmware 0a5c21729c37a40af16492085e4537aa94ffdd3e.
• Python exact-head GitHub CI, CodeQL, and CircleCI are running.
• Firmware exact-head PR CI and push CI are running.
• Local full Python emulator suite: 432 passed, 47 documented skips, 0 failures/errors.
• The two new emulator regressions produced exactly 10 OLED PNGs in three manifested transaction sequences, with no setup or policy-cleanup frames.
• Final combined test counts, PDF frame reconciliation, ARM hashes/parity, and release firmware SHA-256 remain pending exact-head firmware artifacts.

The previous 616-test/619-frame evidence and firmware hash belong to superseded heads.

After exact-head evidence is reconciled and this PR receives independent approval, merge it, verify the durable master tree, repin firmware #458, and rerun final firmware CI before hardware/storage checks or signing. No merge, tag, or signing is authorized by this evidence.

[BitHighlander]

Exact-head evidence for c0dcb40880e1feaff3485185bc9428516678b8c9:

• lint: passed
• CodeQL (Python and Actions): passed
• integration: passed
• CircleCI emulator build/test: passed
• the four new Solana OLED A/B regressions also pass against firmware #458 candidate 48f995e6b7152f1f5fdebe26b6b0d05449627489

This PR still requires independent approval. After merge, firmware #458 must verify and repin the durable master commit and rerun its complete exact-head gate; the current firmware candidate is not signing authorization.

[BitHighlander]

Ready for review

Python #219 at 36e052dd is green and is pinned by firmware #458 at 2d2153ba, whose full exact-head CI also passes.

There are no known code or CI blockers. After approval and merge, firmware will repin the final commit and run the last CI and hardware check.

[BitHighlander]

Native pseudo-address regressions are pushed at Python 36bc792368800aa6e976e5333bd43654522fa2e3 and pinned by firmware #458 at 0a5c21729c37a40af16492085e4537aa94ffdd3e.

The new tests pin the exact Unknown token value OLED review for chain-257 standard transfer, approve, and OutputAddressType_TRANSFER calls to 0xeeee..eeee. Locally, the full emulator suite is green (432 passed, 47 documented skips), and the new cases produce exactly 10 OLED PNGs in three manifested sequences with no setup/policy residue.

Python and firmware exact-head CI are in progress. The previous 616-test/619-frame evidence and release firmware hash are superseded; final report, OLED, ARM hash, and parity claims remain pending the new firmware artifacts.

No merge, tag, or signing was performed.

[Copilot]

Pull request overview

This PR updates the Python-based firmware test suite and CI plumbing for the KeepKey 7.14.2 presign-security changes, emphasizing offline/deterministic fixtures, strict network denial, and stronger OLED evidence capture/reporting.

Changes:

• Replace live transaction/network dependencies with an offline fixture manifest + enforcement (and CI verification).
• Add/extend 7.14.2 regression tests for display disclosure, refusal/terminal-state boundaries, and chain-ID/metadata isolation.
• Harden OLED capture and reporting so evidence is complete, deterministic, and fail-closed.

Reviewed changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/zcash_rpc.py	Removes a live-node RPC helper (eliminates hardcoded network dependency).
tests/txcache/manifest.json	Adds authoritative fixture manifest describing all offline tx fixtures.
tests/txcache/insight_dash_tx_acb3b7f259429989fc9c51ae4a5e3e3eab0723dceb21577533ac7c4b4ba4db5d.json	Removes an unused tx fixture JSON.
tests/txcache/insight_bitcoin_tx_4a405a771b1c16af9059e01aa1de19ae1e143da6a5a8d130d1591875a93f9e0c.json	Removes an unused tx fixture JSON.
tests/tx_fixture_manifest.py	Introduces tooling to build/check the deterministic tx fixture manifest and reject network-capable test code.
tests/test_verify_typed_data.py	Makes JSON fixture loading cwd-independent (but should also specify encoding).
tests/test_tx_fixture_integrity.py	Adds tests to prove fixture integrity and that missing fixtures never fall back to HTTP.
tests/test_sign_typed_data.py	Makes JSON fixture loading cwd-independent (but should also specify encoding).
tests/test_protection_levels.py	Gates expected response ordering on semantic firmware version (7.14.2 auth ordering).
tests/test_network_policy.py	Adds explicit tests asserting non-emulator transports/HTTP are denied.
tests/test_msg_ton_display_disclosure.py	Adds TON display-disclosure regression tests.
tests/test_msg_solana_signtx.py	Updates Solana signing tests for 7.14.2 policy/disclosure behaviors.
tests/test_msg_solana_instruction_disclosure.py	Adds adversarial Solana instruction disclosure A/B controls.
tests/test_msg_solana_display_disclosure.py	Adds Solana message/memo display-disclosure regression tests.
tests/test_msg_signtx_ethereum_erc20.py	Adds 7.14.2 refusal expectations for unlimited approvals.
tests/test_msg_signing_boundaries.py	Adds terminal-state/bounds tests for signing stream correctness.
tests/test_msg_ripple_get_address.py	Adds a “show address” OLED retention case.
tests/test_msg_resetdevice.py	Improves OLED capture correctness for reset ceremony pages and adds 18/24-word cases.
tests/test_msg_recoverydevice_cipher.py	Adds OLED capture timing improvements and a terminal-failure recovery boundary test.
tests/test_msg_ping.py	Adds a cancellation-terminal boundary regression test.
tests/test_msg_osmosis_validation.py	Adds input validation refusal tests for missing/empty Osmosis fields.
tests/test_msg_ethereum_signtx.py	Adds chain-ID isolation and “unrenderable amount” refusal tests; strengthens blind-sign policy assertions.
tests/test_msg_ethereum_signtx_xfer.py	Adds transfer/asset binding and chain-ID width binding tests.
tests/test_msg_ethereum_getaddress.py	Adds a “show address” OLED retention case.
tests/test_msg_ethereum_erc20_approve.py	Adds 7.14.2 refusal expectations for unlimited approvals.
tests/test_msg_ethereum_data_disclosure.py	Adds calldata hash tail A/B tests ensuring streamed bytes are bound to OLED review.
tests/test_msg_display_disclosure.py	Strengthens ScreenRecorder capture correctness + groups/manifesting for OLED evidence.
tests/test_msg_confirm_data_disclosure.py	Adds confirm_data disclosure A/B tests across BTC/EOS/Omni paths.
tests/test_msg_changepin.py	Adds explicit OLED capture calls around PIN-change flows.
tests/conftest.py	Implements screenshot selection, skipped-evidence cleanup, and deny-external-network autouse fixture + manifest checks.
tests/common.py	Configures offline fixtures, adds screenshot reset helper, and introduces semantic firmware version helpers.
scripts/generate-test-report.py	Improves JUnit accounting, screenshot evidence inclusion, and metadata plumbing for release reports.
keepkeylib/tx_api.py	Adds offline-only fixture mode and fail-closed errors when fixtures are missing/malformed.
keepkeylib/client.py	Makes OLED capture fail-closed, adds settled/animated capture helpers, and improves ButtonRequest evidence timing.
.github/workflows/ci.yml	Adds fixture manifest verification and network egress gating during pytest.
.circleci/standalone-emulator-tests.sh	Adds a standalone emulator test runner script for CircleCI lane separation.
.circleci/config.yml	Uses the standalone emulator test runner and clarifies evidence separation from firmware CI.

Suppressed comments (1)

tests/test_msg_solana_signtx.py:295

• These negative-path assertions currently accept any CallException, which can hide unrelated transport/timeout errors. Assert at least a stable substring in the error to ensure the refusal reason is the policy/opacity gate being tested.

        with pytest.raises(CallException):
            self.client.call(messages.SolanaSignTx(
                address_n=parse_path("m/44'/501'/0'/0'"), raw_tx=raw_tx))

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

tests/common.py:57

• reset_screenshot_capture() only deletes top-level btn*.png/frames.json, but leaves any screenshot_group subdirectories (and their PNGs/frames.json) intact. Re-running screenshot tests in a non-clean workspace can therefore retain stale OLED evidence or trigger spurious manifest mismatches in later phases. Consider clearing the entire per-test directory contents (including subdirectories) before resetting screenshot_id.

    os.makedirs(screenshot_dir, exist_ok=True)
    for name in os.listdir(screenshot_dir):
        if (name.startswith('btn') and name.endswith('.png') and
                len(name) == len('btn00000.png')) or name == 'frames.json':
            os.unlink(os.path.join(screenshot_dir, name))

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.