Native decoder test suite with real meter payload corpus#1206
Merged
Conversation
Establish host-side decoder testing (`pio test -e native`), which never actually linked before, and drive it with a corpus of real captured meter frames. Test corpus (test/payloads/) - 164 real frames mined from GitHub issues and the maintainer's mail, organised by manufacturer with per-type READMEs and a manifest. - Decryption keys kept out of git (keys.local.json, gitignored); injected via GitHub Actions secrets. keymap.json links 15 fixtures to verified keys. Native test harness (test/test_decoder/) - decoder_harness mirrors PassiveMeterCommunicator's unwrap+dispatch (HDLC/LLC/MBUS/GBT/DLMS/DSMR -> IEC6205675/LNG/LNG2/IEC6205621), incl. multi-segment M-Bus reassembly. - Golden-master sweep over all unencrypted fixtures + per-meter tests. - Encrypted end-to-end decode tests (gh73/501/787/905) via native mbedTLS; self-skip when mbedTLS or keys are unavailable. - test/stubs/: minimal Arduino/WString/EEPROM/Timezone/PROGMEM shims so the decoder builds on host; platformio native env gets test_build_src + a pre-build probe that enables mbedTLS when libmbedtls-dev is present. Production changes - GcmParser: native (NATIVE_TEST + HAVE_MBEDTLS) AES-GCM decrypt branch. - DsmrParser: accept LF as well as CRLF telegram line endings. - GbtParser: include <stdlib.h> (was relying on a transitive include). CI - build.yml and pull-request.yml install libmbedtls-dev and run the native tests, passing key secrets for the encrypted-decode tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- GcmParser native branch: select the mbedtls_gcm_starts/update API by MBEDTLS_VERSION_MAJOR. Ubuntu's libmbedtls-dev (CI) is 2.x with the older 4-arg update; 3.x uses the 6-arg form. Auth path is identical on both. - Remove reporter real names / private email identifiers from the payload READMEs and test comments (keep public GitHub issue references). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Harness now reassembles multi-frame GBT (General Block Transfer): each block rides in its own HDLC frame, so the unwrap loop jumps frame-to-frame (resetting the per-read length budget) until the final block. Added landis-gyr/gh740.hex (the three gh740 blocks concatenated) which now decodes to a full L&G list (id 63326413). - Removed 22 fixtures that begin mid-stream (no frame boundary at byte 0): clipped/leading-flag-missing captures and raw application-data dumps. These never decoded and aren't representative wire data. - Regenerated manifest/fixtures/golden and refreshed README counts (143 files). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gh787-* (Iskraemeco) and gh73-* (Kamstrup) were keyed via keymap.json but not flagged in their per-manufacturer tables. Add the 🔑 marker and include the keyed count in each header line, consistent with gh501/gh905. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- test_encrypted_framing_no_key probes every encrypted fixture we have no key for through the transport framing (HDLC/M-Bus/LLC) and the GCM-header parse with a dummy key, asserting no crash and that frames reaching the GCM layer yield a system title. Needs no mbedTLS (system title is read pre-decryption). - GcmParser: guard the ciphertext length (len - authkeylen - 5) and the auth-tag read against underflow on short/garbage lengths — this smoke test hit a stack blow-up (VLA) when a wrong/dummy key meets a cleartext DSMR telegram. - Reclassify elgama/gh1177-1.txt as unencrypted: it's the decoded cleartext telegram (not a GCM frame), so it decodes directly and joins the golden sweep. - gen_fixtures_header.py emits ENC_NOKEY; harness gains a probe entry point. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Branch pushes are now validated by the PR workflow (build per env + native tests), so the Build workflow only needs to run on main and release tags. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🔧 PR Build ArtifactsVersion: All environments built successfully. Download the zip files:
|
gskjold
added a commit
that referenced
this pull request
Jun 18, 2026
…1210) When an encryption key is configured but the meter sends a plaintext DSMR/P1 telegram, DSMRParser routes the telegram to GCMParser. The first cleartext byte ('0' == 0x30 == 48) was read as the system-title length and memcpy'd into the 8-byte ctx.system_title and 12-byte initialization_vector, overflowing the stack and rebooting the device in a loop (reboot reason "Software reset (3/0)"). Reported for a Kamstrup OMNIA (KAM5) in Denmark: POW-P1 up ~10s then reboot; disabling encryption worked around it. The #1206 hardening guarded the ciphertext length but not the system-title length. Reject system titles longer than 8 bytes (the DLMS maximum) before the memcpy, turning the crash into a clean GCM_DECRYPT_FAILED that the existing error handling reports gracefully. Adds a native regression test feeding the reported plaintext frame to a key-configured decoder, and enables AddressSanitizer on the native test env so this class of overflow fails the suite deterministically. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stands up host-side decoder unit testing (
pio test -e native) — which never actually linked before — and drives it with a corpus of 164 real captured meter frames mined from GitHub issues and the maintainer's email archives.13 test cases, all green, covering every unencrypted frame (golden-master) plus end-to-end decryption of the keyed encrypted frames via native mbedTLS.
What's here
Payload corpus —
test/payloads/manifest.json.keys/keys.local.json, gitignored) and injected via GitHub Actions secrets.keys/keymap.jsonlinks 15 fixtures to verified keys.Native test harness —
test/test_decoder/decoder_harnessmirrorsPassiveMeterCommunicator's unwrap + dispatch (HDLC/LLC/MBUS/GBT/DLMS/DSMR → IEC6205675/LNG/LNG2/IEC6205621), incl. multi-segment M-Bus reassembly.test/stubs/Arduino/WString/EEPROM/Timezone/PROGMEM shims; native env getstest_build_src+ a pre-build mbedTLS probe.Production changes
GcmParser: native (NATIVE_TEST+HAVE_MBEDTLS) AES-GCM decrypt branch.DsmrParser: accept LF as well as CRLF telegram line endings.GbtParser:#include <stdlib.h>(was relying on a transitive include).CI
build.ymlandpull-request.ymlinstalllibmbedtls-devand runpio test -e native, passing key secrets for the encrypted-decode tests.Before merging
keys/README.mdreferences a few reporters by name (from support email); review if that should be anonymized for a public repo.🤖 Generated with Claude Code