You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit d2ddf39
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: CHANGELOG.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,6 +10,9 @@ Version numbers follow [Semantic Versioning](https://semver.org/).
10
10
## [Unreleased] — 2.0.0
11
11
12
12
### Fixed
13
+
- **`setBklen()` rounded to an even value, which is only sufficient for a 2-byte symbol; `uint8_t` codebooks at a bklen not divisible by 4 silently decoded to wrong symbols.** The reverse codebook occupies `4*(2*32) + sizeof(T)*bklen` bytes and the bitstream begins immediately after it, so the bitstream is 4-aligned only when `sizeof(T)*bklen` is. Even-rounding satisfies that for `uint16_t` (the case it was written for) and `uint32_t` is aligned at any bklen, but a `uint8_t` book needs a multiple of **4**. At bklen 30 the bitstream landed at offset 2 mod 4, the decode kernel read it as `uint32*`, and a lossless round-trip returned symbol 0 where symbol 1 began — **reporting success**. Worse than wrong data: the misaligned access poisons the CUDA context, so every later allocation in the process fails with `cudaMalloc failed: misaligned address`, pointing nowhere near this stage (observed as 5 unrelated Huffman tests failing after the first). `setBklen()` now rounds up until `sizeof(T)*bklen % 4 == 0`. Not reachable through any shipped preset — they use bklen 256 and 1024, both aligned — so this was latent. `deserializeHeader()` takes the archive's bklen verbatim rather than rounding it (decode must reproduce the encoder's layout, so rounding there would describe a different codebook) and instead **throws** on an unaligned value, since such archives are malformed at the source. Test `HuffmanStage.PerBlockFallsBackToAdaptiveOnOverlongCode` uses bklen 30 deliberately and covers both this and the fallback below.
14
+
- **A histogram that could not fit the 27-bit code field failed the whole compression instead of falling back.** `PerBlock` threw whenever some symbol needed a code wider than `HuffmanWord<4>`'s field. Failing loudly was right compared to the silent clamping it replaced, but it is the wrong answer for *data* — it cost the cuSZ preset every wide-dynamic-range field in the corpus (HACC/vy, HACC/xx, EXAFEL/data all hard-failed at multiple error bounds). The stage now falls back to an `Adaptive` (frequency-floored) book, which is **not** a relaxation of the error bound — the bound belongs to the quantizer, and flooring frequencies changes only how symbols are spelled. Measured: HACC/vy at 1e-2 gives CR 15.27x at 44.78 dB, HACC/xx at 1e-3 gives 13.79x at 64.77 dB, both with `max_abs_err` exactly equal to the bound, and both deterministic across repeated runs. `buildAdaptiveBook()` halves the floor shift until the book fits and shift 0 is uniform, so the fallback always terminates. The fallback is sticky per stage, logs one `FZ_LOG(WARN)`, and is queryable via `getAdaptiveFallbackUsed()`; the configured `getBookSource()` is left untouched so it still reports what was asked for. A **user-supplied** frequency table that cannot be coded is a configuration error and still throws (test HF30 unchanged).
15
+
-**The CLI installed no log sink unless `-v` was passed, so every `FZ_LOG(WARN)` in the library went nowhere.** WARN is exactly the level used for "recoverable, but you need to know" — outlier drops, codebook fallbacks, deprecated-mode remapping — and none of it reached the user or the benchmark logs. This project has already paid for that twice: the outlier-overflow corruption was invisible for precisely this reason and had to be promoted to a `throw` to be seen at all. Stderr logging is now on at WARN by default; `-v`/`-vv`/`-vvv` raise it to INFO/DEBUG/TRACE as before. Turning it on immediately surfaced two pre-existing warnings that had never been visible in practice: the `Fine` encode falling back to the coarse path, and `MemoryPool` growing past its configured size.
13
16
- **`QuantizerStage<double, …>` quantized at float32 precision, silently violating the error bound on f64 fields.** All four ABS/NOA forward kernels rounded with an unconditional `__float2int_rn(static_cast<float>(x) * static_cast<float>(ebx2_r))`, and the inverse reconstructed through a `float` — so `TInput = double` was nominal: the stage is templated and instantiated for `double`, but the arithmetic was never wider than f32. Any f64 field whose range is small next to its offset is unrepresentable that way. S3D/N2 spans 1.10e-5 about 0.7369, where the float32 spacing (5.96e-08) is **54x** the `abs_eb` a 1e-4 NOA bound asks for (1.103e-09) — the rounding error alone blew the bound before quantization did any work. Four cells (`cuszp2`/`cuszp3` x {plain, outlier}) round-tripped S3D/N2 at 48 dB with `status: ok` and `err_over_bound` **81.08 on all four**, the identical figure being the tell that the shared quantizer, not the coders, was at fault. Observed `max_abs_err` 8.941e-08 ~ 1.5x the float32 spacing, i.e. two roundings. Kernels now round via `quantRound<TInput>` (`__double2int_rn` for f64) and compare/reconstruct in `TInput`; `fabsf` on the threshold check was the same downcast and is gone. **Reachable only in `linear_mode`** — the radius-clamped path needs `|x|/abs_eb < 2*quant_radius` for an inlier, which forces `abs_eb` above the f32 spacing at `|x|`, so the radius check always escalates to an outlier first. That is why the cuSZp-style presets were hit and the others were not. The bound is also carried at full precision now: `computeValueBase()` returns `TInput` instead of narrowing the NOA range to `float`, and `QuantizerConfig` gained `abs_error_bound_f64` / `value_base_f64`, because decode computes `recon = q * 2*abs_eb` with `q ~ 3.3e8` — a 6e-08 *relative* error in a float `abs_eb` lands as 4.4e-08 absolute, 40x the bound. Old headers (<= 56 B) leave the new fields 0 and fall back to the float copies. **Throughput is unchanged**: the kernel is memory-bound, and a 64 Mi-element probe over f32/f64 x linear/radius x ABS/NOA moves every case by under 1.2%, within run-to-run noise (f64 linear NOA 0.4943 -> 0.4941 ms; f32 linear ABS 0.2275 -> 0.2271). Tests `QuantizerLinear.DoubleTinyRangeLargeOffsetRespectsBound` (fails at 61.4x over bound without the fix) and `QuantizerNOA.RefusesBoundBelowInputTypeSpacing`.
14
17
- **`LorenzoQuantStage` narrowed a computed f64 error bound to float32 twice, violating the bound on relative-mode f64 fields.** Its quantization kernel was already correct (`round(input[i] * ebx2_r)` in `TInput`), but the bound reaching that kernel was not: `computeValueBase()` returned `float`, so `abs_eb = user_eb * value_base` was formed from a narrowed range, and `LorenzoQuantConfig::error_bound` then serialized the result as a `float`, so the decompressor read back a bound with ~6e-08 relative error. **Only reachable in a relative mode** — under `ABS` the bound comes straight from `config_.error_bound`, which is already a `float`, so the header narrowing is a no-op there and an ABS test passes vacuously. `cusz` on S3D/N2 (f64, values ~0.7369, range 1.10e-5) at a 1e-4 `rel_range` bound came back **22.73x over bound at 53.27 dB with `status: ok`**. Both narrowings are independently causal, isolated by reverting one at a time on the real cell: with neither fix 22.73x / 53.27 dB, with only the `TInput` value_base **6.62x / 64.98 dB**, with both **1.0x / 84.75 dB**. `computeValueBase()` now returns `TInput`, `computed_value_base_` is `TInput`, and the header carries `error_bound_f64` / `value_base_f64` (pre-2026-08-08 headers leave them 0 and fall back to the float copies). The v2-field check now compares against the literal 44-byte v2 size instead of `sizeof(LorenzoQuantConfig)`, which the new fields would otherwise have pushed past every existing archive. Test `LorenzoQuantHeader.DoubleNoaTinyRangeLargeOffsetRespectsBound` — a **file** round-trip, deliberately: an in-process compress/decompress reuses the live stage object and never reads the serialized header, so it cannot observe this at all. Fails at 11.9x over bound without the fix.
15
18
-**Relative error-bound modes accepted bounds no implementation could satisfy.** NOA/PREL derive `abs_eb = eb * value_base`, which can land below the input type's own resolution at the data's magnitude — the input's round-trip error through `TInput` then exceeds the bound before any compression happens. `QuantizerStage` and `LorenzoQuantStage` now compare `abs_eb` against `epsilon(TInput) * max(|data|)` (the scan already had `max(|data|)`; `computeValueBase()` just discarded it) and throw rather than emitting a stream that decodes cleanly into out-of-bound values.
0 commit comments