Exploring fast decimal floating-point compression in Rust: benchmark observations and SIMD throughput from fastalp

[x-at-01]

In real-time stream processing and event-driven analytical storage (such as financial tickers, cryptocurrency order books, network metrics, and IoT sensor streams), high-frequency floating-point metrics represent a substantial portion of in-memory state and LSM-tree SSTable storage. Traditional floating-point codecs like Gorilla or Chimp rely on sequential bitstreams with variable-length bit-shifts, which can constrain single-core decompression throughput during vectorized stream scans and state checkpoints.

Over the past months, I have been developing fastalp (available on crates.io), an implementation and optimization of the Adaptive Lossless Floating-Point (ALP) compression algorithm written in pure safe Rust. Instead of XORing successive IEEE 754 representations, it samples block values to detect the decimal exponent e, transforms floats into exact integers via multiplication by 10^e, and encodes them using Frame-of-Reference (FOR) bit-packing.

Key architectural characteristics:

• Decimal integer mapping: Converts decimal floating-point numbers into integers without loss of precision. Any non-exact values are handled via an exception buffer.
• Vectorized SIMD decoding: Bit-unpacking processes fixed-width integer vectors directly into CPU registers, delivering 55 to 77 GB/s decompression throughput on modern x86_64 / aarch64 architectures.
• Zero-allocation API: Provides compress_into and decompress_into methods to eliminate heap allocations in hot storage engine loops.
• Differential mode: Includes an adaptive Delta-before-ALP path for monotonic counters and trending metrics.
• Zero expansion guarantee: Automatically falls back to uncompressed raw storage when encountering high-entropy noise.

Benchmark results measured on standard time-series datasets (weather telemetry, stock market prices, disk metrics) with 1,000 double-precision values per batch:

Algorithm	Compressed Size (bits/val)	Compression Latency (µs / 1000 vals)	Decompression Latency (µs / 1000 vals)	Decompression Throughput
fastalp	16.34	2.255	0.423	2364 Mval/s
Chimp128	17.29	8.631	9.270	108 Mval/s
Patas	21.51	3.820	5.140	195 Mval/s
Gorilla	52.70	6.042	5.920	169 Mval/s

In these measurements, fastalp achieved 16.34 bits/val while completing decompression in 0.423 µs per 1,000 values (2.36 billion values per second on a single core).

I hope these benchmark figures and architectural observations offer helpful reference for storage engine encoding evaluations and block codec discussions in RisingWave.

[x-at-01]

Performance update: fastalp v0.1.37 release

I have updated the benchmark observations with the latest fastalp v0.1.37 release (published on crates.io with source code at fastalp).

Key improvements and updated metrics:

1. Decompression throughput: reached 32.53 GB/s (a 14.7% increase from 28.36 GB/s), compared to reference C++ ALP at ~20.0 GB/s and Gorilla / Chimp decoders at 1.5 ~ 2.5 GB/s. The speedup comes from zero-cost monomorphized decoder traits and direct raw pointer writes without temporary slice allocations.

2. Compression ratio: across the 37 standard time-series datasets from the SIGMOD 2024 ALP benchmark, the total compressed volume dropped from 104,465 bytes to 93,909 bytes (a 10.1% footprint reduction and +11.2% compression ratio improvement, reaching 3.23x overall and 6.99x geometric mean). Smooth datasets with continuous trends reach up to 431x via relaxed delta threshold evaluation.

3. Compression throughput: 4.87 GB/s with full dynamic parameter sampling, and 6.02 GB/s for pure kernel encoding with cached parameters (vs 5.45 GB/s for reference C++ ALP).

4. Production-grade memory soundness: full zero-allocation slice APIs (compress_into and decompress_into_raw) with bounded expansion on incompressible streams.