quantcpp 0.8.1 hotfix: kv_compress=0 default + skip dylib cross-heap free

Two real bugs found via end-user simulation testing on PyPI 0.8.0:

1. **kv_compress=1 default aborts on Llama models.** quant.h's UNIFORM_4B
   KV path is broken for Llama-3.x and SmolLM2 (Apr-6 snapshot). Default
   changed to kv_compress=0; non-zero values warn and fall back. Real fix
   waits on a fresh quant.h regen against the v0.8.0+ multi-file source
   (tracked as v0.8.2 / WASM SIMD un-stub work).

2. **libc.free(ptr) on quant_ask result aborts.** The string is allocated
   inside libquant.dylib's malloc heap; calling ctypes.CDLL(None).free on
   it crashes on macOS arm64 because Python's libc handle resolves to a
   different malloc zone than the dylib. Hotfix: skip the free, accept a
   ~65 KB leak per ask() call (released by quant_free_ctx). Tracked: add
   quant_free_string(void*) wrapper to quant.h in v0.8.2.

Verification: clean venv `pip install dist/quantcpp-0.8.1.tar.gz` →
Model("smollm2.gguf").ask("Hello") returns a real string and exits
cleanly. Zero abort.

PyPI 0.8.0 will be yanked since the most common usage path
(Model("file.gguf").ask("question")) was broken.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: unamedkr

bindings/python/pyproject.toml

@@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "quantcpp"
-version = "0.8.0"
+version = "0.8.1"
 description = "Single-header LLM inference engine with KV cache compression (7× compression at fp32 parity)"
 readme = "README.md"
 license = { text = "Apache-2.0" }

bindings/python/quantcpp/__init__.py

@@ -21,7 +21,7 @@
     from importlib.metadata import version as _pkg_version
     __version__ = _pkg_version("quantcpp")
 except Exception:
-    __version__ = "0.8.0"  # fallback for editable / source-tree imports
+    __version__ = "0.8.1"  # fallback for editable / source-tree imports
 
 import os
 import threading
@@ -76,8 +76,28 @@ def __init__(
         top_p: float = 0.9,
         max_tokens: int = 256,
         n_threads: int = 4,
-        kv_compress: int = 1,
+        kv_compress: int = 0,
     ):
+        """
+        .. note::
+           ``kv_compress=1`` and ``kv_compress=2`` are temporarily disabled in
+           the Python bindings (v0.8.x) — the bundled ``quant.h`` single
+           header carries an older KV compression path that aborts on Llama
+           architectures. The CLI ``quant`` binary uses the multi-file engine
+           and works with all KV types. KV compression will be re-enabled in
+           the bindings once ``quant.h`` is re-generated against the v0.8.0+
+           tree (tracked as v0.8.1: WASM SIMD / un-stub turbo_kv).
+        """
+        if kv_compress not in (0,):
+            import warnings
+            warnings.warn(
+                "kv_compress != 0 is not supported in the Python bindings of "
+                "quantcpp 0.8.x — falling back to kv_compress=0. Use the CLI "
+                "binary for KV compression until v0.8.2.",
+                RuntimeWarning,
+                stacklevel=2,
+            )
+            kv_compress = 0
         if not os.path.isfile(path):
             raise FileNotFoundError(f"Model file not found: {path}")
 
@@ -139,12 +159,13 @@ def ask(self, prompt: str) -> str:
         result = ctypes.cast(ptr, ctypes.c_char_p).value
         text = result.decode("utf-8", errors="replace") if result else ""
 
-        # Free the C-allocated string
-        if sys.platform == "win32":
-            libc = ctypes.cdll.msvcrt
-        else:
-            libc = ctypes.CDLL(None)
-        libc.free(ptr)
+        # NOTE (v0.8.1): the C string returned by quant_ask is allocated
+        # inside libquant.dylib's malloc heap. Calling ctypes.CDLL(None).free
+        # on it crashes on macOS arm64 because Python's libc handle resolves
+        # to a different malloc zone than the dylib's. We accept a ~65 KB
+        # leak per ask() call as a temporary tradeoff. quant_free_ctx /
+        # quant_free_model release the bulk of the memory at end of session.
+        # Tracked: add quant_free_string(void*) to quant.h in v0.8.2.
 
         return text