GPU-Accelerated Synaptic Scoring

[sbharatjoshi]

🗣️ Discussion: GPU-Accelerated Synaptic Scoring

Research discussion about accelerating Spector's 6-phase fused scoring pipeline using GPU computation.

---

Current Performance

The SIMD scoring pipeline (Panama Vector API) achieves 8-16x throughput on CPU. But at scale (>1M memories), scoring becomes a bottleneck.

GPU Acceleration Options

Approach	Maturity	Integration
TornadoVM	Emerging	JVM bytecode → OpenCL/CUDA
JCuda	Stable	Direct CUDA bindings
Panama + libcuda	Experimental	FFM to CUDA driver API
ONNX Runtime	Stable	Pre-compiled scoring models
Triton Inference Server	Stable	External gRPC service

What Would We Accelerate?

1. Cosine similarity — Batch dot products of query vector vs all memory vectors
2. Fused scoring — 6-dimension weighted aggregation across all candidates
3. HNSW beam search — Distance computations during graph traversal
4. Hebbian propagation — Sparse matrix-vector multiplication for activation spreading

Questions for Discussion

• TornadoVM (pure JVM) vs JCuda (NVIDIA-only) — which path?
• Should GPU be optional with CPU fallback (detected at startup)?
• What's the minimum dataset size where GPU beats CPU SIMD?
• Should we consider Apple Metal / AMD ROCm support?
• Is it worth the complexity, or should we focus on CPU SIMD optimization first?

Related Issues

#56 GPU-Accelerated Scoring, #85 HebbianGraph CSR Layout

---

We'd especially love input from GPU computing experts!