LLM inference systems engineer. I build the systems that serve large language models: GPU kernels, KV-cache stores, prefill/decode disaggregation, cache-aware routing, and the Kubernetes underneath.
Senior Software Engineer (MLOps) at Multiverse Computing · San Sebastián, Spain · Remote, EU/UK hours
mibrahim.dev · Writing · CV · LinkedIn · Papers · Email
A sharded KV-cache tier: each prompt chunk c belongs to rank c mod 8 and lives in that rank's own Valkey instance. How it got GLM-5.3 65% more throughput on 5 nodes instead of 11 →
- 11 → 5 nodes. Re-architected serving for a frontier-scale MoE model (GLM-5.3, FP8) on NVIDIA B300 as prefill/decode disaggregation: the same 200k-token SLA with 65% more throughput, 3.6× per node.
- 124× lower p99. KV-cache tail latency from 83 s to 0.67 s, found with a production replay, a cost model and queueing analysis (paper).
- +34% prefill throughput on 8× B300 from a sharded KV store and an upstream FlashInfer kernel fix; DeepEP for SM103 now builds in vLLM's Docker image.
- 99% cache hits. KV-cache-aware routing across replicas, up from 79–89%; warm P50 from 9.8 s to 1.3 s.
- 7.5× throughput. Llama 3.3 70B on H200 with CUDA graphs, speculative decoding, FlashAttention and FP8.
- More throughput from fewer GPUs: profiling and fixing GLM-5.3 prefill on NVIDIA B300 · Oct 2026 · 23 min
A sharded KV store, a FlashInfer kernel fix, one Valkey instance per rank and asynchronous loading. - vLLM v0.11.0: Technical Highlights & Optimizations · Nov 2025
- DeepSeek-V2: MLA Explained · Sep 2025
Fixes found in production, submitted upstream with tests: 27 pull requests to vLLM, LMCache and vLLM production-stack. Selected:
| Project | Pull request | Status |
|---|---|---|
| vLLM | #61829 Build DeepEP for SM103 (B300/GB300) in the Docker image | merged |
| vLLM | #61900 Make the DeepEP high-throughput SM count configurable | open |
| LMCache | #5545 Shard the MLA KV store across prefill-context-parallel ranks | open |
| LMCache | #5553 Async loading for the sharded PCP store | open |
| LMCache | #5548 {rank} in remote_url selects a per-worker remote instance |
open |
| LMCache | #5547 Native clients: don't lose completions drained before the future is registered | open |
| LMCache | #5551 RESP GET of a missing key is a miss, not a hang | open |
| LMCache | #5727 Key the offload server socket by world rank under context parallelism | open |
| vLLM | #59603 Forward failed KV loads to the LMCache scheduler side | open |
| vLLM | #60538 PCP: replicate prefills too short for every rank | open |
| production-stack | #1121 Aggregate engine stats over data-parallel engines | open |
| production-stack | #1118 Read vllm:kv_cache_usage_perc for KV-cache usage |
merged |
| production-stack | #1108 Load-aware routing: score bursts against live load | merged |
| production-stack | #1107 KV lookup for chat completions, with fallback | merged |
When the KV Cache Is Slower Than the GPU: Scaling Remote KV Tiers for Context-Parallel Prefill
M. Ibrahim · arXiv:2610.21500, 2026
A cost model for when a remote KV-cache hit beats recompute on B300, and a sharded, rank-partitioned KV tier that cuts hit-heavy p99 from 83 s to 0.67 s. All changes submitted upstream to LMCache.
CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks
A. Tomut, S. S. Jahromi, A. Sarkar, … M. Ibrahim, … R. Orús · arXiv:2401.14109, 2024
I led the 6-person engineering team that built the 70%-compressed LLM, adopted by 3 enterprise clients for edge deployment.
| Role | Company | |
|---|---|---|
| Mid/Senior Software Engineer, MLOps / LLMOps | Multiverse Computing | Aug 2024 – present |
| Junior Software Engineer | Multiverse Computing | Jun 2023 – Jul 2024 |
| Software Engineer | Bank Al Habib Ltd. | Sep 2022 – Jun 2023 |
MS Computer Science, Georgia Institute of Technology · BE Electrical & Computer Engineering, NED University
LLM serving vLLM, LMCache, NIXL, FlashInfer, prefill/decode disaggregation, KV-cache offload and routing, tensor/data/expert/context parallelism, speculative decoding, FP8/NVFP4, CUDA graphs
Performance Nsight Systems/Compute, NCCL, py-spy, cost and queueing models, SLA benchmarking
Platform Kubernetes, Helm, Argo CD, Karpenter, KEDA, Istio/Envoy, AWS (SageMaker HyperPod, EKS, EFA), Azure ML, Terraform, SkyPilot
Languages Python, C++, Go, Rust, TypeScript, Java, SQL
- AWS Certified Machine Learning – Specialty (2025)
- Microsoft Azure AI Engineer Associate (2025)
- AWS Certified Developer – Associate
- TensorFlow Certified Developer
Also: AWS Certification Subject Matter Expert for the GenAI Developer exam, and an agentic-AI workshop at AWS Madrid (rated 4.6/5).