KV cache storage solution

San Jose – November 06, 2026 -- Supermicro, IBM and NVIDIA have launched a joint KV cache storage solution that delivered up to 56x faster Time-to-First-Token (TTFT) at 130,000-token prompts in testing with a 120-billion-parameter large language model.

Test results show TTFT rising 0.377 seconds with cache versus 29.55 seconds without

As input length grew from 30,000 to 130,000 tokens, TTFT increased by only 0.377 seconds with persistent KV cache. Without it, TTFT increased by 29.55 seconds. The figures come from joint testing by Supermicro and NVIDIA and are detailed in an IBM Redbook.

The companies also ran cluster-level validation covering high-concurrency and noisy-neighbor network traffic scenarios.