Grateful to the BentoML team for integrating LMCache — excited to help boost LLM inference together!

CacheGen(arxiv.org/abs/2310.07240) lets you store KV caches on disk or AWS S3 and load them way faster than recomputing! Modern LLMs use long contexts, but reprocessing these every time is slow and resource-intensive. While engines like vLLM (and LMCache) can cache contexts in

We're thrilled to share an integration between KServe and llm-d, bringing powerful, scalable LLM serving to Kubernetes. Our Red Hat + AI & 🌶 team is integrating llm-d, a Kubernetes-native distributed inference framework, into KServe. This is all about combining the best of both

Looking forward to seeing LMCache on stage at Cloud Native K8s AI Day! 🚀 Don’t miss the KServe Next session for cutting-edge LLM serving insights.

8 KV-Cache Systems You Can’t Afford to Miss in 2025 By 2025, KV-cache has evolved from a “nice-to-have” optimization into a critical layer for high-performance large language model (LLM) serving. From GPU-resident paging tricks to persistent, cross-node cache sharing, the

Fastest inference engine for LLMs! LMCache is an LLM serving engine that reduce Time to First Token (TTFT) and increase throughput, especially under long-context scenarios. 100% Open Source

love how Ashutosh frames the problem, ask us any questions about kvcache!

Mark your calendars! Excited for the first FastAGI meetup featuring incredible speakers on AI infra & agents 🚀 Looking forward to the discussions and energy at LMCache Lab!

Thanks to Alex @ New Port AI invite us to introduce lmcache at the Bay Area Generative AI Builders Meetup! More details here: lu.ma/5sdoeg1y

🚀 Exciting to see LMCache x Mooncake being discussed at the vLLM Shanghai Meetup! The ecosystem around vLLM is evolving fast — from distributed inference to hardware optimizations — and cache innovations like this will be key to unlocking the next level of efficiency &

LMCache Lab in vLLM Singapore meetup!