🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

🚀 Hello, Kimi K2! Open-Source Agentic Model! 🔹 1T total / 32B active MoE model 🔹 SOTA on SWE Bench Verified, Tau2 & AceBench among open models 🔹Strong in coding and agentic tasks 🐤 Multimodal & thought-mode not supported for now With Kimi K2, advanced agentic intelligence

Huge thanks to the MoonCake team for bringing day 0 support for Kimi K2 in SGLang and KTransformers, including the integration of the new reasoning parser!

SGLang is currently the only open-source LLM serving engine validated with PD disaggregation + large-scale EP on the H200 cluster with over 100 GPUs. Huge thanks to the MoonCake Kimi.ai team for helping verify it even before release!

Kimi Kimi.ai K2 on SGLang: As a trillion-parameter model, long context on 1–2 H200/B200 nodes struggles. Use SGLang’s PD disaggregation + large-scale EP—validated on 100+ H200s by MoonCake Team. New SOTA is here—start using it!

Kimi.ai K2 shares the same architecture as DeepSeek R1, with experts increased from 256 to 384—so all SGLang optimizations work out of the box. PD disaggregation + large-scale EP was validated on 100+ H200s by MoonCake Team before release. Always trust SGLang! 🚀

🚀Summer Fest Day 3: Cost-Effective MoE Inference on CPU from Intel PyTorch team Deploying 671B DeepSeek R1 with zero GPUs? SGLang now supports high-performance CPU-only inference on Intel Xeon 6—enabling billion-scale MoE models like DeepSeek to run on commodity CPU servers.

Congrats Mingfei and the team! Thanks to the excellent collaborations between LMSYS Org and Intel! #intel #sglang #xeon

NVILA is available in SGLang👏🏻

This work is done by my PhD student Zijian and researchers at Nvidia! Ligeng Zhu Song Han Big congrats! We are also integrating VLM into AReaL.