vLLM is finally addressing a long-standing problem: startup times 35s -> 2s for CUDA graph capture is a great reduction!

The Hugging Face Transformers ↔️ vLLM integration just leveled up: Vision-Language Models are now supported out of the box! If the model is integrated into Transformers, you can now run it directly with vLLM. github.com/vllm-project/v… Great work Raushan Turganbay 👏

The new transformers release comes w/ a surprise: kernels support ⚡️ It integrates deeply with precompiled kernels on the HF Hub. - opt-in, automatic kernels for your hardware and software - kernels like FA2/3 w/o compilation - community-built kernels, for inference & training

We have recently merged fast processors for many models, the speed-up in Qwen-VL series is 🔥 you get speed-up up to 3x on CPU and 26x on GPU 🤯 you don't have to do anything, this is enabled by default 🥳

Did you know you can now run your own AI Job on the Hugging Face infrastructure? Introducing `hf jobs`, the latest addition to the Hugging Face CLI. A quick thread to get you all started! 🧵⤵️

When Sam Altman told me at the AI summit in Paris that they were serious about releasing open-source models & asked what would be useful, I couldn’t believe it. But six months of collaboration later, here it is: Welcome to OSS-GPT on Hugging Face! It comes in two sizes, for both

Welcome back, OpenAI ! Day-0 support llama.cpp with MXFP4, let it rock 🚀🤘

OpenAI just released GPT-OSS: An Open Source Language Model on Hugging Face Open source meaning: 💸 Free 🔒 Private 🔧 Customizable

Head over to gpt-oss.com to try the official demo of OpenAI's gpt-oss models powered by Hugging Face!

For all you lazy lads! OpenAI's latest model in a short video by Dylan! 💥

And just like that, OpenAI gpt-oss is now the number one trending model on Hugging Face, out of almost 2M open models 🚀 People sometimes forget that they've already transformed the field: GPT-2, released back in 2019 is HF's most downloaded text-generation model ever, and

Hugging Face Explained in 45 seconds

Want to deploy open models using vLLM as the inference engine? We just released a step-by-step guide on how to do it with Hugging Face Inference Endpoints, now available in the vLLM docs. let the gpus go brrr

I've been wanting to do this for a really long time... vLLM is now fully formatted using ruff! 🚀 This change makes the codebase more readable and uses stronger tooling to keep it that way. Kudos to Python Software Foundation for the Black code format and to Astral for ruff!

It's not as exciting as BERT support... but the Hugging Face Transformers backend for vLLM now supports mixture-of-expert (MoE) models at full speed! 🚀 Install both packages from source and take it for a spin!

🚀 Excited to share our work on batch-invariant inference in vLLM! Now you can get identical results regardless of batch size with just one flag: VLLM_BATCH_INVARIANT=1 No more subtle differences between bs=1 and bs=N (including prefill!). Let's dive into how we built this 🧵👇

🔥 We're thrilled to announce 𝚑𝚞𝚐𝚐𝚒𝚗𝚐𝚏𝚊𝚌𝚎_𝚑𝚞𝚋 v1.0! After five years of development, this foundational release is packed with A fully modernized HTTP backend and a complete, from-the-ground-up CLI revamp! $ pip install huggingface_hub --upgrade 🧵highly recommend

If you missed my talk at Ray Summit last week, fear not! I'll be giving it again in Paris at next week's vLLM meetup 🎙️