Join us for a deep dive into Zero-Shot Named Entity Recognition with GLiNeR presented by Ihor Stepanov on Tuesday, August 26th. Thanks to our Retrieval and Search program leads Mayank Rakesh and Avinab Neogy for organizing this session ✨ Learn more: cohere.com/events/Cohere-…

🚀 LLM Compressor v0.7.0 is here! This release brings powerful new features for quantizing large language models, including transform support (QuIP, SpinQuant), mixed precision compression, improved MoE handling with Llama4 support, and more. Full blog: developers.redhat.com/articles/2025/…

Let's break down intelligent inference serving. Traditional serving uses basic round-robin load balancing where requests are sent to the "next" pod. Intelligent inference serving makes scheduling decisions based on AI-specific workload signals. Let's dig into what this means.

🚀 Excited to announce QuTLASS v0.1.0 🎉 QuTLASS is a high-performance library for low-precision deep learning kernels, following NVIDIA CUTLASS. The new release brings 4-bit NVFP4 microscaling and fast transforms to NVIDIA Blackwell GPUs (including the B200!) [1/N]

Wow, thanks to Charles 🎉 Frye , you can understand internals of vLLM with a live notebook from Modal 🥰

Someone at a16z claimed a few weeks ago that 80% of Bay Area startups are building on Chinese open source models. The graphic below shows Chinese model downloads exceeding US models on HuggingFace.

🚀 Thrilled to announce GuideLLM v0.3.0! This release is highlighted by a brand new Web UI, containerized benchmarking, and powerful dataset preprocessing. GuideLLM GitHub: github.com/vllm-project/g… (Thread 👇)

IBM just released small swiss army knife for the document models: granite-docling-258M 🔥 not only a document converter but also can do document question answering, understand multiple languages 🤯 with Apache 2.0 license 👏

small models are the frontier now.

As a researcher at a frontier lab I’m often surprised by how unaware of current AI progress public discussions are. I wrote a post to summarize studies of recent progress, and what we should expect in the next 1-2 years: julian.ac/blog/2025/09/2…

much more convinced after getting my own results: LoRA with rank=1 learns (and generalizes) as well as full-tuning while saving 43% vRAM usage! allows me to RL bigger models with limited resources😆 script: github.com/sail-sg/oat/bl…

LLM Compressor 0.8.0 is here, with extended support for Qwen3-Next and Qwen3-VL models, improved GPTQ accuracy, and more flexible quantization workflows. Explore what’s new in this release 👇

If you’re building with open source AI, join Red Hat AI Day of Learning next week (Oct 16) for deep dives into vLLM, LLM Compressor, agentic AI, scaling inference, and more. Free & virtual → redhat.com/en/events/webi…

IBM Granite team released Granite 4 Nano models 1B variant outperforms Qwen3-1.7B with fewer params on a mix of tasks from math to coding 👏

⛵Marin 32B Base (mantis) is done training! It is the best open-source base model (beating OLMo 2 32B Base) and it’s even close to the best comparably-sized open-weight base models, Gemma 3 27B PT and Qwen 2.5 32B Base. Ranking across 19 benchmarks:

Good news - we'll be live streaming the first official vLLM meetup in Europe from Zürich. Thu, Nov 6 at 11:30am ET / 8:30am PT / 5:30pm CET Hear from vLLM maintainers and contributors at Red Hat, IBM, and Mistral AI covering quantization, hybrid models, distributed

Hybrid models like Qwen3-Next, Nemotron Nano 2 and Granite 4.0 are now fully supported in vLLM! Check out our latest blog from the vLLM team at IBM to learn how the vLLM community has elevated hybrid models from experimental hacks in V0 to first-class citizens in V1. 🔗

The latest Kimi K2 Thinking model is officially released in the compressed-tensors (INT4A16) format, enabling faster, more efficient reasoning and tool-use at scale. INT4A16 delivers ~2x speedup with minimal accuracy loss. It's ideal for 256K context and agentic tasks. Kudos to

Today, we are officially open-sourcing a set of high-quality speculator models on the Hugging Face Hub. Our first release includes Llamas, Qwens, and gpt-oss. In practice, you can expect 1.5–2.5× speedups on average, with some workloads seeing more than 4× improvements!

So cool to see Artificial Analysis add openness in their analysis!