Morgan Stanley Research says OpenAI makes up around $330B of the $880B total future contract value (RPO) tied to Microsoft, Oracle, and CoreWeave, so a lot of supplier growth depends directly on OpenAI’s stability. That means about 66% of Oracle’s and about 40% of CoreWeave’s

Ray Summit is going to be excellent. Can't wait to hear from xAI, Perplexity, Cursor, Thinking Machines, Physical Intelligence, Applied Intuition, Prime Intellect, vLLM, and so many others. Some major themes this year: - Reinforcement learning infra - Multimodal data (lots of

Team is cooking 🧑‍🍳

We’ve found the root cause of AWS outage today. Sorry guys.

What a terrible night for on-calls 🛏️

Hi SF folks! I’ll be speaking at this meetup on inference systems this Thursday, 10/23 @ 6PM. It should be a cool event (other speakers from Meta Superintelligence, SGLang, and the Laude Institute) Hope to see you there 🤠 luma.com/brn77bc8?tk=JI…

Today we’re donating Ray to The Linux Foundation under the PyTorch Foundation with PyTorch + vLLM, strengthening the open compute fabric for AI. Ensures long-term neutrality, open governance, and ecosystem alignment. Blog: na2.hubs.ly/H01JydX0 Ray Summit (Nov 3–5, SF):

We’re excited to welcome Ray to the PyTorch Foundation 👋 ray is an open source distributed computing framework for #AI workloads, including data processing, model training and inference at scale. By contributing Ray to the PyTorch Foundation, Anyscale

Ray ♥️ PyTorch foundation: Open-source drivers of modern AI infrastructure

🚀 Excited to share our work on batch-invariant inference in vLLM! Now you can get identical results regardless of batch size with just one flag: VLLM_BATCH_INVARIANT=1 No more subtle differences between bs=1 and bs=N (including prefill!). Let's dive into how we built this 🧵👇

I enjoyed speaking at #PyTorchCon today. Wanted to share one slide from my talk about open source AI infra. This is about how Ray and vLLM work together. LLM inference is growing more and more complex, and doing a good job with LLM inference means working across layers and

Fortunate to be part of two (!) foundation projects (vLLM and ray) that have great synergy with each other. The Ray + vLLM + PyTorch stack is coming together. Congratulations, Ray!

I'm hiring for a new engineering role working directly with me to support our most sophisticated customers. Looking for someone who wants to work across the AI / AI infra stack, write / debug a ton of code, work directly with customers, move / learn super fast. DM me.

Infra that ships. Ideas that scale. Keynotes at #RaySummit 2025: • Chelsea Finn (robot learning, meta-learning) • Jimmy Ba (Adam optimizer, DL/RL) • Peter Ludwig (safe AI at scale) Nov 3–5 SF Save your seat: 🔗na2.hubs.ly/H01MYqw0

🚀 Introducing UCCL-EP: A portable, efficient Expert Parallelism framework that brings DeepEP-level GPU-driven communication with the same APIs to any cloud or hardware — AWS EFA, AMD GPUs, Broadcom NICs and beyond. Blog: uccl-project.github.io/posts/uccl-ep/ Code: github.com/uccl-project/u…

A couple of months ago we were perplexed by slow MoE communication perf on cloud (e.g. EFA with perplexity kernels). So we built UCCL-EP — an efficient GPU-driven EP library that runs on public clouds (e.g. AWS EFA) and heterogeneous GPUs/NICs, with the same APIs as DeepEP.

🔥Ray Summit 2025 will be one of the biggest events for vLLM this year, with 10+ talks centered around vLLM! Looking forward to see you there. 🤩Use our discount code (limited time only!) RAYVLLM50 anyscale.com/ray-summit/2025

Except that the network latency will kill the utility of the aggregate power.