🦙 We're excited to host Meta Llama-3 8b and 70b on Anyscale Endpoints! ➕ Fine-tuning, JSON mode and function calling support coming soon as well! Pricing: - 8B: $0.15 / Million tokens - 70B: $1.00 / Million tokens

Not all questions need GPT-4! We introduce RouteLLM – a routing framework based on human preference data that directs simple queries to a cheaper model. With data augmentation techniques, RouteLLM achieves cost reductions of over 85% on MT Bench and 45% on MMLU while

So excited that our work is finally out! A joint work with my awesome collaborators: isaac ong Vincent Wei-Lin Chiang Tianhao Wu Joey Gonzalez Waleed Kadous Ion Stoica Anyscale LMSYS Org #Anyscale #UCBerkeley #LLM #RouteLLM #OpenSource #MachineLearning #AICommunity #AIResearch

Collaboration with LMSYS Org. Step-by-step instructions for building your own model router. Key steps: 1. Generating labeled data 2. Fine-tune an LLM-based classifier 3. Run offline evals The whole thing takes about 120 minutes. Overall goal is to direct "simple" queries to

In 5 of 8 recent conversations, ML platform leaders told me that their top priority over the next 6 months is to enable training on more data (e.g., an order of magnitude more). Why? Scaling laws. The idea that larger models + data + compute can lead to better results (not just

Excited to announce our partnership with Google Cloud integrating Anyscale RayTurbo natively into Google Kubernetes Engine (GKE)! 🚀 Together, we're creating a unified, distributed operating system tailored specifically for AI workloads – simplifying development and

Following constructive collaboration with our U.S. colleagues, Syrians can now reconnect with the world. We especially appreciate the leadership of Ambassador Tom Barrack and his team. We call on American tech firms to act swiftly: restore services, empower millions, and join us in one