Evaluated Llama 3.1 70B (Fireworks AI) as well and is performant at the same level as Mistral Large 2 128B. - Llama is outperforming on GSM8k (math) while Mistral is outperforming on MATH benchmark.😅 - Llama > Mistral (reasoning tasks) - Mistral > Llama (Q&A tasks)

Hallucinations in AI-assisted coding are a significant challenge for developers. Join Daniel Loman from Groq Inc and Anish Shah from Weights & Biases as they share strategies to overcome this issue in our technical webinar on August 6. Register now: streamyard.com/watch/hikcAgep…

I will be talking tomorrow on- - LLM landscape - Need for structured output - function calling, json, constrained decoding, more - RAG - LLM system evaluation - Weights & Biases Weave for building LLM applications correctly If it excites you, consider showing up. 💫⭐🌟

Stoked to partner with the team to host 7 lectures, a lot of agents creativity needed 📺 wandb.me/hackercup NeurIPS Meta Hacker Cup AI track gonna be 🔥

Join me tomorrow. I'm presentimg a codegen agent featuring RAG-based episodic memory and reflexion. Whether you're aiming to enhance your LLM toolkit or just curious about the latest techniques, this talk will deliver essential insights. 🔗events.bizzabo.com/NeurIPShackerc…

Join us on September 10 to learn how to build production-ready RAG systems. Learn from Anish Shah about optimizing pipelines, enhancing queries, and scaling solutions for real-world applications. Ideal for tech leads and product managers driving AI innovation. Register now:

I've been evaluating chunking mechanisms while prepping our RAG course's data ingestion lesson. Structured, semantic, and syntactic chunking each uniquely impact RAG performance. Learn to choose the right approach for your use case: wandb.me/rag-course

⚡️ AI Hacker Cup Lightning Comp Today we're kicking off a ⚡️ 7-day competition to solve all 5 of the 2023 practice Hacker Cup challenges with Mistral AI models Our current baseline is 2/5 with the starter RAG agent (with reflection) Mistral AI api access provided Details👇

Today I gave LlamaIndex 🦙's workflows a go to tackle the NeurIPS AI hackercup competition. I created a Workflow to iterate from an initial solution, run the generated solution, and check against the expected output. It is effortless to define steps with the expected inputs and

Defining your grading criteria for LLM outputs is an organic process that evolves the more time you spend on it In “Who Validates the Validators” Shreya Shankar et al highlight this This weekend in our SF office a good chunk % of hackers will learn this 😃 lu.ma/judge

Announcing our new RAG++ course, now available in collaboration with @CohereAI and @Weaviate_io. Created for engineers looking to build production-ready RAG systems. The course covers everything from evaluation strategies and data preprocessing to advanced retrieval techniques

📣 I am happy to announce that Weights & Biases Weave is now integrated with Instructor (jason liu)! 🧶 Weave will automatically capture traces for Instructor. To start tracking, call `weave.init()` and use the library as normal. 👉 Learn more at weave-docs.wandb.ai/guides/integra…

The new Weights & Biases RAG++ course uses the v2 API and the new command-r models. In addition to learning about nuanced RAG techniques, you also get to work with the new API in the course colabs thanks to the credits from cohere. Register here: wandb.me/rag

🏆🏆🏆🏆 Four papers at COLM were awarded outstanding paper awards. Congratulations to the authors!

Looking to go beyond basic LLM experiments and build real, production-grade applications? We have two free virtual courses designed for AI engineers of all levels who want to master RAG and LLM evaluations. Here’s what you’ll learn 🧵