Rewarding to see Gorilla API-Bench and the Berkeley Function Calling Leaderboard (BFCL) cement themselves as fundamental evals for LLMs 🫡

Excited to announce our $2.75 million seed round from investors like Paul Graham himself, Nat Friedman, Daniel Gross, Y Combinator, and many more ⚡️

Some tactics to build LLM apps consisting of multiple components: - test every sub-step to minimize cascading failures - reference-based evaluation of sub-components using synthetic data - cache LLM calls to quickly iterate on independent components More details incl. synthetic

Tested out Microsoft Azure 's LLMLingua prompt compression over the weekend. Avg ~2.5x compression, 50% cost saving, and almost similar performance on a RAG task. Even though context lengths are increasing, compression could provide a few interesting upsides. Colab notebook below 👇

cohere 's actually pretty awesome. More folks should be exploring their models. PareaAI , now has auto-instrumentation for the Cohere py sdk 🚀

🚀 New deep dive notebook on PareaAI experiments and LLM evals 📝🔬. I cover some of the key functionalities illustrating the power and flexibility of our API. 🔽 Link in comments 🔽

Run more tests! youtube.com/clip/UgkxYSbBA…

Saturdays are for doc upgrades

If anyone is worried about my future after OpenAI steamrolled the marketing vehicle for my consulting business, please retweet.

Garry Tan it's not won yet (it still feels very nascant), we are building CodeStory and believe that many versions of the code generation vision will play out here.

A must read

Applied AI Office Hours x.com/i/broadcasts/1…

woops looks like I accidentally added support for all languages in Aide 👀

.PareaAI also looks like a good LLM monitoring tool and is open source