The bugs I encounter most with LLMs in production are related to data drift.

This is acute for LLMs b/c of all the moving parts: prompts, RAG, functions, etc. There is a classic ML technique that works for detecting drift. I explain it in this post: hamel.dev/blog/posts/dri…

Finished a blog post on using Axolotl for the first time. drchrislevy.github.io/posts/intro_fi…
Thanks Jeremy Howard for recommending the tool, Hamel Husain for docs, Wing Lian (caseus) and others for the great library, Maxime Labonne for your blog posts which I used as a guide.

Peak C fomo rn

We've reached the point where models are becoming as persuasive as human writers.

There's a clear model size x persuasion scaling trend, and it doesn't look like it is plateauing.

Looks like C/CUDA is the new hotness … began with CUDA mode and now Karpathy

In my experience Automated prompt optimizers are in the same class as off the shelf automated evals

It can become a dangerous comfort blanket to make you feel like you are making progress but in the long term it becomes tech debt

Take your prompts and run.

With $10k on the line, you have to ask why no magical prompt optimizer solved this instead of a human manually grinding. Surely it would be great advertising. 🤔

Getting excited about our Haystack talk about integrating LTR into Reddit search.

Hope to see you there!

haystackconf.com/us2024/talk-4/

😮 This is so friggin’ cool. As the circles reach the horizon, they trigger a kalimba sample. Settings allow us to tweak the motion, and generate trippy polyrhythms.

polyrhythmic-rings.vercel.app

(🎵 Sound on for video!)

Livestream w/ Hamel Husain and Eugene Yan scheduled for 4pm EST!

youtube.com/watch?v=qhOwho…

It cannot be over-stated that evaluation driven development is key to building robust LLM products, quickly.

Great blog post by Hamel Husain that not only explains 'why', but gives practical tips on 'how'

hamel.dev/blog/posts/eva…

Early days for me in training some models with Axolotl . Thanks Hamel Husain for starting more documentation. I was struggling to figure out what was in last_run_prepared directory for the pre-processed dataset, until I read this. openaccess-ai-collective.github.io/axolotl/docs/i…

New open model by Cohere is here! C4AI Command R+ 🔥

128K context length
104b parameters
Multilingual
Conversational Tool support and RAG capabilities

Model hf.co/CohereForAI/c4…
Cohere playground: dashboard.cohere.com/playground/chat
Demo: huggingface.co/spaces/CohereF…

A resource I keep coming back to often is the Hugging Face Trainer example zoo I made awhile back. Want a quick way to just 'run the trainer' and verify your setup works? Or just see minimal working examples that can be substituted for any model? Check it out…

Tbh getting CUDA to work is like hunting
Also takes 2 weeks

Eugene loves evals
This means I love Eugene 🤝

I've been trying several evals to find those that correlate well with use cases and discriminative enough for prod.

Here's an opinionated take on what works, focusing on classification, summarization, translation, copyright regurgitation, and toxicity.

eugeneyan.com/writing/evals/

after using what I consider to be AGI to help me code for over a year now; I am not really afraid of too large disruptive effects caused by AGI