If someone ever asks me about fine-tuning language models. Hamel is the guy I would invite.

Just from the guest speakers, you'd get a 5 figure workshop amount of value for something that basically fits under your learning & development budget

maven.com/parlance-labs/…

just learned about Agile. my response:

a chief reason for fine-tuning is to get rid of slop in the outputs of your pipelines

Aran stays on top of current research in lots of areas, if you need someone 'in the know' I'm sure he'd be an excellent person to talk to!

AI Dev tools: if i can’t swipe a credit card, read your docs and get started myself - ngmi

Developers cannot take a 30 min meeting . That costs 10-100x more than a one month subscription to your tool

Does anyone have LLM evaluation tools/vendors that really like that are useful in domain specific contexts?

I'm looking for vendors that have ALL of the below:

1. Observability
2. Allow you to write your own assertions and LLM judges
3. Bootsrap the creation of #2…

This is tldraw in Jupyter with a sketch of a Blender scene. Hitting the 'MakeReal' button will produce Python Code that generates the Blender scene:

Who is building this

sota RAG in 2024

It's Hugging Face Accelerate release time and there are a TONof exciting features to get through: new optimizers, FP8 fixes, DataLoader improvements, documentation, and so much more!

For a quickread, check out the full notes: github.com/huggingface/ac…

Otherwise let's dig in🧵

Maven's top AI course just added a ton of new guest speakers.

Incredible talent convening on Maven to teach LLM Fine-Tuning:

- Wing Lian: Creator of Axolotl library for LLM fine-tuning
- Shreya Shankar: LLMOps and LLM Evaluations researcher
- Zachary Mueller: Lead maintainer…

brushed up my personal site and brain dumped a post on 🎶musicgen-songstarter-v0.2🎶

It covers:
- 🧠my thought process/motivation behind it
- ✏️notes on my previous experiments over the last 9 months
- 👀 training deets, Weights & Biases logs w/ hparams

nateraw.com/posts/training…

Classic example of overfitting to the validation set re: LLMs, when I started working with Phillip Carter I found few-shot examples from the validation set in the prompt (we fixed it!).

There are lots of reasons for a separate eval set. Overfitting can come in many forms.

Has someone created materials around “fundamentals of ML for AI Engineers”, not focused on building models but things like evaluations, error analysis, etc

Maybe something already exists? I don’t want to do it lol - looking for a resource I can share with people

📺Tune in next week as Sebastian Raschka and I riff on 'Developing and Training LLMs From Scratch' in a live podcast recording for Vanishing Gradients Podcast 💫

lu.ma/build-llms-fro…

This will likely be a sprawling convo in which we tell you everything you need to know about LLMs, but were too…