I am diving into LLM fine-tuning.

There is a lack of deep tech content on fine-tuning:

• how it works
• why it works
• what it does to an LLM, etc.

There is a ton of high-level stuff, however.

I want to grok the first principles of fine-tuning.

If you have an excellent…

I studied word embeddings today.

Mainly, how LLMs like GPT-4 convert input text into input embeddings.

It’s simpler than I expected.

There are five key steps:

1. Convert input text to input tokens.

2. Map tokens to token IDs. Common vocab size is ~50K tokens.

3. Create…

Great tutorial by Joschka Braun on DSPy.

I have yet to spend much time exploring DSPy.

But that is about to change.