Heading “back” to Massachusetts Institute of Technology (MIT) for a talk!

If you are on campus, it will be at 32D-463 (Star) 4pm ET tomorrow (5/2).

New paper from Reka 🔥 (yes an actual paper).

This time we're releasing part of our internal evals which we call Vibe-Eval 😃 This comprises of a hard set which imo is pretty challenging for frontier models today.

The fun part here is that we constructed it by trying to…

i do have a soft spot for gpt2

I'm getting used to AI surpassing me in more areas, much like how I trust Google Maps over me.

Even two years ago, it was so easy to look at the model generation and grade it myself. Now it is quite difficult for some domains (e.g. GPQA eval). Such a humbling experience.

The first lecture of our Stanford University CS25 V4 Transformers course (cs25.stanford.edu) is now released! Check it out here: youtube.com/watch?v=fKMB5U….

We (the instructors) gave a brief intro and overview of the history of NLP, Transformers and how they work, and their impact. We

Leverage dilemma: if you are truly leveraged, you benefit greatly even if you don't work hard. But if you do work hard, the additional benefit will be so significant that it is too costly not to work hard

Congrats to Yi Tay and the reka team on this launch!

In the tech report i see this huge spike in the loss curve. Hope you did not lose much sleep when that happened Yi Tay

People don’t like to repeat because they don’t feel like making progress. But repetition is necessary for deeper understanding. E.g.
- Re-reading books
- Repeating the thought process understanding a new concept

Unfortunate side effect of over-reliance on quantitative metrics

Really excited about the expansion to Asia!

🔥Exciting news -- GPT-4-Turbo has just reclaimed the No. 1 spot on the Arena leaderboard again! Woah!

We collect over 8K user votes from diverse domains and observe its strong coding & reasoning capability over others. Hats off to OpenAI for this incredible launch!

To offer…

Excited to see our 🍮Flan-Palm🌴 work finally published in Journal of Machine Learning Research 2024!

Looking back, I see this work as pushing hard on scaling: post-training data, models, prompting, & eval.

We brought together the methods and findings of many awesome prior works, scaled them up, and…

Flan-2 is published in JMLR jmlr.org/papers/v25/23-…. I think it's a nice piece of history.

The work scaled instruction tuning with respect to model size and finetuning tasks, which both improved performance. Our MMLU was 75%, SOTA when the paper came out in Oct 2022.

Our…

i mean it when i say that the Apple Vision Pro will be a game changer for education

here’s how I studied the heart 5 years ago vs how I can study it today, credit to the visionOS app Insight Heart

the contrast in experience and comprehension can’t be denied

Great to see such detailed descriptions of challenges training large models from scratch. Such knowledge is extremely valuable and scarce. Hope more people share their unique experience!

Many people learn about the tools just enough to get the job done. I prefer to dive deeper; understanding my tools in detail makes my work much more fun.

Not sure if it's good or bad. Just more fun! Perhaps that’s what truly matters in the end.

I strive not to be too organized because doing so misses a lot of deep lessons that tend to compound in the long run.

I sometimes work on things that don’t generate output for some time. From a highly organized person’s perspective, I am not being “productive” and this is a

Being brutally honest with oneself is difficult, especially when it requires facing harsh reality. Here is how I strive for self-honesty. I observe myself as if I were a ghost floating above. And in doing so, I replace the subject from “I” to “this monkey'. For example

Inner…