🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

It was with great confusion that I just realized at my old age, that the Greek word sycophant means ‘insincere flatterer’. In modern Greek (as in ancient) it means ‘slanderer’ and I was confused about GPT-4 spreading slander.

Great work on Phi4. This seems to be the best open weights model for reasoning, beating the previous best QWQ 32B even though it’s only 14B

Data-centric workshop at ICML 2025. Awesome speakers and panelists. (And they also kindly invited me)

Really excited to be participating in the UC Berkeley entrepreneurs mixer event- with amazing researchers, funders and founders.

The Berkeley entrepreneurs student club has some cool alums that started a few small startups.

Very cool result: KV cache compression can be done with compressed sensing: store keys and values as sparse combinations of some dictionary vectors. Interestingly, the dictionary is universal across inputs (but learned for each model).

I thought of DSPy as a prompt optimization tool. But it can optimize the weights of multi-component AI systems too, including GRPO for multi-turn and tool calling, see this very interesting new addition: dspy.GRPO

GroupNorm (normalizing groups of channels) considered harmful. It kills the relative means of different channels as nicely explained here.

A good example how predicting one token requires reasoning.

A New benchmark for visual reasoning just came out.

Congratulations LMArena !

Very interesting paper showing that releasing embeddings of text is almost the same as releasing the text itself. The universality of embedding geometry for different models and datasets is still puzzling me.

Incredible progress on Multi-turn RL by Berkeley NovaSky team! They get very good results in Text-to-SQL on the Spider benchmark. The agent learns to explore the database to answer questions very efficiently. Quick highlights: Multi-turn RL learns faster and generalizes better

In 2019 introducing BERT into a lecture required custom university attire.