Jeffrey Li πŸ’™πŸ’› (@askerlee) 's Twitter Profile
Jeffrey Li πŸ’™πŸ’›

@askerlee

Machine Learning researcher. Veritas.

ID: 109251162

calendar_today28-01-2010 12:53:20

6,6K Tweet

2,2K Followers

1,1K Following

Jeffrey Li πŸ’™πŸ’› (@askerlee) 's Twitter Profile Photo

"Spectral autoregression" is a useful but overly-simplified anology of diffusion. IMO diffusion training is constructing countless feature/semantic neighborhoods, and inference is randomly walking along these neighborhoods. The guidance signal suggests the trajectory of the walk

Jeffrey Li πŸ’™πŸ’› (@askerlee) 's Twitter Profile Photo

Nice discovery and maybe we can "scale up" the conclusion: maybe at the end of the day, every LLM, including the largest ones nowadays, is a "small model", in the sense that they have difficulty in generating certain challenging CoTs

Jeffrey Li πŸ’™πŸ’› (@askerlee) 's Twitter Profile Photo

Alignment really gets interesting these days. On the surface this might look like self-consciousness, however I'd hypothesize that the prompt makes R1 enter some kind of role-playing mode so it fakes its viewpoints.

Jeffrey Li πŸ’™πŸ’› (@askerlee) 's Twitter Profile Photo

One of the best episodes of Dwarkesh's interviews. I like that Karpathy always offers thought-provoking insights whenever he pushes back on an idea or a question.