Yi Tay
@YiTayML
chief scientist / cofounder @RekaAILabs 🫠
past: research scientist @google brain 🤯
currently learning to be a dad 🍼
ID:790033937531703296
http://yitay.net 23-10-2016 03:35:43
3,0K Tweets
29,0K Followers
97 Following
Enjoyed this paper that plots emergent abilities with pretraining loss on the x-axis, which is actually a suggestion that Oriol Vinyals also made a few years back: arxiv.org/abs/2403.15796
The paper uses intermediate checkpoints to plot a variety of pretraining losses. For some
One year since I posted this so here's an update! Adding Donovan Ong to the list of notable Singaporean researchers/engineers doing great work in AI and LLMs. He helped train Reka's (@RekaAILabs) series of OP models (Core, Flash, Edge) so he deserves to be on this list! 🔥