OLMo-7b is finally out 🎉, and we are releasing everything; weights, intermediate checkpoints, training code and logs, training data and toolkit, evaluation and adaptation code and data.
Most of it has been released, and the rest is coming soon. OLMo-65b and Adapted OLMo-7b are
LLMs have problems that are difficult to identify and explain. To understand them better, we need to study the relationship between LLMs and their pre-training data, and with the OLMo release, Ai2 helps make this possible!
sciencenews.org/article/genera…
I’m happy to share that I’m starting a new position as Founding Research Engineer Spiffy AI !
This is a totally new adventure for me. I am very excited to join such an amazing team and work together on challenging problems!
Released torchdistill v1.1.0 last night!
⚗️ Key updates ⚗️
- PyTorch 2.2.1 support
- 3 new KD methods
- Custom low-level loss support
github.com/yoshitomo-mats…
For the new KD methods (SRD w/ Roy Miles), we reproduced ImageNet results (2 wins, 1 lose)
yoshitomo-matsubara.net/torchdistill/b…
