Announcing Yarn-Mistral-7b-128k! You heard right, 128k (and 64k) context length for Mistral 🥳 🤗128k: huggingface.co/NousResearch/Y… 📜v2: arxiv.org/abs/2309.00071 Special thanks to LAION for the compute support via FZ Jülich-JSC Along with Bowen Peng EnricoShippole Honglu Fan

Announcing StripedHyena 7B — an open source model using an architecture that goes beyond Transformers achieving faster performance and longer context. It builds on the lessons learned in past year designing efficient sequence modeling architectures. together.ai/blog/stripedhy…

Proud to announce that YaRN (arxiv.org/abs/2309.00071) got accepted to ICLR 2024! Our very own Bowen Peng will be in Vienna to present it 🥳 To celebrate, we're releasing YaRN versions of Upstage's SOLAR model 🤗 huggingface.co/NousResearch/Y… 🤗 huggingface.co/NousResearch/Y…

What if you could use all the computing power in the world to train a shared, open source AI model? Preliminary report: github.com/NousResearch/D… Nous Research is proud to release a preliminary report on DisTrO (Distributed Training Over-the-Internet) a family of

Just finished recording a 2 hr podcast with the Nous Research DisTrO team about their upcoming paper. Haven't been this excited in a while. We are entering a new era in distributed systems H/t to Teknium (e/λ) for putting this on my radar!