Finally! Thanks Leaderboard! 🤗

Up next on Hugging Face! Coming to you this week:

- New fine-tuned Llama-3-70B models
- New fine-tuned Llamixtral-3 models (Mixture of Llama-3 in 24B and 47B)
- New fine-tuned Qwen1.5-32B models

The shortlist so far! Vote people!😎

1. Llamixtral-Nx8B
2. MixLlama-Nx8B
3. LlaMoE-Nx8B
4. Llama-3-Nx8B

Why the name (Mix)tral? Only Mixtral, Deepseek MoE, and Qwen MoE architectures are supported. So, I chose Mixtral to create my MoE based on the Llama-3-8B models.

If you were to create a MoE version of the Llama-3 model, what would you name it?

1. LlaMix3
2. MixLlama
3. Llamixtral
4. Mixtrallama
5. Llama-3-Nx8B

Earlier today I submitted this model to the Open LLM Leaderboard. There is room to improve in our quest to extend Llama-3 context length. 🚀

The weekend looms—what's our play? Another daring venture with the Llama-3-70B model! Dive in, my friend, the deep end beckons!

Expect more Llama-3-70B models next week on Hugging Face 🚀