.Ori Goshen, Co-Founder + Co-CEO of AI21 Labs, talks about growth opportunities for the company following a $208 million Series C funding round, and shares his perspective on the future of AI on #NYSEFloorTalk with Judy Khan Shaw

Building a RAG solution is easy. Building a great one is not.

In our guest blog on Streamlit, our team explores the intricacies of how AI21's Contextual Answers Task-Specific Model & our RAG Engine generate context-based answers grounded in your proprietary organizational data.…

Paper of Jamba: A Hybrid Transformer-Mamba Language Model

Jamba, a novel architecture which combines Attention and Mamba layers, with MoE modules, and an open implementation of it, reaching state-of-the-art performance and supporting long contexts.

We showed how Jamba provides…

Jamba

A Hybrid Transformer-Mamba Language Model

present Jamba, a new base large language model based on a novel hybrid Transformer-Mamba mixture-of-experts (MoE) architecture. Specifically, Jamba interleaves blocks of Transformer and Mamba layers, enjoying the benefits of

Last Week AI21 Labs released the production-scale Mamba implementation, and today, they released their paper. 🧐
Jamba introduces a new hybrid Transformer-Mamba mixture-of-experts architecture offering state-of-the-art performance but with significant improvements on long…

Yesterday AI21 Labs released Jamba, the first production-scale Mamba implementation as a hybrid SSM-Transformer MoE 🐍 And today, you can already finetune it with Hugging Face TRL.

Alexander Doria shared a working Qlora script using 4-bit quantization on an A100 GPU (for now,…

🥹 Jamba is truly amazing!

Everyone speaks about Long Context. But it's been mostly useful for ingesting in-context learning.

But Jamba seems to be the first Model offering a great throughput even for higher context!

This Jamba overview is a great way to quickly understand the novel features Jamba brings to the dev community. Thanks Ai Flux!

I played a little with Jamba: it looks like an amazing model.

In terms of architecture, the MoE implementation is very close to Mixtral's. What's great about it is that it hasn't been fine-tuned. Curious to see how much improvement we can get through SFT.

I made a little…

Extremely cool new model release from AI21 Labs - Jamba - and it's not even a transformer! It's a hybrid model that combines Mamba (structured state space model), transformer layers and MoE technique, and it's a first production-grade Mamba based model!

* It's a 52B MoE with 12B…

incredibly impressed by AI21 Labs' Jamba today. This is the first legitimate Mixtral-killer we've seen and it came out of 'nowhere':

buttondown.email/ainews/archive…

They've helped me redefine my idea of a model 'weight class' from 'number of parameters' (increasingly outdated with MoEs…

OMG! Jamba! 3X throughput and fits 140K context on a single GPU!

Find our new model on Hugging Face.