We know that Jamba 1.5 models are the fastest, but the question is - how fast? Artificial Analysis tested our models to find out 😎The image below shows the throughput for various models (with prompt length = 10K tokens). Jamba 1.5 models are a whole lot faster – and that speed

Verified by Artificial Analysis, Cerebras Inference is capable of serving Llama 3.1 70B at 450 tokens/sec and Llama 3.1 8B at 1,850 tokens/sec!

More details in this thread from Artificial Analysis. x.com/ArtificialAnly…

I've been playing with SambaNova Systems's API serving fast Llama 3.1 405B tokens. Really cool to see leading model running at speed. Congrats to Samba Nova for hitting a 114 tokens/sec speed record (and also thanks Kunle Olukotun for getting me an API key!) sambanova.ai/blog/speed-rec…

Verified by Artificial Analysis, Cerebras Inference achieves 1,850 tokens/sec on Llama 3.1 8B and 450 tokens/sec on Llama 3.1 70B! By dramatically reducing processing time, we're enabling more complex AI workflows and enhancing real-time LLM intelligence. This includes a new class

Cerebras Inference has the industry’s best pricing for high-speed inference - 10c per million tokens for Llama3.1- 8B - 60c per million tokens for Llama3.1- 70B Try it today: inference.cerebras.ai

Great post from Meta that provides insights into the growth of their open-source Llama ecosystem and how AI is being used, featuring 5 case studies from large enterprises. Data from Artificial Analysis is referenced and was used to provide independent evidence that the Llama