Finally launched x.ai! The mathematics of deep learning is profound, beautiful, and unreasonably effective. Developing the "theory of everything" for large neural networks will be central to taking AI to the next level. Conversely, this AI will enable everyone

For anyone who’s interested, here is the code github.com/bazingagin/npc…. btw, I’m the author of the paper and thanks Luke Gessler for digging my paper out of that many ACL papers lol 😂

That's a wrap! The Waterloo (Waterloo's Cheriton School of Computer Science) team had fun attending the ACL 2023 Conference in Toronto, Canada! #ACL2023NLP 🇨🇦 We would like to congratulate ralphtang.eth Linqing Liu Gin Jiang Jimmy Lin et al. for winning the Best Paper Award at ACL 2023!!🏆 Next stop is SIGIR 2023.

If you want to move past the AI hype and learn some real fundamental basics behind today's learning algorithms there's no better choice than MacKay's "Information Theory, Inference and Learning Algorithms". You can read the book for free on the official website:

📢The costs for training (L)LMs skyrocketed 🚀 in recent years, motivating efficient training algorithms. However, when pre-training BERT and T5 models with a fixed compute budget, we find their gains vanish compared to a baseline with a fully-decayed learning rate! 1/5

Very excited to share our latest work: 🤔 Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation w/ Edward Kim, Beyza Ermiş, Marzieh Fadaee, Sara Hooker 🔗: arxiv.org/abs/2310.14424

Announcing Mixtral 8x7B mistral.ai/news/mixtral-o… and our early developer platform mistral.ai/news/la-platef…. Very proud of the team!

x.ai/blog/grok-os

Meet DBRX, a new sota open llm from Databricks. It's a 132B MoE with 36B active params trained from scratch on 12T tokens. It sets a new bar on all the standard benchmarks, and - as an MoE - inference is blazingly fast. Simply put, it's the model your data has been waiting for.

At Databricks, we've built an awesome model training and tuning stack. We now used it to release DBRX, the best open source LLM on standard benchmarks to date, exceeding GPT-3.5 while running 2x faster than Llama-70B. databricks.com/blog/introduci…

Today we released an open source model, DBRX, that beats all previous open source models on the standard benchmarks. The model itself is a Mixture of Experts (MoE), that's roughly twice the brains (132B) but half the cost (36B) of Llama2-70B. Making it both smart and cheap. Since

magnet:?xt=urn:btih:9238b09245d0d8cd915be09927769d5f7584c1c9&dn=mixtral-8x22b&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=http%3A%2F%https://t.co/OdtBUsbeV5%3A1337%2Fannounce

Official now, very proud of the team! Apache 2.0 and instructed versions for your pleasure, available today on la Plateforme mistral.ai/news/mixtral-8…

Excited to work on this code autocompletion model that supercharge your coding experience!

Character AI is serving 20,000 QPS. Here are the technologies we use to serve hyper-efficiently. [research.character.ai/optimizing-inf… ]

We are thrilled to announce the milestone release of SGLang Runtime v0.2, featuring significant inference optimizations after months of hard work. It achieves up to 2.1x higher throughput compared to TRT-LLM and up to 3.8x higher throughput compared to vLLM. It consistently

Evaluating LLMs in enterprise domains can be challenging. In this post, we share how our applied AI team synthesized high-quality code tests for specific libraries to enhance system performance. Joint work with MatthewHayes Matei Zaharia Ritendra Datta!

The world model is taking shape... 🌐

Thrilled to kick off the Gemini 2.0 era with Gemini 2.0 Flash, an update to our workhorse model that outperforms even 1.5 Pro at twice the speed. It has really great multilingual skills, and can natively call tools, like Google Search. It’s the first release in the Gemini 2.0

Good post from Balaji on the "verification gap". You could see it as there being two modes in creation. Borrowing GAN terminology: 1) generation and 2) discrimination. e.g. painting - you make a brush stroke (1) and then you look for a while to see if you improved the