A 🧵 on my favorite, influential works on 'Data Measurements'

🚂 Datasets drive AI progress
📚 But... massive datasets remain impenetrable & poorly understood for *years*
🔍 Data forensics uncover their mysteries

1/

Perhaps the highest praise one can give a data auditing paper: working on this paper fundamentally changed how I think about web-crawled datasets, assumptions I made about them, and how I practice data collection and cleaning.

very proud to share a faster & stronger rerank model today!! Please try it out and let us know your thoughts

Introducing Rerank 3: our newest foundation model purpose built to enhance enterprise search and Retrieval Augmented Generation (RAG) systems, enabling accurate retrieval of multi-aspect and semi-structured data in 100+ languages. txt.cohere.com/rerank-3/

Perhaps the most significant new feature is search over semi-structured data like tables and json objects. This is something existing search stacks really struggle with and Rerank 3 provides a dramatic lift in performance.

Introducing Rerank 3! Our latest model focused on powering much more complex and accurate search.

It's the fastest, cheapest, and highest performance reranker that exists. We're really excited to see how this model influences RAG applications and search stacks.

Join our community-led Geo Regional Asia Group on Wednesday, April 17th as they host Saquib Sarfraz for a presentation on 'Efficient Clustering & Visualization with FINCH & h-NNE: Large-Scale Data Understanding in NLP & Vision Applications'

Learn more: cohere.com/events/c4ai-Sa…

Rerank 3 is the best reranker for AI and software transitioning to AI.

4k context length, search within structures (emails, invoices, JSON documents, code, and tables), still multilingual and fast

extremely proud of Sylvie Shi Nils Reimers elliott c for this work. thank you…

Last summer we launched our Research Grants Program! Each grant provides subsidized access to the Cohere API, enabling researchers to use large-scale models for experimentation & testing.

Find out more and apply here: txt.cohere.com/c4ai-research-…

🧵 LLMs are most useful if they can serve users in their language. We highlight performance in three widely spoken non-Latin script languages:
- 🇰🇷 Korean
- 🇯🇵 Japanese
- 🇨🇳 Chinese

Command R+ outperforms Claude 3 Sonnet and Mistral Large and is competitive with GPT 4-Turbo on…

Command R+ has strong multilingual capabilities. Its tokenizer also compresses multilingual text much better than other tokenizers. For example, in comparison the OpenAI tokenizer uses:
- 1.18x more tokens for Portuguese
- 1.54x more tokens for Chinese
- 1.67x more tokens for…

I am excited to announce Quotient AI!

Quotient enables developers to evaluate, improve and ship high quality AI products through fast, real-world, data-backed experimentation.

We're throwing 4 hackathons at each of our offices around the world!! If you're in NYC, London, Toronto, or SF come hang out with us and build with Command R and R+ 🛠️

Get ready for Cohere Build Day 🌐

Join our exclusive developer workshop in 4 cities! Dive into our new Command R & R+ models and learn from Cohere experts to develop enterprise-grade AI solutions.

Submit your interest to attend now:
info.cohere.ai/cohere-build-d…

💯let’s do this. So overdue.

Speaking of model releases; come work with Sara Hooker and I :) jobs.lever.co/cohere/3dbae8b…

let's gooo

There are many LLM evals, but the Arena is by far the most trustworthy.

Command R+ is now officially the best open weights model! 🔥

Go chat with it here: huggingface.co/spaces/CohereF…

Command R+ is suuuuuuuper hot🔥
Climb up to #6 within 1 week on the leaderboard. Congrats to Cohere For AI

There's still time to register for our next Fireside Chat, tomorrow, April 10th featuring Sebastian Ruder, Research Scientist at Cohere & C4AI Research Scientist, Ahmet Üstün for a discussion on 'Exploring the Frontiers of Multilingual AI' 🔥

Register now: zoom.us/webinar/regist…