If you want to influence the future, now is the time to release a really good benchmark. x.com/jamescham/stat…

One reason I wish more humanities-oriented people would engage with AI is that models are writers, trained on words, producing words. There are strengths & weaknesses in the models that can only be seen if you engage deeply with them as writers. They do not show up in benchmarks

hardmaru Sakana AI How about applying to the automated production of reproducibility reports of existing papers? This should be easier than producing entirely new papers, and be quite useful in that this is a type of work that we probably don't do enough as a community.

This awesome team of University of Toronto students is about to demo the AI Tutor Chatbot they created over the course of the last couple of months as part of their course. I’m very proud of the effort you put in guys 👏🎉 Human Feedback Foundation AI Tinkerers #TorontoAI Mozilla

Join our next Paper Club session AI Tinkerers Human Feedback Foundation - almost 700 people registered for this online session with the authors. Elena Yunusov - I just added Paper2Audio to the page so you can listen to a summary of the paper.

747 registered for this week! Paper Club - AI Tinkerers Human Feedback Foundation - come hang with Cas (Stephen Casper) and learn about the AI Agent Index ✈️

Where are AI agents deployed today? USA. shared via research by Cas (Stephen Casper) on AI Tinkerers Paper Club call just now Human Feedback Foundation Composio Karan Vaidya

At my org Human Feedback Foundation we: - Produce Paper Club feat. AI researchers doing talks on their papers for AI/ML builders - Build AI Tinkerers Toronto community via meetups, tech workshops, etc - Collaborate with leading universities on AI in Production mentorship program

Ok this kind of exploded, in a good way. Our very first large meetup was hosted by Shopify youtube.com/watch?v=PVOwXC… Happy to be back for our next event on April 24th toronto.aitinkerers.org/p/ai-tinkerers… Supporting AI/ML builders in this city and beyond via Human Feedback Foundation x AI Tinkerers 🚀

OpenAI's CPO Kevin Weil: Why You Need Evals to Build AI Products "The easiest way to think about it is as a quiz or a test to gauge how well a model knows a certain set of subject material, or how good it is at responding to a set of questions. Evals basically perform as

Loved the Lenny Rachitsky ep with Guillermo Rauch . Some non-obvious insights/themes that stood out to me: Idea velocity going wild: AI lets you test, iterate, and fail way faster. Roles are blurring, silos collapsing: AI means PMs, Designers, Marketers can actually build stuff, fewer

📊 Introducing Prolific’s AI user experience leaderboard—the most reliable framework for evaluating AI against human preferences. We’re thrilled to share our benchmark on Hugging Face, which assesses how well language models handle real-world tasks based on user experiences. 👇

This evaluation approach means we can tune #AI output to human preferences for better accuracy, alignment, and advancement. Special thanks to Hugging Face, Human Feedback Foundation, MLCommons, Collective Intelligence Project, and Factored for supporting the development of this benchmark.

Glad to see Elena Yunusov here!

Why we need orgs like Human Feedback Foundation and open source benchmarks to measure and track these changes in model behavior and performance.

Had a great time talking about evaluations for language and embodied agents at the Paper Club by AI Tinkerers . Thank you to all the people who tuned in and asked great questions! Human Feedback Foundation paperclub.aitinkerers.org/p/join-our-pap…

🇨🇦 From Toronto to London 🇬🇧 We’ve had another month full of exciting discussions with AI/ML builders at AI Tinkerers! We headed to the Toronto AI Agents meetup at ada's HQ, also catching up with our partner and the meetup series producer Human Feedback Foundation.

Proud to help launch RAISE — a first-of-its-kind program to help Canadian nonprofits adopt AI responsibly. Led by Human Feedback Foundation w/ Creative Destruction Lab (CDL) & support from DIGITAL. A big step forward for #ResponsibleAI + #SocialImpact. 🔗 humanfeedback.io #AIforGood

CDL, Human Feedback Foundation, and the Dais are proud to jointly announce the launch of the Responsible AI Adoption for Social Impact (RAISE) pilot program, a national initiative to help nonprofit organizations across Canada adopt AI responsibly and effectively. globenewswire.com/news-release/2…

AI can help nonprofits scale impact. Is your organization ready? Learn how to build responsible AI policies that align with your mission. Register by August 5: tmu-emarketplace.paymytuition.com/essentials-of-… #nonprofit #impact #digitalskills