🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Finally got around to publishing the largest per-parcel California property tax dataset - on Kaggle kaggle.com/datasets/iwebs… These are the numbers behind the CA property tax map, which still gets a surprising amount of traffic and data requests officialdata.org/ca-property-ta…

promptfoo has passed 250,000 evals + thousands of users from companies like Microsoft, Salesforce, Intel. Open source, developer-first, organic grass-fed LLM evals (choose the best prompt and model) If you are serious about deploying LLMs, check it out! promptfoo.dev

The Eta Aquariids peak this weekend, when the Earth passes through a dust cloud left by Halley's Comet meteorshowers.org

For all our LLM (and many ML) projects at Shopify we standardized on promptfoo.dev for writing evals. That has caused a lot of great progress speedups. Highly recommended if you are looking for good eval system. Fun realization: TDD is alive and well in the ML world!

promptfoo's 0.62.0 changelog is very crowded for just 1 week of work. Shoutout to maltais (Shopify), Rohit Agarwal (Portkey), Peli de Halleux (Microsoft), @dlssrt, Michael D'Angelo and many others not on Twitter. Open source makes the world go round 💪

Just added automated LLM red teaming/vulnerability scanning to promptfoo! Uncovers app-specific jailbreaks and failure modes... Completely open source promptfoo.dev/llm-vulnerabil…

This is a picture of the 9.8 km-long moon Daphnis traveling inside the Keeler Gap in Saturn's A ring, captured with Cassini's narrow-angle camera 14 years ago on July 5, 2010

We all knew image generators can be jailbroken to generate violent and graphic content. It turns out the jailbreak process can be fully automated :o Write-up here (open-source) with examples from OpenAI's Dall-E promptfoo.dev/blog/jailbreak…

Had a great chat with Anjney Midha 🇺🇸 on the finer points of AI safety and security

Democratizing Generative AI Red Teams a16z.com/podcast/securi… < Really good discussion with Ian Webster (of promptfoo ) and Anjney Midha 🇺🇸 about securing AI products at the app layer, rather than focusing on models.

Perseids peak tonight! meteorshowers.org is once again on NASA APOD :)

Doing online support can be a drag, but sometimes you get gems like this

I wrote a deep dive blogpost about how promptfoo helps us eval our AI features! semgrep.dev/blog/2024/does…

Anthropic has been quietly publishing top notch content on LLM fundamentals. Lots of great examples of using Promptfoo for evals in this new course!

❤️ Love seeing promptfoo + Google AI Studio Gemini 2.5 experiments in the wild! Check out this example benchmark that gauges models' ability to help people in need apply for SNAP benefits: propel.app/insights/build…

Exploring Promptfoo via Dave Guarino’s SNAP evals simonwillison.net/2025/Apr/24/ex…