Dam this trending hard over the weekends DeepEval

No marketing stunts, only grind

In DeepEval 's latest release ⭐ (v3.1.9), we're continuing our mission to build the evaluation infrastructure for the open-source world. This update brings a new form of test case, called Arena — our new way to allow developers to run comparisons across different iterations of

Loving the message Confident AI

A Python decorator is all you need to trace LLM apps (open-source). Most LLM evals treat the app like an end-to-end black box. But LLM apps need component-level evals and tracing since the issue can be anywhere inside the box, like the retriever, tool call, or the LLM itself.

🚧 One of the biggest barriers to LLM evaluation today? The steep learning curve. You’re expected to: - Understand how each metric works - Interpret what the results actually mean - Compare across multiple (often conflicting) metrics And if you get it wrong? You might end up

Thrilled to announce that DeepEval now supports LiteLLM as a native model provider. Developers can now use hundreds of models without having to change the LiteLLM (YC W23) models they're already using. More to come in the following days.

In the week of June DeepEval decided to embrace OpenAI's API messages format for evaluating multi-turn conversations - and as a result almost tripled the total number of conversational evals ran from all our users. As a result we're releasing a native OpenAI integration later

The most neglected pages on Confident AI are the settings, organization, and project pages. Kritin Vongthongsri and I sat down over the weekend and refactored + redesigned 10k lines of code to make things more professional. Does this look better now?

DeepEval LangChain CrewAI LlamaIndex 🦙 ..with llm and tool span visible on the platform docs for tracing your llm apps on Confident AI : documentation.confident-ai.com/docs/llm-traci…

You guys loved G-Eval, so we shipped Multimodal G-Eval. You can now evaluate any multimodal task (text2image, computer/browser use, etc) in plain english. We've also added 1. gpt-4.1 and o4-mini suppport for all multimodal metrics on DeepEval 2. Image support on Confident AI

DeepEval on trending again, seems like the recent component-level evals is definitely something people need for evaluating AI agents.

Too lazy to sift through 100+ test cases? Confident AI just dropped 🪄AI Insights Board🪄. This means running evals with DeepEval instantly tells you: 1. What are your LLM app's strengths and weaknesses 2. Areas your LLM consistently struggles with 3. The exact model and

One of the most beautiful things about talking to users is you realize where they are struggling. We're making two major releases at DeepEval this week to address the most common issues people face when running evals.

At Confident AI, we say no to vibe coding, because we’re all about “vibes” coding.

Don’t get frustrated by writing print statements and endlessly scrolling terminal logs to debug your LangChain (and LangGraph) app. Trace your agent’s execution steps in production on Confident AI using our callback handler, with just two lines of code. Documentation:

🤖 Two LLM outputs walk into an arena… Only one leaves with the crown 👑 ⚔️ Pairwise battles ⚖️ Elo-style scoring 🙈 Blind trials 🧠 LLMs judging LLMs No complex metrics. Just ask: which one is better? confident-ai.com/blog/llm-arena…

🚀 Shipped Integration You can now trace your CrewAI apps on the Confident AI platform. By just adding 2 lines of code in your app, you can get the entire execution steps of your agent in the form of a single trace. Leverage your LLM application performance using

One of the greatest blocker to LLM evaluation? Communication between engineers and domain experts when curating datasets. When engineers gets put on an AI project, they aren't necessarily experts in the domain they're building for. In fact, the humans safeguarding AI responses

.@Confident_AI's DeepTeam is an open-source red teaming framework for AI agents. Test for memory leaks, goal hijacking, and decision flaws across 40+ attack types & vulnerabilities. Congrats on the launch, Jeffrey 🐬 confident-ai.com and Kritin Vongthongsri! github.com/confident-ai/d…