🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Now that we are fully immersed in the year of Agentic AI, nobody has created a benchmark to evaluate how well these agents work together...until now. 👇 @Salesforce's CRMArena-Pro builds upon CRMArena with nineteen expert-validated tasks across sales, service, and 'configure,

CRMArena-Pro reveals why enterprise AI deployment remains challenging—many top-performing agents struggle significantly on real-world business tasks. 👇Full technical breakdown from our research lead Kung-Hsiang Steeve Huang below. #EnterpriseAI #AgenticAI #EGI

AI agents are rapidly integrating into various industries, however their full potential remains underutilized due to performance inconsistencies and enterprise hesitation. To alleviate this issues, we introduce <CRMArena-Pro>, a novel enterprise-agent benchmark for holistic and

Does RL truly expand a model’s reasoning🧠capabilities? Contrary to recent claims, the answer is yes—if you push RL training long enough! Introducing ProRL 😎, a novel training recipe that scales RL to >2k steps, empowering the world’s leading 1.5B reasoning model💥and offering

Salesforce AI Introduces CRMArena-Pro: The First Multi-Turn and Enterprise-Grade Benchmark for LLM Agents: ift.tt/zs73bwj

Salesforce AI Launches CRMArena-Pro: A Game-Changer for Evaluating LLM Agents in Business #CRMArenaPro #LLMAgents #SalesforceAI #CustomerExperience #DataPrivacy itinai.com/salesforce-ai-… Understanding CRMArena-Pro: A New Benchmark for LLM Agents Salesforce AI has introduced CRM…

Salesforce AI Introduces CRMArena-Pro: The First Multi-Turn and Enterprise-Grade Benchmark for LLM Agents #AI #MachineLearning #IoT #LLM marktechpost.com/2025/06/05/sal…...

Thanks Marktechpost AI Research News ⚡ for covering CRMArena-Pro 🙏 Our new benchmark reveals that even the best LLM agents achieve only ~58% success rate on realistic business tasks, dropping to 35% in multi-turn scenarios. Also, confidentiality awareness is nearly non-existent across all models

Synthesized data for #EnterpriseAI evaluation is an ethical imperative. CRMArena-Pro lets us rigorously test agents in a real-life business environment—a messy, multi-step, complex world—without putting sensitive data at risk. Proud of the team's work toward safer, more

🚀 Discover how LLMs perceive their knowledge boundaries across languages in our #ACL2025 main paper! 🌍 By probing LLMs’ internal representations, we reveal key insights on where knowledge boundaries are encoded & propose a training-free method to combat cross-lingual

Another paper drop, this time from Salesforce: "These results underscore a significant gap between current LLM capabilities and real-world enterprise demands, highlighting needs for improved multi-turn reasoning, confidentiality adherence, and versatile skill acquisition."

🚀We are thrilled to launch 'Lingshu' – A Generalist Medical Multi-modal Foundation Model! 🩻 🌟 Highlights of Lingshu: ⚕️ Unified knowledge across 12+ imaging modalities (X-Ray, CT, MRI & more!). 🧠 Enhanced reasoning & reduced hallucinations via novel data curation and

Im really excited to be presenting this work in Greece! 🏛️ As generative text models start reshaping how we search for information, understanding their societal impact is more important than ever. 🔎 If you’ll be at #ACMFAccT2025, let’s grab a coffee and chat! ☕️

1/10🎉New paper on AI Agent and LLM judge safety "Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflows" As AI agents become increasingly autonomous, they often rely on feedback from judges (evaluators). These judges evaluate, critique, and

🚀 I'm looking for full-time research scientist jobs on foundation models! I study pre-training and post-training of foundation models, and LLM-based coding agents. The figure highlights my research/publications. Please DM me if there is any good fit! Highly appreciated!

arxiv.org/abs/2505.18878 Salesforce tried LLMs in Real Business Scenarios and Found Disappointing Performance Even from the Best

Andrej Karpathy Great share as usual! Just read this related piece where a study showed issues with LLM-based agents not recognizing sensitive information and not adhering to appropriate data handling protocols: theregister.com/2025/06/16/sal… paper: arxiv.org/abs/2505.18878

4/ I’m actually bullish medium term involving AI in customer experience. But IT depts must educate themselves. The details on CRMArenaPro and the gap between LLMs / enterprise CRM needs in a major new paper by Salesforce AI Research’s Kung-Hsiang Steeve Huang + team: arxiv.org/abs/2505.18878