If you've got ideas for the future of RAG evaluation, consider submitting to the Eval4RAG workshop!

🔊 The second workshop on Large Language Models for Evaluation in IR (#LLM4Eval), co-located with #WSDM2025, is now accepting submissions! If your work explores or leverages LLM capabilities for evaluation, we encourage you to submit and share your research.

The call for papers is now open for the second workshop of #LLM4Eval: 💡 See CFP: llm4eval.github.io/WSDM2025/cfp/ 🗓 Submission deadline: January 20, 2025

At 15:00 on 13th January, Francesco Luigi De Faveri kdf_7 from University of Padua Università di Padova will give an #IRTalk entitled "Preserving Privacy in Information Retrieval: Leveraging Differential Privacy Across Domains". Details: samoa.dcs.gla.ac.uk/events/viewtal… UofG Computing Science Glasgow IR Group

🚨 New Pre-Print! 🚨 Reviewer 2 has once again asked for DL’19, what can you say in rebuttal?  We have re-annotated DL’19 in the form of classic evaluation stability studies. Work done with Maik Fröbe, Harry Scells, Ferdinand Schlatt, Guglielmo Faggioli, Saber Zerhoudi, Sean MacAvaney, Eugene Yang 🧵

Today is the submission deadline! Don’t forget to submit your paper.

🏨 🛏️ The #sigir2025 HOTELS are now available! sigir2025.dei.unipd.it/recommended-ho… sigir2025.dei.unipd.it/accomodation.h… ⚠️ WARNING Sunday 13 July there will be the Iron Maiden concert in Padua (the only Italian date). Accommodations are expected to fill up quickly, so be proactive in booking 🤟

📢 Call for Papers: LLM4EVAL @ SIGIR '25 - The 3rd Workshop on Large Language Models for Evaluation in IR! SIGIR 2025 🗓️ Submission: April 23, 2025 (AoE) 🔗 Full CFP: llm4eval.github.io/SIGIR2025/cfp/ 🌐 Workshop: llm4eval.github.io/SIGIR2025/

In this series, we move BEYOND search relevance to tackle: ✨ Personalized search evaluation ✨ Conversational interfaces ✨ Holistic assessment of integrated experiences ✨ Bias & fairness in LLM evaluations

A large contingent of our staff & students UofG Computing Science will soon be traveling to #ECIR2025. We'll be presenting our latest work in #search & #RecSys throughout the 5-day conference. Looking forward to connecting with colleagues and friends in Lucca! ECIR2025 UoG IDA Section, Glasgow CS

📢 The final schedule for the Query Performance Prediction (QPP++ 2025) Workshop at ECIR2025 is now live! 📅 Join us on 10th April 2025: qppworkshop.github.io 🎤 Keynote by Debasis Ganguly : "The Role of Query Performance Prediction in Developing Adaptive Search and RAG Systems"

Looking fwd to my talk on 10th ECIR2025, where I'll be giving a perspective on the potentials of applying #QPP to improve #RAG.

🎉Happy to share that two papers have been accepted at #SIGIR2025 "A Comparative Study of Large Language Models and Traditional Privacy Measures to Evaluate Query Obfuscation Approaches" w/ Guglielmo Faggioli , Nicola Ferro. “Evaluating MDCU in IR” w/ Guglielmo Faggioli , Nicola Ferro, K. Järvelin.

Congratulations to Manish Chandra Debasis Ganguly and Iadh Ounis for being awarded the Best Paper Award at #ECIR2025 for their work entitled “One size doesn’t fit all: Predicting the number of examples for in-context learning”. The paper will be presented Wednesday morning in Lucca.

LLMs are reshaping how we evaluate ranked lists, conversations, and even simulate users. At #LLM4Eval @ #SIGIR2025, we’re spotlighting 3 special themes: 🧠 Beyond query-doc 🗣️ Simulating users 🧪 Synthesizing evaluation data 👇 Curious? Read on & submit by April 23! #LLMs #IR

🧵 Theme 2: 🗣️ Simulating users: What if LLMs could act as realistic user proxies? We are looking for papers on simulating user behavior, preferences, context, or personas for evaluation.

Glad to be organizing the third series of #LLM4Eval @ #SIGIR2025! We have just announced three special themes we are especially excited about: beyond query doc evaluation, simulating users, and synthesizing evaluation data. Submit by April 23 🧠🗣️🧪

Chuan Meng ECIR2025 Delighted to deliver the keynote at the QPP++ workshop. Thanks a lot to the organizers Chuan Meng Guglielmo Faggioli Josiane Mothe for inviting me. Slides: tinyurl.com/4t23dj2y

We will host multiple breakout discussions to pick your brains 🧠 on the future challenges and research on this topic. Let's shape the LLMEval research agenda. Interested? Already have thoughts and ideas? Then, submit your extended abstracts or just join us in the workshop.

📢 Deadline extended! Submit to #LLM4Eval @ #SIGIR2025 by April 30 (AoE) We welcome work on LLM-based evaluation, including: 🧠 beyond query-doc 🗣️ simulating users 🧪 synthetic eval data 🔗 llm4eval.github.io/SIGIR2025/cfp/ #CFP #IR #LLMs #SIGIR2025