🚨 New Pre-Print! 🚨 Reviewer 2 has once again asked for DL’19, what can you say in rebuttal?  We have re-annotated DL’19 in the form of classic evaluation stability studies. Work done with Maik Fröbe, Harry Scells, Ferdinand Schlatt, Guglielmo Faggioli, Saber Zerhoudi, Sean MacAvaney, Eugene Yang 🧵

We re-annotated DL’19. The results? 👇

I've just ported the excellent monoELECTRA-{base, large} reranker models from Ferdinand Schlatt & the research network Webis Group to Sentence Transformers! These models were introduced in the Rank-DistiLLM paper, and distilled from LLMs like RankZephyr and RankGPT4. Details in 🧵

Now accepted at #SIGIR2025! looking forward to discussing evaluation with LLMs at #ECIR2025 this week and of course in Padua! In the mean time skim this thread.

Next up at #ECIR2025, Maik Fröbe presenting his fantastic work on corpus sub sampling and how to more efficiently evaluate retrieval systems

It was a really pleasant surprise to learn that our paper “Efficient Constant-Space Multi-Vector Retrieval” aka ConstBERT, co-authored with Sean MacAvaney and Nicola Tonellotto received the Best Short Paper Honourable Mention at ECIR 2025! #ECIR2025 #IR #Pinecone

Thank you Carlos Lassance for shout-out of Lightning IR in the LSR tutorial at #SIGIR2025 If you want to fine your own LSR models, check out our framework at github.com/webis-de/light…

Now it’s Andrew Parry presenting the reproducibility efforts of a large team of researchers in relation to the shelf life of test collections #sigir2025

Really like this work, if you haven't read it yet, have a look: arxiv.org/pdf/2502.20937 PSA from Ian Soboroff! move to Krippendorff's Alpha if you want to check the annotator agreement!

Want to know how to make bi-encoders more than 3x faster with a new backbone encoder model? Check out our talk on the Token-Independent Text Encoder (TITE) #SIGIR2025 in the efficiency track. It pools vectors within the model to improve efficiency dl.acm.org/doi/10.1145/37…

Happy to share that our paper "The Viability of Crowdsourcing for RAG Evaluation" received the Best Paper Honourable Mention at #SIGIR2025! Very grateful to the community for recognizing our work on improving RAG evaluation. 📄 webis.de/publications.h…

The fourth ReNeuIR Workshop @ #SIGIR2025 is about to start. Join us! Program of the day is here: reneuir.org/program.html

Honored to win the ICTIR Best Paper Honorable Mention Award for "Axioms for Retrieval-Augmented Generation"! Our new axioms are integrated with ir_axioms: github.com/webis-de/ir_ax… Nice to see axiomatic IR gaining momentum.