Are open-sourced LLMs really good? 👀 We introduce FLASK🧪, a fine-grained evaluation based on skill sets! Even SOTA open-sourced LLMs such as LLaMA2 Chat 70B lag behind proprietary LLMs for some abilities. 🤯 Paper: arxiv.org/abs/2307.10928 Demo: kaistai.github.io/FLASK

After spending just 20 minutes with the Mistral AI model, I am shocked by how unsafe it is. It is very rare these days to see a new model so readily reply to even the most malicious instructions. I am super excited about open-source LLMs, but this can't be it! Examples below 🧵

😀We're looking for a talented researcher to join our team at Naver Labs Europe (NAVER LABS Europe) , working on LLMs and Retrieval!😃 Please apply here: europe.naverlabs.com/job/research-s… !

📢 Open position! PostDoc position in #LLM powered conversational agents NAVER LABS Europe Grenoble, France. UTTER ***Please share*** Start date: September Duration: 1yr More info & how to apply: europe.naverlabs.com/job/postdoc-ll…

We offer this 1y postdoc to work with us on the UTTER EU project on LLM-based agents ! Come work with us on 1 or several of these topics: i] managing uncertainty and ambiguity ii] improving the use of conversational context iii] ensuring the safety and alignment of LLMs.

What’s a good baseline for RAG? 🤔 The literature shows consistent differences in experimental setups, retrievers, datasets, and metrics. So, we built the BERGEN library github.com/naver/bergen to enhance reproducibility and identify strong baselines : 🧵 NAVER LABS Europe

🧵 Excited to share PRISM, our new methodology for auditing biases in Large Language Models, as part of @PHAWM_project w/Leif Azzopardi. This flexible, role-based framework can be tailored to audit biases on political, social, or any dimensions. Responsible Ai UK arxiv.org/abs/2410.18906

📢 Job Alert 📢 NAVER LABS Europe My team is looking for a Research Scientist in Visual Representation Learning - More info tinyurl.com/3jz935fk

If you are at #EMNLP2024 and interested in RAG, come to discuss our BERGEN library! Vassilina Nikoulina will present the BERGEN poster tomorrow, Nov 13, 16:00-17:30, location: Jasmine Repo: github.com/naver/bergen Paper: aclanthology.org/2024.findings-… NAVER LABS Europe #NLProc #RAG

We have an open research internship position at Naver Labs Europe, in Grenoble, France! Apply to work on improving #RAG for multilingual and/or multi-domain settings! europe.naverlabs.com/job/3/ Vassilina could tell about it at her poster tomorrow #EMNLP2024, Nov 13, 16:00-17:30

I'll present two papers at the main COLING 2025 conference next week: (1) ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models (w/ Thibaut Thonet and Jos Rozen) - see arxiv.org/abs/2403.20262 and github.com/utter-project/…

We are #hiring for Internship: Advanced Constraint Processing in LLMs at Naver Labs Europe. We are attending ⁦COLING 2025⁩ if you would like to meet! - via #Whova event app

🚨 Excited about ML/NLP and looking for a research internship on controlled text generation? Come work with us on advanced constraint processing in large language models at NAVER LABS Europe! ✨ Learn more and apply here: europe.naverlabs.com/job/2-7/

Excited to share that Provence is accepted to #ICLR2025! Provence is a method for training an efficient & high-performing context pruner for #RAG, either standalone or combined with a reranker huggingface.co/blog/nadiinchi… w/ @thibault_formal Vassilina Nikoulina Stéphane Clinchant NAVER LABS Europe

Join AutoMin 2025 - 3d Run of the Automatic Minuting Shared Task at #SIGDIAL2025 to push the limits of LLMs in 📌 Meeting Summarization 📌 Meeting Question Answering 📅 Workshop: Aug. 28, 2025 ufal.github.io/automin-2025/ #NLP #LLMs #AI #AutoMin

🚀 AutoMin 2025 – 3rd Edition of the Automatic Minuting Shared Task at #SIGDIAL2025! NAVER LABS Europe & UTTER push LLMs in: 📌 Meeting Summarization 📌 Meeting QA 📅 Aug 28, 2025 Join the future of NLP, AI & Meeting Understanding! 🔗 ufal.github.io/automin-2025/ #NLP #LLM #UTTER

Still possible to join this shared task !

🎉 Delighted to share that our work “FaST: Feature-aware Sampling and Tuning for Personalized Preference Alignment with Limited Data” has been accepted to #EMNLP2025 main! 🧑🏻‍💻 Joint work with Germán Kruszewski, Jos Rozen, Pierre Erbacher, Marc Dymetman 📄 Preprint: arxiv.org/abs/2508.04698