Interested in a ph.d. position in NLP, in the beautiful city of Copenhagen? We're hiring for a project on the topic of Explainability and Factuality in Language Modelling, at AAU Copenhagen. AAU TECH Department of Computer Science, Aalborg University #NLProc #LLM Apply via this link: stillinger.aau.dk/phd-stillinger…

Our latest typology #NLProc paper was accepted to eaclmeeting main, with Emi who visited us from Mila - Institut québécois d'IA @McGillU, and Esther. We derive continuous word order features from treebanks, better reflecting the variability of language: arxiv.org/abs/2402.01513 Department of Computer Science, Aalborg University

New #NLProc paper on ArXiv! More and more papers in NLP claim to evaluate on ‘typologically diverse’ languages. But what does this even mean? In our new paper (with Wessel Poelman, Miryam de Lhoneux/ @mdlhx.bsky.social and Johannes Bjerva), we systematically such claims. arxiv.org/abs/2402.04222 1/🧵

Can open-source LLMs execute *chains of instructions* in a single query? Not so well, we found. However, they can learn this ability by: - augmenting examples from public SFT mixtures with chains of instructions automatically - performing *sequential instruction tuning* on them.

The memory in Transformers grows linearly with the sequence length at inference time. In SSMs it is constant, but often at the expense of performance. We introduce Dynamic Memory Compression (DMC) where we retrofit LLMs to compress their KV cache while preserving performance

Next week I’ll be in Malta 🇲🇹 to present our work on Improving Generalization in Semantic Parsing by Increasing Natural Language Variation at #EACL2024! 1/3

Introducing Zero-Shot Tokenizer Transfer (ZeTT) ⚡ ZeTT frees language models from their tokenizer, allowing you to use any model with any tokenizer, with little or no extra training. Super excited to (finally!) share the first project of my PhD🧵

Today I am joining NVIDIA part-time as a visiting professor I could not imagine a better place to explore new efficient architectures for LLMs and diffusion I am looking forward to collaborating with so many talented researchers!

Reimagining table representation! In our new #ACL2024NLP paper we introduce PixT3: a family of image-based Table-to-Text Generation models that scale better at generating text from large tables, outperforming traditional text-based baselines. arxiv.org/abs/2311.09808

Completed my research stay at The University of Edinburgh, supported by Otto Mønsteds Fond! Investigated linguistic variation using multi-agent communication. Gained insights, networked with top researchers, and made new friends! Edoardo Ponti Johannes Bjerva AAU TECH Department of Computer Science, Aalborg University EdinburghNLP

BabyLM is looking for organizers to join our team! If you are a mid-stage graduate student, interested in (sample-efficient) language modeling or cognitive science, you could be a great fit! Find out more information, and fill out our interest form here: forms.gle/uCf62Nk6mmQhhx…

Got to present our work in progress on leveraging adapters for machine translation of Creole languages MRL #EMNLP2024 🚀 Stay tuned for more on Creole language MT! aclanthology.org/2024.mrl-1.17/ @v4mer @heather_nlp @prajdabre1 Johannes Bjerva

I'm now officially looking for a post-doc position, starting in Spring! I would be happy to pursue my work on training dynamics, interpretability, or even more specifically on LM representations and character-level models Feel free to reach out!