Is your model struggling with high latency and huge memory costs for real-time sequence processing?

🚀 Introducing STAR: A transformer-based model for streaming seq2seq transduction with compression.
arxiv.org/abs/2402.01172
#NLProc #Streaming #Seq2seq #Compression #SpeechToText

Very happy to give this talk! Feel free to join!

We love DPO for its elegance and simplicity. So, we are making it even better! By eliminating the reference model, the loss function becomes contrastive and we call it CPO (Contrastive Preference Optimization). It's even more effective at our target task than DPO!

Opening up a new generation of machine translation leveraging the power of LLMs! It's now in #ICLR2024 (w/ Haoran Xu , Amr Sharaf , Hany Hassan Awadalla ).

Teaser: Another breakthrough is coming, soonish..

So happy to share that our paper 'The Trickle-down Impact of Reward (In-)consistency on RLHF' (arxiv.org/abs/2309.16155…) has been accepted by ICLR this year. #ICLR #RLHF

I believe that we should explore/enhance RLHF through a data-centric perspective! JHU CLSP

Very proud that ALMA has been accepted at ICLR 2024! JHU CLSP Microsoft Research
arxiv.org/abs/2309.11674

Multi-language mastery: minimized hardware, maximized efficiency! Johns Hopkins computer scientists (feat. Haoran Xu & Kenton Murray) introduce a new method to reduce the size of multilingual language models. hub.jhu.edu/2023/12/07/mul…

How to generate translation within the target domain only with some monolingual data? Check Weiting (Steven) Tan's new paper!

“Condensing Multilingual Knowledge with Lightweight Language-Specific Modules”

Draft: arxiv.org/abs/2305.13993
By: Haoran Xu and authors

TLDR: We propose lightweight but parameter-efficient language-specific modules and further fuse multilingual knowledge in a shared module.

Is In-Context Learning (ICL) equivalent to Gradient Descent (GD)? There is a common belief that applying ICL in #LLM functions like GD-based fine-tuning. But does this hold in real-world LLMs? 🤔

Find out in our latest paper: arxiv.org/abs/2310.08540

Trilled to share that our paper has been accepted at the main conference of EMNLP 2023! 🎉 EMNLP 2024 JHU CLSP

Happy to share our #EMNLP findings paper: Flatness-Aware Prompt Selection Improves Accuracy and Sample Efficiency: arxiv.org/abs/2305.10713

(1/5) The standard MLE objective is notoriously vulnerable to noise! How can we make LLMs robust to noise in the training data? 🤔

We propose Error Norm Truncation (ENT), a modified training objective that ignores noisy tokens in the training corpus.

📰: arxiv.org/abs/2310.00840

'Does the consistency/robustness of reward model matter in RLHF?' Check our latest work🥳! arxiv.org/pdf/2309.16155…

Had a lot excitement with this work! Feel so lucky to have Sihao Chen and Daniel Khashabi 🕊️ as collaborators!