“What should I work on?” is a question we hear more & more often from NLP students, during a time when the media rhetoric is that “it’s been all solved” Turns out there are many NLP research areas rich for exploration—here is our answer from 20+ students arxiv.org/abs/2305.12544

🧩 Introducing PuzzleVQA: A dataset consisting of puzzles based on abstract patterns. It evaluates large multimodal models using fundamental concepts like colors, numbers, sizes, and shapes. 🔍 Our experiments reveal these models struggle to generalize with simple abstract

🚨 [NAACL 2024] Thrilled to share our latest research paper SEALING (SElf-Adaptive sampLING), a study on the frame sampling methods for video question answering task on image--text models (ITMs, or VLMs). 📖 paper: arxiv.org/abs/2307.04192 💻 code: github.com/declare-lab/Se…

``HyperTTS: Parameter Efficient Adaptation in Text to Speech using Hypernetworks,'' Yingting Li, Rishabh Bhardwaj, Ambuj Mehrish, Bo Cheng, Soujanya Poria, ift.tt/geBK4H2

🎉𝐓𝐚𝐧𝐠𝐨 𝟐: A new text-to-audio model that aligns diffusion-based generations🎧 🔍Tango 2 from Declare-lab captures concepts and their temporal ordering in the generated audio, enhancing the audio-text alignment (one event followed by another event in generation)

📢 I added the new Tango-2 text-to-audio model to the Image-to-SFX space on Hugging Face, please try it and compare with alternatives —› huggingface.co/spaces/fffilon…

Proud moment as we find our research "Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense" has been recognized with the Social Impact Award at #NAACL2024. Kudos to all the co-authors!

DARWIN is a decode-time alignment technique that uses a reward-guided tree search framework to align the LLM and achieve comparable performance to preference optimization on 2 instruction following benchmarks. Paper: Reward Steering with Evolutionary Heuristics for Decoding-time

``Improving Text-To-Audio Models with Synthetic Captions,'' Zhifeng Kong, Sang-gil Lee, Deepanway Ghosal, Navonil Majumder, Ambuj Mehrish, Rafael Valle, Soujanya Poria, Bryan Catanzaro, ift.tt/DyCxpJE

Synthetic labels are amazing! Do you need an audio labelling machine? Audio Flamingo checkpoints are available on github.com/NVIDIA/audio-f… ...and pre-training with synthetic labels from Audio Flamingo gives large improvements in text-to-audio models arxiv.org/abs/2406.15487

#indoml2024 gearing up with a stellar set of speakers (indoml.in) Bhramar Mukherjee Ranjay Krishna Chandrajit Bajaj Soujanya Poria Munmun De Choudhury, PhD Arindam Banerjee ... stay tuned for more updates ...

Can LLM generate novel and valid research hypotheses, and therefore work as a copilot for scientists? For the first time, we find that it CAN!!! Accordingly we propose a new paradigm for research with a research copilot. Join us at poster session, Monday 12:45 pm! #ACL2024

For me, it was the highlight of the trip. #ACL2024