Unfortunately, I couldn't attend COLING 2024 in person, but here's a shoutout to our two contributions! 🎉📖 #COLING2024 #ResearchHighlights 1. Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction. [paper]

Cool work! Purely end-to-end multimodal transformation may be hard, but such kind of integration of different cross-modal encoders is absolutely a practical solution. 🔥

Nice work by Minghao Wu ! Mimicking human expert behaviours in some specific tasks (Like last year we encouraged LLMs to human-likely evaluate translations by analyzing errors, refer to x.com/liangdingNLP/s…) and implementing them with carefully designed LLM agents is a

Thanks so much for your post Aran Komatsuzaki In this work, we present a 🔥zero-shot promoting strategy🔥 to enhance the 🔥math&reason🔥performance, where we pushed the SOTA performance of several best LLMs.

When we should use RAG and how to maximise its effectiveness? Check out Shiyu Ni ‘s recent work👏

Liang Ding Indeed! Well, you never know what becomes relevant. I believe diffusion models and multimodal tokenization techniques are non-autoregressive generation as well. Speculative decoding reminds me of techniques that came up in this context too. Let's keep moving and building models!

🌸Thanks for highlighting our TransAgents work in your GitHub repo! Andrew Ng 🪐We're also committed to integrating Language Agents into real-world translation service. Paper: arxiv.org/pdf/2405.11804 Our Demo is also on the way: github.com/minghao-wu/tra…

🔥🔥🔥We release our new work "Demystifying the Compression of Mixture-of-Experts Through a Unified Framework" (with code), where we aim to find the best recipe to compress the MoE in LLMs, and in this work, by using the combined strategy of Expert Slimming and Expert Trimming

Transformers have long been criticized for the computational complexity of their attention mechanisms. Are these computations worth it? Please check out Shwai He ‘s new work🎊

My analysis for Llama 3.1 1. 15.6T tokens, Tools & Multilingual 2. Llama arch + new RoPE 3. fp16 & static fp8 quant for 405b 4. Dedicated pad token 5. <|python_tag|><|eom_id|> for tools? 6. Roberta to classify good quality data 7. 6 staged 800B tokens long context expansion

AI System Achieves Silver Medal-level score in IMO The International Mathematical Olympiad (IMO) is the oldest, largest & most prestigious competition for young mathematicians. Every year, countries send their top young mathematicians to take a 6 problem test spanning two days.

Sorry to miss ACL, but welcome to check out our work on using LLM to evaluate translation with Error Analysis prompt ( EA is one of the earliest work to incorporate human experience such as MQM into LM based NLG eval and was published at ACL’23 oral), feel free to talk with

Introducing *Transfusion* - a unified approach for training models that can generate both text and images. arxiv.org/pdf/2408.11039 Transfusion combines language modeling (next token prediction) with diffusion to train a single transformer over mixed-modality sequences. This