✨A Spark of Vision-Language Intelligence! We introduce DnD-Transformer, a new auto-regressive image gen model beats GPT/Llama w/o extra cost. AR gen beats diffusion in joint VL modeling in a self-supervised way! Github: github.com/chenllliang/Dn… Paper: huggingface.co/papers/2410.01…

🤔How much potential do LLMs have for self-acceleration through layer sparsity? 🚀 🚨 Excited to share our latest work: SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration. Arxiv: arxiv.org/abs/2410.06916 🧵1/n

🤔 Are LLMs Ready for Real-World Data Science Challenges? 🚀 We’ve just open-sourced our #EMNLP2024 work DA-Code, a cutting-edge benchmark designed to push LLMs to their limits in real-world data science tasks. Get involved and challenge your models! da-code-bench.github.io

(Perhaps a bit late) Excited to announce our survey on ICL has been accepted to #EMNLP2024 main conf and been cited 1,000+ times! Thanks to all collaborators and contributors to this field! We've updated the survey arxiv.org/abs/2301.00234. Excited to keep pushing boundaries!

How to deploy a 100B model on your CPU devices? 🔥 Excited to introduce bitnet.cpp, our inference framework for BitNet b1.58 🚀🚀 github.com/microsoft/bitn…

About to arrive in #Miami 🌴 after a 30-hour flight for #EMNLP2024! Excited to see new and old friends :) I’d love to chat about data synthesis and deep reasoning for LLMs (or anything else) —feel free to reach out!

🎆Survey of the Year: 𝐍𝐞𝐱𝐭 𝐓𝐨𝐤𝐞𝐧 𝐏𝐫𝐞𝐝𝐢𝐜𝐭𝐢𝐨𝐧 𝐓𝐨𝐰𝐚𝐫𝐝𝐬 𝐌𝐮𝐥𝐭𝐢𝐦𝐨𝐝𝐚𝐥 𝐈𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐜𝐞: 𝐀 𝐂𝐨𝐦𝐩𝐫𝐞𝐡𝐞𝐧𝐬𝐢𝐯𝐞 𝐒𝐮𝐫𝐯𝐞𝐲 arXiv: arxiv.org/abs/2412.18619 HugFace: huggingface.co/papers/2412.18… Github: github.com/LMM101/Awesome…

Proud to introduce our latest work “Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey” as our new year gift for the multimodal learning community! Paper: huggingface.co/papers/2412.18… Github: github.com/LMM101/Awesome…

Excited to introduce BitNet b1.58 2B4T — the first large-scale, native 1-bit LLM🚀🚀 BitNet achieves performance on par with leading full-precision LLMs — and it’s blazingly fast⚡️⚡️uses much lower memory🎉 Everything is open-sourced — run it on GPU or your Macbook 🖥️⚙️

So happy to reunite with old and new friends at ICLR! Had an amazing time exploring Singapore too! 🌟🇸🇬 #ICLR2025

Thanks to elvis for sharing our work!

All my labmates and I from the PKU Computational Linguistics Lab will be in #Suzhou for #EMNLP2025 (Nov 3–9) ! Looking forward to meeting old and new friends. Always happy to grab a coffee and chat 🥰

🚀 We propose Generative Adversarial Distillation (GAD) 🤖 Designed to perform on-policy distillation from proprietary black-box LLMs. ➡️ Requires neither access to teacher logits nor alignment of tokenizer vocabularies. (1/n)