🔬 Introducing HPC-AI Lab NUS! We're dedicated to High-Performance Computing (HPC), ML Systems, and AI Applications, including LLM, CV, etc. Stay tuned for exciting discussions with leading scholars - everyone is welcome to join!

📢 Join us for the 1st HPC-AI Lab Public Seminar featuring Dr. Enze XIE from Huawei Noah’s Ark Lab! 🌟 🔗 Registration: forms.gle/G54J9VkvVPYgqg… 🗓️ Date/Time: 14 Jun. 2023 (Wednesday), 1 PM to 2 PM 📍 Venue: Online via Zoom Don't miss out on this exciting seminar!

📢 Join us for the HPC-AI Lab Public Seminar featuring IEEE Fellow Dr. Stan Z. Li! 🌟 🔗 Registration: forms.gle/5bcDLV6B7eX3SK… 🗓️ Date/Time: 27 Jun. 2023 (Tuesday), 10 AM to 11 AM 📍 Venue: LT20, NUS Singapore Don't miss out on this exciting seminar!

I am happy to share that our paper won the Outstanding Paper Award of ACL. We propose CAME to simultaneously achieve two goals: fast convergence as in traditional adaptive methods, and low memory usage as in LLM training. arxiv.org/abs/2307.02047

🎉 Exciting news! Our Lab has two papers accepted at ACL 2023! 📚✨ We're thrilled to announce that our CAME optimizer has won the Outstanding Paper award! 🏆 Congratulations to the entire team for their remarkable achievement! 🥳 #ACL2023

Our work DREAM has been accepted by ICCV-2023 #ICCV2025. We are the first to explore the matching efficiency in dataset distillation and speed up the previous works more than 8 times without performance drop. Check out our DREAM in Github: github.com/lyq312318224/D…

Excited to introduce our #ICCV2023 paper Dataset Quantization (DQ). DQ achieves lossless training performances with 2% data keep ratio on language tasks and 60% data keep ratio on vision tasks. Just check out our paper and project: openaccess.thecvf.com/content/ICCV20… github.com/magic-research…

Excited to share our #ICCV2023 paper: Fine-tuning Vision-Language Models without Zero-Shot Transfer Degradation (ZSCL). ZSCL outperforms the pre-trained model on downstream tasks and maintains its zero-shot transferability to other tasks. paper: openaccess.thecvf.com/content/ICCV20… blog:

Time flies! I got my PhD from Berkeley 1218 days ago. My first PhD student is graduating. That is my first achievement :-)

📢 Join us for the HPC-AI Lab Public Seminar! 🔗 Registration: forms.gle/4anywqoXtSumHu… 🗓️ Date/Time: 29 Nov. 2023 (Wednesday), 1 PM to 2 PM 📍 Online via Zoom

I am happy to share that our paper has been accepted by ICLR as an ORAL paper (1.2% acceptance rate). InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruning arxiv.org/abs/2303.04947 InfoBatch randomly prunes a portion of less informative samples based on the

InfoBatch is accepted as Oral to ICLR'24! 🔥 InfoBatch prunes data on the fly and speedups 20%~40% on img classification, semantic segmentation, MAE, Diffusion, LLM instruction tunning.🧵3 ArXiv: arxiv.org/abs/2307.02047 Blog: zhengzangw.github.io/blogs/infobatch Code: github.com/NUS-HPC-AI-Lab…

Neural Network Diffusion Diffusion models have achieved remarkable success in image and video generation. In this work, we demonstrate that diffusion models can also generate high-performing neural network parameters. Our approach is simple, utilizing an autoencoder and a

Exciting News from Open-Sora! 🚀 They've just made the ENTIRE suite of their video-generation model open source! Dive into the world of cutting-edge AI with access to model weights, comprehensive training source code, and detailed architecture insights. Start building your dream

Training-free Video Enhancement: Achieved 🎉 Nice work with Xuanlei Zhao Wenqi Shaw Victor.Kai Wang @VitaGroupUT Yang You et al. Non-trivial enhancement, training-free, and plug-and-play 🥳 Blog: oahzxl.github.io/Enhance_A_Vide… (🧵1/6)

Generating ~200 million parameters in just minutes! 🥳 Excited to share our work with Doven Tang , ZHAO WANGBO , and Yang You: 'Recurrent Diffusion for Large-Scale Parameter Generation' (RPG for short). Example: Obtain customized models using prompts (see below). (🧵1/8)

Congrats to our lab member Ziming Liu !

Customizing Your LLMs in seconds using prompts🥳! Excited to share our latest work with HPC-AI Lab, VITA Group, Konstantin Schürholt, Yang You, Michael Bronstein, Damian Borth : Drag-and-Drop LLMs(DnD). 2 features: tuning-free, comparable or even better than full-shot tuning.(🧵1/8)