State-of-the-art CIFAR-10 results for kernel methods! Now on arXiv:Convolutional Deep Kernel Machines. Link here: arxiv.org/abs/2309.09814. Read on for the abridged version. (1/10)

New preprint out on geographic location encoding using spherical harmonics and sinusoidal representations! 🌐➡️🌏 Work led by Marc Rußwurm and w/ Esther Rolf, Robin Zbinden and devistuia. 📄Paper: arxiv.org/abs/2310.06743 💻Code: github.com/marccoru/locat… 🧵Thread: 👇 1/11

A highlight of the #NeurIPS2023 meetup in Cambridge for me was Carl Rasmussen's keynote on "Halting Climate Change". Spoiler: it has nothing to do with AI. Worth a watch: youtube.com/watch?v=naFaQs…

The first "Research Focus" blog post of 2024 from Microsoft Research highlights papers from couple of internship projects that I co-supervised with Siân Lindley and James Hensman last summer. Blog post: microsoft.com/en-us/research…

Internship opportunity: come and work with me on the future of knowledge bases and efficient language modelling. jobs.careers.microsoft.com/global/en/job/…

We're hiring!🚀 The Alexandria team (microsoft.com/en-us/research…) at Microsoft Research (Cambridge, UK) is looking for summer #interns. Come work with us on KB-powered LLMs, generative knowledge linking, transformer compression, & other exciting problems More info: jobs.careers.microsoft.com/global/en/job/…

The SIAM community mourns the passing of former SIAM President Nick Higham. We extend our condolences to his family, colleagues, and friends. He will be sorely missed.

It was such a pleasure to work with Saleh Ashkboos and @max_croci on this method for LLM compression during Saleh's internship last year.

Come and work with us to make Language Models more efficient. A Research Residency at Microsoft Research and 365 Research in Cambridge, UK. jobs.careers.microsoft.com/global/en/job/…

[1/7] Happy to release 🥕QuaRot, a post-training quantization scheme that enables 4-bit inference of LLMs by removing the outlier features. With Amirkeivan Mohtashami @max_croci Dan Alistarh Torsten Hoefler 🇨🇭 James Hensman and others Paper: arxiv.org/abs/2404.00456 Code: github.com/spcl/QuaRot

Saleh Ashkboos Amirkeivan Mohtashami @max_croci Dan Alistarh Torsten Hoefler 🇨🇭 James Hensman Awesome project. Just did an installation video of QuaRot locally. Thanks for the great project. youtu.be/r7FvhB1WgI8?si…

We are hiring a senior researcher in ML for healthcare at MSR Cambridge (UK)! The position is in my team, so if you get it you will work with me (is this a pro or a con? do not answer). Focus is multimodal (~vision-language) models for radiology! Link: jobs.careers.microsoft.com/global/en/job/…

One week left to apply for a Health Data Science Post Doc with me as part of the @ai4cihub! bath.ac.uk/jobs/Vacancy.a… There is lots of flexibility in methodology you could pursue and an unbeatable team of collaborators!

Our team is growing. Come and join us as part of the MSR AI Residency program to work on efficiency in deep learning. jobs.careers.microsoft.com/global/en/shar…

🔵New paper!🔵 Our latest work on Pyramid Vector Quantization for LLMs achieves state-of-the-art post-training quantization with a Pareto-optimal trade-off between performance, bits per weight, and bits per activation. A thread. 👇 1/15

Ever thought about using matrix-exp's or log-det's in large-scale ML (think PDEs/GGN matrices with >1M rows)? I have, and maybe you have, too—but gradients? That's where it gets tricky. Not anymore! We have a #NeurIPS2024 spotlight: "Gradients of Functions of Large Matrices."🧵

We're recruiting interns for summer '25. Come and work with me and the team on efficiency in AI systems. jobs.careers.microsoft.com/global/en/job/…

Excited to share our new work, "TurboAttention", a comprehensive approach for efficient quantized attention in LLMs! We introduce FlashQ, a headwise progressive quantization technique for both KV cache compression and quantized attention execution, and Sparsity-based Softmax

Today we bring the latest #DeepSeek distilled models to #Copilot+ PC’s, where they really shine due to the devices’ efficiency and power. This performance is unprecedented. It’s so enabling to put this class of #AI reasoning running on your Copilot+ PC! blogs.windows.com/windowsdevelop…