New optimization paper alert! "Directional Smoothness and Gradient Methods: Convergence and Adaptivity". We develop new sub-optimality bounds for gradient descent that depend on the local smoothness (curvature) along the optimization path.

Btw this large scary math symbol is just code

Good tip for paper writing: just do a basic keyword search on Google scholar. 9 times out of 10 I find stuff I wasn't aware of before starting a project.

Honestly some of my best writing is for double blind reviews that in all likelihood, only a single person will ever read.

Whether or not arxiv should allow this (nonsense) - what's the point of this document? Nobody reading it learns anything.

Was mentally unprepared for the reveal here, and honestly still reeling.

I'll be at ICML, mainly to present at workshops: Fine-tuning LLMs with user-level differential privacy (arxiv.org/abs/2407.07737) - TF2M workshop Scalable Map reduce algorithms in JAX (arxiv.org/abs/2403.07128) - WANT workshop Come say hi!

Adding to this - if anyone at ICML Conference want to chat, even if we've never met before, just ping me.

Incredible - so well-deserved Steve.

I am at ICML 2024 (ICML Conference) in Vienna. Come to our poster #1209 (today, 1:30pm-3pm, Hall C 4-9) on theory for Independent Subnetwork Training (IST). Led by my PhD student Egor Shulgin (Egor Shulgin); who is also here this week!

Great talk from Zach Mueller at the WANT@ICML workshop on designing libraries for model training/inference. My favorite part - judging usability by external contributions, both number and speed with which they are developed.

Probably my favorite talk to give to date on Hugging Face Accelerate: an honest conversation on designing frameworks, why you should avoid “magical code”, how higher level abstractions in the long-term might not be beneficial, and how Accelerate can be a teaching library on how

I believe in a future... when people will stop using images of conventionally hot chicks to show how good their AI models are 🙏🏼

Incredibly cool. I wish there had been more submissions to the competition overall (both tracks) but I think the AlgoPerf competition is one of the best things to happen to optimization research in years.

Review like everyone is watching.

Speaking as someone who is ostensibly on vacation this week - the sub-week timelines for this stuff drives me up the wall. I have authors trying to rewrite their paper en masse in their response. Why is this a reasonable system?

Raygun is an inspiration and I mean this unironically.

BREAKING! @FrancescaGino's ridiculous defamation lawsuit against Data Colada from last year has been dismissed by the court! This is an important victory.. if this had been allowed to move forward it would have had a chilling effect on whistleblowers! storage.courtlistener.com/recap/gov.usco…