Katie Everett (@_katieeverett) 's Twitter Profile
Katie Everett

@_katieeverett

Machine learning researcher at @GoogleDeepMind (via Brain) + PhD student @MIT. Previously @chorus cofounder, @twitter, @MIT.

ID: 1666204921

calendar_today12-08-2013 22:38:09

51 Tweet

554 Followers

548 Following

Damien Ferbach (@damien_ferbach) 's Twitter Profile Photo
Damien Ferbach
@damien_ferbach

3 months ago

It's very difficult to improve the *exponent* in scaling laws for loss vs compute, especially by changing the optimizer! Our new paper shows that scaling momentum correctly can *provably* improve the scaling exponent on a theoretical model. Empirically, it works on LSTMs too!

It's very difficult to improve the *exponent* in scaling laws for loss vs compute, especially by changing the optimizer! Our new paper shows that scaling momentum correctly can *provably* improve the scaling exponent on a theoretical model. Empirically, it works on LSTMs too!