The biggest take-away for me from this work was that training neural nets with SGD does something akin to implicit model selection. The interesting thing is that, for this to work, the order of the data _really_ matters, unlike in exact Bayesian inference.