Ferenc Huszár: As part of DeepMind's mission to sol...| Robocrunch AI

As part of DeepMind's mission to solve intelliegence, we halved the learning rate every 2,500 optimization steps.
— Ferenc Huszár (@fhuszar) April 29, 2022

Fantastic and accessible read from @cHHillee about why ML efficiency is hard to encapsulate with a single objective. "you can understand efficient of your deep learning regime as consisting of 3 different components..." Compute, Memory and Everything else

Shared by Sara Hooker at 3/16/2022

Implicit Rank-Minimizing Autoencoder (PyTorch implementation of IRMAE) https://t.co/SPlib09ANw #deeplearning #machinelearning #ml #ai #neuralnetworks #datascience #PyTorch

Shared by PyTorch Best Practices at 11/28/2021

Shrinking parameter count while maintaining quality is also exciting for reducing both training and inference costs for powerful models. S/O to @borgeaud_s, @arthurmensch, @jordanhoffmann, @drjwrae, @erich_elsen, @laurentsifre, and the rest of the authors at DeepMind! (15/15)

Shared by MosaicML at 3/2/2022

learned_optimization: Meta-learning optimizers and more with JAX https://t.co/fRG0Y5t6Em

Shared by Aran Komatsuzaki at 3/23/2022

Like standard deep learning automatically learns representations of the data, DL 2.0 also jointly learns the architectures, optimizers, SSL pipelines, etc to find good tradeoffs of the specified user objectives. Thanks to my amazing team for their help in this journey! 3/3

Shared by Frank Hutter at 3/17/2022

There's nothing sacred about the math behind deep learning. Standard benchmarks combine hacks and tricks we barely understand to reach the tiny space of neural nets that *actually* work. At @MosaicML, we're changing that math to get to the same performance but faster and cheaper.

Shared by Jonathan Frankle at 3/16/2022

768