Stephan Hoyer    @shoyer    11/24/2021      

Gradient checkpointing (aka rematerialization) is an easy trick that can save massive amounts of memory for calculating gradients. If you differentiate through computation involving long iterative processes (like ODE solving), learn it and make it part of your toolkit! 👇🧵
    44         346




Mark Saroufim    @marksaroufim    11/25/2021      

This README was the best visual explanation of gradient checkpointing I’ve seen Made it click for me. Hopefully someone else finds it useful too.

Rosanne Liu    @savvyRL    11/24/2021      

Hahaha my new earworm “these are gradients, gradients of your loss” Listen to the full song!
    1         7

ServiceNow    @servicenow    11/19/2021      

From 48 days to 48 hours Learn how @NadroOficial reduced customer registration time— and transformed its processes— through the #NowPlatform.
    3         11

Yann LeCun    @ylecun    11/23/2021      

People who think evolution works through random mutations and selection need to explain how intelligent life appeared using nothing else. Clearly, any optimization process is more efficient if it uses some sort of gradient estimation.
    1         24