For #GivingTuesday2021 I got an email from an elite private school with a $15B endowment and $58K tuition asking for donations alongside organizations that feed the poor.

To a theoretical computer scientist, $1 and $1,000,000 are equivalent because they are both O(1). This equivalence explains the career decisions that theoretical computer scientists have made.

You’re equivalent to a carpenter as long as the cost and time difference is bounded by a multiplicative constant.

I wish it was...Looks like a great study. I've done benchmarking of optimizers for their impacts on model robustness, but not for generalization, and definitely not at this scale! Phew...I'm happy that Adam is still a reasonable choice because I don't want to change my code 😬