Brandon Rohrer (@_brohrer_)

Brandon Rohrer @_brohrer_

Data science. Machine learning. Algorithm and feature design. Currently iRobot, formerly Facebook, Microsoft, DuPont Pioneer, Sandia Labs, MIT. he/him

Tweets by Brandon Rohrer

Artisanal architectures, small batched and locally sourced

Shared by Brandon Rohrer at 5/14/2022

If you're headed into an ML interview 1) May God have mercy on your soul 2) Check out this super solid online book from @chipro, Introduction to Machine Learning Interviews https://t.co/NmHfDmWxpO

Shared by Brandon Rohrer at 5/13/2022

One page into Street Coder and @esesci pulls no punches: “There is a reason for every level of abstraction we now have, and it’s not that we are masochists, with the exception of Haskell programmers.” h/t @vboykis

Shared by Brandon Rohrer at 5/8/2022

Shoutout to my interviewers, all of whom were thoughtful and respectful. Any weaknesses of the interviewing process I experienced are cultural and systemic. The conversations I had were all very positive.

Shared by Brandon Rohrer at 4/12/2022

I interviewed for data scientist and machine learning engineer positions at both the staff and senior staff level. I even interviewed for a product manager role.

Shared by Brandon Rohrer at 4/10/2022

New SOTA for parameter efficiency on Fashion MNIST at 90% accuracy: 3,423 Solution uses 4-layer Sharpened Cosine Similarity network in a Mixer-style architecture. Also, there's a snazzy leaderboard to track it on. https://t.co/Vk3iNVndB2

Shared by Brandon Rohrer at 3/18/2022

🤯 ConvMixer takes parameter efficiency to new levels. 103K parameters to get CIFAR-10 to 91% 21.1M parameters to get ImageNet top-1 to 80% Amazing stuff from @ashertrockman and @zicokolter

Shared by Brandon Rohrer at 3/13/2022

100% this. <self promotion> I actually created some courses where you start with NumPy and build fully connected NNs, autoencoders, ConvNets, and a deep learning framework. As of a few months ago they’re all free. https://t.co/Jai38mVi1f </self promotion>

Shared by Brandon Rohrer at 3/13/2022

The latest contribution to the Sharpened Cosine Similarity project is a PyTorch Lightning demo from @PSodmann. It shows off the streamlined coding patterns @PyTorchLightnin is famous for. https://t.co/KZUuaOoAF1

Shared by Brandon Rohrer at 3/12/2022

For those of you following the rapidly unfolding development of Sharpened Cosine Similarity (SCS) as an alternative to convolution in deep learning architectures, here the latest advances. https://t.co/DR033Gr8zT

Shared by Brandon Rohrer at 2/20/2022

My favorite part of this is how dead simple the model is: SCS -> AbsPool -> SCS -> AbsPool -> SCS -> AbsPool -> Flatten -> Linear No batch norm, no ReLU, no dropout

Shared by Brandon Rohrer at 2/17/2022

Here's an image classification model that gets 18.4% error on CIFAR 10 with only 68k parameters using Sharpened Cosine Similarity. https://t.co/PgcclFZ8Q6

Shared by Brandon Rohrer at 2/16/2022

As the overlap increases, it's getting tougher to tell a machine learning engineer from a data scientist. Here's a simple test. Show them an improvement on a state-of-the-art algorithm. One will ask: Is it scalable? The other: Is it significant?

Shared by Brandon Rohrer at 2/16/2022

So which one of you is going to write the dataviz tool that applies neural style transfer to bar charts?

Shared by Brandon Rohrer at 2/15/2022

Want to see something cool? Sharpened Cosine Similarity (CosSim) is an alternative to Convolution for building features in neural networks. It doesn't need normalization, dropout, or activation functions and it performs as well as ConvNets with 10x-100x more parameters.

Shared by Brandon Rohrer at 1/30/2022

If you've always wondered how Transformers work but know nothing about machine learning, I wrote this peek behind the curtain for you. (Actually I wrote it for me, but you might find it useful.) Here's the beta release. https://t.co/VaFt8md9hJ

Shared by Brandon Rohrer at 11/23/2021

The state of the art in autonomous driving is Roomba. https://t.co/uBX0Z6XMqk

Shared by Brandon Rohrer at 11/14/2021

Next stop on the tour is embeddings. These are what save transformers from being impossibly unwieldy. https://t.co/NeqtbwO9xw

Shared by Brandon Rohrer at 11/11/2021

The selective-second-order-with-skips model is useful way to think about what the decoder part of transformers do. It captures what models like OpenAI's GPT-3 and GitHub's Copilot are doing. It doesn't tell the complete story, but it represents the central thrust of it.

Shared by Brandon Rohrer at 11/6/2021

Movie trailer voice: In a post-backpropagation world neural network training algorithms start with all connection weights at zero and only a sparse few escape. This is their story.

Shared by Brandon Rohrer at 11/5/2021

Now here's the section on second order Markov models, which consider the two most recent words in the sequence instead of just one. https://t.co/B5oMcVba2V

Shared by Brandon Rohrer at 11/4/2021

The human labeler is a bottleneck putting ML into practice. @svlevine offers some ideas for how to skirt them in reinforcement learning.

Shared by Brandon Rohrer at 10/26/2021

The analog to the Turing complete computer language is the drawing complete plotting package. With enough effort, it can control every pixel on the screen. @matplotlib is one of these. For example, here’s how you can draw a fish. Step one, sketch a fish on paper.

Shared by Brandon Rohrer at 10/6/2021

Don’t have time to read the entirety of algorithmic fairness Twitter? Not to worry! @KLdivergence boiled down the whole of it to a single thread here, just for you.

Shared by Brandon Rohrer at 10/5/2021

I’m never again going to be able to keep a straight face while reading economic analysis based on the assumption of a rational agent.

Shared by Brandon Rohrer at 10/1/2021

Keeping track of how three dimensional arrays change as your code runs is harder. You can start by rotating them a little bit. https://t.co/nJ05RfdXgc

Shared by Brandon Rohrer at 9/28/2021

93% of machine learning research is predicated on the assumption that benchmark data set labels are accurate - that they represent ground truth. It turns out that's not the case.

Shared by Brandon Rohrer at 9/5/2021

At the office: This meeting could have been an email. In production: This deep neural network could have been k-nearest neighbors.

Shared by Brandon Rohrer at 8/31/2021

An approachable guide to building ML models for research. Covers the five stages of of the machine learning process which, I was surprised to discover, are not denial, anger, bargaining, depression, acceptance. https://t.co/XImgM90f5Z h/t @hardmaru, @vboykis

Shared by Brandon Rohrer at 8/8/2021

Feeling bored with machine learning? Here's how to bring back the magic. 1. Choose a problem family. Bonus points if it's something other than image classification or next-word prediction. 2. Choose an application area. Bonus points if it's something other than e-commerce.

Shared by Brandon Rohrer at 5/8/2021