David Pfau: The data on which the big transforme...| Robocrunch AI

The data on which the big transformer was trained was generated from independently trained agents, was it not? There is no actual RL being done to train the final agent, it's all amortizing other agents, and each of the RL-trained agents was trained on a single environment.
— David Pfau (@pfau) May 13, 2022

I dunno why you’re focused on the RL framing though? No one said it was trained with RL. It’s a multitask/multimodal agent trained w supervised learning and some folks presumably didn’t expect that to be “easy” by just putting it all in serial form

Shared by Miles Brundage at 5/13/2022

Unsupervised skill learning allows RL agents to learn useful behaviors in the absence of task-specific rewards. But beneath its promising facade lies a dark secret… a challenging exploration problem! Check out our #iclr2022 Spotlight to learn more: https://t.co/l6VS7h55Pl. 🧵⬇️

Shared by DJ Strouse at 3/17/2022

I'm excited about large general agents but I don't quite understand this paper. Surely you can fit many experts into a transformer with BC. The difficulty is to then learn new tasks faster. But 1000 expert demos to swing up a cart pole is worse than training PPO from scratch?

Shared by Danijar Hafner at 5/12/2022

This feels remarkable! Transformers are 🔥 It is not clear if posing the RL problem as Sequence prediction is true to the original problem, but the breadth of results is noteworthy! In transformers, we trust!

Shared by Animesh Garg at 5/12/2022

LMs can learn via inference alone through demonstrations -- but how does it work? We find that LMs do not really need correct input-output pairs. Randomly replacing labels in the demonstrations barely hurts performance, consistently over 12 models. https://t.co/O3T7TtHxr2

Shared by Sewon Min at 2/28/2022

It's hard to compare unsupervised RL algorithms. That is why researchers @UCBerkeley and @nyuniversity proposed the Unsupervised Reinforcement Learning Benchmark. And they open-source code for eight leading unsupervised RL methods: https://t.co/7W1W6mTHi5

Shared by TheSequence at 2/25/2022

768