Language modeling on raw bytes by Albert Gu A new version of hierarchical networks (H-Nets) where it finds 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute.