论文与标准

Scaling Laws for Neural Language Models

在受控实验中研究语言模型损失随参数量、数据量和训练计算变化的经验幂律。

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, Dario Amodei · 2020