论文与标准

Training data-efficient image transformers & distillation through attention

分类标记和蒸馏标记共同经过编码器，两个输出头分别学习真实标签和教师预测。

Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, Hervé Jégou · 2021 · Proceedings of the 38th International Conference on Machine Learning, PMLR 139:10347-10357