Distilling the Knowledge in a Neural Network

Abstract

This paper introduces and evaluates the idea described in “Distilling the Knowledge in a Neural Network”, and reports empirical results that helped shape subsequent work in distillation, compression.