BEiT: BERT Pre-Training of Image Transformers

Abstract

This paper introduces and evaluates the idea described in “BEiT: BERT Pre-Training of Image Transformers”, and reports empirical results that helped shape subsequent work in vision, transformers.