ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Abstract

This paper introduces and evaluates the idea described in “ALBERT: A Lite BERT for Self-supervised Learning of Language Representations”, and reports empirical results that helped shape subsequent work in nlp, efficiency.