Longformer: The Long-Document Transformer

Abstract

This paper introduces and evaluates the idea described in “Longformer: The Long-Document Transformer”, and reports empirical results that helped shape subsequent work in nlp, long-context.