Adjusting BERT’s Pooling Layer for Large-Scale Multi-Label Text Classification (original) (raw)

Text, Speech, and Dialogue, 2020

Abstract

In this paper, we present our experiments with BERT models in the task of Large-scale Multi-label Text Classification (LMTC). In the LMTC task, each text document can have multiple class labels, while the total number of classes is in the order of thousands. We propose a pooling layer architecture on top of BERT models, which improves the quality of classification by using information from the standard [CLS] token in combination with pooled sequence output. We demonstrate the improvements on Wikipedia datasets in three different languages using public pre-trained BERT models.

Pavel Ircing hasn't uploaded this paper.

Let Pavel know you want this paper to be uploaded.

Ask for this paper to be uploaded.