AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification

By Ronghui You, Ziye Wang, Suyang Dai, Hiroshi Mamitsuka, Shanfeng Zhu. Neural Information Processing Systems, 2018

303 citations recorded by the source corpus.

Extreme multi-label text classification (XMTC) is an important problem in the era of {\it big data}, for tagging a given text with the most relevant multiple labels from an extremely large-scale label set. XMTC can be found in many applications, such as item categorization, web page tagging, and news annotation. Traditionally most methods used bag-of-words (BOW) as inputs, ignoring word context as well as deep semantic information. Recent attempts to overcome the problems of BOW by deep learning still suffer from 1) failing to capture the important subtext for each label and 2) lack of scalability against the huge number of labels. We propose a new label tree-based deep learning model for XMTC, called AttentionXML, with two unique features: 1) a multi-label attention mechanism with raw text as input, which allows to capture the most relevant part of text to each label; and 2) a shallow and wide probabilistic label tree (PLT), which allows to handle millions of labels, especially for "tail labels". We empirically compared the performance of AttentionXML with those of eight state-of-the-art methods over six benchmark datasets, including Amazon-3M with around 3 million labels. AttentionXML outperformed all competing methods under all experimental settings. Experimental results also show that AttentionXML achieved the best performance against tail labels among label tree-based methods. The code and datasets are available at \url{http://github.com/yourh/AttentionXML} .

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/5249b572fb4c3deab2c14a47336e29dc00b140e9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A Survey on Text Classification: From Shallow to Deep Learning (2020, unknown): “…[74] HL, M icro − F 1 RCV1-V2, AAPD deep contextualized representation [66] Accuracy, F1 SQuAD, SNLI, SRL, SST-5, etc. a label tree-based model [131] P@k, N@k, PSP@k EUR-Lex, Amazon-670K, etc. knowledge powered attention [132] Accuracy Weibo, Product Review, etc. bi-directional block…”
A Survey on Text Classification: From Traditional to Deep Learning (2020, unknown): “The SICK contains almost 10,000 English sentence pairs.”
The Emerging Trends of Multi-Label Learning (2020, background): “developed for the Challenging MLC problems, such as Extreme MLC [31], [31], [32], [33], partial and weakly-supervised MLC [19], [23], [23], [34], MLC with unseen labels [35], [36], [36].”
Attention mechanism in neural networks: where it comes and where it goes (2022, methodology): “AttentionXML is a tree-based model for extreme multi-label text classification [166].”
Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering (2021, unknown): “The Fudan University (“ dmiip fdu ”) team also relied on their previous “ AttentionXML ” [1], “ DeepMeSH ” [30], and “ BERTMeSH ” models [46].”