Hierarchical document categorization with support vector machines

By Lijuan Cai, Thomas Hofmann. 2004

442 citations recorded by the source corpus.

Automatically categorizing documents into pre-defined topic hierarchies or taxonomies is a crucial step in knowledge and content management. Standard machine learning techniques like Support Vector Machines and related large margin methods have been successfully applied for this task, albeit the fact that they ignore the inter-class relationships. In this paper, we propose a novel hierarchical classification method that generalizes Support Vector Machine learning and that is based on discriminant functions that are structured in a way that mirrors the class hierarchy. Our method can work with arbitrary, not necessarily singly connected taxonomies and can deal with task-specific loss functions. All parameters are learned jointly by optimizing a common objective function corresponding to a regularized upper bound on the empirical loss. We present experimental results on the WIPO-alpha patent collection to show the competitiveness of our approach.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN (2018, background): “For example, some hierarchical cost-sensitive loss can be developed [5].”
Hierarchical Semi-supervised Classification with Incomplete Class Hierarchies (2016, background): “There has been a lot of research done in the area of supervised hierarchical classiﬁcation [6, 16, 24].”
On Taxonomies for Multi-class Image Categorization (2012, unknown): “This task fits well into the popular structured learning framework (Taskar et al. 2004; Tsochantaridis et al. 2005) which has recently seen many applications among them in particular document classification with taxonomies (Cai and Hofmann 2004).”
Multilabel classification via calibrated label ranking (2008, background): “(2006) consider the hierarchical multilabel learning scenario, a combination of hierarchical classification (Cai and Hofmann 2004) and multilabel learning.”
Multitask Learning without Label Correspondences (2010, methodology): “One instance as considered in [19] is to use a feature map φ(y) ∈ R for k nodes in the DAG (excluding the root node) and associate with every label y the vector describing the path from the root node to y, ignoring the root node itself.”
Domain-specific Knowledge Extraction from the Web of Data (2018, background): “Another body of related work is in the area of document/text classification into pre-defined topic hierarchies or taxonomies using machine learning techniques, such as [Cai and Hofmann 2004] and [Hao et al. 2007].”
Multi-instance multi-label learning (2008, background): “Most studies on multi-label learning focus on text categorization [22,33,39,47,56, 65, 74], and several studies aim to improve the performance of text categorization systems by exploiting additional information given by the hierarchical structure of classes [14, 15, 53] or unlabeled data [43].”
Recursive regularization for large-scale classification with hierarchical and graphical dependencies (2013, methodology): “The more principled methods include the large-margin models by [31],[6],[26],[9],[7] where the discriminant functions take the contributions from all nodes along the path to the root, and the model parameters are jointly learned to minimize a global loss over the hierarchy.”
Support vector machines classification with a very large-scale taxonomy (2005, methodology): “Hierarchical SVMs refer to those methods that decompose the training tasks according to the structure of the taxonomy [ 4 ][5][6][10][17][19].”
A scalable modular convex solver for regularized risk minimization (2007, background): “Let us discuss the following two cases: Ontologies for Structured Estimation: For hierarchical labels, e.g. whenever we deal with an ontology [7], we can use a decomposition of the coeﬃcient vector along the hierarchy of categories.”
Large Margin Methods for Structured and Interdependent Output Variables (2005, methodology): “In that respect our approach follows the work of Collins (2002) on perceptron learning with a similar class of discriminant functions.”
Hierarchical Bayesian Inference and Recursive Regularization for Large-Scale Classification (2015, methodology): “The more principled methods include the large-margin models by Tsochantaridis et al. [2006], Cai and Hofmann [2004], Rousu et al. [2006], Dekel et al. [2004], and to minimize a global loss over the hierarchy.”
Bayesian models for Large-scale Hierarchical Classification (2012, background): “[20, 6] proposed large-margin discriminative methods where the discriminant function at each node takes the contributions from all nodes along the path to the root node, and the parameters are jointly learned to minimize a global loss over the hierarchy.”
Weakly-Supervised Hierarchical Text Classification (2018, unknown): “(Cai and Hofmann 2004) define hierarchical loss function and apply cost-sensitive learning to generalize SVM learning for hierarchical classification.”
Kernel-Based Learning of Hierarchical Multilabel Classification Models (2006, methodology): “Cai and Hofmann (2004) defined a weighted version of the loss that can take into account factors such as subscription loads of nodes.”
Cross-Validation Optimization for Large Scale Structured Classification Kernel Methods (2008, methodology): “The results are quite similar to the ones of Cai and Hofmann (2004), obtained with a support vector machine variant.”
A Review on Multi-Label Learning Algorithms (2014, background): “Furthermore, there are alternative multi-label settings where other than a single instance each example is represented by a bag of instances [201], [202] or graphs [87], [89], or extra ontology knowledge might exist on the label space such as hierarchy structure [2], [4], [14], [17], [124], [125], [”