Breaking the Closed World Assumption in Text Classification

By Geli Fei, B. Liu. 2016

139 citations recorded by the source corpus.

Existing research on multiclass text classific a-tion mostly makes the closed world assum p-tion, which focus es on designing accurate classifiers under the assumption that all test classes are known at training time. A more r e-alistic scenario is to expect unseen classes dur ing testing ( open world ) . In this case, the goal is to design a learning system that class i-fies documents of the known classes into their respective cl asses and also to reject doc u-ments from unknown classes . This problem is called open ( world ) classification . This paper approach es the problem by reducing the open space risk while balancing the empirical risk . It proposes to use a new learn ing strategy, called c enter - based similarity (CBS) space learning (or CBS learning ) , to provide a novel solution to the problem . Extensive e xper i-ments across two datasets show that CBS learning give s promising results on multiclass open text classification compared to state - of - the - art baselines.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bing Liu
2. Hu Xu
3. Lei Shu
4. B. Liu
5. Zixuan Ke
6. Sahisnu Mazumder
7. Eric Robertson
8. Shuai Wang

In Information Systems

1. B. Liu
2. Bing Liu
3. Hu Xu
4. Lei Shu
5. Geli Fei
6. Shuai Wang
7. Hua Xu
8. Ting-En Lin

Citations in the corpus

• A Two-phase Prototypical Network Model for Incremental Few-shot Relation Classification (2020, background): “They are always based on the closed-world assumption (Fei and Liu, 2016) that the relation expressed in the query instances must have appeared in the pre-defined relation set.”
• DOC: Deep Open Classification of Text Documents (2017, methodology): “We perform evaluation using two publicly available datasets, which are exactly the same datasets used in (Fei and Liu, 2016).”
• Learning on the Job: Online Lifelong and Continual Learning (2020, background): “Several researchers have studied this problem, e.g., Scheirer et al. (2013), Bendale and Boult, 2015; Fei and Liu (2016), De Rosa et al (2016), and Xu et al. (2019).”
• Learning Cumulatively to Become More Knowledgeable (2016, methodology): “Given open space formulations of the traditional SVM and 1-vs-Set Machine, we can see that both methods label an unlimited area as positively labeled space, while the formulation in [14] reduces it to a bounded spherical area.”
• Recent Advances in Open Set Recognition: A Survey (2018, methodology): “We refer the reader to [68]–[70] for more details.”
• Unknown Intent Detection Using Gaussian Mixture Model with an Application to Zero-shot Intent Classification (2020, methodology): “While there have been some pioneering works studying the open-world classification problem in natural language processing (Fei and Liu, 2016; Shu et al., 2017), very few methods are designed for unknown intent detection.”
• Classification-Reconstruction Learning for Open-Set Recognition (2018, background): “Center-based similarity space models [7] represent data by their similarity to class centroids in order to tighten the distributions of positive data.”
• Self-Initiated Open World Learning for Autonomous AI Agents (2021, background): “Classic machine learning (ML) makes closed world assumption, which means that what are seen by the system in testing or deployment must have been seen in training (Fei and Liu 2016; Liu 2020), i.”
• Deep Unknown Intent Detection with Margin Loss (2019, methodology): “We follow the validation setting in (Fei and Liu, 2016; Shu et al., 2017) by keeping some classes in training as unknown and integrate them back during testing.”
• Unseen Class Discovery in Open-world Classification (2018, methodology): “Since many existing methods exist for (1) and (3) (Scheirer et al., 2013; 2014; Bendale & Boult, 2015; Fei & Liu, 2016; Bendale & Boult, 2016), we will not study them in this paper.”
• Towards Knowledgeable Supervised Lifelong Learning Systems (2020, methodology): “The approach implements the concept of open-world classification (Fei & Liu, 2016), a paradigm that relies on maintaining an unknown category along with existing categories or classes.”
• HRN: A Holistic Approach to One Class Learning (2020, methodology): “On broader impact, we believe that our holistic one-class learning can help positive and unlabeled (PU) learning [52], open-world learning (or out-of-distribution detection ) [23], and continual learning [11, 63] as all these learning paradigms need to face unseen/novel situations.”
• Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP (2021, methodology): “The earlier methods are mainly based on SVMs (Scheirer et al. 2012; Scheirer, Jain, and Boult 2014; Fei and Liu 2016).”