Towards a Universal Continuous Knowledge Base

By Gang Chen, Maosong Sun, Yang Liu. 2020

3 citations recorded by the source corpus.

In artificial intelligence, knowledge is the information required by an intelligent system to accomplish tasks. While traditional knowledge bases use discrete, symbolic representations, detecting knowledge encoded in the continuous representations learned from data has received increasing attention recently. In this work, we propose a method for building a continuous knowledge base that can store knowledge imported from multiple, diverse neural networks. The key idea of our approach is to define an interface for each neural network and cast knowledge transferring as a function simulation problem. Preliminary experiments on text classification show promising results: we first import the knowledge encoded in an RNN model and a CNN model to the knowledge base, from which the fused knowledge is exported back to the RNN model, achieving a higher classification accuracy than the original RNN model. With the continuous knowledge base, it is also easy to achieve knowledge distillation and transfer learning. Our work opens the door to building a universal continuous knowledge base to collect, store, and organize all continuous knowledge encoded in different neural networks trained for different AI tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhiyuan Liu
2. Minlie Huang
3. Maosong Sun
4. Jie Tang
5. Xu Han
6. Xipeng Qiu
7. Ji-rong Wen
8. Wayne Xin Zhao

In Computer Vision and Pattern Recognition

1. Zhiyuan Liu
2. Ji-rong Wen
3. Yang Liu
4. Xu Han
5. Wayne Xin Zhao
6. Minlie Huang
7. Jie Tang
8. Xiao Liu

Citations in the corpus

• Pre-Trained Models: Past, Present and Future (2021, methodology): “To learn the fine-grained alignment between image regions and words, UNITER further proposes a word-region alignment task in the way of Optimal Transport (Chen et al., 2020c), which first finds a sparse matching between image regions and words, and then minimizes the alignment distance.”