Memory, Show the Way: Memory Based Few Shot Word Representation Learning
By Jingyuan Sun, Shaonan Wang, Chengqing Zong. 2018
16 citations recorded by the source corpus.
Distributional semantic models (DSMs) generally require sufficient examples for a word to learn a high quality representation. This is in stark contrast with human who can guess the meaning of a word from one or a few referents only. In this paper, we propose Mem2Vec, a memory based embedding learning method capable of acquiring high quality word representations from fairly limited context. Our method directly adapts the representations produced by a DSM with a longterm memory to guide its guess of a novel word. Based on a pre-trained embedding space, the proposed method delivers impressive performance on two challenging few-shot word similarity tasks. Embeddings learned with our method also lead to considerable improvements over strong baselines on NER and sentiment classification.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jiajun Zhang
- Chengqing Zong
- James T. Kwok
- Lionel Ming-shuan Ni
- Yaqing Wang
- Jingyuan Sun
- Shaonan Wang
- Quanming Yao
Citations in the corpus
- Distill and Replay for Continual Language Learning (2020, background): “For example, a network fitted on a first task tends to forget how to perform on it after trained on a sequential new task (Sun et al., 2018; Shin et al., 2017).”
- Generalizing from a Few Examples (2019, methodology): “Example applications include parsing [64], translation [65], sentence completion (which fills in the blanks using a word chosen from a provided set) [97, 138], sentiment classification from short reviews [150, 157], user intent classification for dialog systems [157], criminal charge prediction [61]”