Nonparametric Method for Data-driven Image Captioning
By Rebecca Mason, Eugene Charniak. 2014
105 citations recorded by the source corpus.
We present a nonparametric density estimation technique for image caption generation. Data-driven matching methods have shown to be effective for a variety of complex problems in Computer Vision. These methods reduce an inference problem for an unknown image to finding an existing labeled image which is semantically similar. However, related approaches for image caption generation (Ordonez et al., 2011; Kuznetsova et al., 2012) are hampered by noisy estimations of visual content and poor alignment between images and human-written captions. Our work addresses this challenge by estimating a word frequency representation of the visual content of a query image. This allows us to cast caption generation as an extractive summarization problem. Our model strongly outperforms two state-ofthe-art caption extraction systems according to human judgments of caption relevance.
Subfields: Computer Vision and Pattern Recognition, Natural Language Generation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Trevor Darrell
- Dacheng Tao
- Xiaogang Wang
- Jitendra Malik
- Alan L. Yuille
- Ming-Hsuan Yang
- Cordelia Schmid
In Natural Language Generation
- Saurabh Gupta
- C. Lawrence Zitnick
- Ramakrishna Vedantam
- Vicente Ordonez
- Devi Parikh
- Li Fei-Fei
- Qi Wu
- Arnau Ramisa
Citations in the corpus
- Microsoft COCO Captions: Data Collection and Evaluation Server (2015, background): “The automatic generation of captions for images is a long standing and challenging problem in artificial intelligence [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19].”
- Text2Scene: Generating Compositional Scenes From Textual Descriptions (2018, background): “Most research on visually descriptive language has focused on generating captions from images [5, 21, 16, 14, 31, 32, 22, 2].”
- Obj2Text: Generating Visually Descriptive Language from Object Layouts (2017, background): “Our work is also related to other language generation tasks such as image and video captioning (Farhadi et al., 2010; Ordonez et al., 2011; Mason and Charniak, 2014; Ordonez et al., 2015; Xu et al., 2015; Donahue et al., 2015; Mao et al., 2015; Fang et al., 2015).”