Challenges and Prospects in Vision and Language Research
By Kushal Kafle, Robik Shrestha, Christopher Kanan. 2019
43 citations recorded by the source corpus.
Language grounded image understanding tasks have often been proposed as a method for evaluating progress in artificial intelligence. Ideally, these tasks should test a plethora of capabilities that integrate computer vision, reasoning, and natural language understanding. However, the datasets and evaluation procedures used in these tasks are replete with flaws which allows the vision and language (V&L) algorithms to achieve a good performance without a robust understanding of vision and language. We argue for this position based on several recent studies in V&L literature and our own observations of dataset bias, robustness, and spurious correlations. Finally, we propose that several of these challenges can be mitigated by creation of carefully designed benchmarks.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Christopher Kanan
- Robik Shrestha
- Kushal Kafle
- Scott D. Cohen
- Brian L. Price
- Dietrich Klakow
- Aditya Mogadala
- Marimuthu Kalimuthu
In Computer Vision and Pattern Recognition
- Christopher Kanan
- Brian L. Price
- Scott D. Cohen
- Robik Shrestha
- Kushal Kafle
- Dietrich Klakow
- Marimuthu Kalimuthu
- Aditya Mogadala
Citations in the corpus
- Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods (2019, background): “Also, there is an interest to understand limitations (Kafle et al., 2019) in integration of vision and language research.”
- Answering Questions about Data Visualizations using Efficient Bimodal Fusion (2019, background): “Like other VQA tasks [19], our results suggest harder datasets are needed.”