Asking the Difficult Questions: Goal-Oriented Visual Question Generation via Intermediate Rewards
By Junjie Zhang, Qi Wu, Chunhua Shen, Jian Zhang, Jianfeng Lu, Anton van den Hengel. 2017
29 citations recorded by the source corpus.
Despite significant progress in a variety of vision-and-language problems, developing a method capable of asking intelligent, goal-oriented questions about images is proven to be an inscrutable challenge. Towards this end, we propose a Deep Reinforcement Learning framework based on three new intermediate rewards, namely goal-achieved, progressive and informativeness that encourage the generation of succinct questions, which in turn uncover valuable information towards the overall goal. By directly optimizing for questions that work quickly towards fulfilling the overall goal, we avoid the tendency of existing methods to generate long series of insane queries that add little value. We evaluate our model on the GuessWhat?! dataset and show that the resulting questions can help a standard Guesser identify a specific object in an image at a much higher success rate.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
In Reinforcement Learning
Citations in the corpus
- Towards a Better Metric for Evaluating Question Generation Systems (2018, background): “Apart from generating questions from text there is also research on generating questions from images (Jain et al., 2017; Li et al., 2017; Zhang et al., 2017) and knowledge base (Serban et al., 2016; Reddy et al., 2017).”