Sheffield MultiMT: Using Object Posterior Predictions for Multimodal Machine Translation
By Pranava Swaroop Madhyastha, Josiah Wang, Lucia Specia. 2017
17 citations recorded by the source corpus.
This paper describes the University of Sheffield’s submission to the WMT17 Multimodal Machine Translation shared task. We participated in Task 1 to develop an MT system to translate an image description from English to German and French, given its corresponding image. Our proposed systems are based on the state-of-the-art Neural Machine Translation approach. We investigate the effect of replacing the commonly-used image embeddings with an estimated posterior probability prediction for 1,000 object categories in the images.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Machine Translation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Pranava Swaroop Madhyastha
- Loïc Barrault
- Lucia Specia
- Ozan Caglayan
- Desmond Elliott
- Samuel R. Bowman
- Mohit Bansal
- Kyunghyun Cho
In Computer Vision and Pattern Recognition
- Pranava Swaroop Madhyastha
- Loïc Barrault
- Ozan Caglayan
- Lucia Specia
- Desmond Elliott
- Mohit Bansal
- Ákos Kádár
- Kyunghyun Cho
In Multimodal Learning
- Pranava Swaroop Madhyastha
- Loïc Barrault
- Ozan Caglayan
- Lucia Specia
- Desmond Elliott
- Florian Metze
- Ákos Kádár
- Ramon Sanabria
In Machine Translation
- Lucia Specia
- Loïc Barrault
- Pranava Swaroop Madhyastha
- Ozan Caglayan
- Desmond Elliott
- Kashif Shah
- Trevor Cohn
- Daniel Beck
Citations in the corpus
- Probing the Need for Visual Context in Multimodal Machine Translation (2019, background): “, 2018) (ii) cross-modal interactions with spatially-unaware global features (Calixto and Liu, 2017; Ma et al., 2017; Caglayan et al., 2017a; Madhyastha et al., 2017) and (iii) the integration of regional features from object detection networks (Huang et al.”