Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solution

By Ron Zhu. 2020

97 citations recorded by the source corpus.

Hateful meme detection is a new research area recently brought out that requires both visual, linguistic understanding of the meme and some background knowledge to performing well on the task. This technical report summarises the first place solution of the Hateful Meme Detection Challenge 2020, which extending state-of-the-art visual-linguistic transformers to tackle this problem. At the end of the report, we also point out the shortcomings and possible directions for improving the current methodology.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Social Psychology, Multimodal Learning, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tejas Srinivasan
2. Shrikanth S. Narayanan
3. Georgios Chochlakis
4. Jesse Thomason

In Computer Vision and Pattern Recognition

1. Tejas Srinivasan
2. Shrikanth S. Narayanan
3. Georgios Chochlakis
4. Jesse Thomason

In Social Psychology

1. Tejas Srinivasan
2. Shrikanth S. Narayanan
3. Georgios Chochlakis
4. Jesse Thomason

In Multimodal Learning

1. Tejas Srinivasan
2. Shrikanth S. Narayanan
3. Georgios Chochlakis
4. Jesse Thomason

In AI Ethics, Fairness & Society

1. Tejas Srinivasan
2. Shrikanth S. Narayanan
3. Georgios Chochlakis
4. Jesse Thomason

Citations in the corpus

• VAuLT: Augmenting the Vision-and-Language Transformer with the Propagation of Deep Language Representations (2022, methodology): “For modifications of multimodal transformers, we can look at VL-BERT+ (Zhu 2020), where data such as race and gender extracted from the images as well as web-recommended tags are fed to VL-BERT.”