MMQA: A Multi-domain Multi-lingual Question-Answering Framework for English and Hindi

By Deepak Kumar Gupta, Surabhi Kumari, Asif Ekbal, Pushpak Bhattacharyya. 2018

59 citations recorded by the source corpus.

In this paper, we assess the challenges for multi-domain, multi-lingual question answering, create necessary resources for benchmarking and develop a baseline model. We curate 500 articles in six different domains from the web. These articles form a comparable corpora of 250 English documents and 250 Hindi documents. From these comparable corpora, we have created 5 ; 495 question-answer pairs with the questions and answers, both being in English and Hindi. The question can be both factoid or short descriptive types. The answers are categorized in 6 coarse and 63 finer types. To the best of our knowledge, this is the very first attempt towards creating multi-domain, multi-lingual question answering evaluation involving English and Hindi. We develop a deep learning based model for classifying an input question into the coarse and finer categories depending upon the expected answer. Answers are extracted through similarity computation and subsequent ranking. For factoid question, we obtain an MRR value of 49 : 10 % and for short descriptive question, we obtain a BLEU score of 41 : 37 %. Evaluation of question classification model shows the accuracies of 90 : 12 % and 80 : 30 % for coarse and finer classes, respectively.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages (2020, methodology): “…al., 2019), a semi-automatic Italian translation of SQuAD (Croce et al., 2018), ARCD—an Arabic reading comprehension dataset (Mozannar et al., 2019), a Hindi-English parallel dataset in a SQuAD-like setting (Gupta et al., 2018), and a Chinese–English dataset focused on visual QA (Gao et al., 2015).”
The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants (2023, background): “…et al., 2019), Bulgarian (Hardalov et al., 2019), French (d’Hoffschmidt et al., 2020), German (Möller et al., 2021), Hindi (Anuranjana et al., 2019; Gupta et al., 2018), Italian (Croce et al., 2018), Russian (Efimov et al., 2020; Shavrina et al., 2020), and Tibetan (Sun et al., 2021), amongst…”
iNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages (2020, background): “…(Akhtar et al., 2017), word analogy (Grave et al., 2018), text classification, sentiment analysis (Akhtar et al., 2016; Mukku and Mamidi, 2017), paraphrase detection (Anand Kumar et al., 2016), QA (Clark et al., 2020; Gupta et al., 2018), discourse mode classification (Dhanwal et al., 2020), etc. .”
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering (2020, background): “0 for Korean (Lim et al., 2019), and MMQA for Hindi (Gupta et al., 2018).”