MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment

By P. M. McCarthy, Scott Jarvis. 2010

936 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Developmental and Educational Psychology, Natural Language Processing, Evaluation & Benchmarks, Computational Linguistics & Semantics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Developmental and Educational Psychology

In Natural Language Processing

In Evaluation & Benchmarks

In Computational Linguistics & Semantics

Citations in the corpus

Evaluation of Text Generation: A Survey (2020, methodology): “The HD-D and its variants (McCarthy & Jarvis, 2010) have been used to measure the diversity in story generation (McCarthy & Jarvis, 2010) and summarization tasks (Crossley et al., 2019).”
TEXT CHARACTERISTICS OF TASK INPUT AND DIFFICULTY IN SECOND LANGUAGE LISTENING COMPREHENSION (2013, methodology): “On the basis of a careful validation study, McCarthy and Jarvis ( 2010 ) recommend using a combination of indices, including the measure of textual lexical diversity and the D-formula (Malvern & Richards, 1997 ), to adequately capture this construct.”
The relationship between vocabulary and writing quality in three genres (2012, methodology): “Diversity Vocabulary diversity was calculated using the Measure of Textual Lexical Diversity (MTLD, McCarthy & Jarvis, 2010), included as part of the Gramulator software program (McCarthy, Watanabi, & Lamkin, in press).”
The tool for the automatic analysis of text cohesion (TAACO): Automatic assessment of local, global, and text cohesion (2015, background): “TTR indices have demonstrated positive relations with measures of cohesion in previous studies (Crossley & McNamara, 2014; McCarthy & Jarvis, 2010), but generally they demonstrate negative relations with measures of text coherence (Crossley & McNamara, 2010; McNamara et al. 2010a, b).”
Identity Deception Detection (2017, background): “Lexical diversity This set includes four lexical diversity metrics, including type/token ratio (McCarthy and Jarvis, 2010), mean word frequency, and the Yule’s I and K indexes (Oakes, 2000).”
UKP: Computing Semantic Textual Similarity by Combining Multiple Content Similarity Measures (2012, background): “We also include a number of measures which capture statistical properties of texts, such as typetoken ratio (TTR) (Templin, 1957) and sequential TTR (McCarthy and Jarvis, 2010).”
A multifaceted approach to investigating pre-task planning effects on paired oral test performance (2014, methodology): “In addition, the study used the Measure of Textual Lexical Diversity (MTLD: McCarthy & Jarvis, 2010) to analyze lexical complexity.”
Automatically Assessing Lexical Sophistication: Indices, Tools, Findings, and Application (2015, methodology): “The validation of TAALES is guided by the following research questions: 1 Recent studies (e.g., McCarthy & Jarvis, 2010) indicate that indices of lexical diversity are likely related to text cohesion (i.e., word repetition).”
Capturing the Diversity in Lexical Diversity. (2013, background): “…Simpson’s index nor the Gini-Simpson index appears to have been adopted as a measure of lexical diversity (but see Baayen, 2001), although some new measures of lexical diversity, such as the vocd D measure and HD-D are built on similar principles of probability (see McCarthy & Jarvis, 2007, 2010).”
Enhancing Chat Language Models by Scaling High-quality Instructional Conversations (2023, methodology): “Lexical diversity is calculated by averaging the MTLD score (McCarthy and Jarvis, 2010) over each utterance with LexicalRichness7.”
Effects of Task Complexity on L2 Writing Behaviors and Linguistic Complexity (2017, methodology): “Lexical variability was assessed by Malvern and Richards’s (1997) D formula and the measure of textual lexical diversity (MTLD; McCarthy & Jarvis, 2010).”
The Effects of Complexity, Accuracy, and Fluency on Communicative Adequacy in Oral Task Performance (2014, methodology): “It was also not possible to supplement D with the measure of textual lexical diversity (MTLD) as recommended by McCarthy and Jarvis (2010). The MTLD tool requires a minimum of 100 tokens, and 43.”
Assessing Lexical Proficiency Using Analytic Ratings: A Case for Collocation Accuracy (2014, methodology): “However, in addition to measuring breadth of knowledge, lexical diversity can be considered an individual construct (i.e. a measure outside of breadth and depth of lexical knowledge; Baba 2009) as well as a measure of discourse cohesion (McCarthy and Jarvis 2010).”
Assessing the Validity of Lexical Diversity Indices Using Direct Judgements (2020, methodology): “McCarthy and Jarvis (2010) dealt with this issue by estimating the final factor length based on the parameters of the remaining words.”