TSNLP: Test Suites for Natural Language Processing
By S. Lehmann, Stephan Oepen, Sylvie Regnier-Prost, K. Netter, Veronika Lux, J. Klein, Kirsten Falkedal, F. Fouvry, Dominique Estival, Eva Dauphin and 4 more. 1996
The growing language technology industry needs measurement tools to allow researchers, engineers, managers, and customers to track development, evaluate and assure quality, and assess suitability for a variety of applications.The TSNLP (Test Suites for Natural Language Processing) project has investigated various aspects of the construction, maintenance and application of systematic test suites as diagnostic and evaluation tools for NLP applications. The paper summarizes the motivation and main results of TSNLP: besides the solid methodological foundation of the project, TSNLP has produced substantial (i.e. larger than any existing general test suites) multi-purpose and multi-user test suites for three European languages together with a set of specialized tools that facilitate the construction, extension, maintenance, retrieval, and customization of the test data.The publicly available results of TSNLP represent a valuable linguistic resource that has the potential of providing a wide-spread pre-standard diagnostic and evaluation tool for both developers and users of NLP applications.
Subfields: Natural Language Processing, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Yonatan Belinkov
- James R. Glass
- Nadir Durrani
- Hassan Sajjad
- Fahim Dalvi
- Percy Liang
- Ellie Pavlick
- John Hewitt
In Evaluation & Benchmarks
Citations in the corpus
- Analysis Methods in Neural Language Processing: A Survey (2018, methodology): “In MT evaluation, Burchardt et al. (2017) reported results using a large test suite covering 120 phenomena, partly based on Lehmann et al. (1996).”