Principled Frameworks for Evaluating Ethics in NLP Systems
By Shrimai Prabhumoye, Elijah Mayfield, Alan W. Black. 2019
8 citations recorded by the source corpus.
We critique recent work on ethics in natural language processing. Those discussions have focused on data collection, experimental design, and interventions in modeling. But we argue that we ought to first understand the frameworks of ethics that are being used to evaluate the fairness and justice of algorithmic systems. Here, we begin that discussion by outlining deontological and consequentialist ethics, and make predictions on the research agenda prioritized by each.
Subfields: Natural Language Processing, Evaluation & Benchmarks, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Ruslan Salakhutdinov
- Shrimai Prabhumoye
- Alan W. Black
- Brendon Boldt
- Mohit Bansal
- Jianfeng Gao
- Jason Aaron Edward Weston
- Percy Liang
In Evaluation & Benchmarks
- Ruslan Salakhutdinov
- Alan W. Black
- Shrimai Prabhumoye
- David M. Blei
- Matthew Henderson
- Matthew D. Hoffman
In AI Ethics, Fairness & Society
Citations in the corpus
- Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…et al. (2019); Gonen and Goldberg (2019); Lauscher and Glavaš (2019); Loukina et al. (2019); Mayfield et al. (2019); Mirzaev et al. (2019); Prabhumoye et al. (2019); Ruane et al. (2019); Sedoc and Ungar (2019); Sun et al. (2019); Nissim et al. (2020); Rozado (2020); Shah et al. (2020);…”
- Case Study: Deontological Ethics in NLP (2020, background): “…this discussion and made contributions towards ethical frameworks and design practices (Leidner and Plachouras, 2017; Parra Escartı́n et al., 2017; Prabhumoye et al., 2019; Schnoebelen, 2017; Schmaltz, 2018), data handling practices (Lewis et al., 2017; Mieskes, 2017) and specific domains like…”