Subjective Natural Language Problems: Motivations, Applications, Characterizations, and Implications

88 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation (2022, neutral): “However, the disagreements between annotators may embed valuable nuances about the task [4, 36].”
Towards Measuring the Representation of Subjective Global Opinions in Language Models (2023, unknown): “However, understanding the model behaviour in settings that involve ambiguity is crucial to identifying and mitigating potential biases in order to build models that respect human diversity (Selbst et al., 2019; Alm, 2011).”
Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations (2021, background): “However, in many subjective tasks, there often exists no such single “right” answer (Alm, 2011) and enforcing a single ground truth sacriﬁces the valuable nuances embedded in annotator’s assessments of the stimuli and their disagreements (Aroyo and Welty, 2013; Cheplygina and Pluim, 2018)…”
Whose Opinions Do Language Models Reflect? (2023, background): “There has been a long-standing push within the NLP community to consider the subjective and affective dimensions of language in evaluating models (Alm, 2011).”