A Practical Guide to Sentiment Annotation: Challenges and Solutions
By Saif M. Mohammad. 2016
113 citations recorded by the source corpus.
Sentences and tweets are often annotated for sentiment simply by asking respondents to label them as positive, negative, or neutral. This works well for simple expressions of sentiment; however, for many other types of sentences, respondents are unsure of how to annotate, and produce inconsistent labels. In this paper, we outline several types of sentences that are particularly challenging for manual sentiment annotation. Next we propose two annotation schemes that address these challenges, and list benefits and limitations for both.
Subfields: Natural Language Processing, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Kai-Wei Chang
- Nanyun Peng
- Emily Sheng
- Premkumar Natarajan
- John P. McCrae
- Bharathi Raja Chakravarthi
- Ruba Priyadharshini
- Vigneshwaran Muralidaran
In Evaluation & Benchmarks
- Kai-Wei Chang
- Nanyun Peng
- Emily Sheng
- Premkumar Natarajan
- Bharathi Raja Chakravarthi
- John P. McCrae
- Ruba Priyadharshini
- Vigneshwaran Muralidaran
Citations in the corpus
- The Woman Worked as a Babysitter: On Biases in Language Generation (2019, methodology): “The sentiment and regard annotation guidelines are adapted from Mohammad (2016)’s sentiment annotation guidelines.”
- Challenges in Sentiment Analysis (2017, methodology): “See Mohammad (2016a) for annotation schemes that address some of these challenges.”
- Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text (2020, methodology): “TamilMixSentiment was developed with guidelines following the work of Mohammad (2016) and without annotating the word level language tag.”