Annotating Named Entities in Twitter Data with Crowdsourcing

By Timothy W. Finin, William Murnane, Anand Karandikar, Nicholas Keller, Justin Martineau, Mark Dredze. Mturk@HLT-NAACL, 2010

356 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/f2537e72219933ee97a57a5b35e74e59bf3a25e7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Piggyback: Using Search Engines for Robust Cross-Domain Named Entity Recognition (2011, background): “Lawson et al. (2010), Finin et al. (2010), and Yetisgen-Yildiz et al. (2010) investigate how to best use Amazon Mechanical Turk (AMT) for NER.”
Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings (2015, background): “, 2012a), with recent efforts turning towards social media (Finin et al., 2010; Liu et al., 2011; Ritter et al., 2011; Fromreide et al., 2014; Li et al., 2012; Liu et al., 2012).”
Beyond Mechanical Turk: An Analysis of Paid Crowd Work Platforms (2015, background): “If many workers fail gold tests, the job is suspended (Finin et al., 2010; Negri & Mehdad, 2010).”
Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments (2010, background): “However, the bulk of this work eschews the standard pipeline of tools which might enable a richer linguistic analysis; such tools are typically trained on newstext and have been shown to perform poorly on Twitter (Finin et al., 2010).”
What to do about bad language on the internet (2013, background): “In the case of Twitter, the limit of 140 characters for each message is frequently cited as an explanation for bad language (Finin et al., 2010).”
Between Subjectivity and Imposition (2020, background): “However, these systems require, in many cases, the intervention of human discretion in their deployment [2, 64], and even more frequently, in their creation [18, 22, 35, 36, 39, 48, 51, 61, 66, 67].”
A Survey of Crowdsourcing Systems (2011, background): “email datasets and medical named entities [29], [20], [49], [89].”
Online Hate Interpretation Varies by Country, But More by Individual: A Statistical Analysis Using Crowdsourced Ratings (2018, methodology): “FigureEight), the crowdsourcing platform used by several researchers [28]–[30].”
Recognizing Named Entities in Tweets (2011, methodology): “Finin et al. (2010) use Amazons Mechanical Turk service 2 and CrowdFlower 3 to annotate named entities in tweets and train a CRF model to evaluate the effectiveness of human labeling.”
ZenCrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking (2012, methodology): “In the context of entity identification, crowdsourcing has been used by Finn et al [16] to annotate entities in Twitter.”
Analysis of named entity recognition and linking for tweets (2014, unknown): “ image URLs embedded in the corpus suggest that it was collected during September 2010. The second is the gold standard data created through a Twitter annotation experiment with crowdsourcing at UMBC [22] (441 tweets comprising 7;037 tokens). The third is the dataset developed for the Making Sense o”
GERBIL - Benchmarking Named Entity Recognition and Linking consistently (2017, methodology): “[19] which was created using crowdsourcing of simple entity annotations over Twitter microposts as well as the WSDM2012/Meij dataset [2] which also describes tweets but these were annotated using only two an-”
Corpus Annotation through Crowdsourcing: Towards Best Practice Guidelines (2014, background): “Others create corpora of special resource types such as emails (Lawson et al., 2010), twitter feeds (Finin et al., 2010), augmented and alternative communication texts (Vertanen and Kristensson, 2011).”
Crowd Truth: Harnessing disagreement in crowdsourcing a relation extraction gold standard (2013, background): “All this led us to evaluate crowdsourcing as an approach for generating gold standard, following a growing community of machine learning and NLP research [7, 5].”
Adaptive Co-attention Network for Named Entity Recognition in Tweets (2018, background): “…of the existing NER datasets only focus on entity classes: Person, Location, Organization, such as NER datasets construct by Derczynski et al. (2016) and Finin et al. (2010), or focus on entity classes: Person, Location, Organization and Misc, such as CoNLL2003(Tjong Kim Sang and De Meulder 2003).”
Twitter Catches The Flu: Detecting Influenza Epidemics using Twitter (2011, background): “Twitter can provide suitable material for many applications such as named entity recognition (NER) (Finin et al., 2010) and sentiment analysis (Barbosa and Feng, 2010).”