TweetMotif: Exploratory Search and Topic Summarization for Twitter

By Brendan T. O'Connor, Michel Krieger, David Ahn. 2010

433 citations recorded by the source corpus.

We present TweetMotif, an exploratory search applica- tion for Twitter. Unlike traditional approaches to in- formation retrieval, which present a simple list of mes- sages, TweetMotif groups messages by frequent signif- icant terms — a result set’s subtopics — which facili- tate navigation and drilldown through a faceted search interface. The topic extraction system is based on syn- tactic ﬁltering, language modeling, near-duplicate de- tection, and set cover heuristics. We have used Tweet- Motif to deﬂate rumors, uncover scams, summarize sentiment, and track political protests in real-time. A demo of TweetMotif, plus its source code, is available at http://tweetmotif.com.

Subfields: Artificial Intelligence, Information Systems, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Statistical and Nonlinear Physics

Citations in the corpus

"How Old Do You Think I Am?" A Study of Language and Age in Twitter (2013, methodology): “Tokenization is done using the tool by (O’Connor, Krieger, and Ahn 2010).”
Adversarial Removal of Demographic Attributes from Text Data (2018, methodology): “Preprocessing We tokenize each tweet using twokenize , a twitter speciﬁc tokenizer (O’Connor et al., 2010; Owoputi et al., 2013), and discard duplicate tweets and tweets with less than three to-kens.”
Steering Output Style and Topic in Neural Response Generation (2017, neutral): “Twitter specific tokenizer (O’Connor et al., 2010).”
Sensing Trending Topics in Twitter (2013, methodology): “Similar approaches based on textual similarity and can be found in the literature [18], [19].”
Demographic Dialectal Variation in Social Media: A Case Study of African-American English (2016, methodology): “We use the twokenize Twitter-speciﬁc tokenizer for the messages, which separates emoticons, symbols and URLs from the text (O’Connor et al., 2010b; Owoputi et al., 2013) 21 and use the space-separated tokenizations as input to SyntaxNet, allowing it to tokenize further.”
A System for Real-time Twitter Sentiment Analysis of 2012 U.S. Presidential Election Cycle (2012, methodology): “We compared several Twitter-specific tokenizers, such as TweetMotif (O'Connor et al., 2010) and found Christopher Potts’ basic Twitter tokenizer best suited as our base.”
Twitter Universal Dependency Parsing for African-American and Mainstream American English (2018, background): “…to do this well, it would require a more involved segmentation model over the dozens or even hundreds of alternate spellings each of the above can take; 5 we instead rely on Owoputi et al. (2013); O’Connor et al. (2010)’s rule-based tokenizer that never attempts to segment within such shortenings.”
A Latent Variable Model for Geographic Lexical Variation (2010, methodology): “Informal text from mobile phones is challenging to tokenize; we adapt a publicly available tokenizer5 originally developed for Twitter (O’Connor et al., 2010), which preserves emoticons and blocks of punctuation and other symbols as tokens.”
Diffusion of Lexical Change in Social Media (2012, methodology): “Text was tokenized using the twokenize.py script, 3 which is designed to be robust to social media phenomena that confuse other tokenizers, such as emoticons [29].”
A survey on opinion mining and sentiment analysis: Tasks, approaches and applications (2015, methodology): “Reference Name of the tool Purpose [251] TweetMotif Tokenization of tweets [204], [250] POS tagger Twitter POS tagger [244] TweetNLP(6) Twitter natural language processing [245] Lancaster stemming algorithm Stemmer [246] GNU Aspell Spell Checker [247] Snowball English stemmer [248] Stanford Log-line”