Averaging Gone Wrong: Using Time-Aware Analyses to Better Understand Behavior
By Samuel Barbosa, Dan Cosley, Amit Sharma, Roberto M. Cesar Jr.. 2016
29 citations recorded by the source corpus.
Online communities provide a fertile ground for analyzing people's behavior and improving our understanding of social processes. Because both people and communities change over time, we argue that analyses of these communities that take time into account will lead to deeper and more accurate results. Using Reddit as an example, we study the evolution of users based on comment and submission data from 2007 to 2014. Even using one of the simplest temporal differences between users---yearly cohorts---we find wide differences in people's behavior, including comment activity, effort, and survival. Further, not accounting for time can lead us to misinterpret important phenomena. For instance, we observe that average comment length decreases over any fixed period of time, but comment length in each cohort of users steadily increases during the same period after an abrupt initial drop, an example of Simpson's Paradox. Dividing cohorts into sub-cohorts based on the survival time in the community provides further insights; in particular, longer-lived users start at a higher activity level and make more and shorter comments than those who leave earlier. These findings both give more insight into user evolution in Reddit in particular, and raise a number of interesting questions around studying online behavior going forward.
Subfields: Communication, Sociology and Political Science, Statistics and Probability, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Communication
In Sociology and Political Science
In Statistics and Probability
- Kristina Lerman
- Nazanin Alipourfard
- Peter G. Fennell
- Aram Galstyan
- Fred Morstatter
- Nripsuta Ani Saxena
- Ninareh Mehrabi
In Evaluation & Benchmarks
Citations in the corpus
- A Survey on Bias and Fairness in Machine Learning (2019, neutral): “itudinal Data Fallacy. Observational studies often treat cross-sectional data as if it were longitudinal, which may create biases due to Simpson’s paradox. As an example, analysis of bulk Reddit data [9] revealed that comment length decreased over time on average. However, bulk data represented a cr”
- Can you Trust the Trend?: Discovering Simpson's Paradoxes in Social Data (2018, background): “A study [2] of Reddit found that while it may appear that average comment length on decreases over any fixed period of time, when data is disaggregated into groups based on the year user joined Reddit, comment length within each group increases during the same time period.”
- Computational social scientist beware: Simpson’s paradox in behavioral data (2017, neutral): “For example, a study of Reddit [2] found that average comment length decreased over time.”