Strategies for evaluating information visualization tools: multi-dimensional in-depth long-term case studies

By Ben Shneiderman, Catherine Plaisant. Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization, 2006

511 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Sociology and Political Science, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/6e77a3fce66982106066636634549ea219450d78

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Sociology and Political Science

In Human-Computer Interaction

Citations in the corpus

Interacting with Predictions: Visual Inspection of Black-box Machine Learning Models (2016, background): “measuring task time completion or number of errors) are often insufficient to evaluate visualization systems [6, 35, 38].”
Empirical Studies in Information Visualization: Seven Scenarios (2012, unknown): “Two examples of long-term case studies in visualization evaluation are: Multidimensional In-depth Long-term Case studies (MILCs) [76] and insight-based evaluations [70].”
Evaluating Information Visualizations (2008, background): “While there is growing recognition in our research community that evaluation information visualization is difficult [55, 57, 67], the recognition of this difficulty has not in itself provided immediate answers of how to approach this problem.”
LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks (2016, background): “Shneiderman and Plaisant [26] propose strategies for multi-dimensional in-depth long-term case (MILC) studies to evaluate information visualization tools.”
EventAction (2019, methodology): “Following the Multi-dimensional In-depth Long-term Case studies (MILCs) procedure [67] all our case studies were conducted with real users and using real-world datasets.”
Mastering the Information Age - Solving Problems with Visual Analytics (2010, background): “Shneiderman and Plaisant[101] argue for the need for longitudinal studies as There are many types of evaluation including, longitudinal studies, surveys, interviews, observation and think-aloud studies an effective means for understanding the issues involved in tool adoption.”
Design Study Methodology: Reflections from the Trenches and the Stacks (2012, methodology): “Finally, many proposed evaluation methods address the specific needs of validating the usefulness of visualization tools such as the multidimensional in-depth long-term case study approach [80], the insight-based method [68, 69], and grounded evaluation [33].”
Interpreting informational cues: An explorative study on information use among prospective homebuyers (2009, unknown): “In recent years, researchers have become interested in studying how experienced professionals change their working strategies as they gain familiarity with a tool over weeks and months (Shneiderman and Plaisant, 2006).”
A Systematic Review on the Practice of Evaluating Visualization (2013, background): “’ In visualization, case studies are commonly defined as “detailed reporting about a small number of individuals working on their own problems, in their normal environment” [62].”
Mental Models, Visual Reasoning and Interaction in Information Visualization: A Top-down Perspective (2010, background): “For example, verbal analysis of the mental modeling process is applicable when expert users engage in long-term collaborations with visualization researchers [50].”
A Multi-Level Typology of Abstract Visualization Tasks (2013, methodology): “We plan to validate the typology’s evaluative power by assessing its utility a multi-level code set for labelling user behaviour, a common practice in observational studies of open-ended visualization use; these include longitudinal insightbased studies [48] and multidimensional in-depth long-term c”