The effects of transparency on trust in and acceptance of a content-based art recommender
By Henriette S. M. Cramer, Vanessa Evers, Satyan Ramlal, Maarten van Someren, Lloyd Rutledge, Natalia Stash, Lora Aroyo, Bob J. Wielinga. 2008
480 citations recorded by the source corpus.
The increasing availability of (digital) cultural heritage artefacts offers great potential for increased access to art content, but also necessitates tools to help users deal with such abundance of information. User-adaptive art recommender systems aim to present their users with art content tailored to their interests. These systems try to adapt to the user based on feedback from the user on which artworks he or she finds interesting. Users need to be able to depend on the system to competently adapt to their feedback and find the artworks that are most interesting to them. This paper investigates the influence of transparency on user trust in and acceptance of content-based recommender systems. A between-subject experiment (N = 60) evaluated interaction with three versions of a content-based art recommender in the cultural heritage domain. This recommender system provides users with artworks that are of interest to them, based on their ratings of other artworks. Version 1 was not transparent, version 2 explained to the user why a recommendation had been made and version 3 showed a rating of how certain the system was that a recommendation would be of interest to the user. Results show that explaining to the user why a recommendation was made increased acceptance of the recommendations. Trust in the system itself was not improved by transparency. Showing how certain the system was of a recommendation did not influence trust and acceptance. A number of guidelines for design of recommender systems in the cultural heritage domain have been derived from the study’s results.
Subfields: Cognitive Neuroscience, Museology, Social Psychology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Cognitive Neuroscience
- Donghee Shin
- W. Bruce Croft
- Judith Masthoff
- Jo Vermeulen
- Yongfeng Zhang
- Mohan Kankanhalli
- Yiqun Liu
- Martijn C. Willemsen
In Social Psychology
- Janghee Cho
- Emilee J. Rader
- Judith Masthoff
- Todd Kulesza
- Brian Y. Lim
- Donghee Shin
- René F. Kizilcec
- Kelley Cotter
Citations in the corpus
- Explainable Recommendation: A Survey and New Perspectives (2018, background): “[2012], Cramer et al. [2008]). Based on these content features, content-based recommendation is usually intuitive to be explained by generating feature-based explanations. Content-based recommendations can be based on various item features/characteristics depending on each application scenario. For ”
- User Perceptions of Algorithmic Decisions in the Personalized AI System:Perceptual Evaluation of Fairness, Accountability, Transparency, and Explainability (2020, background): “There is an increasing call for transparency and explainability to make algorithm comprehensible to those affected by it (Cramer et al., 2008).”
- Principles of Explanatory Debugging to Personalize Interactive Machine Learning (2015, background): “Other work has studied the impact of machine-generated explanations on users’ mental models, but did not explore whether this helped participants’ personalize their learning systems [8, 25, 30].”
- Recommender Systems (2016, unknown): “Numerous online methods for trust evaluation are discussed in [171, 175, 248, 486].”
- The Impact of POMDP-Generated Explanations on Trust and Performance in Human-Robot Teams (2016, background): “However, disuse is a realistic and common problem in human-automation interaction [27] and often linked to lack of transparency [5].”
- Explanation-Based Human Debugging of NLP Models: A Survey (2021, background): “, when explanations are used to promote system acceptance (Cramer et al., 2008), to verify the model reasoning (Caruana et al.”
- Explanations as Mechanisms for Supporting Algorithmic Transparency (2018, background): “Explanations improve system usability and overall performance, and promote more positive user perceptions and acceptance of the system [10, 24, 21, 37].”
- F1 Is Not Enough! Models and Evaluation towards User-Centered Explainable Question Answering (2020, background): “…improve user satisfaction and confidence (Sinha and Swearingen, 2002; Biran and McKeown, 2017) and, thus, increase the acceptance of automatic systems (Herlocker et al., 2000; Cramer et al., 2008), this aspect is very important when evaluating models that should be applied in reallife scenarios.”
- Evaluating Recommendation Systems (2011, background): “whether the system recommendations are reasonable in a user study [22, 12, 11, 47].”
- Explaining Recommendations: Design and Evaluation (2015, background): “CHIP [20] “Why is ‘The Tailor’s Workshop recommended to you’? Because it has the following themes in common with artworks that you like: * Everyday Life * Clothes .”
- Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models (2023, background): “This continuous feedback loop can contribute to the ongoing improvement and refinement of AI models, ensuring that they remain fair, accurate, and relevant to users’ needs [148, 149, 150, 151].”
- Designing and Evaluating Explanations for Recommender Systems (2011, background): “[20] evaluated the acceptance of recommended items in terms of how many recommended items were present in a final selection of six favorites.”
- Evaluating Recommender Systems with User Experiments (2015, background): “Research has found that users like explanations [50], and that they increase users’ understanding of the recommendation process [41, 117], their trust in the quality of the recommendations, and the competence and benevolence of the system [24, 36, 119] (more on credibility and trust can be found in ”
- Machine Learning Interpretability: A Survey on Methods and Metrics (2019, background): “With respect to interpretability itself, historically speaking, there has been sporadic interest in explanations of intelligent systems since the 1970s, beginning with attention on expert systems [11–13]; to, a decade after, neural networks [14]; and then to recommendation systems in the 2000s [15,1”
- User Trust in Intelligent Systems: A Journey Over Time (2016, methodology): “System ability, control, predictability and transparency have been identified in existing research as some of the key factors that influence users’ trust in intelligent systems [2, 5, 12, 14].”
- How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface (2016, background): “Another experiment found ‘why’ explanations to increase recommendation acceptance but not trust in the system [6].”