Understanding Blind People's Experiences with Computer-Generated Captions of Social Media Images

By H. MacLeod, Cynthia L. Bennett, Meredith Ringel Morris, Edward Cutrell. 2017

222 citations recorded by the source corpus.

Subfields: Cognitive Neuroscience, Computer Science Applications, Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Science Applications

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Understanding and Evaluating Racial Biases in Image Captioning (2021, background): “Prior works [63, 51] fnd that, in certain contexts, people who are blind or low vision want racial descriptors to be included.”
VizWiz Grand Challenge: Answering Visual Questions from Blind People (2018, background): “In addition, social media platforms, such as Facebook and Twitter, provide people a way to maintain connections with friends by enabling them to identify and tag friends in posted images as well as respond to images automatically described to them [26, 41].”
Object Hallucination in Image Captioning (2018, background): “In another study, (MacLeod et al., 2017) analyzed how visually impaired people react to automatic image captions.”
Captioning Images Taken by People Who Are Blind (2020, background): “Given the clear wish from people who are blind to receive descriptions of images [3, 10, 11, 13, 34, 39, 47], many human-in-theloop [2, 3, 5, 35, 42, 46] and automated services [1, 4] have emerged to do so.”
“It’s Complicated”: Negotiating Accessibility and (Mis)Representation in Image Descriptions of Race, Gender, and Disability (2021, background): “Additionally, there is some evidence that screen reader users may over trust AI-generated image descriptions, raising the need for more public education about AI’s limitations [68, 124].”
Partially-Supervised Image Captioning (2018, background): “This limitation severely hinders the use of these models in real applications, for example as assistants for people with impaired vision [12].”
"Person, Shoes, Tree. Is the Person Naked?" What People with Vision Impairments Want in Image Descriptions (2020, background): “Others focused on BLV participants’ experience with descriptions for images presented on social media platforms and how BLV users perceive automatically generated captions [25, 43, 44].”
Let there be a clock on the beach: Reducing Object Hallucination in Image Captioning (2021, background): “This is especially true for visually impaired people where they prefer correctness over coverage [29] for obvious reasons.”
Aligning Large Multimodal Models with Factually Augmented RLHF (2023, result): “…the absence of a negative feedback mechanism preventing the model from identifying hallucinations in its output, and the potential limitations of our Fact-RLHF reward model, especially when compared against the high-capacity oracle models in previous successful studies (Touvron et al., 2023b).”
Concadia: Tackling image accessibility with context (2021, background): “, 2018), which greatly reduces their value when it comes to making inaccessible images accessible (MacLeod et al., 2017; Morris et al., 2016; Dognin et al., 2020; Salisbury et al., 2017).”
What is the point of fairness? (2019, background): “Though developers of many identifying technologies clarify that their use is meant to support not replace human decisionmaking, we know that technology is often pedestalized; that technological and scientific ways of “knowing” are treated as superior to the alternative, and frequently deferred to ev”
DreamLLM: Synergistic Multimodal Comprehension and Creation (2023, background): “…hallucination, a phenomenon where MLLMs generate non-existent objects or identities in images, significantly compromises their multimodal comprehension capabilities (Dai et al., 2023b; Liu et al., 2023a; Gunjal et al., 2023) and may pose safety risks (MacLeod et al., 2017; Rohrbach et al., 2018).”
Evaluating Object Hallucination in Large Vision-Language Models (2023, background): “It is generally believed that the hal-lucination would degrade the model performance and greatly harm the user experiences in real-world applications (MacLeod et al., 2017; Ji et al., 2022).”