Report Decoding Sound and Imagery Content in Early Visual Cortex

By Petra Vetter, Fraser W. Smith, Lars Muckli. 2014

236 citations recorded by the source corpus.

Summary Human early visual cortex was traditionally thought to process simple visual features such as orientation, contrast, and spatial frequency via feedforward input from the lateral geniculate nucleus (e.g., [1]). However, the role of nonretinal influence on early visual cortex is so far insufficiently investigated despite much evidence that feedback connections greatly outnumber feedforward connections [2‐5]. Here, we explored in five fMRI experiments how information originating from audition and imagery affects the brain activity patterns in early visual cortex in the absence of any feedforward visual stimulation. We show that category-specific information from both complex natural sounds and imagery can be read out from early visual cortex activity in blindfolded participants. The coding of nonretinal information in the activity patterns of early visual cortex is common across actual auditory perception and imagery and may be mediated by higher-level multisensory areas. Furthermore, this coding is robust to mild manipulations of attention and working memory but affected by orthogonal, cognitively demanding visuospatial processing. Crucially, the information fed down to early visual cortex is category specific and generalizes to sound exemplars of the same category, providing evidence for abstract information feedback rather than precise pictorial feedback. Our results suggest that early visual cortex receives nonretinal input from other brain areas when it is generated by auditory perception and/or imagery, and this input carries common abstract information. Our findings are compatible with feedback of predictive information to the earliest visual input level (e.g., [6]), in line with predictive coding models [7‐10].

View Paper: https://www.semanticscholar.org/paper/b35b9c1a4c4dbc38141a482e212aee88d16a0892

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Functional connectivity of visual cortex in the blind follows retinotopic organization principles (2015, background): “In contrast, there is ample evidence that peripheral V1 is also connected to non-visual cortices and responds to auditory information and attention in sighted people (Eckert et al., 2008; Cate et al., 2009; Beer et al., 2011; Spierer et al., 2013; Vetter et al., 2014).”
The multisensory function of the human primary visual cortex. (2016, background): “Notably, very recently, Vetter et al. (2014) demonstrated that it is possible to decode the category of natural sounds heard by participants based on the patterns of activity within the primary visual cortex (see also de Haas et al. (2013) for results of impaired decoding of visual stimuli within…”
A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes (2014, unknown): “Additional sources of variation include attention (Kamitani and Tong, 2005), reward expectation (Serences, 2008), the perception of coherent shape (Murray et al., 2002), global context (Joo et al., 2012), and even auditory stimulation (Vetter et al., 2014).”
Single-trial multisensory memories affect later auditory and visual object discrimination. (2015, unknown): “The more robust impact of visual information on subsequent auditory object recognition suggests that visual information can lead to category specific predictive activations within auditory object processing areas, similar to what has been reported by Vetter et al. (2014) for auditory information.”