Explaining the Effect of Likelihood Manipulation and Prior through a Neural Network of the Audiovisual Perception of Space

Ursino, Mauro; Cuppini, Cristiano; Magosso, Elisa; Beierholm, Ulrik; Shams, Ladan

Abstract

Results in the recent literature suggest that multisensory integration in the brain follows the rules of Bayesian inference. However, how neural circuits can realize such inference and how it can be learned from experience is still the subject of active research. The aim of this work is to use a recent neurocomputational model to investigate how the likelihood and prior can be encoded in synapses, and how they affect audio-visual perception, in a variety of conditions characterized by different experience, different cue reliabilities and temporal asynchrony. The model considers two unisensory networks (auditory and visual) with plastic receptive fields and plastic crossmodal synapses, trained during a learning period. During training visual and auditory stimuli are more frequent and more tuned close to the fovea. Model simulations after training have been performed in crossmodal conditions to assess the auditory and visual perception bias: visual stimuli were positioned at different azimuth (±10° from the fovea) coupled with an auditory stimulus at various audio-visual distances (±20°). The cue reliability has been altered by using visual stimuli with two different contrast levels. Model predictions are compared with behavioral data. Results show that model predictions agree with behavioral data, in a variety of conditions characterized by a different role of prior and likelihood. Finally, the effect of a different unimodal or crossmodal prior, re-learning, temporal correlation among input stimuli, and visual damage (hemianopia) are tested, to reveal the possible use of the model in the clarification of important multisensory problems.

Citation

Ursino, M., Cuppini, C., Magosso, E., Beierholm, U., & Shams, L. (2019). Explaining the Effect of Likelihood Manipulation and Prior through a Neural Network of the Audiovisual Perception of Space. Multisensory Research, 32(2), 111-144. https://doi.org/10.1163/22134808-20191324

• Journal Article Type: Article
• Acceptance Date: Jan 4, 2019
• Online Publication Date: Feb 13, 2019
• Publication Date: Jan 31, 2019
• Deposit Date: Jan 8, 2019
• Publicly Available Date: Feb 13, 2020
• Journal: Multisensory Research
• Print ISSN: 2213-4794
• Electronic ISSN: 2213-4808
• Publisher: Brill Academic Publishers
• Peer Reviewed: Peer Reviewed
• Volume: 32
• Issue: 2
• Pages: 111-144
• DOI: https://doi.org/10.1163/22134808-20191324