Audiovisual Capture with Ambiguous Audiovisual Stimuli

Abstract

Audiovisual capture happens when information across modalities get fused into a coherent percept. Ambiguous multi-modal stimuli have the potential to be powerful tools to observe such effects. We used such stimuli made of temporally synchronized and spatially co-localized visual flashes and auditory tones. The flashes produced bistable apparent motion and the tones produced ambiguous streaming. We measured strong interferences between perceptual decisions in each modality, a case of audiovisual capture. However, does this mean that audiovisual capture occurs before bistable decision? We argue that this is not the case, as the interference had a slow temporal dynamics and was modulated by audiovisual congruence, suggestive of high-level factors such as attention or intention. We propose a framework to integrate bistability and audiovisual capture, which distinguishes between “what” competes and “how” it competes (Hupé et al., 2008). The audiovisual interactions may be the result of contextual influences on neural representations (“what” competes), quite independent from the causal mechanisms of perceptual switches (“how” it competes). This framework predicts that audiovisual capture can bias bistability especially if modalities are congruent (Sato et al., 2007), but that is fundamentally distinct in nature from the bistable competition mechanism.