Spatial and Semantic Processing between Audition and Vision: An Event-Related Potential Study

Abstract

Using a crossmodal priming paradigm, this study investigated how the brain bound the spatial and semantic features in multisensory processing. The visual stimuli (pictures of animals) were presented after the auditory stimuli (sounds of animals), and the stimuli from different modalities may match spatially (or semantically) or not. Participants were required to detect the head orientation of the visual target (an oddball paradigm). The event-related potentials (ERPs) to the visual stimuli was enhanced by spatial attention (150–170 ms) irrespectively of semantic information. The early crossmodal attention effect for the visual stimuli was more negative in the spatial-congruent condition than in the spatial-incongruent condition. By contrast, the later effects of spatial ERPs were significant only for the semantic- congruent condition (250–300 ms). These findings indicated that spatial attention modulated early visual processing, and semantic and spatial features were simultaneously used to orient attention and modulate later processing stages.