The effect of cognitive factors on cross-modal synchrony perception

TitleThe effect of cognitive factors on cross-modal synchrony perception
Publication TypeThesis
Year of Publication2016
AuthorsIkumi N, Soto-Faraco S
Academic DepartmentDepartment of Experimental and Health Sciences
Number of Pages184
Date Published11/2016
UniversityUniversitat Pompeu Fabra
Thesis TypePhD
Keywordsmultisensory, synchrony perception, temporal recalibration

Perception in our everyday life takes place in multisensory environments, and hence involves the processing of a multitude of signals captured by various sensory modalities. Given the different nature of the signals, understanding how all the information is combined in the brain to form an integrated percept is not straightforward. One of the paramount questions is how the considerable timing differences between sensory information processing are managed. For example, in the last years there has been a tremendous surge in interest to understand how the perceptual system elicits the subjective impression of synchrony for stimuli coming from different sensory modalities. Yet, most evidence so far concerns stimulus-related properties in simple contexts. The present dissertation addresses the influence of cognitive factors and participants’ inner state (such as attention, action task demands, and ongoing brain rhythms) on synchrony perception between audio-visual events. In the first two studies of the dissertation, we have addressed the role of selective attention and action during cross-modal temporal recalibration. The results of these studies provide evidence that subjective simultaneity can be strongly modulated as a function of the focus of the observer’s endogenous attention, in otherwise identical stimulation conditions. In the third study, we have recorded electroencephalographic activity while participants performed an audio-visual simultaneity judgment task for stimuli presented at different asynchronies. Our results show that the phase of low frequency neural oscillations, reflecting brain states prior to the occurrence of an audio-visual event, can predict perceptual variability in synchrony judgments. Overall, our results shed new light on how cognitive factors can modulate multisensory perception.