Cross-Modal Associations Between Harmonic Dissonance and Visual Roughness

Abstract

Visual associations with auditory stimuli have been the subject of numerous studies. Colour, shape, size, and several other parameters have been linked to musical elements like timbre and pitch. In this article, we aim to examine the relationship between harmonisations with varying degrees of dissonance and visual roughness. Based on past research in which high sensory dissonance was associated with angular shapes, we argued that nontonal and highly dissonant harmonisations will be associated with angular and rough images, while more consonant stimuli will be associated with the images of low visual roughness. A fixed melody was harmonised in 7 different styles, including highly tonal, nontonal, and random variations. Through a listening task, musically trained participants rated the stimuli in terms of enjoyment, familiarity, and matched them to images of variable roughness. The overall consonance of the stimuli was calculated using two distinct models (Harrison & Pearce, 2020; Wang et al., 2013 ) and a variant of the aggregate dyadic consonance index ( Huron, 1994 ). Our results demonstrate that dissonance, as calculated by all models, was highly correlated with visual roughness, and enjoyment and familiarity followed expected patterns compared to tonal and nontonal stimuli. In addition to sensory dissonance, however, it appears that other factors, such as the typicality of chord progressions and the sense of tonality may also influence this cross-modal interaction.