Perceived Congruence between the Changing Patterns of a Visual Image and Pitch of a Sound

Abstract

Perceived congruence between the changing patterns of a visual image and pitch of a sound was examined by a series of rating experiments. Vertical correspondence of direction between the movement of an image and the pitch shift of a sound had a strong effect on the perceived congruence; the combination of a rising image and an ascending pitch, and that of a falling image and a descending pitch, created higher congruence than the alternative combinations. The combination of an enlarging image and an ascending pitch, and that of a reducing image and a descending pitch, also created higher congruence. Furthermore, the combination of a movement sliding from left to right and an ascending pitch, and that of a movement sliding from right to left and a descending pitch, created higher congruence. When the visual transformation pattern was a complex of these three kinds of movements, the vertical direction transformation was the most effective factor in determining congruent pitch pattern. The next most effective transformation factor was the enlarging and reducing image. The transformation of right and left direction movement was not strongly effective in determining the perceived congruence between complex visual movements and pitch patterns.