Multimodal text-image synergy in representing interpersonal relations in picture books

Abstract

Picture books as multimodal narratives contain rich data for analysis of multimodal meaning-making. Verbal and visual modes offer various choices for representing different interpersonal relations. In this paper, I focus on the following research question: how visual and verbal resources are used to construct interactive meanings between the reader and characters and between the characters of picture books. In my analysis, I follow Halliday’s social semiotic approach to language within systemic-functional linguistics, according to which every semiotic system operates at three levels corresponding to ideational, interpersonal and textual metafunctions. I focus on the interpersonal level, using methodology and typologies from Kress and van Leeuwen’s (2006) visual grammar, further developed by Painter, Martin, and Unsworth (2012) for the analysis of picture books. Visual resources used for representing interpersonal relations are connected with the categories of contact, social distance, attitude, pathos, affect, ambience, and graduation. Verbal means of realizing interpersonal relations include, among others, mood structures and attitudinal lexis. Martin and White’s (2005) appraisal theory is applied to classify appraisal resources according to three systems: attitude, engagement and graduation. I have analysed interpersonal text-image relationship in two picture books: The Paper Bag Princess by Munsch (2005) and Snow White in New York by French (1989). The analysis of visual resources at the interpersonal level has demonstrated a predominance of long shots, direct eye contact with a reader is rare with the exception of several close-ups. Front-on images situate characters in an equal position with readers, creating a sense of involvement. Overall, the interactions between the visual and verbal components create a synergetic effect in representing interpersonal relations in picture books.

Keywords

• multimodality
• text-image relationship
• picture book
• visual grammar
• interpersonal metafunction