It is easy to envision a classroom that relies on the use of a print textbook and resources that primarily use printed words and visual images to represent meaning. However, print resources are changing in ways that are reflective of the multiple ways, or modes, that are used to communicate within digital contexts. Recall from Chapter 1 that modes of communication encompass all forms of expression, including “Visual Meanings (images, page layouts, screen formats); Audio Meanings (music, sound effects); Gestural Meanings (body language, sensuality); Spatial Meanings (the meanings of environmental spaces, architectural spaces); and Multimodal Meanings” (New London Group, 1996, p. 80). Though children’s literature, especially picture books, rely mainly on print and visual modes (i.e., words combined with pictures), there are growing numbers of children’s books that creatively incorporate audio, gestural, and spatial modes as well. Multimodal texts are capable of drawing on students’ strengths and preparing them for a multimodal society where individuals communicate through audio, gestural, visual, spatial, and print resources, as well as various combinations of these modes.

Each mode has its own capacity to communicate, or potential to make meaning, which is called an affordance (Kress & Jewitt, 2003; Kress & Van Leeuwen, 2001). Basically, this means that each mode communicates the same idea in a different way than any other mode. For example, an individual may communicate a story about a cat by telling the story in words, moving around the room, using sounds, or acting out the story with no words or sounds. Each version might communicate a particular part of the story especially well, while another part may not be communicated as well using that mode. The idea that modes have different affordances, or potential to make meaning, suggests that some modes of communication are better suited for some tasks than others. When modes are integrated, their combination also contributes to an overall meaning that could not be achieved by the use of any one mode on its own. Even within a mode, the materials used or the format of the communicated message can contribute differently to the understanding of the message. For example, a written message carries different meaning if it is written in sand versus carved in stone (Kress & Van Leeuwen, 2001).

The meaning-making potential of a mode also depends on how a society or group of individuals values that particular mode or how that mode is used within that society in different situations and contexts (Kress & Jewitt, 2003). As individuals understand the potential usefulness of a mode of communication within the context of their culture, they can choose the modes that most appropriately express their message. Thinking back to the example of the cat story, not only does a particular mode communicate the story differently, the choice of a mode may be appropriate in some circumstances but not in others. For example, it would be more appropriate for a small child to act out the story while moving around the room and meowing than a college professor teaching an English class!

Discussions such as this related to how humans communicate and value various modes is grounded in a larger field known as social semiotics. Social semiotics essentially explains how humans make sense of the world and communicate with each other through all ways that are socially meaningful, such as by drawing, creating visuals, talking, making gestures, engaging in dance and movement, creating architecture, and singing and making music (Lemke, 1990). Additionally, as societies or cultural groups adapt over time, they place different value on various modes people use to make meaning. Schools are important in shaping the value placed on different modes in a society; their overemphasis on reading and written language systems marginalizes other valuable forms of expression and, likewise, students who have talents and abilities in these other forms (Eisner, 1991). Eisner explained that being able to understand messages communicated through multiple modes is central to three important educational aims, including increasing the variety and depth of meaning people make in their lives, developing cognitive potential, and providing educational equity in our schools.

When teachers and students begin to understand the potential of each mode, more options become available to understand and create meaning. Students make choices on a daily basis as far as the mode used to communicate, as well as the medium or format of a message. For example, students choose a medium when they decide to send an email or a text message, share a tweet, a picture, or a song, or create a video. The medium chosen often dictates the format of the message—a text might use shorthand or emoticons while an email would use full words. If they understand the potential of each of the modes, they can make choices to create and understand messages more fully. Authors and publishers of children’s literature are also aware of these choices, and the literature they produce is certainly influenced by the knowledge that students’ communication preferences are both flexible and dynamic.

Teachers can facilitate learning in the classroom that allows all of the above to be possible, such as students having knowledge of modes to make the best choices to express their messages, students with talents and abilities in areas beyond print and linguistic forms to have a valued mode to express themselves in the classroom, and students having the ability to fully understand the messages that are communicated through various modes and their combinations throughout society. An understanding of how modes work together in texts is thus necessary for those preparing to enter the teaching profession. Children’s book authors and illustrators are able to offer more multimodal experiences for readers that extend beyond the combination of print and visual modes to include audio, gestural, and spatial modes. As multimodal texts are viewed, readers make meaning by experiencing integrated and cohesive texts that draw on the potential of multiple modes of meaning. Teachers must therefore understand how modes work together within texts in order to prepare students to understand and make meaning with a wider variety of texts and communicate through a wider variety of modes.