Comprehension and learning through multimedia : integrative processing of text and illustrations

• Main Author: Roy, Marguerite Claire.
• Other Authors: Frederiksen, Carl H. (advisor)
• Format: Others
• Language: en
• Published: McGill University 2000
• Subjects: Visual education.; Multimedia systems.; Integration (Theory of knowledge)
• Online Access: http://digitool.Library.McGill.CA:80/R/?func=dbin-jump-full&object_id=37826

The comprehension of illustrations and text was studied from a cognitive discourse processing perspective. Typically, learners must construct conceptual knowledge representations that integrate different types of information from diverse sources and modalities (e.g., text and illustrations). Currently, little is known about how such integrative processing works in "multimedia" learning situations. This study focused on the semantic representations that low prior knowledge learners constructed as they read from text and static illustrations presenting multiple types of information (structural, functional, and energy) describing a functional system. Both the text and the illustrations were modified so that structural information would be highlighted over other information types. === Participants were twenty-four undergraduate engineering students who had little prior knowledge of the target domain (the human visual system), but were experienced in learning about functional systems using texts and illustrations. Six students were randomly assigned to each of four presentation conditions: (a) text only, (b) illustrations only, (c) text with controlled access to illustrations, or (d) text with free access to illustrations. === The materials were presented individually in a computer environment which recorded and timed all information accessed. Participants provided on-line interpretations as they read, post-input verbal and visual free recalls of the materials, and responses to integrative comprehension questions. Planned comparisons were used to contrast: (a) the two text with illustrations groups, (b) the combined text with illustrations groups to the text only group, (c) the text with illustrations groups to the illustrations only group, and (d) the processing of information which was privileged (structure) to other information. === The results indicated that the text and illustrations each provided mutually constraining information that functioned together to support comprehension. Illustrations aided the construction and elaboration of mental models by providing an external context that supported more active conceptual processing and integration of information. Text aided both literal and high level comprehension by communicating the meaning of illustrations and signaling what information was important. These results support perspectives on situated learning which emphasize the role that discourse plays in comprehending knowledge in environments involving more than one external information source. Future research is recommended to extend such findings to other populations of learners and materials.