Expertise reversal for iconic representations in sciencevisualizations

Bruce D. Homer Æ Jan L. Plass

Received: 12 January 2009 / Accepted: 3 August 2009� Springer Science+Business Media B.V. 2009

Abstract The influence of prior knowledge and cognitive development on the effec-

tiveness of iconic representations in science visualizations was examined. Middle and high

school students (N = 186) were given narrated visualizations of two chemistry topics:

Kinetic Molecular Theory (Day 1) and Ideal Gas Laws (Day 2). For half of the visual-

izations, iconic representations of key information were added. Results indicated a main

effect of prior knowledge on learning in Day 1. In Day 2, a three-way interaction was

found between prior knowledge, age group and icons: icons were effective for all middle

school students and for high school students with low prior knowledge, but were not

effective for high school students with high prior knowledge. These findings indicate that

the expertise reversal effect can be mediated by cognitive development and other factors,

not just domain specific prior knowledge.

Keywords Simulation � Expertise reversal � Science visualization �Multimedia learning �Cognitive development � Cognitive load

Introduction

How should science visualizations be designed to support learners of different ages and

different levels of prior knowledge? For the purpose of this paper, we define science

visualizations as visual representations of models that describe natural phenomena.

Examples of such representations are charts, graphs, maps, and pictures that can be pre-

sented as static images, animations, or simulations. Whereas static images are viewed by

B. D. Homer (&)Program in Educational Psychology, The Graduate Center, CUNY, 365 Fifth Avenue,New York, NY 10016-4309, USAe-mail: BHomer@gc.cuny.edu

J. L. PlassNew York University, New York, NY, USA

123

Instr SciDOI 10.1007/s11251-009-9108-7

the learner with little interaction and mostly do not visualize the dynamic nature of the

content, animations depict dynamic phenomena but typically only afford the learners

control over the pacing of the presentation. Simulations, on the other hand, allow learners

to explore these phenomena and their dynamic properties through the manipulation of

parameters of the model (Lee et al. 2006). Research has shown that science visualizations

are most effective when they are designed under consideration of the specific cognitive and

affective needs of the target audience (Plass et al. 2009a; Rieber 2005). This applies to the

information design, i.e., the type and format of the representation of the information chosen

by the designer, and the interaction design of these materials, i.e., the types of learner

physical and cognitive engagement they afford.

The existing research on learning from science visualizations has been investigating

information design features such as how to highlight important information through cueing

(de Koning et al. 2007; Lowe 2003) and color coding (Keller et al. 2006), and how to

dynamically link and integrate multiple representations with one another (van der Meij and

de Jong 2006). Research on interaction design features of dynamic visualizations has

focused on the level of learner control (Hasler et al. 2007; Niemiec et al. 1996; Swaak and

de Jong 2001) and the kind of content manipulation that best facilitate learning (Bodemer

et al. 2004; Bodemer et al. 2005; Plass et al. 2007; Rieber 1991). It has also investigated

what kind of guidance should be provided to learners (de Jong 2005; de Jong and van

Joolingen 1998) and what kind of feedback they should receive during the simulation

exploration (Moreno 2004; Moreno and Mayer 2005; Rieber et al. 1996; Rieber 1996;

Rieber et al. 2004).

Research has also identified prior knowledge as a significant determinant of the

effectiveness of educational interventions in general (Kalyuga 2005), and as a predictor of

the impact of specific design elements of educational simulations in particular (Plass et al.

2009b). Rather than revealing a simple linear relationship of prior knowledge and learning

outcomes, this work has shown that learners with different levels of prior knowledge

require different designs of educational materials, and that materials that are effective for

learners with a low level of prior knowledge can hinder learners with high prior knowl-

edge, and vice versa (Kalyuga et al. 2003; Kalyuga 2005).

The present study focuses on the use of iconic representations as one specific aspect of

the information design of science visualizations and investigated differences in the

effectiveness of iconic representations for middle school v. high school learners and

learners with different levels of prior knowledge in the subject matter. Below we will first

summarize findings from previous research on the information representation in visual

materials for science education and then discuss how the expertise reversal effect can be

applied to information design considerations such as the use of iconic representations in

science visualizations.

Representation of information

In order to improve the design of visual materials for science education, we have been

studying the use of icons to represent key information in science visualizations. Consider

the example of a simulation of the Ideal Gas Laws shown in Fig. 1. The gas laws describe

the relationship of temperature, volume, and pressure for an ideal gas. The version of the

simulation presented in Fig. 1 uses icons to represent these gas properties, which are the

key information in the simulation, by using burners to represent temperature, and weights

for pressure.

B. D. Homer, J. L. Plass

123

These icons were included to help novice learners comprehend the visual representa-

tions in the simulation by compensating for their lack of domain-specific prior knowledge.

Previous research has shown that the addition of icons in science simulations can reduce

cognitive load, especially for learners with low prior knowledge (Lee et al. 2006).

The distinction between iconic and symbolic representations is based on Peirce’s (1955)

classification of signs (representations) that increase in complexity and abstraction. Peirce

defines icons as the most basic type of signs that relies on physical resemblance to convey

meaning. Indices are signs that acquire their meaning through temporal or spatial prox-

imity between the sign and its referent. In contrast, symbols are abstract, arbitrary signs that

rely on social convention for meaning (Bertin 1983). Based on this semiotic approach, the

Integrative Theory of Text and Picture Comprehension distinguishes descriptive repre-

sentations (symbols) and depictive representations (icons) as the two major types of

knowledge representation in educational materials (Schnotz and Bannert 2003; Schnotz

2005).

There is evidence that a learner’s developmental state affects how a sign is interpreted,

and that symbolic understanding progresses from iconic to symbolic with development and

learning. For example, when young children first start using language, words are indexical

in nature and intimately tied to their immediate environment (Nelson 1996). For a young

child, the word ‘‘cat’’ refers not to some abstract concept, but rather to the orange tabby

that lives next door. As children have more experience with hearing the word cat at

different times and in different contexts, they come to acquire the more abstract meaning of

the word used by adults in their culture. This means that language only gradually becomes

fully symbolic, and that a true understanding of symbols is not possible without a certain

level of experience and prior knowledge (Homer and Nelson 2005). Deacon’s (1997) has

argued that, regardless of age, learners progress through the trajectory from iconic to

Fig. 1 Ideal Gas Law simulation with iconic representations for key semantic information about pressure(represented by weights) and temperature (represented by flames)

Expertise reversal for icon effectiveness

123

symbolic understanding of signs whenever they acquire symbolic representations in a new

subject matter domain.

Applied to the design of educational science visualizations, this suggests that the

addition of icons can be beneficial for learners with low prior knowledge in the subject

matter area because they may be more easily understood than symbolic representations that

require knowledge of the field to be comprehended. On the other hand, it could also be

argued that adding icons to a display that already presents information in symbolic form

(e.g., adding the burner and weight icons to the simulation that is already labeled with the

text symbols temperature and pressure, respectively, see Fig. 1) is redundant and would

therefore increase the visual complexity of the display and hinder learning.

Existing research on this question for dynamic visualizations has found that adding

iconic representations of the key information in a science simulation can improve learning

(Lee et al. 2006). Similar effects have been found in a study of different representation

types for feedback in simulations, which has been found to be more effective when it is

provided in graphical rather than textual form (Rieber 1996; Rieber et al. 1996, 2004). This

finding is consistent with research on constructing mental models that showed that written

and pictorial instructions were equally effective for building simple molecular models, but

that for the construction of complex molecules the iconic representations (pictorial

directions) were more effective than symbolic representations (written directions) (Carlson

et al. 2003). An open question is whether the effect of icons on comprehension only applies

to science simulations or whether it can also be expanded to static images with narrations.

Further research on the use of iconic representations indicated that the type of sign

chosen by the designer of visual environments to represent key information impacts out-

comes for learners with different levels of prior knowledge. For example, in a series of two

studies with 171 participants, high school students used a simulation to study KineticMolecular Theory. Students either received a simulation with icons, or a version where key

information was displayed using symbolic representations only (Plass et al. 2009a, b). In

addition, complexity of the simulation was manipulated by adding or removing an auto-

matically updating chart with the simulation data. Results indicated that adding iconic

representations to the chemistry simulations was more effective for learners with low prior

knowledge than for those with high prior knowledge, especially when the complexity of

the materials was high (i.e., the updating chart was shown). This effect is referred to in the

literature as expertise reversal.

Expertise reversal

One of the most solid findings in recent educational psychology research is the expertise

reversal effect (Kalyuga 2007). This effect describes the fact that prior knowledge mod-

erates the effectiveness of instructional materials such that interventions designed for

learners with low prior knowledge may be ineffective or even have negative effects for

learners with high prior knowledge in the subject matter, and vice versa (Kalyuga 2007;

Kalyuga et al. 2003). The effect, grounded in memory research that suggests that working

memory capacity is limited (Miller 1956), is a specialized version of general cognitive

science research on expert-novice differences (e.g., Chase and Simon 1973; De Groot

1965).

Learners without prior knowledge in a particular domain have to process large amounts

of new information, which may easily exceed their available memory capacity (Kalyuga

2007). In response to this high cognitive demand, instructional designers add specific

B. D. Homer, J. L. Plass

123

design elements that support these learners in the process of knowledge construction.

Experts, on the other hand, as the result of their experience with the subject matter, have a

large number of domain-specific knowledge structures in long-term memory. The orga-

nized long-term memory knowledge structures, which are described as prior knowledge,

can reduce working memory limitations of instructional materials by chunking bits of

information together into a single, higher-level element (Chi et al. 1982). This allows

subject-matter experts to skip steps during problem solving and not require additional

learning support and guidance (Kalyuga 2007). As a result, research on the expertise

reversal effect has found that instructional design techniques that are effective with less

knowledgeable learners can be less effective or even inhibiting when used by more

experienced learners, and vice versa (Kalyuga 2006). For example, if guidance for low

prior knowledge learners is included in the instruction, then high prior knowledge learners,

who do not need this guidance, have to process what for them is redundant information. If

materials are designed for high prior knowledge learners, and do not include such guid-

ance, then low prior knowledge learners experience this material as too difficult, i.e., they

experience a high cognitive load and will not be able to process all information and

construct corresponding mental models.

Applied to visual displays of information, the effectiveness of specific information

design features, and their working memory requirements, depends on specific character-

istics of learners, in particular, on their prior knowledge (e.g., Kalyuga 2006, 2007; Lee

et al. 2006; Mayer and Sims 1994; Plass et al. 2003). For example, research on the effect of

learner characteristics on learning from text and visual displays has shown that the

advantages of diagrams depend on student domain-specific knowledge and experience

(e.g., Hegarty and Just 1993; Kalyuga 2007; Lowe 1993; Schnotz et al. 1993; Schnotz and

Rasch (2005). Similar expertise reversal effects for dynamic visual representations such as

simulations have been found for labeled simulations and diagrams (Ollerenshaw et al.

1997); see Kalyuga (2007) for a detailed review.

Learners with low prior knowledge may have difficulty processing visual information

because their lack of background knowledge makes it difficult for them to interpret the

visual information with their limited available working memory resources. In related

research with science simulations, we have found an expertise reversal effect for icons:

Adding iconic representations to a science simulation is effective for learners with low

prior knowledge, but can lead to reduced learning for learners with high prior knowledge

(Plass et al. 2009a, b). A high mental load is required for novice learners to interpret the

meaning of symbolic representations because symbols implicitly assume that learners have

existing knowledge in the domain. Once learners have acquired sufficient knowledge in a

particular domain, the working memory load required for the processing of this infor-

mation is reduced, allowing for effective learning from more abstract, symbolic repre-

sentations by tapping into relevant schematic representations that the learners already hold

in their long-term memory.

The present study

In our previous work, we have examined the effects of adding iconic representations to

science simulations. Simulations require learners to manipulate one or more parameters in

the model, which can add to the complexity of a learning situation, creating a higher

cognitive load, particularly for inexperienced learners. We have found that the addition of

iconic representations to simulations can enhance learning for inexperienced learners, but

Expertise reversal for icon effectiveness

123

may hinder learning for more experienced learners, creating an expertise reversal effect

(Lee et al. 2006; Plass et al. 2009a, b). In other words, in high-load situations, redundant

iconic representations help learners with lower prior knowledge but can hinder learning for

learners with higher prior knowledge.

The current study investigated the effects of adding iconic representations to scientific

visualizations that induce a lower cognitive load than simulations. Specifically, icons were

added to narrated visualizations, in which learners were presented with still images of the

optimal steps for exploring a particular scientific phenomenon. Applying the modality

effect, the explanation that accompanied the images was presented as a narration rather

than as on-screen text in order to further reduce cognitive load (Low and Sweller 2005).

Although there is some debate about whether this more direct instruction is preferable to

the more exploratory learning of simulations, this was not the question being addressed.

Instead, we wanted to find out whether the expertise reversal for iconic representations that

has been found for simulations would also be found for narrated visualizations as an

example of materials inducing lower cognitive load. The interactive nature of simulations

requires a certain amount of cognitive resources, which adds to the overall cognitive load

of learning with simulations and may hinder learning, particularly for learners with low

prior knowledge. The limited interactivity of narrated visualizations reduces cognitive load

and may benefit learners, particularly learners with low prior knowledge. This means that

learners with the lowest prior knowledge, who may not be able to learn from interactive

simulations, may be able to learn from narrated visualizations. This may then increase the

expertise reversal effect of icons, particularly for low prior knowledge learners.

A second issue addressed in the current study was whether or not developmental factors

influence the expertise reversal for iconic representations. Previous research has not sep-

arated developmental factors from a prior knowledge, however, younger learners, even if

they have high levels of specific content knowledge, may lack more general symbolic

competencies or learning skills that would make the addition of iconic representations

desirable. For example, executive functions, which allow learners to plan, monitor and

evaluate their behaviors, peak around age 14 (Riek 2007). This suggests that there should

be differences in learning that are independent of prior knowledge for students who are

younger than 14 compared to students who are older than 14, particularly in situations

requiring self regulated learning. Therefore, the present study included children who were

at different developmental levels, i.e., middle school children as well as high school

children.

A final way in which the current study extends previous research is by examining the

effects of multiple exposures to similar scientific visualizations. Previous studies have

involved a single exposure to one simulation. However, the representation of information

employed in scientific visualizations, such as the ones used for this research, constitutes a

kind of ‘visual language’ in itself. If used in a one-time exposure, we would expect

different levels of prior knowledge both in this visual language and in the subject matter.

Using two visualizations of two different topics with a consistent visual language, where

the first visualization represents a pre-training in the comprehension of this visual lan-

guage, allowed us to study the effect of differences in learners’ prior knowledge in the

subject matter, not in the visual language itself.

In the current study, students from two age groups (middle school and high school) were

presented with two narrated chemistry visualizations on two separate days. On Day 1, the

students received a visualization of Kinetic Molecular Theory (based on Plass et al. 2009a, b),

and on Day 2, the students received a visualization of the Ideal Gas Laws (based on Lee et al.

2006). Both visualizations used a similar interface and presented key information in the same

B. D. Homer, J. L. Plass

123

manner. The students were given these computer-based materials during their regular

chemistry class, with each session lasting for one class period. For both topics, a test of prior

knowledge was given at the beginning of the class so that students’ expertise in the specific

subject matter could be assessed. After a brief introduction, the students were allowed to

interact with the narrated visualization at their own pace. All students were then given a

posttest that assessed their learning at three levels: Comprehension, which is understanding of

key factual information that was presented in the visualizations; Application, which is the

ability to apply key concepts presented in the visualizations; and Transfer, which is the ability

to use what was learned in the visualizations to make sense of novel situations. Although in

general, greater expertise reversal effects are expected for higher-levels of learning, this is

susceptible to floor effects. In other words, if the students are showing little evidence of

learning at the transfer level, then no expertise reversal effect would be possible at this level of

learning. In our previous work, expertise reversal effects have been found for comprehension,

but not for transfer (Lee et al. 2006; Plass et al. 2009a, b).

Based on our prior work, we predicted an expertise reversal effect for icons: we

hypothesized that the addition of iconic representations would enhance learning for the

learners with low prior knowledge, but have little effect or perhaps even hinder learning for

learners with high prior knowledge. Because the narrated visualization on Day 1 and Day 2

had a similar visual design and interface, we predicted that the students would find it easier

to use the visualization on Day 2, which would create a greater expertise reversal effect on

Day 2 than on Day 1. In other words, during Day 1, the added icons may help provide a

context for the students–including students with high prior knowledge–make sense of the

visualization, which means that the expertise reversal effect would be smaller for Day 1.

We also hypothesized that the expertise reversal effect would be reduced or eliminated for

the younger children who may lack more general symbolic competencies, even if they have

content knowledge. In other words, we predicted that the icons would help the younger

children, regardless of their prior knowledge. Finally, we hypothesized that because overall

the students had low levels of prior knowledge about chemistry, most of the learning that

occurred would be at the level of comprehension and, therefore, this is where the effects

would primarily be found.

Method

Participants and design

The participants for this study were students from a public high school (N = 159;

approximately 41% female) and a public middle school (N = 27; approximately 61%

female) in rural Texas. The high school is a large, central school to which several smaller

middle schools, including the middle school in our current study, provide students. A

majority of the students were of Hispanic decent (88%) with the remaining students being

White (non-Hispanic) (10%), African-American (1%) or ‘‘other’’ (1%). The middle school

students ranged from 10–13 years of age (M = 12) and high school students from 16–

18 years of age (M = 17) and had not studied any materials related to the content of the

study materials (i.e., kinetic molecular theory and ideal gas laws).

The study involved two factors. The Icon factor (SYMB, ICON) described whether

participants received treatments with or without iconic representations. The Developmentfactor (MS, HS) described whether participants were at an earlier developmental stage (i.e.,

attending the middle school) or a later developmental stage (i.e., attending high school).

Expertise reversal for icon effectiveness

123

For each level of the Development factor, participants were randomly assigned to one of

the two levels of the Icon factor–narrated visualization with symbolic representations only,

i.e., without icons (MS-SYMB, HS-SYMB), or narrated visualization with added icons for

key information (MS-ICON, HS-ICON).

Materials

For each participant, the paper-and-pencil materials consisted of a demographic infor-

mation questionnaire. The demographic information questionnaire elicited participants’

gender, grade level, number of chemistry classes taken, and level of comfort using

computers.

The computer-based instructional materials included narrated visualizations that were

based on two different but related simulations, both developed using Macromedia Flash

MX 2004 (Macromedia 2004), delivered on a web page, and viewed by learners on desktop

PCs. The first topic was the Kinetic Molecular Theory of Heat (KMT), which showed a

visual display of a cylinder containing gas molecules. Pressure was represented through the

text symbol pressure and the numeric value of the current pressure. Sliders showed that the

temperature of the gas and the number of gas particles was variable. Temperature was

represented by the text symbol temperature and the numeric value of the current tem-

perature, and the number of particles by the letter N and the numeric value. In the symbolicversion of the visualization, essential information was presented in symbolic format (e.g.,

only letters and numbers were given to indicate pressure and temperature), while in the

icon version, iconic representations were added to represent the same essential informa-

tion: burners were added as iconic representations of the temperature, and a pressure gauge

as iconic representation of pressure. Figures 2 and 3 are screenshots from the KMT

materials with and without icons.

A second visualization covered the topic of the Ideal Gas Laws (IGL), which describes

the interrelationship between temperature, pressure, and volume of an ideal gas, i.e., a gas

in which all collisions between atoms or molecules are perfectly elastic and in which there

are no attractive forces between molecules. The IGL materials were designed using

interface elements consistent with the design of the KMT materials to avoid usability

problems, see Figs. 4 and 5.

For both visualizations, an experienced science educator designed a sequence of steps of

manipulating the gas properties that constituted a meaningful exploration of each topic.

The steps were narrated by a professional speaker, based on a script provided by the

science educator, and included an introduction and explanation of the various elements of

the visualization (e.g., the pressure gauge). All materials were reviewed by additional

science educators and scientists and revised for scientific accuracy and optimal exploration

strategy.

For each of the two topics, knowledge pre-tests were administered before the treatment.

The pre-test for the KMT consisted of seven multiple-choice items and the pre-test for the

IGL of eight multiple-choice items. A sample item from the pre-test for the IGL was, ‘‘If

the pressure of a gas is maintained at the same level, how does the volume of a gas sample

change if you increase its temperature?’’ Learners received one point for each correct

answer, and the two pre-test scores were computed as the sum of all points received in each

of the respective tests.

The learning outcome measures for each of the two topics consisted of a comprehension

test, a knowledge application test, and a transfer test. The comprehension tests were

B. D. Homer, J. L. Plass

123

Fig. 2 Kinetic Molecular Theory narrated visualization without iconic representations

Fig. 3 Kinetic Molecular Theory narrated visualization with iconic representations

Expertise reversal for icon effectiveness

123

comprised of multiple-choice items that assessed learners’ understanding of key concepts

presented in the visualization (e.g., for KMT, ‘‘Which of the following images accurately

shows an increase in the number of particles?’’). The comprehension test for the KMT

consisted of 17 items; the comprehension test for the IGL of 16 items. Students received

one point for each correct answer, and the two comprehension test scores were computed

as the sum of all points received in each of the respective tests.

The knowledge application test consisted of multiple-choice items that were designed to

measure whether learners could apply the concept covered by the visualizations, for

example, ‘‘If the temperature is held constant, what change in the volume of a gas

Fig. 4 Ideal Gas Law narrated visualization without iconic representations

Fig. 5 Ideal Gas Law narrated visualization with iconic representations

B. D. Homer, J. L. Plass

123

container will decrease the gas pressure from 10 atm to 5 atm?’’ For the KMT, this test

consisted of 11 items, and for the IGL of 9 items. Students received one point for each

correct answer, and the two knowledge application test scores were computed as the sum of

all points received in each of the respective tests.

The transfer tests employed open-ended items in which students were asked to write

possible solutions to problems that required them to make inferences based on their mental

model of the KMT and IGL, respectively. For example, and item from the IGL transfer test

was, ‘‘On a very hot summer day, your friend realizes that he had left an aerosol can (e.g.,

air freshener) in his car. It was exposed to the sun and it now very hot. Describe as many

ways as you can think of how to prevent the aerosol from exploding. Explain your answer

using the Gas Laws.’’ For the KMT, this test consisted of 5 items, and for the IGL of 6

items. All items were scored by two independent raters, and scoring disagreements were

resolved through discussion. Students received up to three points for each correct answer,

and the two knowledge transfer test scores were computed as the sum of all points received

in each of the respective tests. Although there was no limit on how many answers could be

generated, the maximum score for this measure did not exceed 15.5 (KMT) and 17 (IGL).

Procedure

Participants were tested in groups of approximately 20–30 students in grade 7 (middle

school) and grades 11 and 12 (high school). In both schools, the study took place in the

science classroom, where each participant worked on an individual computer equipped

with headphones.

On Day 1, participants were provided with an overview of the study. Once consent to

participate had been obtained, participants completed the demographics questionnaire on

paper and the KMT knowledge pre-test on the computer. Participants were then randomly

assigned to one of the two treatment conditions (i.e., iconic vs. symbolic) and spent

approximately 20 min learning about the kinetic molecular theory using the narrated

visualizations. Finally, participants completed the comprehension post-test, knowledge

application post-test, and transfer knowledge post-test for KMT. All students completed

the procedure for Day 1 within a 50 min class period.

On Day 2, participants first completed the IGL knowledge pre-test on the computer

and then began learning from the narrated visualizations for the Ideal Gas Laws. They

were randomly assigned to one of the two treatment conditions (i.e., iconic vs. symbolic)

and spent approximately 20 min learning about the IGL. This means that participants

who were assigned to one group (symbolic or iconic) on Day 1 were either assigned to

the same or a different group on Day 2, resulting in groups who received the same

treatment on Days 1 and 2 (iconic or symbolic) and groups who received different

treatments (iconic then symbolic and symbolic then iconic). After the treatment, par-

ticipants completed the comprehension post-test, knowledge application post-test, and

transfer knowledge post-test for IGL. All students completed the procedure for Day 2

within a 50 min class period.

Results

The data were initially screen to remove any outliers in posttest scores (i.e., any score

that was more than 2 standard deviations from the mean). In addition, participants’ time

spent engaging with the narrated visualization was examined to remove data from any

Expertise reversal for icon effectiveness

123

participants who had spent 0 time interacting with the materials and, therefore, did not

receive any treatment. The screening procedures resulted in data from 8 high school

students being omitted in Day 1, and data from 8 high school students and one middle

school student being omitted in Day 2. Mean scores and standard deviations are reported

in Table 1.

T-tests were then conducted to determine if the middle school and high school students

differed significantly on either prior knowledge (as indicated by the KMT and IGL pretest

scores) or previous chemistry experience (as indicated by number of prior chemistry

courses). The results indicated no significant differences between the two age groups on

either Day 1 pretest scores, t (157) = .455, n.s., or Day 2 pretest scores, t (157) = .16, n.s.

The two age groups also did not differ in the number of previous chemistry courses that

they had taken, t (99) = 1.22, n.s. In both age groups, students had taken an average of one

previous course in chemistry. This suggests that developmental factors rather than level of

domain specific prior knowledge or experience in chemistry would explain any differences

found between the two age groups.

Two sets of analyses were conducted, one for each day of the intervention. For both Day 1

and Day 2 data, a separate 2 (age group: middle school vs. high school) 9 2 (representational

format: added icons or no icons) analysis of covariance (ANCOVA) with prior knowledge

(i.e., score on the pretest for that day) entered as a covariate was conducted for each level of

learning (i.e., recall, comprehension and transfer). The model tested in the ANCOVAs

included main effects for each factor, two-way interactions between prior knowledge 9 age

group and prior knowledge 9 icon, and the three way interaction (prior knowledge 9 age

group 9 icon). In order to be included in the analyses, students had to have completed at least

75% of the posttest (i.e., have fewer than 25% of the questions unanswered). This resulted in a

sample size (and df) that varied slightly for each of the analyses.

For Day 1 Recall, only prior knowledge was found to have a significant effect,

F (1, 149) = 17.50, p \ .001, gp2 = .11. For comprehension, a significant effect was found

only for Pretest score, F (1, 149) = 17.50, p \ .001, gp2 = .11. Score on the pretest

Table 1 Pretest and post tests (recall, comprehension and transfer) mean scores (and SD) for Day 1 andDay 2

Pretest Recall Comprehension Transfer

Day1

Middle school 4.3 (0.9) 18.3 (1.7) 8.8 (1.7) 6.9 (3.6)

Icon

Symbolic 3.5 (1.4) 14.3 (3.5) 9.0 (1.1) 5.2 (2.5)

High school 2.7 (1.5) 15.7 (2.5) 8.3 (1.5) 6.8 (3.8)

Icon

Symbolic 3.9 (0.7) 15.7 (3.0) 7.3 (2.9) 6.5 (5.6)

Day 2

Middle school 5.4 (0.7) 12.6 (2.4) 5.2 (1.2) 8.4 (5.6)

Icon

Symbolic 5.0 (.6) 11.8 (2.0) 4.2 (1.7) 4.8 (1.8)

High school 5.3 (0.6) 10.7 (7.5) 5.3 (2.5) 5.5 (4.5)

Icon

Symbolic 5.0 (1.8) 11.0 (3.4) 4.9 (2.8) 6.6 (3.6)

B. D. Homer, J. L. Plass

123

was positively and significantly correlated with score on the comprehension test,

R2 (157) = .13, p \ .001. None of the other factors or interactions were significant for

recall. For Day 1 knowledge application, once again, only pretest score was found to be

significant, F (7, 151) = 10.99, p = .001, gp2 = .07. Score on the pretest was positively

and significantly correlated with score on the knowledge application test, R2 (159) = .16,

p \ .001. None of the other main effects or interactions for comprehension were found to

be significant. Finally, for Day 1 knowledge transfer, none of the individual factors were

found to be significant.

For Day 2 comprehension, a significant main effect of pretest scores was found, F (1,

139) = 6.76, p = .01, gp2 = .05. Score on the pretest was positively and significantly

correlated with score on the comprehension test, R2 (157) = .43, p \ .001. There was also

a significant 3-way interaction between prior knowledge, age group and representational

format, F (1, 139) = 5.16, p = .006, gp2 = .03. This 3-way interaction was further

explored by conducting separate ANCOVAs for the middle school and high school chil-

dren. These data are presented in Figs. 6 and 7. For the middle school children, the

interaction between icon and pretest scores was significant, F (1, 18) = 6.89, p = .017,

gp2 = .28. As illustrated in Fig. 6, at all levels of prior knowledge, students in the icon

group did better than those in the symbolic group, and this difference increased with prior

knowledge. In other words, the slope for the icon group was steeper than for the symbolic

group. For the high school students, prior knowledge (as indicated by score on the pretest)

was a significant factor, F (1, 89) = 21.12, p \ .001, gp2 = .19. The interaction between

icon and prior knowledge was marginally significant, F (1, 89) = 3.11, p = .08. As

Fig. 6 The effects of added icons on the relation between prior knowledge and posttest comprehensionscore for middle school students

Expertise reversal for icon effectiveness

123

illustrated in Fig. 7, children with lower prior knowledge had better comprehension in the

icon condition, while children with higher prior knowledge had better comprehension in

the symbolic condition.

For Day 2 knowledge application, only pretest score was significant, F (1, 138) = 7.84,

p = .006. Score on the pretest was positively and significantly correlated with score on the Day

2 application test, R2 (158) = .36, p \ .001. None of the other factors or interactions in the

model were significant. Finally, none of the factors were found to be significant for transfer.

Discussion and conclusion

The goal of this research was to study whether the effectiveness of iconic representations in

narrated science visualizations would show similar patterns as we have found in our

previous research on the effectiveness of iconic representations in science simulations. In

these previous studies, there was an overall positive effect for icons under conditions of

high cognitive load, and we were able to show that iconic representations are especially

beneficial for inexperienced learners (Lee et al. 2006; Plass et al. 2007; 2009a, b).

The present research extended these findings in three important ways: First, in order to

study the effectiveness of iconic representation in materials that induced less cognitive load

than computer-based simulations, we devised narrated static visualizations of the same

content that did not require learners to explore the simulation on their own, but did allow

them to control the pacing of the presentation. Second, we investigated if learners’

Fig. 7 The effects of added icons on the relation between prior knowledge and posttest comprehensionscore for high school students

B. D. Homer, J. L. Plass

123

cognitive development affected the way in which level of prior knowledge influenced the

effectiveness of iconic representations. To that end, we included learners from two

developmental groups in this study: middle school students (grade 7) and high school

students (grades 11 and 12). Finally, instead of administering one brief intervention of a

single exposure to the learning material, we devised a 2 day sequence in which learners

were given two different visualizations for two separate, but related topics.

Learning outcomes were assessed on three different levels: comprehension, knowledge

application, and knowledge transfer. Our results indicated that during Day 2 of testing,

students’ level of cognitive development did affect the way in which iconic representations

interacted with prior knowledge to affect learning outcomes. Whereas the high school

students with lower levels of prior knowledge tended to have better comprehension with

the visualization that included added iconic representations, the high school students with

higher levels of prior knowledge tended to have better comprehension with the visuali-

zation without the added icons. Although this interaction was only marginally significant

(p = .08), it suggests an expertise reversal effect for icons in narrated visualizations

similar to what we have found in our previous work with simulations. In contrast, the

middle school students tended to have better comprehension outcomes in the Day 2 nar-

rated visualizations with added icons, regardless of their level of prior knowledge. There

was a significant icon by prior knowledge interaction for the middle school students,

indicating that the students with higher levels of prior knowledge were showing even

greater benefits of icons than were the students with lower levels of prior knowledge, who

showed much less of a difference between their learning outcomes with and without the

icons (see Fig. 6). In their research on students’ ability to benefit from hyperlinks added to

learning materials, Seufert et al. (2006) found that the effectiveness of the help provided by

the hyperlinks was dependent upon the prior knowledge of the learners: Whereas learners

with low levels of prior knowledge did not have sufficient knowledge to benefit from the

help, and learners high levels of prior knowledge did not need the added help, it was

learners whose prior knowledge fell in the middle who demonstrated the greatest benefit of

added help. Similarly, in the current study, the middle school students with low prior

knowledge may not have had enough prior knowledge of chemistry and related concepts to

benefit from the help provided by the icons in the narrated visualizations.

It is interesting to note that these effects were found only for the Day 2 narrated

visualizations. In our previous work with simulations, we found that added icons were only

beneficial in comparatively more complex learning environments, with little or no effect of

icons in less complex learning environment (Plass et al. 2009a, b). In the current study, the

Day 1 narrated animation, which involved 2 variables, was less complex than the Day 2

animation, which involved 3 variables. Because there was a logical sequence for intro-

ducing the concepts presented in the animations, order of animation was not counter-

balanced. This makes it difficult to make claims about why the interactions were found in

Day 2 but not Day 1 since order and complexity were confounded. Nonetheless, it seems

more likely that the complexity of the learning environment is responsible for the dif-

ference, rather than task order. However, this should be explored further in future research.

This study has important theoretical and practical implications. On the theoretical side,

perhaps the most intriguing finding is that age mediates the expertise reversal effect.

Although some caution is warranted in interpreting this finding due to the differences in

size between the smaller middle school sample and larger (and presumably more hetero-

geneous) high school sample, the finding is nonetheless intriguing. It suggests that factors

other than domain specific prior knowledge create the ‘‘expertise’’ that affects learning.

Expertise reversal for icon effectiveness

123

Although the current study found age to interact with the expertise reversal effect,

further research is needed to identify exactly what factors are changing with age to cause

this effect. No differences were found between the middle school and high school students’

scores on the pretests, which assessed understanding of specific topics in chemistry (i.e.,

KMT and IGL). This finding was interpreted as indicating that there was no difference

between the age groups in their knowledge of chemistry. However, it is likely that even

though the high school students did not have greater knowledge of the specific topics

assessed in the pretest, they did have greater knowledge of other aspects of chemistry or

greater general scientific knowledge that may have facilitated learning. Additional research

is needed to determine if more general chemistry or scientific knowledge is what is causing

age to mediate the expertise reversal effect. If both general chemistry and scientific

knowledge can be ruled out, then further research is warranted to identify other cognitive

changes occurring between middle school and high school that are responsible for the

influence of age on the expertise reversal effect.

A likely factor is metacognitive skills, which undergo critical developmental changes

between middle school and high school and create important differences in how children

interact with learning environments. There is considerable evidence that metacognitive

development influences learning (Hacker et al. 1998), and recent research has found stu-

dents’ metacognitive skills to be better than general intellectual ability in predicting

learning outcomes from interacting with computer based learning environments (Prins

et al. 2006). General symbolic competencies also increase with age and may be influencing

how the students in the present study were learning from the narrated animations. As

children get older, they become more flexible in their ability to use and interpret signs

(Homer and Hayward 2008). This may allow the older children in the current study to more

quickly grasp the meaning of the visual representations used in the narrated visualizations,

even without the added icons. Related to this, there may have been differences in

knowledge between the middle school and high school students in other domains that

affected learning. For example, compared to the middle school students, the older students,

particularly those who did well on the chemistry pretests, most likely had more knowledge

of mathematics, which may have enabled them to better understand the narrated animation.

Further research is needed to determine how factors such as metacognitive skills, symbolic

abilities, and knowledge in domains such as mathematics influence learning in dynamic

and static science visualizations.

On the practical side, our research has provided designers of scientific visualizations

with an important principle for their information design, suggesting that the use of icons to

represent key information is useful when learners have low prior knowledge or have not

yet fully developed their symbolic competencies but that icons should be faded for learners

who have prior knowledge in the subject matter or have reached a developmental stage in

which these representations are no longer necessary.

Acknowledgments The authors would like to thank the students and teachers who participated in thisresearch. We would also like to express our gratitude to Juan Barrientos, Yoo Kyung Chang, and MinchiKim for their assistance with data collection and coding. The research presented in this paper was supportedin part by the Institute of Education Sciences (IES), US Department of Education (DoEd) through GrantR305K050140 to Jan L. Plass, Catherine Milne, and Trace Jordan at New York University and Bruce D.Home at The Graduate Center, CUNY. The content of this publication does not necessarily reflect the viewsor policies of IES or DoEd, nor does any mention of trade names, commercial products, or organizationsimply endorsement by the US Government.

B. D. Homer, J. L. Plass

123

References

Bertin, J. (1983). Semiology of graphics: Diagrams, networks, maps. Madison, WI: University of WisconsinPress.

Bodemer, D., Ploetzner, R., Bruchmuller, K., & Hacker, S. (2005). Supporting learning with interactivemultimedia through active integration of representations. Instructional Science, 33, 73–95.

Bodemer, D., Ploetzner, R., Feuerlein, I., & Spada, H. (2004). The active integration of information duringlearning with dynamic and interactive visualizations. Learning and Instruction, 14, 325–341.

Carlson, R., Chandler, P., & Sweller, J. (2003). Learning and understanding science instructional material.Journal of educational psychology, 95(3), 629–640.

Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4, 55–81.Chi, M. T. H., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R. Sternberg (Ed.), Advances

in the psychology of human intelligence (Vol. 1, pp. 17–76). Hillsdale, NJ: Erlbaum.De Groot, A. D. (1965). Thought and choice in chess. The Hague: Mouton.de Jong, T. (2005). The guided discovery principle in multimedia learning. In R. E. Mayer (Ed.), The

Cambridge Handbook of Multimedia Learning (pp. 215–228). New York: Cambridge University Press.de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with computer simulations of

conceptual domains. Review of Educational Research, 68(2), 179–201.de Koning, B., Tabbers, H., Rikers, R., & Paas, F. (2007). Attention cueing as a means to enhance learning

from an animation. Applied Cognitive Psychology, 21, 731–746.Deacon, T. W. (1997). The symbolic species: The co-evolution of language and the brain (1st ed.).

New York: W.W. Norton.Hacker, D. J., Dunlosky, J., & Graesser, A. C. (Eds.). (1998). Metacognition in educational theory and

practice. Mahwah, NJ: Lawrence Erlbaum.Hasler, B. S., Kersten, B., & Sweller, J. (2007). Learner control, cognitive load and instructional animation.

Applied Cognitive Psychology, 21, 713–729.Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams.

Journal of Memory and Language, 32(6), 717–742.Homer, B. D., & Hayward, E. (2008). Cognitive and Representational Development in Children. In K. B.

Cartwright (Ed.), Literacy Processes: Cognitive Flexibility in Learning and Teaching (pp. 19–41).New York, NY: Guilford Publishing.

Homer, B. D., & Nelson, K. (2005). Seeing Objects as Symbols and Symbols as Objects: Language and theDevelopment of Dual Representation. In B. D. Homer & C. Tamis-LeMonda (Eds.), The developmentof social cognition and communication (pp. 29–52). Mahwah, NJ: Lawrence Erlbaum.

Kalyuga, S. (2005). Prior knowledge principle in multimedia learning. In R. Mayer (Ed.), The Cambridgehandbook of multimedia learning (pp. 325–337). New York: Cambridge University Press.

Kalyuga, S. (2006). Instructing and testing advance learners: A cognitive load approach. New York: NovaScience.

Kalyuga, S. (2007). Expertise reversal effect and its implications for learner-tailored instruction. Educa-tional Psychology Review, 19(4), 509–539.

Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). Expertise reversal effect. EducationalPsychologist, 38, 23–31.

Keller, T., Gerjets, P., Scheiter, K., & Garsoffky, B. (2006). Information visualizations for knowledge acquisition:The impact of dimensionality and color coding. Computers in Human Behavior, 22(1), 43–65.

Lee, H., Plass, J. L., & Homer, B. D. (2006). Optimizing cognitive load for learning from computer-basedscience simulations. Journal of Educational Psychology, 98(4), 902–913.

Lowe, R. (1993). Constructing a mental representation from an abstract technical diagram. Learning andInstruction, 3(3), 157–179.

Lowe, R. K. (2003). Animation and learning: selective processing of information in dynamic graphics.Learning and Instruction, 13, 157–176.

Mayer, R. E., & Sims, V. K. (1994). For whom is a picture worth a thousand words? Extensions of adual-coding theory of multimedia learning. Journal of Educational Psychology, 86(3), 389–401.

Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity forprocessing information. Psychological Review, 63, 81–97.

Moreno, R. (2004). Decreasing cognitive load for novice students: Effects of explanatory versus correctivefeedback in discovery-based multimedia. Instructional Science, 32(1–2), 99–113.

Moreno, R., & Mayer, R. E. (2005). Role of guidance, reflection, and interactivity in an agent-basedmultimedia game. Journal of Educational Psychology, 97(1), 117–128.

Nelson, K. (1996). Language in cognitive development: The emergence of the mediated mind. New York:Cambridge University Press.

Expertise reversal for icon effectiveness

123

Niemiec, R., Sikorski, C., & Walberg, H. J. (1996). Learner-control effects: A review of reviews and ameta-analysis. Journal of Educational Computing Research, 15(2), 157–174.

Ollerenshaw, A., Aidman, E., & Kidd, G. (1997). Is an illustration always worth ten thousand words? Effectsof prior knowledge, learning style, and multimedia illustrations on text comprehension. InternationalJournal of Instructional Media, 24, 227–238.

Peirce, C. S. (1955). Logic as semiotic: The theory of signs. In J. Buchler (Ed.), The philosophical writingsof Peirce (pp. 98–110). New York: Dover Books.

Plass, J. L., Chun, D. M., Mayer, R. E., & Leutner, D. (2003). Cognitive load in reading a foreign languagetext with multimedia aids and the influence of verbal and spatial abilities. Computers in HumanBehavior, 19(2), 221–243.

Plass, J. L., Homer, B. D., & Hayward, E. (2009a). Design Factors for Educationally Effective Animationsand Simulations. Journal of Computing in Higher Education, 21(1), 31–61.

Plass, J. L., Homer, B. D., Milne, C., Jordan, T., Kalyuga, S., Kim, M., et al. (2009b). Design Factors forEffective Science Simulations: Representation of Information. International Journal of Gaming andComputer-Mediated Simulations, 1(1), 16–35.

Plass, J. L., Homer, B. D., Milne, C., Jordan, T., Kim, M., & Barrientos, J. (2007). Representational modeand cognitive load: Optimizing the instructional design of science simulations. Featured research paperpresented at the annual convention of the Association for Educational Communication and Technology(AECT) in October, 2007 in Anaheim, CA.

Prins, F. J., Veenman, M. V. J., & Elshout, J. (2006). The impact of intellectual ability and metacognition onlearning: New support for the threshold of problematicity theory. Learning and Instruction, 16, 374–387.

Rieber, L. P. (1991). Effects of visual grouping strategies of computer-animated presentations on selectiveattention in science. Educational Technology Research and Development, 39(4), 5–15.

Rieber, L. P. (1996). Animation as feedback in a computer-based simulation: Representation matters.Educational Technology Research and Development, 44(1), 5–22.

Rieber, L. P. (2005). Multimedia learning in games, simulations, and microworlds. New York, NY:Cambridge University Press.

Rieber, L. P., Smith, M., Al-Ghafry, S., Strickland, B., Chu, G., & Spahi, F. (1996). The role of meaning ininterpreting graphic textual feedback during a computer-based simulation. Computers Education,27(1), 45–58.

Rieber, L. P., Tzeng, S., & Tribble, K. (2004). Discovery learning, representation, and explanation within acomputer-based simulation: Finding the right mix. Learning and Instruction, 14, 307–323.

Riek, J. M. S. (2007). The development of attention regulation in the Wisconsin Card Sorting Task.Developmental Science, 10(5), 664–680.

Schnotz, W. (2005). An integrated model of text and picture comprehension. In R. E. Mayer (Ed.), Thecambridge handbook of multimedia learning (pp. 49–70). New York: Cambridge.

Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representations.Learning and Instruction, 13, 141–156.

Schnotz, W., Picard, E., & Hron, A. (1993). How do successful and unsuccessful learners use texts andgraphics? Learning and Instruction, 3, 181–199.

Schnotz, W., & Rasch, T. (2005). Enabling, Facilitating, and Inhibiting Effects of Animations in MultimediaLearning: Why Reduction of Cognitive Load Can Have Negative Results on Learning. EducationalTechnology Research and Development. Special Issue: Research on Cognitive Load Theory and ItsDesign Implications for E-Learning, 53(3), 47–58.

Seufert, T., Janen, I., & Brunken, R. (2006). The impact of intrinsic cognitive load on the effectiveness ofgraphical help for coherence formation. Computers in Human Behavior, 23, 1055–1071.

Swaak, J., & de Jong, T. (2001). Learner vs. system control in using online support for simulation-baseddiscovery learning. Learning Environments Research, 4, 217–241.

van der Meij, J., & de Jong, T. (2006). Supporting students’ learning with multiple representations in adynamic simulation-based learning environment. Learning and Instruction, 16, 199–212.

B. D. Homer, J. L. Plass

123