Multimedia Systems (2006) 11(3): 236–248DOI 10.1007/s00530-005-0004-y

REGULAR PAPER

Kirsten R. Butcher · Sonal Bhushan ·Tamara Sumner

Multimedia displays for conceptual discovery: information seekingwith strand maps

Published online: 8 February 2006c© Springer-Verlag 2005

Abstract This article explores the use of a multimediasearch interface for digital libraries based on strand maps de-veloped by the American Association for the Advancementof Science. As semantic-spatial displays, strand maps pro-vide a visual organization of relevant conceptual informationthat may promote the use of science content during digitallibrary use. A study was conducted to compare users’ cogni-tive processes during information seeking tasks when usinga multimedia strand maps interface, versus the textual searchinterface currently implemented in the Digital Library forEarth System Education. Quantitative and qualitative datafrom think-aloud protocols revealed that students were morelikely to engage with science content (e.g., analyzing the rel-evance of science concepts with regard to task needs) dur-ing search when using the strand maps interface comparedto those using textual searching. In contrast, students us-ing a textual search interface engaged more frequently withsurface-level information (e.g., the type of a resource regard-less of its science content) during search and retrieval. As amultimedia search interface for digital libraries, strand mapsappear to be promising tools to promote conceptual discov-ery and learning through content-based processes that pro-mote learner engagement with relevant science knowledge.

Keywords Information seeking · Strand maps · Educationaldigital libraries · Science learning · Educational technology

1 Introduction

The continuing growth of online educational resources hasmeant that learners have an increasingly important need for

K. R. Butcher (B) · S. BhushanDigital Library for Earth System Education Program Center,University Corporation for Atmospheric Research, P.O. Box 3000,Boulder, CO 80307E-mail: {kbutcher, sonal}@ucar.edu

T. SumnerDepartment of Computer Science, University of Colorado at Boulder,Campus Box 430, Boulder, CO 80309-0430E-mail: sumner@colorado.edu

discovery methods that help them to explore and retrieverelevant information. As services that bring together, orga-nize, and support connections to diverse collections of edu-cational materials, digital library systems have great promiseas tools to support online learning. However, as success-ful as digital libraries have been to date, retrieval of rele-vant resources remains a significant problem in educationalsettings. Especially in science, subtle differences betweenconcepts can make it difficult for learners without domainexpertise to generate relevant search terms or even to ef-fectively select the appropriate standard vocabulary fromtopic lists. Lacking relevant knowledge, learners often re-sort to selecting keywords from assignments or existing ma-terials [1]. Previous research has found that science edu-cators also experience difficulties locating digital learningmaterials [2]. Many K-12 science educators frequently mustteach out of area in topic domains for which they lack train-ing or confidence [3]. When considering both educators andlearners, the number of individuals who come to digital li-braries with low domain knowledge is strikingly high. Assuch, digital libraries should be designed to serve as cog-nitive tools that support library users to engage in concep-tual learning, in addition to supporting information search[4].

Our research efforts are exploring how conceptualbrowsing interfaces can support learning of science con-cepts during digital library search tasks. We are creatingconcept browsing interfaces and supporting web servicesbased on the science concepts and strand maps developedby the American Association for the Advancement of Sci-ence [5, 6]. These concepts represent nationally recognizedlearning goals developed through a long-term collaborativeprocess involving pedagogical experts, scientists, scienceeducators, and school districts across the country [7, 8].The strand maps provide an ideal foundation for conceptualbrowsing interfaces because they provide a visual represen-tation of science concepts that supports students and educa-tors in making connections between key ideas.

The context for our research is the National ScienceDigital Library (NSDL). NSDL is intended to support

Multimedia displays for conceptual discovery: information seeking with strand maps 237

educational science reforms by providing educators andlearners with online access to a large number of educa-tional materials. NSDL encompasses a number of discipline-specific digital library systems, including the community-led Digital Library for Earth System Education (DLESE).DLESE seeks to support learning and research in Earth sys-tem science education by providing interactive and effec-tive access to high-quality online educational resources forall educational levels in both formal and informal settings.Given the diversity of their user base, NSDL and DLESEface significant challenges in connecting learners to the rel-evant conceptual knowledge and educational interventionsthat are appropriate to their specific informational needs andlearning goals [9, 10].

In this article, we first describe prior work demonstrat-ing learner difficulties with information search and discussthe theoretical rationale for conceptual browsing interfacesas tools to overcome these observed difficulties. Next, wedescribe one of the interfaces we have developed to provideconceptual support for search tasks. Finally, we report theresults of a study investigating the effects of this interfaceon the cognitive processes performed by users when usingthis conceptual browsing interface versus an existing textualsearch interface.

2 The challenge of conceptual discovery

Educational digital libraries face an extraordinary challengein finding successful ways to support conceptual discov-ery. In this article, we use the term conceptual discoveryto refer to the ability of learners to find new informationand ideas that can inform and refine their conceptual un-derstandings of a specific knowledge domain. As such, con-ceptual discovery requires more than simple information re-trieval. It requires learners to make connections betweenthe content of library resources and their specific knowl-edge needs, and to consider the relationship of new infor-mation to the larger knowledge domain. Previous researchillustrates that issues of learners’ prior knowledge, problemdefinition, and conceptual organization must be addressedin order to promote learning during information seekingtasks. However, the same research highlights the shortcom-ings of textual search interfaces for supporting these learnerneeds.

In early stages of knowledge development, learnerslack the requisite knowledge for generating the specializedand varied vocabulary necessary to locate relevant digitalresources using keywords or phrases [11–13]. Kuhlthau[11] examined the information search processes of studentswriting academic papers and found that initial searchstrategies were characterized by confusion, uncertainty,and a lack of focused knowledge. Given their lack ofknowledge, students often turned to physical methods todiscover conceptually relevant information and to gain abetter understanding about the topic over time. For example,

students used card catalogs to find potentially topic-relevantshelves in the library, which they would then browse tolocate information that they had not identified by keywordsearches or other formal strategies [11].

More recent research has found that even quite knowl-edgeable individuals in a domain can be limited by the fail-ure of digital information databases to support conceptualbrowsing and discovery [12]. Kuhlthau and Tama foundthat lawyers doing research preferred printed texts overelectronic retrieval mechanisms largely because electronicdatabases (1) failed to display information in a useful man-ner relevant to the learning task, (2) limited discovery due tostrict requirements of keyword searching, and (3) failed toorganize information in meaningful ways that prevented theuser from becoming lost in digital resources. These resultssuggest that even experts in a content area may benefit frominterfaces that are strategically designed to promote concep-tual discovery through browsing, especially if the support isorganized and presented in ways that are meaningful for aparticular type of knowledge need.

Kuhlthau’s [11, 12] research highlights the challengesof effective information seeking in digital libraries: lack-ing physical strategies that users can fall back on, digitallibrary search interfaces must resolve long-standing prob-lems resulting from discovery methods that rely upon userinput for information retrieval. Because learners lack suffi-cient domain knowledge to generate reasonable search termsand because specialized expertise within a system is oftennecessary to know the correct terms with which resourcesare indexed, there is a fundamental mismatch between users’vocabularies and system vocabulary [14].

2.1 Textual search in digital libraries

The success of textual search interfaces has been consider-able and likely stems from their simplicity and familiarity;users tend to be experienced and comfortable with onlinetextual search methodologies. The existing DLESE searchinterface (www.dlese.org) is an example of a successful tex-tual interface onto a digital library discovery system (seeFig. 1).

As seen in the top panel of Fig. 1, the textbox pro-vided on the left-hand side of the homepage allows learn-ers to search for resources using keywords, phrases, orother terms chosen by the user. DLESE also supports theuse of optional vocabulary terms (as seen in the bottompanel of Fig. 1) to focus the search results based on thetype of resources needed; for example, users can search forresources that include lesson plans and scientific illustra-tions. Experienced digital library users with ample knowl-edge of a science domain may prefer textual search inter-faces, such as the DLESE interface, for their simplicity andefficiency. However, as discussed above, many users in ed-ucational settings do not have adequate science understand-ing and can benefit from interfaces that support conceptualdiscovery.

238 K. R. Butcher et al.

Fig. 1 The Dlese textual search interface.

3 Approach: semantic-spatial displays

Multimedia search interfaces have the potential to changelearners’ interactions with digital libraries by providing mul-tiple forms of cognitive support for exploration and evalua-tion of knowledge and its organization in a domain. Whenappropriately designed, multimedia interfaces can changethe ways that learners pursue information by engaging themearly on with the science content that informs conceptualdiscovery. Although multimedia can include a wide varietyof resources, including interactive visual displays, sounds,pictures, animations, voice recognition, and other complex

sensory experiences, we define multimedia tools as thosethat include at least two modes of sensory experience. Thisdefinition is similar to that proposed by Mayer [15], who ar-gues that multimedia learning involves processing of verbaland visual components. Previous research in cognitive psy-chology and education has repeatedly demonstrated benefitswhen well-designed multimedia tools are used for learning[15–19]; however, research has not addressed whether multi-media interfaces can change the cognitive behaviors of usersseeking information in digital environments.

We are particularly concerned with supporting learn-ers who have limited domain knowledge to engage with

Multimedia displays for conceptual discovery: information seeking with strand maps 239

science content during digital library searches in order topromote opportunities for learning during search tasks. Pre-vious research has shown that qualitatively different searchprocesses characterize experts and domain novices; whereasstudents with little domain knowledge spend more time for-matting and modifying queries, experts spend more timescanning and reading texts [20]. In short, domain novicestend to engage in surface-based processes whereas expertstend to engage in content-based processes. It is not terri-bly surprising that experts are better able to find relevant re-sources by analyzing informational content; however, it doessuggest that less knowledgeable students can better approx-imate expert behavior by attending to science content earlyand often during a search.

Surface-based search and evaluation processes are char-acterized by the tendency to focus on aspects of the inter-face that are not relevant to domain information (e.g., com-ponents of the resource such as pictures or activities regard-less of their science content). In contrast, content-based pro-cesses are characterized by a focus on science content thatis related to the task at hand (e.g., assessing the relevance ofplate tectonics resources in preparing a lesson plan on thecause of earthquakes). Promoting the use of content-basedprocesses during digital library search can be considered animportant first step in promoting domain learning—that is,the progression from surface-based to content-based searchand evaluation processes represents an important step to-ward reasoning with science content. The attempt to matchinformation needs with science concepts characterizes a typeof reasoning with conceptual information that should pro-mote understanding. Further, the increased use of content-based processes rather than surface-based processes charac-terizes a movement toward more expert behaviors that focuson the conceptual relevance of resources during informationseeking.

Semantic-spatial displays refer to a multimedia represen-tation in which information is presented both verbally (thesemantic content) and visually (the spatial representation ofthe content). Although the use of semantic-spatial represen-tations is a relatively novel approach to facilitating content-based search processes in digital libraries, previous work inpsychology and education has addressed the cognitive im-pact of these types of representations for learning.

Knowledge maps refer to a particular type of semantic-spatial display in which content information or ideas arelocated in nodes and labeled links are used to depict therelationships between nodes; as such, knowledge maps re-flect both the conceptual information in a domain and itstheorized cognitive structure. Previous research on knowl-edge maps has found that organized visual representationsof semantic information can change learning processes andlearning outcomes when compared to text-only representa-tions. Knowledge maps repeatedly have been found to facil-itate students’ memories for central ideas (also referred toas the macrostructure of a text) when knowledge maps areused during learning [21–23]. Further, student engagementwith knowledge maps in a domain can transfer to learning in

other formats; explicit prior training with knowledge mapshas been found to improve memory for non-mapped infor-mation studied in a text format [21]. Overall, the body ofresearch evidence on knowledge maps suggests that thesesemantic-spatial tools may help students identify and inte-grate key concepts during learning. Interestingly, the bene-fits of knowledge maps have been shown to be greatest forstudents with low verbal ability or low prior knowledge of adomain [22].

The usefulness of knowledge maps for learners with lim-ited domain knowledge has important potential applicationsfor digital library development. Engaging learners with sci-ence content requires support in helping learners identifyand understand key information. As multimedia displaysthat visually organize conceptual information according tounderlying relationships, semantic-spatial displays may pro-vide the right types of support to promote content-based pro-cessing of domain information by learners with limited do-main knowledge.

Strand maps are an example of semantic-spatial displaysand are similar to knowledge maps in their form and struc-ture (Fig. 2). These maps consist of node-link diagrams or-ganized around topics important to science literacy (e.g.,weather and climate, flow of energy in ecosystems, conser-vation of matter). Each strand map provides an overviewof K-12 learning goals for a particular topic organized into“strands” reflecting key concepts within that map (e.g., heat,water cycle, atmosphere, and climate change are strandswithin the Weather and Climate Map). Each strand is fur-ther cross-referenced by grade level (K-2, 3–5, 6–8, 9–12).High-level descriptions of science concepts are provided inthe nodes, while the links depict how science concepts bothsupport and depend upon each other. These links betweenconcepts illustrate how learners’ understandings should be-come increasingly sophisticated over the course of their ed-ucation. It should be noted that strand maps are not the onlymultimedia interface that could support conceptual discov-ery in digital libraries; we use strand maps as a test casefor assessing the effectiveness of a multimedia search in-terface in supporting conceptual discovery for educationaltasks.

4 The Strand Map Service

To investigate the potential benefits of multimedia interfacesfor conceptual discovery, we have implemented and evalu-ated a system based on the strand maps just described. TheStrand Map Service (the “Service”) supports the needs oftwo audiences: K-12 educators and learners, and digital li-brary developers. These audiences are supported through theprovision of two kinds of public interfaces: (1) graphicalconcept browsing interfaces for end-users, and (2) a pro-grammatic web service interface that allows digital librarydevelopers to easily create concept browsing interfaces foruse in their own libraries.

240 K. R. Butcher et al.

6-8

3-5

K-2

to and from CONSERVATION

OF ENERGY

to and fromCONSERVATION OF MATTER

to and fromSTATES OFMATTER

water cycleheat

Water left in an open container disappears, but water in a closed container does notdisappear. 4B/P3

The sun warms the land, air,and water. 4E/P1

Water can be a liquid or a solid and can go back and forth from one form to the other. If water is turned intoice and then the ice is allowed to melt, the amountof water is the same as it was before freezing. 4B/P2

When liquid water disappears, it turns into a gas (vapor) in the air and can reappear as a liquid when cooled, or as a solid if cooled below the freezing point of water. Clouds and fog are made of tiny droplets of water. 4B/E3

The cycling of water in and out of the atmosphere plays an important role in determining climatic patterns. Water evaporates from the surface of the earth, rises and cools, condenses into rain or snow, and falls again to the surface. The water falling on land collects in rivers and lakes, soil, and porous layers of rock, and much of it flows back into the ocean. 4B/M7

Heat can be transferred through materials by the collisions of atoms or across space by radiation. If the material is fluid, [such as air or water], currents will be set up in it that aid the transfer of heat. 4E/M3

When warmer things are put with cooler ones, the warm ones lose heat and the cool ones gain it until they are all at the same temperature. A warmer object can warm a cooler one by contact or at a distance. 4E/E2

This is a section of a map called “Weather and Climate.” The whole map consists of 22 concepts, 7 of which are shown here. The arrows indicate how one concept supports the ideas in the next concept. Dotted lines show connections to other maps (e.g., Conservation of Matter). The three boxes on the left side form a strand called “heat” and the four boxes on the right side form a strand called “water cycle.” Three grade ranges are shown on the map. For example, the three benchmarks at the bottom of the map are for grades K-2. The full map extends into grades 9-12.

Fig. 2 Excerpt of strand map on weather and climate.

4.1 End-user interfaces

Graphical concept browsing interfaces based on the Serviceenable K-12 educators and learners to:

• Discover educational resources that support selectedconcepts

• Browse concepts and their interconnections by exploringinteractive, concept map visualizations

• Enhance their own content knowledge by using the ser-vice to examine important background information onconcepts, such as related prior research on student con-ceptions and student learning, related educational stan-dards, and assessment strategies to check student under-standing.

A wide variety of end-user interfaces based on conceptmaps and concept browsing can be created using the strandmap service. One example interface, created for DLESE,is shown in Fig. 3. This particular interface was used inthe experiment described later in this article. The right sideof this interface illustrates a “map view” generated by theService; i.e., a visual display of one complete strand map.Users can explore this map view via direct manipulation;selecting strand names or grade-levels will enable the userto “drill down” and explore smaller portions of this map.Learners or teachers can also explore this map, and otherstrand maps, by manipulating the twist-down folders on theleft side of the interface. That is, in addition to providingvisualizations of maps and map components, the Servicealso provides visualizations that support exploring acrossmap boundaries. Finally, the user can elect to retrieve re-sources from DLESE that support a particular concept bypressing one of the “view related resources” links embed-ded in the map nodes. This causes a list of search results

to be displayed containing brief descriptions of relevant re-sources. These brief descriptions include information abouteach resource, such as the resource’s title, URL, grade-level, a short textual summary, and relevant educationalstandards.

4.2 Web service interface

The programmatic web service interface enables digitallibrary developers to easily construct concept browsinginterfaces appropriate to the needs of their specific libraryaudiences using dynamically generated visual componentsprovided by the Service. Figure 4 illustrates the overallService architecture and briefly describes the major compo-nents of the Service. The Service builds on recent advancesin visualization components [24] and programmatic inter-faces to knowledge organization systems [25]. The Serviceis architected to support a “spectrum of interoperability” tomaximize its utility for a broad range of NSDL projects [26].Specifically, library developers can create interfaces andservices by making calls to our web service interface or byharvesting conceptual information using the Open ArchivesInitiative Protocol for Metadata Harvesting (OAI-PMH)server [27].

Rather than creating static presentations of strand maps,the Service middleware generates visualizations of maps andmap components from information modeled in the bench-marks repository. The information modeled in this reposi-tory is drawn from the Benchmarks and Atlas publications[5, 6], as well as other AAAS materials. Library developerscreate concept browsing interfaces by requesting informa-tion from the Service middleware using a web service in-terface: the concept space interchange protocol (CSIP) [28].

Multimedia displays for conceptual discovery: information seeking with strand maps 241

Fig. 3 The strand map search interface implemented in DLESE. Users can choose to explore the conceptual relations depicted or to retrieverelevant resources by clicking one of the concepts

Fig. 4 Architecture of the strand map service

242 K. R. Butcher et al.

CSIP supports three types of requests: (1) service descrip-tion (which returns information about the capabilities andversion of this instance of the Service), (2) submit resource(which is used to submit additional benchmark informationto the Service), and (3) query (which is used to requestAAAS information and visual components useful for creat-ing concept browsing interfaces). Details on the web serviceprotocol are described elsewhere [29, 30].

When the user performs an action in a client interface,the interface makes an information request to the StrandMap Service, such as “retrieve all the concepts associatedwith a particular strand.” The Service returns the requestedinformation as XML or as scalable vector graphics (SVG)[31]. SVG is a technical format for the exchange of graphi-cal information in web-based interfaces [31]. The SVG op-tion enables developers to easily construct concept browsinginterfaces from interactive visual components that are dy-namically generated by the Service. Using this option, thesame information is returned to the interface as in the XMLoption, but it is already embedded in a visual componentthat can be directly displayed and interacted with. The vi-sual component generator (VCG) takes the information ex-tracted from the benchmarks repository and renders it intoa semantic-spatial display of a strand map in SVG format[29, 32].

One design requirement that emerged from discussionswith NSDL developers was that libraries want to choosefrom a flexible range of approaches for indicating corre-spondence between a concept and library resources. There-fore, a Query Registration Service is provided that al-lows digital libraries to specify particular retrieval meth-ods to use when searching their collections for conceptualinformation.

5 The study: assessing the influence of strand maps

A study was conducted in order to determine how differ-ent search interfaces may influence the cognitive searchprocesses of digital library users when performing educa-tionally relevant tasks. A multimedia strand maps interfacewas contrasted with the standard DLESE textual searchinterface. Because the strand maps interface provides theuser with a visual organization of knowledge as well as richsemantic content about the domain, it was hypothesized thatthe strand maps interface would promote increased use ofcontent-based (science-focused) processes when performinginformation seeking tasks in DLESE. In contrast, becausetextual search interfaces are directed by user input we pre-dicted that learners with limited domain knowledge usingthe DLESE textual interface would fall back on surface-levelsearch processes. These users are more likely to focuson operational strategies—such as selecting vocabularyoptions or typing in keywords—to successfully retrieveresources with less stringent attention to relevant sciencecontent.

5.1 Research methodology

Because the strand maps are designed to promote under-standing of concepts important to science education, stu-dents with limited domain knowledge were used to approx-imate educators with low science knowledge engaging intypical educational search tasks. Twelve undergraduate stu-dents from the University of Colorado, Boulder participatedin the study; each received partial credit toward a researchparticipation requirement for an introductory psychologyclass. The average age of participants was 19 (min: 18,max: 22); six participants were male and six were female.In a demographic form completed at the end of the researchsession, all students rated themselves as being comfortableand experienced with computer use; average response was5.5 (min: 4, max: 7) on a scale from 0 (very low) to 7 (veryhigh). In addition, students reported that they frequentlyused digital methods to search for academic information.When asked how often they used a computer to researchinformation related to class topics, average response was5.5 (min: 3, max: 7) on a scale from 0 (never) to 7 (2+ timesa day).

Students were randomly assigned to one of two experi-mental conditions; half the participants performed the studyusing the DLESE textual search interface (shown in Fig. 1)while the other half performed the study using a strand mapsinterface for DLESE resource retrieval (shown in Fig. 3).

On average, the participants took about 30 min to com-plete the study. Participants first were given an informedconsent form, which explained the procedures of the study,stated that the students would be audio-recorded during thetasks, and explained steps that would be taken to ensureconfidentiality of the data. After consent was obtained, stu-dents began the main experimental procedure. Because thestrand maps are particularly relevant for educators, a seriesof four tasks (see Table 1) were developed to represent typ-ical information-seeking needs that an educator may havewhen teaching an unfamiliar topic in Earth science. Eachtask required the participant to take on the role of an ed-ucator in middle or high school with the goal of findinga specific kind of resource that could be used to teach de-sired concepts in class. In order to understand if the strandmaps search interface influenced search processes duringtask completion, participants in both groups were asked tothink aloud as they completed each task. The same set oftasks was used for both groups of participants (the strandmaps group, and the textual search group).

During the think-aloud procedure, participants in bothgroups were asked to talk aloud about their onscreenchoices while using the system, as well as to verbalizethe reasoning behind their actions. Students were askedto explain everything they were doing or trying to do,what they were thinking, and whatever questions, ideasor thoughts they had along the way. The experimenteremphasized that there were no right and wrong answersduring verbalization and that the more the student talked, themore it would help the researchers understand how people

Multimedia displays for conceptual discovery: information seeking with strand maps 243

Table 1 Task descriptions representing information needs that may be faced by educators

Task 1Carl is an 8th grade science teacher. He’s new at this. He used to be a substitute teacher and has just obtained this full time position. He’steaching science for the very first time in his life. In his school, teachers work as a team during the summer months to develop modules andlesson plans for the upcoming school year. Currently, his team is working on modules related to rocks and sediments.The team is going to meet this afternoon. Carl wants to find some information related to sedimentary rock, specifically—some interactivemedia or images on the topic of sedimentary rock to use for teaching his class. He’s heard from his colleagues that DLESE is a good source forsuch materials.

Task 2Jack is a 10th grade science teacher who has volunteered to fill in for Jan, a 7th grade science teacher, while she’s out sick. Jan was supposedto teach her class about changes in the Earth’s surface this week. She suggests Jack come up with a classroom activity based on changes in theEarth’s surface.One of the topics that Jack teaches in his 10th grade class relates to earthquakes. He wants to teach the 7th graders something related to thistopic. Jack often uses DLESE in order to find activities and detailed text on material he teaches in his 10th grade class. He decides to checkout what DLESE has to offer. He wants to find out which concepts he needs to teach the 7th graders, in addition to a classroom activity thatsupport these concepts.

Task 3Sally has just graduated from CU, where she was an education major, and has found a job as a fifth grade teacher. One of her teachingresponsibilities is science. She is preparing material for the science class. One of the topics in the syllabus is composition of rocks. She wantsto teach some age-appropriate concepts in class. She knows that last year the students learned that rocks come in different sizes and shapes.She wants to find teaching materials to help her teach the students how to classify rocks according to their characteristics.

Task 4Josie teaches 11th grade Math, but she’s substituting for her colleague, Alicia, this week, who teaches 11th grade science, who’s on leave.Before she left, Alicia was teaching her class about the age of rocks or fossils. She has left a note for Josie, suggesting that Josie give the classa lab activity on the topic. She also suggests DLESE as a possible source for lab activities on science topics.

work with the system. A normal think-aloud protocol isnondirective [33]; the only probe used by the examiner afterthe initial instructions is when participants stop verbalizingfor some time, at which point they are reminded that theyneed to think aloud. However, because we were interestedin the cognitive search processes in which students wouldengage during task completion, the think-aloud protocolwas modified slightly to elicit cognitively oriented processinformation from students. The experimenter prompted thestudent after vague or incomplete comments with questionssuch as “Why would you say that?” or “How did you arriveat that conclusion?” These prompts were necessary in orderto capture the underlying processes in which users wereengaging during task completion. All verbal protocols wereaudio recorded and transcribed.

5.2 Verbal protocol analysis

Verbal data collected during the think-aloud protocol wereanalyzed for differences in the cognitive processes repre-sented by the verbalizations. After verbal protocols weretranscribed, each protocol was separated into a series ofcomplex propositions for coding. A complex propositionis approximately equal to an idea unit that represents onethought or action for analysis [34, 35]. Two raters codedeach proposition according to the type of process it repre-sented. Protocols were scored twice; after initial scoring,raters discussed category criteria and reviewed all protocolsfor scoring accuracy.

Nine major categories were coded; see Table 2 foran explanation and example of each category. Seven ofthese categories are organized under three major types of

comprehension processes relevant to digital library search:planning, performance, and evaluation. The other two cate-gories were needed to score propositions that lacked con-tent (Filler/Miscellaneous) or were otherwise uncodable(Search–Other). It is important to note that planning, per-formance, and evaluation processes are not linear in na-ture but tend to be used iteratively during an overall searchtask.

As learners begin to plan their digital library searches,they attempt to coordinate their understanding of thetask, the system, and their needs. Thus, statements relatedto these types of planning-relevant processes were sep-arated into two categories: Monitoring and Task propo-sitions. After assessing the task and their understand-ing, learners begin to search for relevant material inthe library. During search performance, learners tend totalk about both what they are seeing on the screen andtheir search strategies. These general categories are re-ferred to as “Interface” and “Strategy” statements, buteach category can occur at a surface level (where state-ments do not reflect science content) or at a content level(where statements directly refer to relevant domain science).Thus, four Search–Performance categories were scored:Interface–Surface, Interface–Content, Strategy–Surface, andStrategy–Content. Finally, once learners retrieved a re-source, they needed to make a decision about its useful-ness during an evaluation phase. The Evaluation categoryrefers to the learner’s overall decision to use or not to usethe resource. Within the evaluation category, reasoning sub-categories reflected the type of rationale that students gavefor their evaluations; these subcategories included reason-ing related to the surface-level characteristics of the re-source (without mention of science content), content-level

244 K. R. Butcher et al.

Table 2 Categories and subcategories for propositional analysis

Category Description Example Relevant stage

Filler orMiscellaneous

Statements with no analyzable content. “Um” or “Let’s see.” N/A

Monitoring Statements that assess one’s understanding “I think that . . .” Search Planning

Task Statements that relate to information or goals given in theexperimental task descriptions.

“He wants to learn about the rockcycle. . .” or “It says they areseventh graders. . .”

Search Planning

Interface–Surface Statements about on-screen information that did not includescience content.

“I see a lot of pictures on thispage. . .”

SearchPerformance

Interface–Content Statements about on-screen information that includereference(s) to science content or knowledge.

“It says that dissolved minerals formsolid rock again.”

SearchPerformance

Search–Surface Statements about search strategies or behaviors reflectingsurface-level information and no science content.

“I need to find pictures so I’m goingto. . .”

SearchPerformance

Search–Content Statements about search strategies or behaviors that includescience content or knowledge.

“I need to find information about thisplate tectonics so I’m going to. . .”

SearchPerformance

Search–Other Statements about search strategies that were otherwiseuncodable.

“I just thought I’d look back. . .” N/A

Evaluation Statements that reflected decisions about informationrelevance or usefulness.

“I think this is too general” or “Thisone seems like a good choice.”

SearchEvaluation

Evaluationrationale(subcategoriesof evaluation)

Reasoning–Surface: Rationale for accepting or rejecting aresource based on superficial characteristics of theresource.

“This has a lab activity.” SearchEvaluation

Reasoning–Content: Rationale for accepting or rejecting aresource based on science content or knowledge

“This talks about the causes forearthquakes in depth.”

SearchEvaluation

Reasoning–Task: Rationale for accepting or rejecting aresource based on the demands of a given task

“This activity would work better forfifth graders.”

SearchEvaluation

Reasoning–Other: Rationale for accepting or rejecting aresource based on criteria other than those listed above.

“This is more like what I did as akid.”

SearchEvaluation

reasoning about the resource (related to science domain in-formation), task-level reasoning (assessing appropriatenessbased on task description), and other reasons that were var-ied and lacked theoretical relevance (e.g., just “liking” a re-source better).

The Filler/Miscellaneous and the Search–Other cate-gories were not included in final analyses due to the un-focused nature of statements they contained. In addition,the Reasoning-Other subcategory was rarely used and there-fore was not included in final analyses. Thus, seven ma-jor categories and three subcategories were analyzed; inter-rater correlations averaged 0.74 for major categories (min:.63 for Task, max: 0.83 for Evaluation) and 0.69 for sub-categories (min: 0.65 for Reasoning–Content, max: 0.75 forReasoning–Task). Table 3 shows a selection of complexpropositions from participant protocols and the coded cat-egories assigned to each statement.

5.3 Results

5.3.1 Quantitative results

In order to compare the two groups (strand maps interfacevs. textual search), the number of complex propositions ofeach type were averaged across raters and tasks. Based onour experimental questions, a repeated measures analysis ofvariance (ANOVA) was used to test for group differencesin the frequency with which students engaged in content-based and surface-based processes (in the interface and strat-egy categories). Results showed a significant two-way in-teraction (F(1,10)=9.2, p<.02, η2

p = 0.47) between processtype (content vs. surface) and condition (strand maps searchvs. textual search) such that students using the strand mapssearch interface engaged in more content-based processesand students using the textual search interface engaged in

Multimedia displays for conceptual discovery: information seeking with strand maps 245

Table 3 Sample complex propositions and their scored cognitive process categories

Complex proposition Coded categoryParticipant 5, Strand maps search interface

He wants to learn about the rock cycle Taskso I guess I’ll go to rocks and sediments right now Strategy–Content

Here’s one that talks about the rock cycle and reforming of rock Interface–ContentI think this one might work well EvaluationIt talks about the rock cycle Reasoning–Content

and mentions minerals Reasoning–Contentand that’s what he wants to teach Reasoning–Task

Participant 4, Textual search interfaceI’m wondering if MonitoringI’m supposed to go to this browse resources and collections Interface–SurfaceI’m going to try to search Strategy–Surface

for like the basic atomic structures Strategy–Contentin minerals Strategy–Content

It says that he wants pictures Taskso I clicked on visual Strategy–Surface

Now there’s a bunch of sedimentary rocks Interface–Content. . .so I guess that would work as far as pictures Reasoning–Surface

Table 4 Means and standard deviations for propositions uttered duringsearch performance for each interface condition

Interface Complex proposition Mean Standardcondition category Mean deviation

Strand maps Interface: Surface 10.1 4.1Strategy: Surface 3.6 2.8

Textual search Interface: Surface 14.7 7.3Strategy: Surface 6.8 4.1

Strand maps Interface: Content 6.9 2.8Strategy: Content 6.8 4.2

Textual Search Interface: Content 3.5 1.0Strategy: Content 3.9 1.6

02468

10121416

Tas

k

Mon

it.

Sur

f(In

t)

Sur

f(S

trat

)

Con

t(In

t)

Con

t(S

trat

)

Eva

l

Rea

s(S

urf)

Rea

s(C

ont)

Rea

s(T

ask)

Search Processes

Co

mp

lex

Pro

po

siti

on

s (M

)

Strand Maps Textual Search

A B C

Fig. 5 Mean number of propositions uttered by strand map and textualsearch groups during (A) planning, (B) performance, and (C) evalua-tion phases

more surface-based processes (see Fig. 5). Means and stan-dard deviations for this effect are seen in Table 4. AdditionalANOVAs were conducted to test for group differences in thefrequency with which students engaged in processes identi-fied by the other categories. There were no group differencesin Monitoring, Task, Evaluation (Fs<1), and Reasoning (alltypes; F<1.8)

As can be seen in Fig. 5, the influence of strand maps as asearch interface appears only during search performance. Inthis stage, students using the strand maps interface tended totalk about science content nearly twice as much as studentsusing the textual search interface. In contrast, student perfor-mance during the search formulation and evaluation stageswas found to be quite consistent across both interface condi-tions.

5.3.2 Qualitative results

The verbal protocols show distinct differences in searchbehaviors demonstrated by students using the strand mapssearch interface and the textual search interface; thesedifferences provide converging evidence for the resultsfound in the quantitative analyses. As seen in Table 5,students using strand maps frequently considered sciencecontent during search. Content-based search behaviorsincluded analyzing strand map content, making connectionsto retrieved resource descriptions, and considering resourceappropriateness. In contrast, students using textual searchwere much more likely to exhibit behaviors related tooperation of the search engine—including selection of key-words for textual search and consideration of the optionalvocabulary terms provided in DLESE.

These differences are highlighted in comments collectedduring task 2 (see Table 5); task 2 asked users to find a class-room activity based on earthquakes for 7th graders. Partic-ipant 1 (using the strand maps interface) reads the contentof the strand map benchmarks and uses this content-basedinformation to guide search behavior; specifically, partici-pant 1 chooses to look for information related to “earth-quakes . . . and volcanic eruptions changing the surface ofthe Earth, creating mountains and ocean basins.” The strandmap supported this participant’s analysis of the informationneeds of the task in light of its relation to meaningful con-ceptual benchmarks in the domain. In contrast, participant

246 K. R. Butcher et al.

Table 5 Sample statements from each task produced by students using the multimedia strand maps interface (left column) and the textual searchinterface (right column)

Multimedia strand maps interface Textual search interface

Task 1P1: I’m reading the sediments that are buried together by dissolved

minerals to form solid rock again, so I’m going to take a look at that.P2: . . .so I’m going to look for photographs of what I’m looking forexactly, and then basically I’m going to search for rock cycle

P9: I’m reading through it, and so it talks about the effects of stress,heat and pressure. It also talks about the minerals, rocks are composedof minerals, these minerals have, so it talks about the minerals of therocks or how minerals can form rocks, or can be formed in rocks. . .

P4: I’m going to just search for models and then, lab activities formodel, and we also want it to be illustrated, models. . .I’m wonderingif I’m supposed to go to this browse resources and collections but . . .now the imagery isn’t selected so I’m wondering if you have to selectthat every time. Photograph, let’s try that.

Task 2P1: . . .so this one telling us about earthquakes. . .Or about the, earth-

quakes changing the, and volcanic eruptions changing the surface ofthe Earth, creating mountains and ocean basins, so I’ll check that oneout.

P2: I’m going to click classroom activity, and then grade level it saysthey’re seventh graders so go to middle, sixth through eighth grade so;and then search for volcanoes; wait, no, earthquakes

P3: . . .this first one says it includes both informational material andinteractive exercises on the faults, seismeticity, and etcetera, so thatlooks like it would be a good one, place to start.

P6: So what I’m going to do in the search engine is put in earthquakesin general, and see what they have. . .

P8: I’m going to resource types to see what that kind of brings up,then on classroom activity.

Task 3P5: This one looks promising; it says rock is composed of differentcombinations of materials; smaller rocks come from the breakage andweathering of bigger rocks and larger rocks. And to classify rocksaccording to their characteristics. Wondering if this is a good choice. . .[The task is] to teach the students how to classify rocks accordingto their characteristics, but . . . this one right underneath this says thatchunks of rocks come in many sizes and shapes, from boldness ofgrains of sand to even smaller, so I’m going to look at this one becauseit might be better.

P6: So, I’m just going to search in general, go ahead and look upcomposition of rocks real quick, see if I come up with anything.

P10: Alright, so I’ll go to the filter again. Go to fifth grade. Rocksand composition. . . need to prepare materials within the classroom. Iwould want a lot of images, I want to try to filter that in somehow,illustration. And I don’t think I need anything under collections orstandards for anything. Click on that, then that. So, kind of composi-tion of rocks, composition of rocks. I’m going to go to the first sitethat’s on there

Task 4P5: I’m reading one right now about thousands of layers of sedimen-

tary rock confirm the long history of the changing of the surface of theEarth and the changing life forms who has remained, whose remainsare found in the successive layers, so I guess that sounds good. So,I’m going to click that link.

P4: . . .so we search for, I would say, let me try fossils first.

P11: I’m going to weathering and erosion, okay, scientific evidenceimplies that some rock near the Earth’s surface is several billion yearsold. That is sort of related to what I’m looking for. Yes, okay I’mclicking the link determining the age of rocks and fossils.

P10: . . .for the eleventh grade, so grade level filter on a high, resourcetype, I guess you want a lab activity, and that, and I’ll start with that, Iguess, with dating rocks.

6—using the textual search interface and lacking concep-tual science support—does not demonstrate connection withscience content during search. Participant 6 falls back onsurface-level processes, selecting a keyword and hoping forresults: “So what I’m going to do in the search engine is putin earthquakes in general, and see what they have. . .”

Similar processes can be seen in task 3 (see Table 5). Par-ticipant 5—using the strand maps interface—analyzes theconceptual information in two benchmarks on the map. Thisparticipant compares the science content of two benchmarksin the map, choosing the benchmark that appears to be betterrelated to the task at hand—“this one right underneath thissays that chunks of rocks come in many sizes and shapes,from boldness of grains of sand to even smaller, so I’m goingto look at this one because it might be better.” Conversely,participant 10 concentrates on finding the combination of

optional vocabulary selections that will return the right typeof resource in light of surface-based considerations, regard-less of scientific content- “I would want a lot of images, Iwant to try to filter that in somehow, illustration.”

Table 5 provides a detailed list of example statementsfrom participants in each experimental condition. Note thatparticipants using the textual search interface rarely talkabout science content or evaluate their textual choices inlight of task demands; most science content found in tex-tual search protocols consists solely of keywords. For theseparticipants, reflection on their information needs mostlyconsists of attention to surface-based features in the formof interface vocabulary options (e.g., illustrations) that canbe selected. In contrast, many statements from students us-ing the multimedia strand maps interface actively considerthe appropriateness of the given science knowledge to the

Multimedia displays for conceptual discovery: information seeking with strand maps 247

information needs of the task. Engaging with rich sciencecontent in the form of strand maps appears to greatly facil-itate the matching process between information needs andpotential resource relevance.

6 Discussion

The results of this study suggest that the multimedia strandmaps interface supported users’ engagement with sciencecontent during search, as demonstrated by their increaseduse of content-based processes during task completion.These users focused on the match between conceptually rel-evant benchmarks and task demands while engaged in dig-ital library search; these processes characterized both theusers’ interaction with aspects of the interface (analyzingscience content appearing on the screen) and their selec-tion of strategies for continued search and navigation (decid-ing what to do or where to go next). In contrast, users whoused a textual search with optional vocabulary selection (theexisting DLESE search interface) were more likely to usesurface-based approaches to retrieving resources. Users us-ing textual search mainly focused on selecting appropriatekeywords from the task descriptions and selecting optionalvocabulary choices (e.g., lab activity) to limit search results.The interaction between process type (content vs. surface)and search interface (strand maps vs. textual) clearly indi-cates that the design of a search interface can influence thecognitive behaviors of users during information seeking.

Current results further suggest that learners with lim-ited domain knowledge can be supported in content-basedsearch processes by a multimedia interface that providesconceptual science information in an easily navigated spa-tial organization. The quantitative and qualitative data col-lected in this research provide converging evidence that themultimedia strand maps interface changed users’ cognitiveprocesses during task completion such that they were morefrequently engaged with science content during informationseeking. We argue that support of content-based (science-focused) processes is particularly important to individualswith limited domain knowledge because it promotes analy-sis and interpretation of science content that is necessary forlearning. In addition, support that engages learners with sci-ence content early in search tasks may better approximateexpert search processes, where informational content is em-phasized over surface-level characteristics. The current re-sults support further exploration of strand maps as an impor-tant type of multimedia interface to promote content-basedcognitive processes through conceptual-browsing behavior.That is, strand maps appear to provide visual means for richexploration of, and engagement with, science content duringdigital library use.

It may be suggested that strand maps provide too muchinformation, essentially leading learners to the correctsearch results by providing a convenient display of con-ceptual knowledge relevant to a particular science domain.

However, results from the user study argue against thissuggestion.

Experimental subjects could easily have scanned the vi-sual displays for relevant keywords, clicked on the nodecontaining a keyword, and focused on surface-level char-acteristics of the returned search results. That is, if strandmaps led learners to relevant search results without concep-tual engagement, there would be no difference between theprocesses demonstrated by learners using the strand mapsversus the textual search. However, our results clearly sup-port the finding that strand maps support qualitatively differ-ent search processes than the textual search interface. Strandmaps can be considered enabling interfaces in the sense thatthey make concepts and their interconnections visible andthus support learners in engaging with the science domain ata more conceptual level, as measured in our experiment bycontent-based processes.

At present, the influence of strand maps on search pro-cesses appears to be limited to those processes that charac-terize user performance of a search rather than its planningor evaluation. It is important to note that the presentationof search results provided by both the strand maps and thetextual search interfaces was identical. Despite differencesin the search interface (strand maps vs. textual search) thatwas used to retrieve a resource list, users saw the same in-formation once the retrieval was complete. Thus, it may bereasonable to expect that users would revert to similar eval-uation strategies without residual influence from the previ-ously used search interface.

The lack of representation difference in the display of re-trieved results characterizes a disconnect in support acrosssearch phases. Continued support of content-engagementduring evaluation may require an interface designed to pro-mote content-processes relevant to evaluation. However, itshould also be noted that participants were not required toexplain their rationale for acceptance or rejection of a re-source during task completion. Thus, it may be that the avail-able data from spontaneous verbalizations are too sparse touncover potential differences in evaluation processes per-formed by participants based on residual influence of strandmaps or textual search interfaces. Further research is neces-sary to determine if evaluation processes can be influencedby interface design. Our results do provide a promising foun-dation for such work by demonstrating that strategically de-signed interfaces can have significant impact on the cogni-tive processes that they are intended to support.

Although the current research found consistent andstrong effects of strand maps as a multimedia tool for dig-ital library search, the nature of the study places limits onthe conclusions that can be made. In this case, we usedundergraduate students as participants in order to replicateusers with limited domain knowledge in an educational do-main. However, it is unclear whether strand maps wouldcontinue to show conceptual benefits for users with higherdomain knowledge. Will more knowledgeable users findthese types of multimedia support helpful or distracting?Future research should address these issues in addition to

248 K. R. Butcher et al.

considering more naturalistic tasks outside a laboratory set-ting; for example, examining search processes and learningoutcomes with motivated educators doing personally rele-vant tasks.

7 Conclusions

Multimedia search interfaces can provide a unique and use-ful tool supporting conceptual discovery, especially for ed-ucational digital libraries. Strand maps show promise as asearch interface to support students’ engagement with sci-entific information and relevant knowledge during informa-tion seeking. Increased use of content-based processes dur-ing search performance may reflect movement toward moreexpert processes and may provide more opportunities forconceptual discovery, leading to learning in a domain. Al-though additional research is needed to understand the ex-tent to which strand maps and other multimedia search in-terfaces can support improved learning outcomes, it is clearthat strategically designed tools can promote users to en-gage in useful cognitive processes during educational searchtasks.

References

1. Macdonald, J., Mason, R.: Information handling skills and re-source based learning in an open university course. Open Learn.13, 38–42 (1998)

2. Deniman, D., Sumner, T., Davis, L., Bhushan, S., Fox, J.: Merg-ing metadata and content-based retrieval methods. J. Digit. Inf. 4(2003) http://jodi.ecs.soton.ac.uk/Articles/v04/i03/Deniman.

3. NRC: National Science Education Standards. Washington, DC:National Research Council (NRC), National Academy Press(1996)

4. Sumner, T., Marlino, M.: Digital Libraries and Educational Prac-tice: A Case for New Models. ACM/IEEE Joint Conference onDigital Libraries (JCDL ’04) Tuscon, AZ, ACM Press (2004)

5. AAAS: Benchmarks for Science Literacy. New York: Project2061, American Association for the Advancement of Science,Oxford University Press (1993)

6. AAAS: Atlas of Science Literacy. Washington, DC: Project 2061,American Association for the Advancement of Science, and theNational Science Teachers Association (2001a)

7. AAAS: Science for All Americans. New York: Project 2061,American Association for the Advancement of Science, OxfordUniversity Press (1989)

8. AAAS: Designs for Science Literacy. New York: Project2061, American Association for the Advancement of Science,Oxford University Press (2001b)

9. Marlino, M., Sumner, T.R., Fulker, D., Manduca, C., Mogk, D.:The Digital Library for Earth System Education: Building Com-munity, Building the Library. Communications of the ACM Spe-cial Issue on Digital Libraries, pp. 80–81 (2001)

10. NSDL: National Science Digital Library Progress Report: Fall2000–Summer 2003. Boulder, CO, NSDL: 29 (2003)

11. Kuhlthau, C.C.: Longitudinal case studies of the informationsearch process of users in libraries. Libr. Inf. Sci. Res. 10, 257–304 (1988)

12. Kuhlthau, C. C., Tama, S.L.: Information search process oflawyers: A call for ‘just for me’ information services. J. Doc. 57,25–43 (2001)

13. Vakkari, P.: A theory of the task-based information retrieval pro-cess: A summary and generalisation of a longitudinal study. J.Doc. 57, 44–60 (2001)

14. Furnas, G.W., Landauer, T.K., Gomez, L.M., Dumais, S.T.: TheVocabulary Problem in Human-System Communication. Com-mun. ACM 30, 964–971 (1987)

15. Mayer, R.E.: Multimedia Learning. Cambridge, CambridgeUniversity Press (2001)

16. Carney, R.N., Levin, J.R.: Pictorial illustrations still improve stu-dents’ learning from text. Educ. Psychol. Rev. 14, 5–26 (2002)

17. Goldman, S.R.: Learning in complex domains: when and why domultiple representations help? Learn. Instr. 13, 239–244 (2003)

18. Mayer, R.E.: The promise of multimedia learning: using the sameinstructional design methods across different media. Lear. Instr.13, 125–139 (2003)

19. Narayanan, N.H., Hegarty, M.: Multimedia design for communi-cation of dynamic information. Int. J. Hum.-Comp. Stud. 57, 279–315 (2002)

20. Marchionini, G.: Information Seeking in Electronic Environ-ments. Cambridge: Cambridge University Press (1995)

21. Chmielewski, T.L., Dansereau, D.F.: Enhancing the recall of text:knowledge mapping training promotes implicit transfer. J. Educ.Psychol. 90, 407–413 (1998)

22. O’Donnell, A.M., Dansereau, D.F., Hall, R.H.: Knowledge mapsas scaffolds for cognitive processing. Educ. Psychol. Rev. 14, 71–86 (2002)

23. Rewey, K.L., Dansereau, D.F., Skaggs, L.P., Hall, R.H., Pitre,U.: Effects of scripted cooperation and knowledge maps on theprocessing of technical material. J. Educ. Psychol. 81, 604–609(1989)

24. North, C., Schneiderman, B.: Snap-together visualization: Canusers construct and operate coordinated visualizations? Int. J. Hu-man Comp. Stud. 53, 715–739 (2000)

25. Binding, C., Tudhope, D.: KOS at your service: Programmatic ac-cess to knowledge organisation systems. J. Digit. Inf. 4 (2004)http://jodi.ecs.soton.ac.uk/Articles/v04/i04/Binding/.

26. Arms, W., Hillmann, D. et al.: A Spectrum of Interoperability:The Site for Science Prototype for the NSDL. D-Lib Magazine. 8(2002) http://www.dlib.org/dlib/january02/arms/01arms.html.

27. OAI: The Open Archives Initiative Protocol for Metadata Har-vesting. OAI-PMH, Open Archives Initiative. Last accessed 2005(2001)

28. Ahmad, F., Bhushan, S., Gu, Q., Sumner, T.: The concept spaceinterchange protocol. ACM/IEEE Joint Conference on Digital Li-braries (JCDL ′04) Tuscon, AZ, ACM Press (2004)

29. Sumner, T., Ahmad, F., Bhushan, S., Gu, Q., Molina, F., Willard,S., Wright, M., Davis, L., Janee, G.: Linking learning goals andeducational resources through interactive concept map visualiza-tions. Int. J. Digit. Libr. 5, 18–24 (2005)

30. Sumner, T., Ahmad, F., Gu, Q., Molina, F., Willard, S., Wright,M., Davis, L., Bhushan, S., Janee, G.: A web service interface forcreating concept browsing interfaces. D-Lib Magazine 10 (2004)

31. W3C: Scalable Vector Graphics (SVG) 1.1 Specification,Ferraiolo, Jon, Fujisawa, Jun, and Jackson, Dean. Last accessed2004 (2003)

32. Gu, Q., Ahmad, F., Molina, F., Sumner, T.: Dynamically gen-erating conceptual browsing interfaces for digital libraries us-ing SVG. Third Annual Conference on Scalable Vector Graphics(SVG Open ′04) Tokyo, Japan (2004)

33. Ericsson, K.A., Simon, H.A.: Protocol Analysis: Verbal Reportsas Data (Rev. ed.), Cambridge, MA, MIT Press (1993)

34. Chi, M.T.H., de Leeuw, N., Chiu, M.-H., LaVancher, C.: Elicit-ing self-explanations improves understanding. Cogn. Sci. 18, 439–477 (1994)

35. Kintsch, W.: Comprehension: A Paradigm for Cognition.Cambridge: Cambridge, University Press (1998)