Information seeking in a multimedia environment by primary school students

Information Seeking in a Multimedia Environment by Primary School Students

Andrew Large Jam&id Beheshti Alain Breuleux

McGill University

Three multimedia CD-ROMs were placed in two grade-six classrooms for the duration of a class project. The 53 students, working in small groups, were free to search them during designated class periods. Sample sessions at the workstations were captured along with a recording of the searchers’ voices. The retrieved information was incorporated by each group into a written assignment, an oral class presentation, and a three-dimensional model. The search strategies were analyzed using a case study methodology. The content of the CD-ROMs and the students’ work were analyzed to identify from which sources the students had drawn their information. The students were able to cope effectively with the physical manipulation of the three different CD-ROM interfaces, although they found certain features easier than others. Constructing effective search strategies was more difficult, and relatively little use was made of the searching in contrast to the browsing facilities. Students demonstrated an ability to extract selectively information and to evaluate sources in terms of usefulness as well as enjoyment and ease of use. Class assignments may have to be re- designed if the full capabilities offered by multimedia instructional products are to be fully exploited.

Multimedia information sources on a wide variety of platforms, including CD- ROMs and World Wide Web sites, are being used by students, alongside tradi-

The unreserved support of the school board, school principal, teachers and, of course, students is gratefully acknowledged. Data collection and analysis would not have been possible without the ded- icated work of the two research assistants, Katalin Simon and especially Shelley Woods. The research was supported by a grant from the Social Sciences and Humanities Research Council, Can- ada.

Direcf all corwspon&~e to: Andrew Large, Graduate School of Library & Information Studies, McGill

University, 3459 McTavish, Montreal, Quebec H3A 1Yl Canada <[email protected]>.

Library & Information Science R esearch, Volume 20, Number 4, pages 343-376 Copyright 0 1998 Ablex Publishing Corporation All rights of reproduction in any form reserved. ISSN: 0740-8188

343

344 Large, Beheshti, 81 Breuleux

tional print sources, in preparation of class projects. These multimedia sources are often consulted in the school library or in the students’ own homes. An alternative, however, is to make them accessible from the classroom itself just as other information sources can be brought into the class. The object of this research was to investigate over several weeks how grade-six students in a primary school would choose to use three CD-ROM-based multimedia information sources in support of a class project on life in the middle ages.

This article reports only upon the retrieval aspects of Year 1 of an ongoing three- year research study. In particular, it seeks answers to three specific questions: do children constitute a distinct user group for information retrieval systems; what kind of interface navigational features are best suited to instructional programs targeted at primary school students; and do multimedia instructional products have a role to play in the classroom?

REVIEW OF THE LITERATURE

Information Seeking

Information seeking has generated a large literature over the years. It is only more recently, however, that children have been studied as a special category of information seekers who may approach an information system differently from adults and whose demands upon that system consequently can be rather different. As Druin and Solomon (1996, p. xii) state, “when developing multimedia environments for children, we as designers must remember that children are not just short adults. We cannot water down multimedia environments designed for adults and expect them to be valuable environments for children.” Negroponte (1995) argues that children, unlike adults, accept as normal the interchangeable and non-linear nature of digital information. It offers them above all an opportunity to explore in hitherto impossible ways.

A number of important studies have now been conducted to investigate children as information seekers, and they are beginning to shed light on how children do find information in large databases of various kinds. Kuhlthau (1989) developed a model of the Information Search Process (ISP) comprising six stages: initiation, selection, exploration, formulation, collection, and presentation. The last three stages were the specific concerns of the current project. Marchionini (1989) explored how primary school children searched for information on a CD-ROM encyclopedia. Using test queries in an experimental setting, he found that, in general, young novice searchers could use command-driven Boolean retrieval software with minimal training, but that they seldom took full advantage of the available retrieval capabilities. Although the students were able to identify key facets in search queries they encountered difficulties in formulating effective search terms. Borgman and her colleagues have investigated the use of OPACs in school environments. They conducted empirical studies of children’s information-

Information Seeking In a Multimedla Environment 345

seeking behavior and found that higher success rates were achieved when they used concrete search terms that matched the catalog’s terminology (Borgman, Hirsh, Walter, & Gallagher, 1995). Although their findings suggest that children are highly successful at finding bibliographic information, Hirsh (1997) concludes that more information retrieval tools designed specifically for children are necessary. In particular, they need help in formulating and articulating search queries that are appropriate for the systems.

Large, Beheshti, Breuleux, and Renaud (1994a) investigated the use by grade- six students of multimedia encyclopedias on CD-ROM. In studying retrieval behavior in an experimental setting, they found that young novice searchers could use an information retrieval system that incorporates several different retrieval paths with minimal training. The students were willing to explore and experiment with the interface, and overall their use was highly interactive. They exhibited considerable skill in selecting search terms, although they did encounter spelling problems. Oliver and Oliver (1996) identified disorientation, navigation inefficiency, and cognitive overload as the most serious problems encountered by children using interactive multimedia information sources in educational settings,

Solomon (1993) investigated OPAC use in a primary school. His methodology has certain similarities with the one adopted by the current authors. He opted for a case study rather than an experimental approach, observing children using the OPAC under normal searching conditions and over an extended period of time. He was struck by the success achieved by even the youngest grade-one students.

Interface Design

Much research has been conducted to elucidate interface design principles, although rarely with children in mind as users. Mandel(1997, p. xi) believes that “the best interface is the one that lets users do what they want to do, when they want to do it, and how they want to do it.” Marchionini (1992) argues that most users want to achieve their search goals with a minimum of cognitive load and a maximum of enjoyment. They will also perform better and continue to use systems that give them pleasure or are interesting. Microsoft’s Windows Inter- face Guidelines (Microsoft Corporation, 1997) remind us that “a well-designed user interface is built on principles and a development process that centers on users and their tasks.” Shneiderman (1992) and Galitz (1997) provide several general design principles to provide the user with both success and satisfaction. Shneiderman (1992) and Head (1997) list the advantages offered by direct manipulation interfaces, but recognize that the graphics can become irksome gimmicks for the frequent user. Beaulieu (1997) has called for further investiga- tion into the cognitive load which different interface environments or features may impose.

346 Large, Beheshti, & Breuleux

Multimedia

Park and Hannafin (1993) argue that designers must expand their perspective to consider teaching and learning methods and models that previously were unfeasi- ble or unavailable. Mayer and Anderson (1992) argue that a research-based theory of how to design multimedia informational tools using words, images, and sound is lacking. Yildiz and Atkins (1992) stress the importance of the learning context, the learning task, and the learners’ characteristics in relation to multimedia design. Reynolds and Ehrlich (1992) and Cates (1992) suggest general outlines for how multimedia might be designed. Large (1996) has reviewed the research literature on the use of computer animation in an instructional environment. Strong evidence is accumulating that children respond more positively than adults to animation. Barron and Atkins (1993) comment upon the paucity of guidelines for instructional designers to follow when incorporating audio into multimedia.

A number of studies, including Levin, Anglin, and Carney (1987), Levin and Lesgold (1978), and Rieber (1990) have suggested that visual information facili- tates learning only when its attributes are selectively applied in relation to the specific learning requirements of a given task. Large, Beheshti, Breuleux, and Renaud (1994b, 1995, 1996) found that still and moving images enhance textual presentation of information, but only under certain circumstances: the content must involve the notion of movement; the information must be conveyed consis- tently by both textual and visual media, and links should be made between the two media; and the learning objective should be at the higher cognitive level of under- standing (as measured, e.g., by the capability to solve a problem or replicate a procedure) rather than a lower cognitive level (e.g., the capability simply to repro- duce accurately the conveyed information). Very little research has been undertaken on the role of sound in multimedia but Mann (1992) has proposed a model for the creation of sound-animation relationships. Janus (1997) describes the process of creating one multimedia product--The World Book Encyclopediu- and discusses some of the design issues encountered. Testing showed that topic searching often provided the youngest users with the most direct answers and so the designers made it the default search.

Methodology

Beaulieu (1997) has argued for the value in interface design of undertaking a more closed type of operational trial with a homogeneous user group which would carry out repeatedly a set of information-seeking tasks or activities. In this study, a homogeneous group of grade-six students from one primary school comprised the users. They were assigned the information-seeking task of researching a project for a class assignment. This involved them searching three CD-ROMs several times per week over a four-week period. A sample of the searches were recorded on videotape (visual images and searchers’ spoken comments at the terminal) for subsequent analysis.

Information Seeking in a Multlmedia Environment 347

This methodology differed from that adopted by the researchers in an earlier study (Large, Beheshti, Breuleux, & Renaud, 1994b, 1995,1996) which opted for an experimental rather than a field environment. The experimental approach enabled quantitative data to be gathered under controlled conditions. Nevertheless, the qualitative case study in the current research offers a number of advantages. First, each student spent around three hours at the computer spread over a four- week period, providing an opportunity potentially to develop searching skills. It is very difficult to extend experimental control over such a long time period (in the previous project students spent around eight minutes per topic at the computer). Second, the searchers had a concrete goal - to collect information for an assignment (and not merely to complete an artificial task set by the experimenter).

The multimedia information sources were made available at three workstations in the classroom itself. They could freely be used by the students during regular sessions time-tabled by the teachers over the course of four weeks (researchers are now realizing that many information seekers conduct multiple searches on the same information problem over a period of time, rather than one search which immediately satisfies their need, and that these searches are connected (Spink, 1996)). The students were allocated to groups of three by the teachers, following the normal practice for such class projects. Although they usually adhered to these groups, group interaction was allowed to take place; no attempt was made by the researchers to hinder inter-group exchanges of information, and certainly on occasion a student from another group would offer advice to a group currently searching at the computer. Students were also free to find information from other sources-both print and electronic-available in the classroom, school or public library, or from home. The students’ objective was to gather information for a class project.

The research was undertaken in an English-language primary school in Quebec, although classes are also regularly conducted in French. Two entire grade-six classes (53 students) participated in the research. They shared an open-plan area and it was decided to work with them both as if they constituted, for the purposes of the research, just one class. Both teachers expressed their satisfaction with this arrangement. It would be difficult to define an average or typical school in which to undertake any kind of research. The particular public school participating in this research project is located in an affluent suburban district of a large, multicultural city. The fact that almost all the students came from homes with computers (and most of the remainder had access to computers from parents’ offices or friends’ homes) is evidence of the relative affluence of the school’s catchment area.

The class project was entitled “The Middle Ages”. Its selection was governed by the content of the grade- six curriculum in the individual school, itself influenced by the policy of the school board in which the school is located and more generally the common curriculum followed by all primary schools in Quebec. The project also had to be agreed upon by the teachers as well as the researchers. Initially the research team had been interested in exploring the use of multimedia in a science project where the motion attributes of video and animation might be put to good use. The final selection of the project, however, was confined by the language of


instruction (some subjects are taught in French rather than in English) and the specific curriculum plans of the grade-six teachers for the semester.

The project’s purpose was to enhance knowledge of the Middle Ages b researching this period and then comparing it with life at the end of the tK 20 century. It comprised three components:

l A written assignment on people in the Middle Ages (chosen from the following list: bishop, knight, monk, noblewoman, nun, pilgrim, serf, shepherd, and troubadour) and the manorial system, alongside a comparison with their modern counterparts;

l An oral presentation, with accompanying poster(s) on three of the following topics - knighthood, armor, weaponry, the Order of the Garter, a noblewoman’s costume, Ring John and the Magna Carta, heraldry, and medieval tools; and

l The construction of a three-dimensional model of a manorial system, including (as a minimum) a castle, church, windmill, fields, and animals.

Three identical workstations were installed in the classroom for the duration of the project. Each comprised a 486 IBM-compatible computer with a SVGA moni- tor and 8x CD-ROM drive, a Microsoft mouse and a Desk Jet 400 Hewlett Packard printer. Three sets of the three multimedia titles were made available at each workstation so that the students could use at any time whichever one they so wished.

The students were given a relatively brief introduction to using each of the three CD-ROM multimedia titles. The objective of this training was to introduce them to the CD-ROMs so that they could begin searching. More extensive training was not offered as all three CD-ROMs are intended for the home or school environment and the interface designers cannot assume that their users will be given training sessions beyond what is provided with the products themselves. The students themselves were then free to explore the interfaces and decide for themselves how exactly they would conduct their searches. As the project extended over four weeks, learning time was available. The 17 groups of students conducted in total approximately 50 hours’ searching during this interval, or on average around three hours per group.

A sample of the searches at one workstation were directly captured on videotape -approximately seven hours were recorded. A microphone was also installed at the table on which this workstation was set up, enabling the searchers’ voices also to be captured synchronously with the screen image on the videotape. The student groups were not assigned to a specific workstation, but took whichever was free. This ensured that almost all groups at one time or another were captured on tape, but no group had all its sessions taped. Either one or two research assistants were present during all search sessions. Students’ comments indicated that observation as well as audio and video recording were unobtrusive. Sometimes a research assistant would sit at the workstation where the search was being taped; at other times she would be at one of the other two workstations, in which case detailed

Information Seeking In a Multimedia Emrironment 349

notes of search strategies were kept. As a consequence, data were collected to some extent from all three workstations. From time to time, the research assistants would ask students why they had taken certain steps as well as to comment upon their strategies.

The group written assignments were photocopied, but without any marks or comments from the teachers (in accordance with the school board’s confidentiality policy). The oral presentations were audiotaped. The models were photographed on 35 millimeter slides and also captured on a digital camera. Teachers and students were asked about sources used for their project other than the three CD- ROMs, which included several other CD-ROMs available either in the School library (Grollier’s Encyclopedia) or at home (Encarta 97 and Compton’s Encyclo- pedia) as well as printed books and encyclopedias in the school and public libraries or at home, and the Internet (from home). All these sources (with the exception of the Internet) were examined to identify sections potentially relevant to the written assignments, oral presentations and models. Finally, a frame analysis technique (DeRemer & Bracewell, 1989; Large, Beheshti, Breuleux, & Renaud, 1996) was employed to match elements within the projects to the sources from which the information had been taken.

A brief subject pre-test was administered to establish the students’ prior knowledge of various aspects of medieval life. Lack of time at the end of the school year prevented post-project interviews being conducted with each student. Instead, the students were asked to complete a written questionnaire comprising closed and open questions intended to ascertain their opinions of the three multimedia sources used (see Appendix). The research assistant briefly explained the purpose and nature of the questionnaire, and was on hand while the students answered it to deal with any questions. All but one student (n = 52) answered it.

The Sources

The overall goal of the research is to establish design criteria for multimedia information sources that can be used in a learning context within the school curriculum. An objective within this goal was to investigate the ways in which students search such multimedia sources. To this end it was decided to select three products whose interfaces and retrieval approaches differ. Given the selection by the teachers of a class project on the Middle Ages, it was also necessary to select products that are relevant to this historical topic. The three CD-ROMs were chosen on their appro- priateness to test different interface designs as well as their suitability for the specific user group and relevance to the subject of the class project.

Encarta 96

A number of general multimedia encyclopedias are available on CD-ROM, any one of which would have served the research purpose. The Encurtu 96 Encyclope- dia (Microsoft Corporation, 1995) was finally chosen because it has garnered very

350 Lsrge, Beheshti, & Breuleux

FIGURE 1 Encarta 96 Sreen

positive reviews (see, e.g., Jacd, 1996; Pack, 1995) and is widely used in schools, but was not available in the specific school involved in the research. Although the Encarta 97 Encyclopedia was already available at the time of the research and also was not available in the school, it was not chosen because it is stored on two discs and necessitates constant disc-switching in the CD-ROM drive.

The Encarta 96 “Home Screen” offers 8 choices, but only one-Find-leads to a search of the database; the others enable users to tour the encyclopedia, create a media show, test their knowledge, and so on (and in practice were not used by any of the students). Selection of the Find option leads to Pinpointer, an alphabetical list of all the 26,386 articles in the encyclopedia. When a search term or phrase is entered, the list automatically scrolls to the closest alphabetic match; alternatively users can simply scroll through the list. In either case, an article is displayed by clicking on a title. Long articles are divided into sections, and section headings can be displayed by opening the Outline menu. Related articles are listed in a second pop-up menu. Articles or sections of articles can be printed. A dictionary can be accessed from a Tools pull-down menu (see Figure 1)

Encarta also offers several methods of conducting more specific searches. These are accessed from the Pinpointer screen. Word search enables a search to be undertaken for occurrences of individual words. Boolean AND, OR and NOT operators

Information Seeking in a Multimedia Environment 351

are available as well as right-hand stem truncation, phrase searching and proximity searching (within eight words). The results of such a word search are presented in Pinpointer as a new list of only those articles (by title) containing the sought word(s) or phrase. Searching can also be restricted to one of nine broad categories (e.g., Geography, History, or Life Science) or to articles containing images or sound (Media). Finally time and place searches can be conducted on a scrollable chronological time line or maps, respectively. Search assistance is provided through a pull-down Help menu or a pop-up “Wizard” that leads the user to a Word, Category, Media, Time or Place search.

Encarfa 96 contained a wealth of information, scattered through many different articles, that is relevant to the students’ project. These include Armor, Artillery, Bishop, Coat of Arms, Heraldry, Knight, Magna Carta, Middle Ages, Monasti- cism, Nobility, Nun, Pilgrim, Seignorialism, Serfdom, Tool, Troubadours and Trouveres, and Warfare.

Castle Explorer

Castle Explorer (1996) is one of a family of products published by DK Multimedia (formerly Dorling Kindersley) and intended for a young audience. It encourages

FIGURE 2 Castle Explorer


information retrieval by exploration of images of the castle itself, various sections of the castle and surrounding countryside, and individual rooms within the castle. Users can zoom in on a selected section of the castle such as the Garrison Quarters or the Middle Bailey, and then scroll horizontally or vertically through a cutaway of that section, clicking on objects to display an explanatory caption or to slide open a portion of roof or wall to reveal the scenes within (see Figure 2). The color- ful images are accompanied by sounds associated with the relevant castle activity. Several video sequences show the insides of various castle rooms.

Several subsidiary navigational aids are offered by Castle Explorer, accessed from a space bar at the bottom of the screen which is activated by clicking on an icon. “Library” reveals six chained books (Health, Society, Warfare, Trades, Crime, and Food); when one is selected the relevant chapter in the book is displayed, the pages of which can be turned forwards and backwards. “Word Search” displays a vertically scrollable index of terms that link the user to the relevant page in a book or the relevant caption from the castle scenes (see Figure 3). Hypertext links are available from “hot” terms within both the books and the captions to a Glossary, and from the captions to relevant pages in the “books”. “,a$ lists the preceding steps taken by the user. Entire screens, active windows or text sequences can be printed or copied from an “Options” menu, and finally “Help” can be chosen.

FIGURE 3 Word Search on Castle Explorer


Castle Explorer includes information on many of the topics required by the students to complete their project. These include armor, clothing, feudal society, heraldry, knights, monks, nuns and monasteries, tools and weapons.

Exploring Castles

Exploring Castles (also confusingly entitled Castles of the British Isles), published by Anglia Multimedia in 1995, relies primarily upon hierarchically organized menus. The main menu comprises: About Castles, Building a Castle, Living in a Castle, Castle Guides, and Castle Data. Selection of any menu item will lead to another level of menu selections: for example, Building a Castle reveals a new menu comprising Occupations, Equipment, Materials, and Building Techniques (see Figure 4). Buttons and icons are provided to undertake operations such as turning pages, returning to the main menu or in places accessing new sections of the information store. Typically, information is displayed on a divided screen, text on the left and still images on the right. On some pages a pop-up window offers further menu choices. Elsewhere, links are provided from one page to another (e.g., from the page on masons within Occupations to the page on stone within Materials). The interface also employs in various places scrollable alphabetical

FIGURE 4 Exploring Castles


lists from which choices can be made , and maps and bar charts. A cutaway vertical plan of a castle allows users to choose any particular floor to learn about, and in one place a time line provides access to periods between 1066 and 1800. Several movie sequences are included with spoken commentaries, as well as a few animation sequences. Marked words throughout the text are linked via hypertext to a glossary. A “Find Text” pop-up window, activated by an icon, allows a search to be undertaken on any word or truncated word within the current main section. Exploring Castles was found particularly useful for its information about tools and weapons.

Navigational and Search Tools on the CD-ROMs

Each CD-ROM exemplifies one primary navigational tool:

Pop-up/pull-down menus: Encarta; Hierarchical menus: Exploring Castles; and Direct manipulation: Castle Explorer.

The various search tools offered by the CD-ROMs are listed in Table 1.

Student Characteristics

The students were,1 1 or 12 years of age; 55% were girls and 45% boys. A majority of their homes (88%) contained a computer, and in these cases the students had access to it (40% daily; 47% weekly; and 13% monthly, on average). Even the ones without a home computer reported computer use at friends or neighbors houses, or a parent’s office. A majority also had used CD-ROMs outside school,

Search Tool

TABLE 1

Search Tools

CD-ROM

Encarta Castle Exdorer Explorino Cast/es

Article title search

Full-text search

Item term search

Hypertext links

Boolean

Word proximity

Truncation

Dynamic browsing

Default search

Secondary search tool

Extensive between Articles

AND, OR, NOT

Up to 8 words

Right-hand

“Word Search”

Limited links from captions to “books” and glossary

Image rotation, navigation, manipulation

Only within each main section

Only to glossary

Right-hand


and all had used CD-ROMs in the School’s Information Technology Laboratory (but not the three CDs used in the project).

The pre-test for knowledge about life in the Middle Ages revealed that the students knew little about this topic before the start of the project; the influence of any such pre-knowledge on the written assignments, oral presentations, or model building could therefore be discounted.

Information-Seeking Behavior

Navigation. Castle Explorer employed the most innovative (or least traditional) interface. Its object-oriented approach encouraged the students to explore the information store in a visual and tactile manner. In order to retrieve information it was necessary to use a variety of manipulation techniques: image rotation (e.g., revolving the 3-D image of the castle on both a horizontal and vertical plane in order to “enter” a particular section such as the lower bailey or the castle surrounds); horizontal and vertical screen scrolling; clicking on objects to reveal what is “inside”, and so on. The students demonstrated impressive competence in undertaking such maneuvers, and required little practice to achieve such dexterity. This is not to say, however, that they found it a simple task to extract information from the CD-ROM.

Ironically, the visual sophistication of the interface proved to be its main down- fall, at least when Castle Explorer was being used to find specific information for a class project rather than to explore in a more serendipitous fashion. The opening screen offers users an opportunity to start a new game or open a saved game. The scenario for this game is “a dangerous secret mission inside the stone walls of Baron Mortimer’s castle” in 14* century Europe. It is intended to introduce users to the CD-ROMs’ interface and to make them explore the castle. In other words, it supports a browsing rather than a searching mode. Not surprisingly, all student groups, on their first encounter with Castle Explorer, chose to undertake the game rather than delve deeper into the interface in order to discover the other search options. Some groups became engrossed in the game and spent more than 45 minutes on it without explicitly identifying relevant information for their project, though it could be argued that they familiarized themselves thoroughly with the castle interior and its environs.

Some students also were sufficiently interested in Castle Explorer to explore it in a less-directed way. The members of one group, for example, were asked by the research assistant why they looked up “executioner” as this was not an occupation included in the assignment. The answer given was that “we were just exploring.”

When exploring the castle, students did open the caption windows, but in many cases they scarcely paused to look at the text before the mouse was once again on the move towards the next visual stimulation. The visual richness of the interface encouraged a high level of interactivity with the interface, but the constantly changing screen left little time for information absorption, let alone contemplation.


By the second or third search session, students were discovering alternative navigational aids such as Library and Word Search. In some cases resort to Word Search was in desperation after castle browsing proved unproductive. In one search, for example, the students were looking for information on weapons. One member of the group asked the others “Where would you hide your weapons?” They decided to look in the castle’s Garrison Quarters, and within it check the prison, but without success. They continued to navigate through the castle, opening the caption at the castle gates, but then quickly closing it (“No, we don’t need to look at the gates”). Another member of the group added: “So we need to look somewhere else then. Come on, we have to find the stuff.” The group discussed various place where they might look before deciding on the Lower Bailey. They opened the caption on the Blacksmith, but quickly decided it is “No use.” Only at this point did one of them conclude: “OK, let’s go to Word Search”. Here they looked up Weapons and could conclude triumphantly: “Yes, we’ve got it.”

On Encartu searches always began from the Main Menu with the Find option. As the following screen most obviously displayed the Pinpointer, almost all searches then continued with a search of the encyclopedias’s article titles via the Pinpointer. There seemed to be an expectation that any search term typed into the Pinpointer search box would find an article with that specific title. As a consequence, terms were often repeatedly entered, each time finding no matching article title. This problem was more severe with Word Search. The consequence of entering a search term here was, again, an alphabetical list of article titles in the Pinpointer window containing the sought term. In many cases the article, judging by its title, appeared to have little relevance to the sought term (and indeed frequently did have little relevance to the actual concept sought by the students). For example, a Word Search on “shepherd” to find information on the role of a medieval shepherd retrieved 83 articles extending over several Pinpointer windows. These are the titles listed on the fast screen: Abel; Akiba ben Joseph; Amado, Jorge; Aristaeus; Assyro-Babylonian Literature; Bagpipe; Barlach, Ernst; Basques; Belgian Tervuren; Brazilian Literature; Browne, William; Bulgaria; Children’s Crusade; and Clare, John. A word search on “garter” turned up COMMON GARTER SNAKE. The group trying this search then opted for a more complex word search, linking two terms with a Boolean AND: “order AND garter.” Very few students, however, used this facility in Encarta.

Students were prepared to experiment with the various navigational aids available on Encurta. For example, a search for “shepherd” began in the Pinpointer index to articles, moved on to the subject categories (Life Sciences, Religion and Philosophy, and then History), then to a word search (after seeking help from the research assistant) that retrieved nine articles, none of which were considered relevant, and finally to the Timeline; only at this point was the search abandoned.

The searches undertaken on Exploring CustZes, of which there were relatively few, tended to be long. The students seemed to have less grasp of direction than on the other two CD-ROMs and moved around the interface with little sense of purpose. The word search capability on Exploring CustZes was not used at all. In


the early stages of the project Exploring Castles was used as frequently as the other two titles, but its usage quickly declined as the project progressed.

Hypertext

Custk Explorer provides hypertext links between words in captions and relevant pages in the “books” and listings in the glossary, but they were infrequently used even though it is clear from some of the taped discussions that students were not always sure of words’ meanings. The articles on Encartu liberally include hypertext links, and they were used from time to time, but not frequently. Examples are a link from the BISHOP article to DIOCESE. Limited hypertext links between terms and glossary entries were also available on Exploring Castles, but they were hardly used at all.

Search Terms

Generally speaking, students employed single-word search terms, but occasionally entered search phrases (e.g., “people of the middle ages”). Search terms were usually directly linked to the project, and in many cases students searched only on the terms used by their teachers to present the project objectives. When asked if they had thought of looking under “sheep” to find material on shepherds, the student answered “No, I never thought of that.” This may have been due, in part, at least to the students being given little background information on the project. This discouraged experimentation with search terms. When one student was asked “Is nobility the same as noblewoman,” the answer was “I’m not sure.”

Help, Dictionary, and Glossary Features

In general, students showed a marked reluctance to utilize the help function on Encurta. In the post-search questionnaire, students were asked if they used the dictionary on Encarta or the glossaries on Castle Explorer and Exploring Castles. More than half the students (64%) responded that they had used the dictionary and 30% could not remember (6% said no). In the case of the glossaries, less than half the students answered they had used them (43%) and 29% could not remember (27% said no). It is impossible to relate these responses to actual use, as only a sample of the searches were captured on video for analysis. Nevertheless, the taped searches cast some doubt on the accuracy of the students’ responses to these two questions. The taped sessions revealed little use of either dictionary or glossary.

Castle Explorer provides hypertext links from terms in the captions to pages in the “books,” and from both captions and “books” to the dictionary, but this facility was infrequently used even though it is clear from some of the taped discussions that students were not always sure of words’ meanings. The articles on Encurtu liberally include hypertext links, and they were used from time to time, but not

358 Large, Beheshti, 81 Breuleux

frequently. Hypertext links were also available on Exploring Castles, but again were little used.

Persistence

When using all three CD-ROMs the students displayed a willingness to persist, often in the face of evidence that the sought information is not to be found. One group, searching on Encarta for the term “noblewoman” in the Pinpointer, first queried themselves as to the spelling before entering “nobleswoman” and then “noble”. The closest article title was “nobility”. This did not satisfy them, and they tried again searching on “noblewoman’s costume” and “noble” before opening the article on NOBLE GASES. Realizing that this was useless, they again searched on “noble’ followed by “woman” before finally deciding: “Ask the person who switches the CDs” (i.e., the research assistant). Students often noticed their own mistakes (likely to be more prevalent in group than individual work). One group was looking initially for “monk” in the Ancient History Category, but eventually they realized their mistake and switched to the Religion Category where they found a link to the article on MONASTICISM. Interestingly, one group touched upon the problem of knowing when enough information has been found on a topic to allow the search to be terminated.

Spelling Errors

None of the interfaces employed a primary search approach that depended upon terms being typed accurately at the keyboard. Although spelling errors were made by the students - shepherd and manor&l in particular caused problems for several students - they did not greatly handicap retrieval, because in most cases inputting a search term merely scrolled an alphabetical index of terms to that approximate location; the term itself was then selected from the list (with its correctly spelled version). In any case, all three interfaces, and especially Castle Explorer and Exploring Castles, offered browsing rather than retrieval techniques that were heavily exploited; a search term therefore was not necessarily required to begin a journey of exploration through the database.

Multimedia

Sound was available on all three CD-ROMs, but was not greatly used. On CastZe Explorer sound was present as a background effect, but otherwise was only used in the secret mission game. The research assistants observed that the students did not appear to pay much attention to it, and this conclusion was supported by the taped sessions. Few of the potentially relevant articles on Encarta incorporated sound. When sound sequences were accessed, it was normally for non-project purposes (e.g., one student listened to the national anthem of his father’s home-

Information Seeking In a Multlmedle Environment 359

land). Some of the video sequences on Exploring Castles included voice commentaries that were heard as the video was viewed.

More surprisingly, visual information did not play a very significant role, and was certainly subordinate to text. In the case of Castle ExpZorer, the interface itself relied heavily on images and therefore inevitably students used visual clues to search for information. Nevertheless, even in this case students spent much time accessing the textual information contained in the chained books, and this was the information that they either printed or took notes on. It was as if the rich visual information was considered interesting, but ultimately not the objective of the search. On Encartu many of the retrieved articles did include still images, but again relatively little attention was paid to them. Captions were rarely read or images expanded. One group found the article on WARFARE and its pictures of medieval weapons and commented, erroneously,: “Oh, cool...oh, we can’t print them, we can only draw them.” In the few cases where video sequences were accessed, they were viewed, but seemingly without any great enthusiasm judging by the recorded comments. One student, however, did respond to a question from the research assistant: “We like the CD-ROM better than a (printed) dictionary- you can look for different things. Some of the dictionaries are really old. We get pictures.”

In the post-project questionnaire students were asked which medium was most helpful in supplying them with information for their assignments. Taking all three CD-ROMs together, 92% answered the text only.

Information Capture

The intention of the students was to collect information for a class project. Not surprisingly, therefore, many searches resulted in data capture of some kind. Most commonly, entire articles or sections of articles were printed (from Encartu) or entire sections of the “chained books” (from Castle Explorer), with little preliminary reading to check for relevance. In some cases students copied sections by hand; the absence of a printing option on Exploring Castles made it essential in this case. Some images were also printed. Downloading was not an option for the students.

The very attractiveness of the object-oriented interface to Castle Explorer deterred students from dwelling for long on any one item. Indeed, it was noticeable that in many searches the cursor was seldom stationary on the screen for more than a couple of seconds. Caption boxes were opened and almost immediately closed; pages in the books were turned back and forward with scarce a pause for text scan- ning, let alone reading. Although Castle Explorer had a print option it was used with relative infrequency. Occasionally students would laboriously copy out sections of a book - typically one student reading and the other writing it down. But even here, the intention was often stronger than the deed: in one search, for example, a student says “We should write this down. Go get a pencil.” But, in fact, they quickly move on to a new image and the note-taking is abandoned.

360 Large, Beheshtl, 81 Breuleux

The absence of printing capabilities on ExpZoring Castles may have contributed to its relative lack of popularity and perceived usefulness, but students did not directly complain of this shortcoming and were willing to take notes by hand. A notepad could be opened on the screen in which a student could type notes for later downloading and printing. Unfortunately, the opened notebook tended to obscure most of the screen, thereby preventing a student from copying displayed information (in any case, students were not able in the classroom to download data for subsequent printing or incorporation into a word processing package).

In contrast, printing was widely used from Encurtu. On some occasions the students did scan an article before printing to check on its length or to highlight relevant sections for printing. In most cases, however, printing the entire article was almost a reflex reaction to retrieving it. The comment “Let’s print it all off and read it after” was expressed by one group but represented a widely held opinion. In a few cases, as with Cad Explorer, students chose to take notes rather than print. When two students were taking notes from the screen, a third one asked “ Why don’t you just print it?’ The answer was “No, because we want to take notes.” Relatively few still images were printed from Encartu even though this facility was available.

Student Assessment

The students’ reactions to the three CD-ROMs were gathered in two ways. First, comments made by the students while actually at one of the three workstations were captured by a microphone. Additionally, on occasions the research assistants did ask the students at the workstation why they had taken certain decisions: in retrospect this technique could have been employed more widely. Second, all students were asked to complete a post-project questionnaire which, in part, gathered assessments of the CD-ROMs.

There can be no doubt that the taped responses of students at the workstations revealed a marked preference for Castle Explorer. Almost without exception the students seemed interested as they navigated from screen to screen. Comments such as “It’s cool”, “Oh, its so cute”, “This one’s the best CD. It’s the coolest,” “That’s awesome,” and “Oh, that’s amazing” abound. Initially they seemed engrossed in the game, in many cases spending a long time on it, but with growing familiarity its attractions waned. The other two CD-ROMs did not elicit such spon- taneous enthusiasm.

Students also expressed their reactions to the three CD-ROMs, as well as other information sources used for the project, in a post-project questionnaire. The responses to the closed question part of this questionnaire are cumulated in Table 2. As in the comments gathered at the workstations, the students answered overwhelmingly that they had most enjoyed using Castle Explorer in comparison with the other two CD-ROMs or other resources such as printed books or the Internet. Encarta also scored well in enjoyment, and these two CD-ROMs scored far higher than Exploring Castles or the non-CD sources. Conversely, very few students least

TA

BL

E 2

P

ost

-Pro

ject

Stu

den

t Q

ues

tion

nai

re*

Prin

t O

ther

prin

ted

Exp

lorin

g C

astle

Q

uest

ions

B

ook

Enc

yclo

paed

ia

mat

eria

ls

Inte

rnet

E

ncar

ta

Cas

tles

Exp

lore

r O

ther

CD

s (A

) F

ound

mos

t 14

9

1 1

29

1 4

2 in

for

for

writ

ten

9

assi

gnm

ent

0”

(8)

Fou

nd m

ost

19

12

1 0

14

1 2

7 4

%

info

r fo

r or

al

s pr

esen

tatio

n

(C)

Fou

nd m

ost

10

1 5

1 3

0 2

0 16

info

r fo

r m

odel

X

rz

(D

) M

ost e

njoy

ed

4 2

1 3

16

3 30

0

5

(E)

Leas

t enj

oyed

12

3

1 0

3 27

2

0 (F

) Mos

t diic

ult

6 1

2 9

; 6

20

5 0

(G)

Leas

t diff

icul

t 12

3

1 B

2

34

1 5

0 z

Not

e: l 52

res

pons

es:

Not

all

stud

ents

ans

wer

ed e

very

que

stio

n, a

nd m

ore

than

one

ans

wer

was

giv

en b

y so

me

stud

ents

to

som

e qu

estio

ns

k c 3 2 s


enjoyed either Castle Explorer or Encarta; the same is not true of Exploring Castles, which was overwhelmingly the least enjoyed.

An explanation for the lack of enthusiasm for Exploring Castles can be sought in the answers to two other questions. Students found it both difficult to use and relatively unhelpful for their written assignment, oral presentation, or model building. In the case of CastZe Explorer, they also found it relatively unhelpful for any of the three assignments, but clearly this did not prevent them enjoying it.

The students were also asked an open-ended question about each CD-ROM. Individual answers differed considerably. The answers can be categorized here by their reference to the students’ enjoyment in using them, ease of navigating the interface, and usefulness for the project (see Table 3).

The student responses to the post-project questionnaire confirmed the evidence from the videotaped search sessions- that information was taken much more from the textual content of the CD-ROMs than from the still images, and that video sequences were of no use in supplying information for the students’ projects.

Despite the availability of the three CD-ROMs in the classroom, students continued to access many other sources. These included other CD-ROMs such as Compton’s Multimedia Encyclopedia, Grolier’s International Encyclopedia and Encarta 97, printed encyclopedias such as the World Book Encyclopedia, and various monographs. One student commented: “It is faster and there is more information on the CD, but if the book is specific then it is better” (he had found a book on armor and considered it more useful than the entries on armor on the various CD-ROMs). Some students also consulted Web sites from their homes. At the end of one search a student told the research assistant that “we found another word for manorial. Now we’re going to the library to look up more on seignorialism.” When asked why they were using Encarta instead of printed encyclopedias, one group answered: “It’s easier. You can just print it out on paper; it’s easier to look up.” Another said: “Information on the computer is better [than books]. You get exactly what you want much faster.”

Student Projects

The written assignments, oral presentations and models have been analyzed to identify the kinds of information from the three CD-ROMs that was incorporated into the students’ work. In the case of the models, it proved difficult to reach any firm conclusions, in part, because it was hard to relate the structures to specific images either in the three CD-ROMs or in the other sources available in the School and which it is known the students used for the written and oral work. The castle in one model does look very much like the castle presented in Castle Explorer, but even here the similarity cannot be convincingly demonstrated. This problem was compounded by the two teachers giving their classes at a late stage in the work specific and slightly different guidelines on model construction (following their normal practice with class projects) such as, for example, that every model should include a windmill, even though the students had not been asked to search on the

Attr

ibut

e E

ncar

ta

TA

BL

E 3

P

ost

-Pro

ject

Stu

den

ts’

Co

mm

ents

CD

-RO

M

Cas

t/e E

xdor

er

Exd

orin

a C

astle

s

Inte

rfac

e

l fa

st a

nd s

impl

e l w

as le

arni

ng m

ade

easy

l w

as n

ot h

elpf

ul

l ty

ped

in th

e to

pic

that

I w

ante

d an

d l in

tera

ctiv

e l ea

sy t

o fin

d in

form

atio

n ab

out

cast

les

tons

of h

elpf

ul in

form

atio

n ca

me

up

l ea

sy t

o ac

cess

l co

mpl

icat

ed t

o us

e l w

hen

you

type

in

a w

ord

it w

ould

giv

e l co

nfus

ing

to f

ind

your

way

aro

und

l di

fficu

lt to

use

bec

ause

you

cou

ldn’

t yo

u in

form

atio

n on

that

wor

d l at

firs

t it w

as h

ard

to u

se b

ut a

fter

it w

as

find

your

way

aro

und

fun

l to

o m

uch

poin

ting

and

clic

king

l ha

rd to

fin

d th

e in

form

atio

n

Use

fuln

ess

l ha

d m

ore

topi

cs

l ar

ticle

s an

d fa

ct b

oxes

l go

od i

nfor

mat

ion

l le

arne

d th

ings

tha

t I n

ever

kne

w

l so

me

of th

e w

ords

I c

ould

not

un

ders

tand

l ha

d pi

ctur

es a

nd v

ideo

s th

at m

ade

it ea

sier

to

unde

rsta

nd w

hat

I was

rea

ding

l fu

n th

e w

ay t

hey

gave

inf

orm

atio

n w

ith

soun

ds a

nd v

ideo

s l ha

d gr

eat

pict

ures

l di

d no

t hel

p m

e m

uch

with

the

info

rmat

ion

l di

d no

t ha

ve a

lot o

f in

form

atio

n th

at I

nee

ded

l ha

d go

od i

nfor

mat

ion

l m

ost o

f the

thi

ngs

I nee

ded

wer

e rig

ht a

t l di

dn’t

have

the

inf

orm

atio

n ou

r gr

oup

s

my

finge

r tip

s ne

eded

l th

e ga

me

was

the

best

par

t of i

t but

I di

dn’t

f

lear

n as

muc

h on

the

gam

e as

on

the

othe

r 3

CD

s 5

l th

e gr

aphi

cs w

ere

neat

m

l th

e pi

ctur

es g

ave

you

info

rmat

ion

whe

n L

you

clic

ked

on t

hem

5 i m

l th

e pi

ctur

es h

elpe

d

l us

eful

l gr

eat

l pr

etty

goo

d l bo

ring

l am

azin

g l in

tere

stin

g E

njoy

men

t l na

useo

us

l a

lot o

f fun

l no

t ver

y go

od

l am

azin

g l in

tere

stin

g l bo

ring

I

364 Large, Beheshtl, 81 Breuleux

CD-ROMs for windmills. The following discussion will therefore be confined to the written group assignment and the oral presentation (including at least one poster), again by groups.

Written assignments were submitted by 17 groups, ranging in length from 6 to 32 pages. All but 6 assignments included illustrations of some kind, but in four cases these were student drawings only. Seven of the assignments included illustrations reproduced from published sources: Encurtu, and Castle Explorer plus other CD-ROMs and books. The assignments dealt with 10 individual topics: two of a conceptual nature (knighthood and the manorial system) and the remaining eight comprised occupations (bishop, knight, monk, nun, pope, serf, shepherd, and troubadour). Although all three CD-ROMs are relevant to the overall theme of medieval life, in terms of these specific topics, Encarta and Castle Explorer contain much more focused information. In one case (shepherd) relevant information is not available on any of the three CD-ROMs, and very little is available on the medieval lives of popes (“pope” was not included by the teachers on their list of possible topics but nevertheless was chosen by one group). Information from the CD-ROMs was used for the remaining eight topics, although in all instances it was supplemented by information from other sources (CD-ROMs, the World Wide Web, and printed books). The students drew upon Encartu for all these eight topics, but found Castle Explorer and especially Exploring CustZes less relevant. Their incorporation of information from the three CD-ROMs differed from student group to student group. In some cases entire sections were copied word for word from the CD-ROM into the assignment, although even here it was unusual (but not unknown) to find that a topic was entirely copied from a single source. In many cases individual sentences rather than entire paragraphs were copied directly from the CD-ROMs. At the other extreme, students on occasion paraphrased ideas found in the CD-ROMs, presenting them in their own words. It was relatively unusual for students to incorporate directly images from the CD-ROMs into their assignments.

Oral presentations to the class were made by 14 groups and covered eight separate topics: heraldry, knighthood, the order of the garter, Ring John, armor, a noblewoman’s costume, tools, and weaponry. All three CD-ROMs were found useful, but again additional sources were also employed for all topics. In some cases entire sections were quoted (without acknowledgment) from the CD-ROMs, and indeed such direct quoting was at least as prevalent in the oral as the written presentations. Information was incorporated into the oral presentation from all three CD-ROMs, and one group showed the class a model of a clay seal that was based on an illustration in Encurtu.

DISCUSSION AND CONCLUSIONS

A case study rather than experimental methodology was chosen so that students could be observed while using multimedia information resources over an extended

Information Seeking in a Multhadla Emflronment 365

period of time and with a real end in mind-finding information to support an actual class project. The students had a real objective to work towards and a context in which to locate their multimedia searching. Undoubtedly, some students still showed less motivation and enthusiasm than others, but this reflected the reality of an actual classroom rather than being colored by students’ reactions to a research team appearing briefly in their lives in order to collect data. If information systems are to be designed for educational purposes then it is essential to observe system usage in a real rather than an artificial environment.

In general, the students worked well in groups, although, in a few instances, individual children found it difficult to cooperate effectively. There was a tendency for certain members of groups to be the searchers while others observed. The observ- ers often contributed valuable suggestions, however, and thereby participated actively in the search process. It was interesting to see a high level of information and expertise sharing among groups. Information about how to use one of the CD- ROMs as well as actual information retrieved from them was readily shared between groups. This may explain why certain kinds of search strategies, and even certain CD-ROMs, were less commonly tried than might have been expected. Once a group had determined that something did or did not work this was likely to be passed on to other groups who would respond accordingly. This might be considered yet another drawback to the case study rather than experimental methodology, but it is an inevitable consequence of working with students in a real classroom.

The students demonstrated a confidence in operating the various interfaces that should no longer surprise adults who have watched the younger generation in control of a mouse. Their dexterity was especially evident with Castle Explorer, which offered the most opportunities for creative exploration. Despite such confidence, however, a number of reservations can be noted about their ability to cope with these three examples of multimedia interfaces.

The least successful interface was that of ExpZoring Castles. Students expressed their reservations about this CD-ROM through their final comments collected in the questionnaire, the infrequency with which they searched this product compared with the other two, and the relative dearth of information incorporated in their written assignments and oral presentations from this title. In the questionnaire, for example, it was rated most difficult to use in comparison with other CD- ROMs, Internet or printed books. No doubt some of this dissatisfaction resulted from a hard-headed judgement as to its usefulness for the project in hand. But Cusrle Explorer provided much less information for their projects than did Encurta; nevertheless, students reported their enthusiasm for using the former over the latter. In other words, they seemed able to separate their evaluation of the usefulness of a CD-ROM from their enjoyment and ease in using it. The interface to Exploring Castles employed a number of different devices: it was inconsistent in structure and operation. Furthermore, it is possible to get lost within its hierarchically structured menus, a problem not helped by the fact that it is possible to exit from any part of the program only to the main menu, and not to an intermedi-


ate menu level. Despite the sophistication of the students in handling interfaces, they did encounter problems with this one. On the other hand, the more radical object-oriented Castle Explorer interface proved very popular with them. It resem- bled much more closely than the other two, of course, a game’s interface.

It can also be observed that despite interface dexterity, students were influenced in their choice of search devices by the way in which these were presented in the interface. More prominent facilities which were given an emphasis in the interface were much more likely to be used than those facilities afforded a secondary location. This can best be illustrated with Encarta. On choosing Find from the main menu, users are presented with the Pinpointer search of article titles. All students opted for this on first using Encarta, and most continued to opt for this as their first move even after repeated experiences with the interface. Searching on subject categories, and especially the powerful word search capability, were much less used, and it is hard to avoid the conclusion that in part this is a result of their lack of prominence. In quantitative terms, analyses of 20 searches revealed that Pinpointer was used 89 times compared with 20 for Subject Categories, three for Timeline and two for Word Search. Hypertext links were only followed on four occasions.

Interface designers cannot necessarily give equal emphasis in the interface to the multiple search tools typically available in the product. Yet they must take account of the probability that novice users (and perhaps even experienced users) are unlikely to exploit retrieval capabilities unless:

l Those capabilities are clearly offered for selection; and l On selection, the advantages of this retrieval strategy versus an alternative

strategy are clearly indicated (ideally in the search results that immediately ensue).

In the case of Encarta, for example, neither of these factors applied to Word Search. Despite its powerful full-text search function, its secondary location coupled with its unhelpful presentation of retrieved information (a list of titles of articles including the sought term) discouraged either initial exploration or, for those few who did try it, subsequent use.

A reluctance to search for individual terms on any of the three CD-ROMs leads to another conclusion: students may demonstrate a confidence and capability in physically manipulating the interface, but their grasp of retrieval strategies is much less well developed. Searching effectively for information via such interfaces is by no means intuitive. When given the opportunity, students preferred browsing to searching strategies. The browsing environment provided by Custk Explorer was more popular than its retrieval features, although the latter were used, especially as students became more familiar with the CD-ROM. On Encartu again, students opted for the browsable Pinpointer title index rather than Word Search. It must be stressed, however, that in both interfaces the browsing features were more prominently displayed than the searching features, thereby encouraging browsing,


especially for the novice user unfamiliar with the full search capabilities. When undertaking searches on Encurta, students would employ repeatedly a search strategy even though it was not producing satisfactory (in their own estimation as well as objectively) results. Frequently, it was not clear to them how they might modify the strategy. This problem was compounded by a belief that if the teacher had asked for information on a particular topic then that information must be present on the CD-ROM and must be retrievable via the actual term used by the teacher to describe the topic (for similar findings with high school students, see Neuman, 1997). The failure to distinguish between a concept and the variety of terms that might be employed to find that concept proved a major cause of search failure. Generally synonyms were not tried when a keyword search failed. Such problems could have been eliminated at the outset by more sustained training of the students by the research team, but this would have generated an artificial environment in which the students were better equipped to use the information tools than would be the case in a typical classroom (or home).

One interest of the research team was to explore whether the students demonstrated a learning curve over several weeks’ use of the CD-ROMs. Inevitably, the students exhibited initial hesitations in operating the interfaces, and false trails were followed. In the case of Castle Explorer, students initially used the game and only later began to explore the other browse and search options. Nevertheless, they very quickly mastered the mechanics of the interfaces and from then onwards used them confidently and competently. At the same time, the problems encountered in designing effective search strategies remained with them throughout the duration of the project. Only in a few cases at the end of the project period is there a sense that new searching skills are being acquired. In order to draw more definite conclusions about a learning curve, a longer case study would have been needed. Certainly, this research does not suggest that students intuitively acquire skills in information strategy formulation simply by dint of time spent at the computer.

The students had not been given prior instruction in information retrieval within their curriculum. As is all too common in Quebec, the school does not have a professional librarian. Within the research project itself, the students were only introduced briefly to the three CD-ROM interfaces and retrieval tools. They were not given lessons in how to construct or modify a search strategy (as this would have differentiated them from typical primary school users). The results of the searches clearly demonstrate the importance of such training if students are to be effective in information retrieval. They did not appreciate the importance of choosing the best search terms to represent their information need, they seldom thought of using synonyms when a search term failed to find the required information, they did not in general show a flair for modifying an unsuccessful strategy, and so on. These failings were in marked contrast to their confidence and capability in coping with three different interfaces at a mechanical, manipulative level.

Despite the availability of multimedia resources with relevant still images, video (with sound) and animation sequences, the students concentrated their attention largely on textual information. This was notwithstanding the fact that the oral

306 Large, Beheshtl, & Breuleux

presentation had to include at least one poster, and that the students had to construct a model of a medieval village. There are several explanations for the relative lack of attention to non-textual information. One CD-ROM did not allow students to print retrieved information. Notes could be made without too much difficulty, but still images are quite a different matter. In the absence of printing capabilities, images cannot be directly used. A second CD-ROM allowed textual but not visual information to be printed. The third CD-ROM did permit images to be printed, but the results were often too small and lacking in clarity to be useful. Theoretically, visual information could have been highly relevant to the model building exercise. None of the CD-ROMs, however, contained a picture of a medieval manor as such. They certainly included images of components of such a manor, such as castles, churches and windmills. That the students did not use them for this purpose can be explained by several factors. First, as commented above, the students were very focused in their searching. If the objective was to find a manor, then it, and it alone, would be sought. Little support is found for the suggestion that younger students may not have the intellectual capacity to ignore non-essential information while browsing (Heller, 1990). It is true that medieval punishment-dungeons, prisons, hangings and pond duckings4id attract attention in the early phases of the project, but overall the students concentrated on the task in hand.

Second, the model seemed to be considered both by the teachers and the students as the least significant part of the project. The model was produced at the very end of the time period allotted and, by then, most students had spent as much time as they wanted on the CD-ROMs. Video and animation can both convey a lot of information, and it may be information that would be difficult to present in textual form. Nevertheless, written, oral, and model-building projects do not provide good scope for the incorporation of such moving visual information. The only remotely relevant sound was incorporated in the video sequences on Exploring Castles, and these sequences were not in fact directly relevant to the project objectives.

The students, in general, were more than willing to print retrieved textual information, although this was less evident in the case of images. This does not support Small and Ferreira (1994) who found that sixth through eighth graders tended to take more notes from print sources than extract information from multimedia sources. In many cases entire sections, uncredited, were copied into their projects, but there seems no reason to think that such a practice is any more prevalent in the case of digital than print sources.

The students were well-focused on the objectives of the assignment as estab- lished by the teachers. They were asked to find information about certain specific people, occupations and equipment, as well as several concepts. They were then asked to relate the medieval manifestations of these entities to their modern equiv- alents. In practice this focused them on retrieving factual information on a number of specific topics and presenting this either as a written project or as an oral presentation. The fact that the latter were first expressed in writing before being read or paraphrased reduced the distinction between the two, at least in terms of the preliminary information collection. To meet these needs, the students required

Information Seeking in a Multimedia Emhonment 369

access to packaged information bundles. This explains the much higher prevalence of information in the written assignments and oral presentations taken from Encartu than either Castle Explorer or Exploring Castles (and indeed the heavy use of other encyclopedic sources, both on CD-ROM and in print). Castle Explorer and Exploring Castles attempt to give students a flavor of life in and around a medieval castle, and in this objective they are in all probability successful. Encurtu has no such aim. As a general encyclopedia, its objective is to deliver information on a wide range of discrete topics, a few of which are relevant to the medieval project. They do not provide an impression of life in the middle ages but they do deliver processed information on specific topics. And this is what the students wanted and needed.

In the final analysis, the class project was not about gaining a feel for life in a bygone age, but about assembling small slabs of concrete factual information to present for assessment. For such tasks, textual information arranged for retrieval through specific index terms will always prove a challenge for multimedia information arranged graphically for browsing and ingesting as a whole rather than through its constituent parts. Moving images and sound cannot easily be represented in traditional school projects. Very considerable cognitive effort must be made to transform moving visual information into static textual information - why bother when sufficient textual information is on hand? Dresang (1997, p. 649) may well be correct in saying that “children have demonstrated their comfort level with far more complexity than adults previously thought possible or appropriate.” But so long as the (adult-assigned) educational tasks can better be accomplished using more traditional linear text presentations then more traditional information sources will continue to play an important role. As Neuman (1997, p. 692) says, “students and teachers alike must develop new conceptions of the best ways to access, evaluate and use multimedia information for learning.” The teachers, then, may have to re-think measurement of learning through class assignments if they are fully to exploit the exciting information presentation possibilities offered by well-designed multimedia instructional products. From the researchers point of view, it was surprising (and even disappointing) to see the strong bias towards Encurtu as the source of project information compared with the more imaginatively designed and executed Castle Explorer and Exploring Castles.

This research is centered upon three important questions:

l Do children constitute a distinct user group for information retrieval systems? This research supports the conclusions reached by the authors cited above in the Review of the Literature section that children are indeed a distinct group. On the one hand, they demonstrate an enthusiasm, confidence and dexterity in interface manipulation that would be the envy of many an adult. They were attracted in particular to the direct manipulation interface of Castle Explorer, and certainly enjoyed exploring its offerings. At the same time, they lack sophistication in language manipulation and search strategy formulation. In lieu of training within the curriculum for information retrieval, it is important

370

.

Large, Beheshti, & Breuleux

that interfaces intended for children should give as much help as possible in this task. For example, a thesaurus that would automatically suggest synonyms would be a great help. In this project the children encountered few problems caused by their spelling errors; this was because in the main users of all three interfaces entered search terms by selecting them from displayed lists. Such an approach largely eliminated what otherwise would have been a major problem. Some search tools were largely ignored despite their potential importance. The most obvious example is the full-text searching capability of Word Search on Encarta. Three modifications can be proposed here. First, Word Search could be more prominently displayed in the interface so that it attracts users’ attention. Second, its purpose and potential retrieval power could be explained more clearly once it has been chosen. Third, the results of a Word Search could be more effectively displayed. Rather than merely listing the article titles that contain the sought word(s), a keyword in context display could be provided that would help the students to judge the significance of their results. ’

Despite these conclusions, some caveats should be placed on children as a distinct user group. First, this research confirmed earlier research (e.g., Large, Beheshti, Breuleux, & Renaud, 1994a, 1994b, 1995,1996) that children cannot be seen as a homogeneous group (any more than other age groups). Some children are better and some worse at information retrieval, and generalisations about overall performance should not mask this important reality. Second, the term “children” can be used to describe people whose age spans at least 12 years (and these are years of rapid and dramatic cognitive development). This research dealt with children on the edge of adolescence-any extrapolation of the findings to other populations-for example, older or younger age groups -must be done with caution. Third, much research with adult users has focused on university students, perhaps because they are the most accessible population for university-based researchers. It would be dangerous to conclude, however, that such adults are representative of the entire adult population. It is possible that the problems encountered by children in such matters as spelling and synonym identification are shared by certain adult groups. More research on wider adult user groups would be useful in this respect. What kind of interface navigational features are best suited to instructional programs targeted at primary school students? The students were able to distinguish between an interface that was fun to use and one that led them to the information they actually needed. Hence, the enthusiasm for Castle Explorer but the greater reliance upon Encarta. Nevertheless, the attractiveness of the direct manipulation interface should be noted by designers. It cannot necessarily compensate for shortcomings in information content (in terms of the

’ The authors are grateful to an anonymous referee for suggesting this third point.

lnformatlon Seeking in a MultimedIa Environment 371

users’ information needs)--children are not naive users-but it will comple- ment and enhance a good content. The interface should be consistent in operation. ExpZoring Castles employs a range of interface navigation tools, and this variety resulted in confusion. Once the interface was deemed unfriendly, word quickly spread around the class and this CD-ROM was then largely ignored.

l Do multimedia instructional products have a role to play in the classroom? The children showed discernment in their selection of information sources for their project-both in their differential use of the three CD-ROMs and more widely in their decisions to draw upon other sources- electronic and print - to support their work. A multimedia source will not be used in preference to a more traditional source unless it meets the need in hand.

Still images, moving images and sound of various kinds can enrich textual presentation of information when used effectively (Large, 1996). Two of the CD- ROMs-Castle Explorer and Exploring Castles-provided rich sources of information on life in medieval times by exploiting effectively such multimedia capabilities. Yet, the third CD-ROM-Erzcarta-with its more traditional arrangement of information was ultimately a richer source for the student assignment. It arranged information under topics in discrete chunks, and this proved the most effective organisation for subsequent retrieval by the students. The assignment asked them to find information on specific topics, a task for which Encarta is ideally suited. It is simply too demanding of the students-both in retrieval and in processing-for them to have made full use of the non-textual information available and relevant to the assignment. How can information in a video sequence, for example, easily be incorporated into a written project? Had the students been asked to write a story about life in a medieval castle, or to write and perform a play on such a theme, the relative use made of the three CD-ROMs (as well as other ancillary sources) might have been very different. Multimedia will only have a full role to play in the curriculum when the assignments take account of its potential.

Postscript

The increasing availability of multimedia information on a variety of platforms including CD-ROM and the Web is encouraging schools to explore these new sources in a learning (rather than playing) context. The students appreciated having the CD-ROMs on hand in the classroom and enjoyed using them. Many asked when they would be able to repeat the experience. In order to exploit them fully, however, it will be necessary to rethink the objectives of class projects. One solution is to have students download information from such sources and incorporate it into their own multimedia projects. Students can utilize simple authoring software to produce new multimedia products on specific topics covered in the class curriculum.

The next phase of this research is to observe how students employ Web resources in furtherance of a classroom project. Here they will have access to a vast reservoir of textual as well as audiovisual information.

AP

PE

ND

IX

Pos

t-Se

arch

Que

stio

nnai

re

You

r nam

e

Nam

es o

f ot

her

grou

p m

embe

rs

Top

ic o

f or

al p

rese

ntat

ion

Plea

se c

heck

off

the

app

ropr

iate

ans

wer

:

(A) f

ound

mos

t inf

o. fo

r w

ritte

n as

sign

men

t fr

om

(B) f

ound

mos

t of i

nfo.

fo

r or

al p

rese

ntat

ion

from

(C) f

ound

mos

t of i

nfo.

fo

r po

ster

from

(D) m

ost e

njoy

ed

usin

a

(E) l

east

enj

oyed

us

ing

(F) m

ost d

iffic

ult’t

o us

e

(G) l

east

diff

icul

t to

US

E

prin

t ot

her

prin

ted

book

en

cycl

opae

dia

mat

eria

ls

Inte

rnet

Enc

arta

‘9

6 C

D

ency

clop

edia

Exp

lorin

g C

astle

s C

ast/e

s C

ast/e

of

the

Brit

ish

Exp

lore

r w

ith

othe

r C

D

Isle

s ga

me

give

na

me

I I I

I to

ok i

nfor

mat

ion

from

: w

ritin

g on

ly

vide

os

pict

ures

I

used

the

CD

Rom

glo

ssar

y

Yes

no

ca

n’t

rem

embe

r I

used

the

CD

Rom

dic

tiona

ry

yes

no

can’

t re

mem

ber

Enc

arta

(C

D e

ncyc

lope

dia)

w

as:

Exp

lori

ng C

astle

s (C

astle

s of

Bri

tish

Isle

s C

D)

was

:

Cas

tle E

xplo

rer

(CD

with

gam

e) w

as:

374 Large, Beheshtl, & Breuleux

REFERENCES

Barron, Ann E., & Atkins, Debra. (1993). Audio instruction in multimedia education: is textual redundancy important? In H. Maurer (Ed.), Educational Multimedia and Hypermedia Annual, 1993: Proceedings of Ed-Media 93, Orlando, Florida, June 1993, (pp. 39-46). Orlando: Association for the Advancement of Computing in Education.

Beaulieu, Michele. (1997). Experiments on interfaces to support query expansion. Journal of Documentation, 53(l), 8-19.

Borgman, Christine. L., Hirsh, Sandra G., Walter, Virginia A., & Gallagher, Andrea L. (1995). Children’s search behavior on browsing and keyword online catalogs: the Science Library Catalog Project. Journal of the American Society for Information Science, 46,663-684.

Cates, Ward Mitchell. (1992). Fifteen principles for designing more effective instructional hypermedia/multimedia products. Educational Technology, 32(12), 5-11.

DeRemer, M., & Bracewell, Robert. (1989). Students’ use of semantic structure in revising their writing. Annual Meeting of the American Educational Research Association, March 27-3 1, 1989, San Francisco.

Dresang, Eliza T. (1997). Influence of the digital environment on literature for youth: Radical change in the handheld book. Library Trends, 45,639-663.

Druin, Allison, & Solomon, Cynthia. (1996). Designing multimedia environments for children. New York: Wiley.

Galitz, Wilbert 0. (1997). Essential guide to user inter$ace design: An introduction to GUI design principles and techniques. New York: Wiley.

Head, A.J. (1997). A question of interface design: How do online service GUIs measure up? Online, 21(3), 20-29.

Heller, R. (1990). The role of hypermedia in education: a look at the research issues. Journal of Research on Computing in Education, 22,431~441.

Hirsh, Sandra G. (1997). How do children find information on different types of tasks? Children’s use of the Science Library Catalog. Library Trends, 45, 725-745.

Jacso, Peter. (1996, April). State-of-the-art multimedia in 1996: The “big four” general encyclopedias on CD-ROM. Computers in Libraries, 26-32.

Janus, Robert J. (1997). From paper and ink to CD-ROM: Digitizing the World Book image. Library Trends, 45,602-622.

Kuhlthau, Carol C. (1989). Information search process: A summary of research and implications for school library media programs. School Library Media Quarterly, 18, 19-25.

Large, Andrew. (1996). Computer animation in an instructional environment. Library & Information Science Research, 18,3-23.

Large, Andrew, Beheshti, Jamshid, Breuleux, Alain, & Renaud, Andre. (1994a). A comparison of information retrieval from print and CD-ROM versions of

Information Seeking in a Multimedia Emhnment 375

an encyclopedia by elementary school children. Information Processing & Management, 30(4), 499-5 13.

Large, Andrew, Beheshti, Jamshid, Breuleux, Alain, & Renaud, Andre. (1994b). Multimedia and comprehension: A cognitive study. JournaZ of the American Society for Information Science, 45,5 15-528.

Large, Andrew, Beheshti, Jamshid, Breuleux, Alain, & Renaud, Andre. (1995). Multimedia and comprehension: the relationship between text, animation and captions. Journal of the American Society for Information Science, 46, 340- 347.

Large, Andrew, Beheshti, Jamshid, Breuleux, Alain, & Renaud, Andre. (1996). The effect of animation in enhancing descriptive and procedural texts in a multimedia learning environment. Journal of the American Society for Information Science, 47,437-448.

Levin, Joel R., Anglin, G. J., & Carney, R. N. (1987). On empirically validating functions of pictures in prose. In D. M. Eillows & H. A. Houghton (Eds.), The psychology of illustration, vol I basic research (pp. 51-85). New York: Springer.

Levin, Joel R., & Lesgold, Alan M. (1978). On pictures in prose. Educational Communication and Technology Journal, 26,233-243.

Logan, Elizabeth. (1990). Cognitive styles and online behavior of novice searchers. Information Processing & Management, 26,503-5 10.

Mandel, Theo. (1997). The elements of user interface design. New York: Wiley. Mann, Bruce L. (1992). The SSF Model: Structuring the functions of the sound

attribute. Canadian Journal of Educational Communication, 21,45-65. Marchionini, Gary. (1989). Information seeking strategies of novices using a full-

text electronic encyclopedia. Journal of the American Society for Information Science, 40,54-66.

Marchionini, Gary. (1992). Interfaces for end-user information seeking. Journal of the American Society for Information Science, 43, 156- 163.

Mayer, Richard E., & Anderson, Richard B. (1992). The instructive animation: Helping students build connections between words and pictures in multimedia learning. Journal of Educational Psychology, 84,444-452.

Microsoft Corporation. (1997). The windows interface guidelines for software design. Available: http//www.microsoft.com/win32dev/uiguide/.

Negroponte, Nicholas. (1995). Being digital. New York: Knopf. Neuman, Delia. (1997). Learning and the digital library. Library Trends, 45,687-

707. Oliver, R., & Oliver, H. (1996). Information access and retrieval with hypermedia

information systems. British Journal of Educational Technology, 27,33-44. Pack, Thomas. (1995). Digital circles of knowledge. Database, Z8(6), 14-23. Park, Innwoo, & Han&in, Michael. (1993). Empirically-based guidelines for the

design of interactive multimedia. Educational Technology Research & Development, 41(3), 63-85.


Reynolds, Lynne, & Ehrlich, Diane (1992). Multimedia in industry and education: A decision model for design. (Educational Resources Information Center Report: ED351 003).

Rieber, Lloyd P. (1990). Using computer animated graphics in science instruction with children. Journal of EducationaZ Psychology, 82(l), 135-140.

Shneiderman, Ben. (1992). Designing the user inter&ce: Strategies for eflective human-computer interaction. 2nd ed. Reading, MA: Addison-Wesley.

Small, R.V., & Ferreira, S.M. (1994). Multimedia technology and the changing nature of research in the school library. Reference Librarian, 44,95-106.

Solomon, Paul. (1993). Children’s information retrieval behavior: Acase analysis of an OPAC. Journal of the American Society for Information Science, 44, 245-264.

Spink, Amanda. (1996). A multiple search session model of end-user behavior: An exploratory study. Journal of the American Society for Information Science, 47,603-609.

Yildiz, Rauf, & Atkins, Madeleine J. (1992). How to evaluate multimedia simulations: Learning from the past. (Educational Resources Information Center Report: ED350 978).

Information seeking in a multimedia environment by primary school students

Documents

Transcript of Information seeking in a multimedia environment by primary school students