Functional Requirements for Online Tools to Support Community-Led Collections Building

14
M. Agosti and C. Thanos (Eds.): ECDL 2002, LNCS 2458, pp. 190-203, 2002. © Springer-Verlag Berlin Heidelberg 2002 Functional Requirements for Online Tools to Support Community-Led Collections Building Michael Khoo 1 , Holly Devaul 3 , and Tamara Sumner 2 1 Department of Communication UCB 270 2 Department of Computer Science UCB 470 University of Colorado, Boulder, CO 80309 U.S.A. {michael.khoo, tamara.sumner}@colorado.edu 3 DLESE (Digital Library for Earth System Education) Program Center University Corporation for Atmospheric Research P.O. Box 3000, Boulder, CO 80307-3000, U.S.A. [email protected] Abstract. The Digital Water Education Library collection (DWEL) is being generated by primary and secondary school teachers in the United States. This complex process involves both individual research and team design, and the use of a variety of online tools, such as an online cataloguing tool. Interactions amongst DWEL members are being ethnographically analysed in order to identify requirements for further development of these tools. The analysis suggests that many DWEL members envision their work as occurring in an integrated environment with stable documents, a situation which is not supported by the current configuration of DWEL tools. The design implications of these findings are reviewed. 1 Introduction: DWEL and Community-Led Collections Building The Digital Water Education Library (DWEL) is a two-year National Science Foundation funded project to provide catalogued records of five hundred online resources related to water and the earth system. The project is collaborating with the Digital Library for Earth System Education (DLESE) and is using their recently- released catalog management tool. This tool facilitates individual contributions of resources as well as coordinated group development of sub-collections. DWEL is experimenting with, and implementing a community-led collections development process that we shall refer to in this article as ‘CLCD.’ In CLCD, disciplinary and special interest groups can design and create collections tailored to their own specific needs. The DWEL collection is being designed by a distributed group of approximately thirty individuals, predominantly primary and secondary school teachers, who share an interest in water-related education topics. This group is sub-divided into four separate working groups, covering kindergarten to year four, years five to eight, years nine to twelve, and informal education. Each of these groups is responsible for selecting, reviewing and cataloguing appropriate online educational resources for their age range. Members are also responsible for designing a series of evaluation and review rubrics for assessing and accessioning resources into the collection, and are thus also engaged in a process of ‘meta-design,’ ‘activities, processes, and objectives to create new media and environments that allow users to act as designers and be creative’ [6].

Transcript of Functional Requirements for Online Tools to Support Community-Led Collections Building

M. Agosti and C. Thanos (Eds.): ECDL 2002, LNCS 2458, pp. 190-203, 2002.© Springer-Verlag Berlin Heidelberg 2002

Functional Requirements for Online Tools to SupportCommunity-Led Collections Building

Michael Khoo1, Holly Devaul3, and Tamara Sumner2

1 Department of Communication UCB 2702 Department of Computer Science UCB 470

University of Colorado, Boulder, CO 80309 U.S.A.{michael.khoo, tamara.sumner}@colorado.edu

3 DLESE (Digital Library for Earth System Education) Program CenterUniversity Corporation for Atmospheric ResearchP.O. Box 3000, Boulder, CO 80307-3000, U.S.A.

[email protected]

Abstract. The Digital Water Education Library collection (DWEL) is beinggenerated by primary and secondary school teachers in the United States. Thiscomplex process involves both individual research and team design, and the useof a variety of online tools, such as an online cataloguing tool. Interactionsamongst DWEL members are being ethnographically analysed in order toidentify requirements for further development of these tools. The analysissuggests that many DWEL members envision their work as occurring in anintegrated environment with stable documents, a situation which is notsupported by the current configuration of DWEL tools. The design implicationsof these findings are reviewed.

1 Introduction: DWEL and Community-Led Collections Building

The Digital Water Education Library (DWEL) is a two-year National ScienceFoundation funded project to provide catalogued records of five hundred onlineresources related to water and the earth system. The project is collaborating with theDigital Library for Earth System Education (DLESE) and is using their recently-released catalog management tool. This tool facilitates individual contributions ofresources as well as coordinated group development of sub-collections. DWEL isexperimenting with, and implementing a community-led collections developmentprocess that we shall refer to in this article as ‘CLCD.’ In CLCD, disciplinary andspecial interest groups can design and create collections tailored to their own specificneeds.

The DWEL collection is being designed by a distributed group of approximatelythirty individuals, predominantly primary and secondary school teachers, who share aninterest in water-related education topics. This group is sub-divided into four separateworking groups, covering kindergarten to year four, years five to eight, years nine totwelve, and informal education. Each of these groups is responsible for selecting,reviewing and cataloguing appropriate online educational resources for their agerange. Members are also responsible for designing a series of evaluation and reviewrubrics for assessing and accessioning resources into the collection, and are thus alsoengaged in a process of ‘meta-design,’ ‘activities, processes, and objectives to createnew media and environments that allow users to act as designers and be creative’ [6].

Functional Requirements for Online Tools 191

In anticipating rapid development of digital library infrastructure in the UnitedStates, it is thought that CLCD models such as DWEL’s offer the possibility of ascalable growth model with economic and administrative advantages over morecentralised systems. In CLCD, many of the tasks required for collections developmentcan be decentralised to user communities, rather than being incrementally accumulatedat a centralised facility. As DWEL is a prototype for these suppositions, it has a built-in component of ethnographic-based independent project evaluation to study theefficacy of CLCD as a design process. This article presents ethnographic data fromthis research that has studied DWEL members’ perceptions of CLCD, and theirexpectations of a close structural integration between the various tools they use tocarry out their tasks.

2 Theoretical Background: The Social Context of TechnologyAdoption

In investigating the implementation of CLCD in DWEL, we assume a complex andnon-linear relationship between a technology and its context of use [12] in which theintroduction of a technology into a social context involves mutual processes of socialand technological co-evolution [e.g. 2, 3, 4]. From this point of view, technology canbe seen as eliciting unpredicted responses in users, responses that can in turn affect theway a technology is restructured and redesigned. Early groupware, for instance, withits potential for reducing the general messiness of human interaction, was thoughtsuitable for supporting and managing focused organisational tasks [5, 15, 17, 22]. Theoverall benefits of groupware adoption have however proved unpredictable [7],particularly ‘in the wild,’ where the technological rubber hits the social andorganisational road [e.g., 9, 11, 18].

Two examples help to illustrate this assertion. In the first one, Harper and Sellenstudied why a new computer intranet at the International Monetary Fund wasinfrequently used [9]. The network was designed to facilitate sharing of the economicdata files used to compile economic reports, and should have made the process ofreport generation easier. The authors found however that each report’s genesis lay inindividual economists focusing on his/her own particular area of expertise, compilingthe statistics that would underpin the final report, work Harper and Sellen characteriseas ‘judgment.’ Only at a later stage in the workflow, characterised as ‘collaboration’and at which the reports are drafted, were these data deemed ‘solid’ enough to beincluded in the IMF’s database. Knowledge generation at the IMF therefore consistedof at least two distinct processes. First, there was an initial division of labour amongstteam members, in which economists worked on their own data, and second, anemergent stage, termed collaboration, in which team members communicated witheach other to produce the final report. Harper and Sellen found that the new IMFintranet was not supportive of the first task. As will be described below, thisconclusion bears some similarity to our analysis of the DWEL workflow.

A second example of unpredictable adoption is drawn from Weatherley et al.’s[23] analysis of an HTML-based anchored discussion forum, the Digital DocumentDiscourse Environment (D3E). In D3E, an electronic document is provided withembedded anchors that link specific elements of that document to discussion threads.The D3E user can see both the parsed document, and the threaded discussions, eitherin overlapping windows or in side-by-side frames. The authors examined two cases of

192 M. Khoo, H. Devaul, and T. Sumner

D3E use in which a number of reviewers under the guidance of an editor reviewedcomplex electronic educational resources. One initial design supposition was that D3Ewould permit reviewers to pursue collaborative, threaded, asynchronous discussions ofeach component of the resource before coming to a conclusion regarding its validity.What was found to occur in practice, however, was cooperative behaviours: ratherthan initiating threaded discussions with their colleagues, each reviewer postedcomments to threads that no other reviewers had posted to. This appeared to be donewith reference to the shared visual outline of the review process afforded by the GUI,which showed who was reviewing each unit of the resource, and what units had yet tobe reviewed. Thus while the intended technological affordance of D3E was to supportcollaboration in task construction, an emergent social affordance appeared to be thatof cooperation in task allocation so as to minimise duplication of effort.

We have presented these two examples to suggest the complexity of theinteraction between technology design and technology use. In the next section, it willbe argued that implementing CLCD in DWEL involves similarly complex processes.To investigate these processes, we describe how DWEL members talk about theirwork and DWEL, and how they see CLCD as working. Particular attention will bepaid to comments that indicate that DWEL members envision their online work asoccurring in a seamless and integrated environment. In the pursuant discussion, thistalk will be treated as evidence of the design requirements that DWEL members feelnecessary to support their work in CLCD.

3 The Case Study: DWEL and Community-Led CollectionsDevelopment

The DWEL project is the subject of ethnographic research that involves iterativecycles of data collection, analysis, and heuristic generation [18]. This research isaimed at supporting project evaluation for project management, and it therefore strivesto generate recommendations for practical interventions as much as theoreticalmodels. As presented in this discussion however, some general theoretical approachesare also emerging from the analysis. Close examination is being paid to the discourseand communication of DWEL members, which is being treated as an activity thatconstitutes the symbolic worlds in which DWEL members operate [e.g. 1, 10, 27].From this perspective, everyday conversation becomes a rich repository ofindividually and collectively held notions of what CLCD, as a technology-supportedsocial process, means to the people involved. The data analysed for this papertherefore include: transcriptions of video tapes of three days of DWEL meetings heldin January, 2002; analysis of one year of DWEL documentation; and interviews withDLESE staff responsible for overseeing various aspects of DWEL project workflow.Note that in the following section, transcriptions from video have been redacted toreduce length, and where necessary pseudonyms have been used.

3.1 DWEL Workflow as a Complex Task

Before describing how DWEL members talk about their work, a brief description ofthat work itself will be provided.

Functional Requirements for Online Tools 193

DWEL members are engaged in a number of different tasks, and work with avariety of online tools that are distributed in both time and space. As such they areengaged in asynchronous distributed interaction. Two main tools are involved. First,intragroup communication is carried out in WebCT, a proprietary distributedgroupware environment designed to support online classrooms and distance learning,that runs in frames in a web browser window [24]. WebCT claims to facilitate and‘transform’ the learning process both for students (for instance by allowing on-demandaccess to course materials and assignments) and for course administrators (for instanceby allowing remote uploading and downloading of course materials, and monitoring ofstudent participation and progress). WebCT allows DWEL members to take part indiscussion threads, post documents, and to message each other. The messages staywithin the WebCT environment and cannot be broadcast to outside accounts.

Second, DWEL catalogue records are created through the DLESE CatalogSystem (the ‘cataloguing tool’), a tool accessed on a server at the DLESE ProgramCenter. The cataloguing tool permits DWEL members to enter bibliometric data asthey apply to an online resource, such as title, URL, date and author (if they can befound!). The tool also requires users to complete a large number of sub-fields, eitherwith free text or with choices from controlled vocabularies. These subfields permit theprecise indexing (and therefore search and retrieval) of a resource according to a widerange of custom terms such as resource type (lab activity, curriculum, visualization,dataset etc.), scientific and educational content, audience or grade range, technicalrequirements, and so on. It is important to note that while catalogue record generation,and also catalogue record viewing, takes place within a web browser interface,catalogue records themselves are not static web pages in the common ‘HTML’ sense,but are generated dynamically from an underlying database.

In some cases, WebCT and the cataloguing tool have conflicting browser versionrequirements. For instance, the cataloguing tool is optimized for Netscape 6.0, butWebCT does not support 6.0 at all. If using Internet Explorer, version 5.5 or higher isnecessary to use the cataloging tool, 5.5 however has a file download bug whenrunning WebCT. Further, the cataloguing tool is in development and requiresscheduled and unscheduled maintenance, and it is recommended that anyone using itshould also keep open an e-mail window to be alerted of impending shut downs, toavoid losing work in progress. Finally, word processors may be used to draft textportions of catalogue records before they are entered into a record.

DWEL cataloguing workflow typically proceeds as follows. First, a browser isused to log on to the WebCT environment, to check for current tasks and instructions(Figure 1a). Second, a browser is then used to look for a suitable online resource (if aPDF is found, a PDF reader will be launched) (Figure 1b). Third, after the resource islocated, a browser is used to access the cataloguing tool; and after checking that theresource is not already catalogued, cataloguing may proceed (Figure 1c). Fourth,cataloguing may also involve use of an e-mail utility (to be warned of shutdowns inthe cataloguing tool) and a word processor (to compose the longer pieces of text thathave to be entered into the cataloguing tool). In Figure 1d, there are five windowsopen: three browser windows (WebCT, the cataloguing tool, and the resource), an e-mail window (Telnet), and a word processor window (Microsoft Word). There arealso five applications open (dial-up software, Adobe Acrobat, Microsoft Word,Telnet, Netscape Navigator), requiring, in addition to the OS, approximately 174 MBof RAM.

194 M. Khoo, H. Devaul, and T. Sumner

Fig. 1a. Checking workflow: The WebCT interface

Fig. 1b. Locating a resource

Functional Requirements for Online Tools 195

Fig. 1c. The cataloguing tool

Fig. 1d. Cataloguing a resource

196 M. Khoo, H. Devaul, and T. Sumner

Table 1. Summary of DWEL cataloguing workflow

1. Checkworkflow

2. Locate aresource

3. Check to see if alreadycatalogued; Check review andscope rubrics

4. Perform cataloguing

Browser:WebCT

Browser:Access WebCTLocate resource

PDF Reader:Read PDF

Browser:Access WebCTView resourceSearch DCSPDF Reader:Read PDF

Browser:Access WebCTCatalog ResourceInput to DCSPDF Reader:Read PDFE-mail utility:Communicate with DCSWord processor:Text composition

This typical workflow for one catalogue entry in DWEL is summarised in Table 1(applications and tasks in italics are optional). Carrying out meta-design tasks (such asthe design of collection scope or review criteria) can add a further level of complexity,in that further WebCT andDCS documents and discussions may have to be accessed.

3.2 DWEL Member Perceptions of DWEL Workflow and Tools

As has just been described, DWEL workflow involves the use of a number of tools.We will now provide examples of how DWEL members talk about the functions of,and the relationships between these tools. We will then discuss what implicationsthese expectations might have for future DWEL design.

The examples are taken from archived DWEL e-mails, and transcripts ofconversations of DWEL members recorded during three days of meetings in January2002. These documents were reviewed and coded for the presence of variousconversational topics. The examples presented below involve specific references byDWEL members to the functioning of WebCT and the cataloguing tool as seamlessand powerful technologies. While DWEL members made implicit references to thissubject on a number of occasions, only the points at which they explicitly and activelyengaged in conversation over it are presented here. Note that the subject of thisparticular coding scheme - the functioning of WebCT and the cataloguing tool asseamless and powerful technologies - has been in turn guided by previousethnographic research at DLESE and DWEL. This previous research, which wasguided significantly by technological frames theory [18], suggested that one commonway in which users (and some designers) can view these digital libraries as operatingis as a ‘bricks-and-mortar’ library that is made more powerful through the applicationof digital technology [13, 14]. This expectation was also found in another study ofdigital library design and implementation [26].

The conversations themselves are drawn from the planning and earlyimplementation stages of the DWEL project (the project itself is scheduled to runfrom January 2002 to December 2003). In the pre-project planning stage in 2001,while an assessment of what teachers wanted from a DWEL collection - scientificaccuracy, ease of use, and so on - was carried out prior to the start of the DWELproject, the operational interaction between DWEL tools was not explicitly defined(this is not surprising, given that designing and evaluating the CLCD process was part

Functional Requirements for Online Tools 197

of the proposed DWEL research agenda). Some high level expectations of what thetools might offer were however apparent. Project e-mails sent in 2001 referred to theneed to ‘[d]evelop good communications structures within and between workinggroups’ (e-mail, 18/7/01), and to ‘setting up our WebCT page (if we decide that is thebest way to communicate and share resources)’ (e-mail, 19/9/01). The groupware-likefeatures of WebCT were described in the latter e-mail, which asked:

Would you like to communicate with your group via the DLESE list serves set upfor K-4, 5-8, 9-12, and informal [groups] or would you like us to set up a ProjectWebCT page with space for each working group to have threaded discussions,post resources, and have access to the discussions from the other working groups(e-mail, 19/9/01).

During January 2002, these expectations – that WebCT could be used to shareboth communication and resources - were expressed in three days of DWEL face-to-face meetings. (The following comments, which have been rendered anonymous, aretaken from transcripts of video tape recordings of these meetings). According to‘Andy,’ for instance:

Andy: [Talking of working group members] What we want them is to be able touse WebCT, share the information, get feedback, so they’re actually using all thetools that we think are going to be vital for communicating throughout the nexttwo years …

Andy: Each of the teams will be responsible for the work being done in theirgroup, for helping to make sure that it’s organised, that people are on task,they’re producing different things, and they’re also sharing resources with eachother, for example through the WebCT site and through the DLESE tools …

While Andy’s second statement indicates an expectation that WebCT would enablethe sharing of resources between DWEL members, the definition of what resourcescould be shared was unclear. It seems to assume that a resource in the cataloguingsystem (i.e. a catalogue record) would also be accessible and perhaps manipulable inthe WebCT environment. Thus Chris, a DWEL designer, also assumed that DWELcatalogue records created on the cataloguing tool would be accessible in WebCT fordiscussion and critique by DWEL members. This is not in fact possible, as records inthe cataloguing tool do not exist in a format easily accessible to browser software, andhe was corrected by another member (Beth) in the following exchange:

Beth: I think that the most problematic aspect will be the fact that [WebCT] is aseparate tool, a separate system from the cataloguing.

Chris: So we could put a link to the cataloguing system in WebCT?

Beth: The problem is, what you want is not a link to the cataloguing system, youwant a link to that record, which is not a URL you can just point to.

198 M. Khoo, H. Devaul, and T. Sumner

Another discussion debated whether WebCT archives were searchable in the sameway that records are searchable in the cataloguing tool. This was raised because it wasthought useful to be able to keep track of what DWEL members were talking about inWebCT. However, while records in the cataloguing tool are searchable across a rangeof controlled vocabularies, WebCT discussions exist only as text files with nocontrolled vocabularies attached, searchable only in terms of the text they contain.This realisation prompted a discussion regarding making sure that when WebCT userspost to discussion threads, abbreviations are not used, and spellings are correct.

Dave: And Chris may correct me if I’m wrong – but I think with the WebCT youcan run a search on text in any of the uh, letters or -

Chris: - threaded discussions -

Dave: - discussions that are put in there, so let’s say if you’re interested inassessment, you can type in the key word ‘assessment,’ and it allows you to justkind of lurk through the archives to see what these guys are talking about … so ifyou want to put in things like a science standard, or a particular content topic orwhatever, you have that ability just to see what they’re doing …

Chris: So we should make sure that people don’t use abbreviations for words.

Andy: Right.

Chris: So that the searches work.

Evelyn: Right, that’s a good point.

Beth: And spelled correctly, good luck.

This perception - that WebCT embodied at least some functions that were similar tothe cataloguing tool, that would allow its use in managing catalogue records at variousstages in the workflow - was also observed in a meeting of the thirty or so people whowould form the various working groups within DWEL:

Chris: [On the WebCT site we have a place] where the working groups can postdocuments, web pages, anything they want, so if one person wants to upload adocument for the rest of the group to download, this is where you have thatcapability … and so I was thinking okay, put up web sites that you areconsidering, and you can put a little description so your co-workers canunderstand why you posted it there …

Andy: This is an area where you can share resources, look at different types ofresources under consideration. One of the queries that we need to look at is whathappens once you’ve reviewed something, where does it go, does it go into a filelabeled ‘reject,’ or perhaps we need to look at this later on and make sure it getsinto DLESE at some point, or it maybe suitable for DWEL but not for your agerange, so we need to set up a series of bins here where resources could be kept,along with notes as to why they might be good for a particular group.

Functional Requirements for Online Tools 199

Here, WebCT is seen as a repository for resources that ‘might come in useful later,’ asort of ‘holding tank’ for DLESE. Chris notes that such a holding tank might beorganised by posting lists of resources to WebCT. However, as a teacher in the grouppointed out, given that there the project’s goal was to generate five hundred resources,plus a large number of ‘also rans’ that might be of interest to other collections:

Teacher: … if we’re all doing this, we’re going to have tons of sites. In thatsearch thing [i.e. the WebCT search tool] will we be able to search and seewhether somebody else has looked at a site?

Chris replied, ‘As long as we have it in HTML format, then it’s searchable,’ but thisassertion was then modified by Beth:

Beth: Wait a minute, I think that it’s very important to realise that there’s twotools that you’re going to have to use in this process. One is WebCT, which isreally your communication, collaboration, and hopefully coordination tool.Another one is the DLESE catalogue system, and this is not only where you enterthe metadata for the resource that you’re considering, but there are certainfacilities within that to tell you that someone’s already grabbed this resource.

As a DLESE project member put it, at least the DLESE catalogue system had beendesigned to try and catch duplicate catalogue records; ‘I would not,’ she told theaudience, ‘recommend you trying to do that [i.e. track duplicate records] just withinthe confines of WebCT, you’ll step all over one another and duplicate the work.’

3.3 Summary of Observations

We suggest that these examples of conversation illustrate that DWEL members oftendo not distinguish between the functionalities of WebCT and the cataloguing tool,even though these are separate tools. Both are perceived to enable somewhat similarfunctions. WebCT in particular is seen as providing, in integrated fashion, bothcommunication, and also the means to manage cataloguing workflow. Further, thepotential of the cataloguing tool to manage and track workflow was overlooked.

4 Discussion

The preceding section presented excerpts from DWEL members’ conversations thatevidenced an expectation of integration across various DWEL tools. In practice, ashas also been described, the DWEL workflow is complex, and involves the use of anumber of separate tools. It was therefore concluded that DWEL members’expectations of tool integration were not being fully realised (or even possible) inpractice. A related conclusion also suggested itself: that if DWEL members had highexpectations of the tools that were at least partly unfounded, they might also find thetools harder to use than they expected. (These initial conclusions gained insignificance when monitoring of the DWEL members’ use of WebCT and thecataloguing tool showed that members were participating unevenly, and some hardlyat all. A subsequent questionnaire, partly informed by the research described here,suggested that many DWEL members were experiencing difficulties in using the tools

200 M. Khoo, H. Devaul, and T. Sumner

(either separately or in combination), especially in pursuit of the task of cataloguingdescribed in Section 3 (above). This research will be reported at a later stage.)

In this section we address therefore (a) short-term strategies for integratingWebCT and the cataloguing tool as they are currently configured, and (b) the longerterm development of a groupware layer that will sit on top of a future releases of thecataloguing tool.

4.1 Short-Term Support Measures for DWEL

One strategy involves generating paper documentation to simplify the workflow forparticipants. A number of studies have found paper to provide significant support forcomplex individual and group tasks [e.g., 16, 19, 20]. Following the initial analysispresented above, and a series of telephone conferences and small meetings involvingDWEL designers, concept maps of workflow and task allocation have been produced,and will be further developed. A more explicit calendar of deliverables has also beendrafted. Supplementary support documentation for the cataloguing tool, such asstructured worksheets for making notes while cataloguing, and a step-by-step outlinewith screen shots, is being considered. It is hoped that these and other measures willhelp participants to develop their understanding of CLCD workflow. Providing paperdocumentation for some of the tasks that require accessing and scrolling through longdocuments can lessen the electronic access load.

Additional training in the capabilities of the cataloguing tool is also suggested, asit has many features that have the potential to fulfill the requirements DWEL membershave been talking about. For instance, participants have expressed the desire to viewother participants’ work, and to have a way to sort and organize catalog records into avariety of groups. Since viewing catalogue records in WebCT is not possible,participants will be informed of and encouraged to use features of the cataloguing toolthat allow all members to view the entire collection in addition to their own work.Individuals will thus be able to work independently in selecting and cataloging theirown resources, but also to view (but not edit) the work of others. Sorting on aparticular concept (e.g. a working group name) can be done if participants insert theirworking group name into their original catalogue record, and there are other searchstrategies available in the DCS that could facilitate discussions about scope andcontent. Finally, the idea of a ‘holding tank’ - that is, of a storage place for records notsuitable for one working group but possibly suitable for another group, or for DLESE- is also possible in the tool, where records whose fate is as yet unknown can be storedindefinitely away from the workflow in one searchable location.

4.2 Long Term Design Considerations for the Cataloguing Tool

This analysis also provides input for future development of the cataloguing tool,which was designed to help coordinate distributed collections development bycommunity members working independently, rather than collaboratively. While theweb-based nature of the tool fully accommodates distributed cataloging and collectionbuilding, social and community interaction needs were not part of the original design.This case study has revealed a number of requirements for possible expansions of thetool. A new module could be built to support the communication functions thatcollection building requires. This module would interact with member’s regular email

Functional Requirements for Online Tools 201

accounts using email groups that also archive as threaded discussions accessible fromthe tool (an HTML page). Participants would be aware of new messages via theirusual email and could choose to store messages locally, or rely on the archive. Hotlinks to the email group addresses, individual members and support personnel wouldalso be available. Document sharing, so that files can be attached, viewed anddownloaded from the module would be enabled. The DLESE website already supportsa posting tool; expanding this to handle document files and images may be areasonable extension of this tool. The cataloging tool would be much improved byadding a spell-checking and character count enforcement to the free-text fields in thecataloging tool. A „smart“ alert feature is envisioned, such that any users currently inthe system could be sent an alert message at any time by the DLESE Program Center,to warn them of unscheduled shutdowns of the cataloguing tool.

Over time, integrating these features into the cataloguing tool will reduce thenumber of applications that must be run simultaneously, eliminate version conflictsbetween tools, and minimize the number of new environments necessary to performthe cataloging and communication tasks of collection building. While the degree ofintegration envisioned by DWEL members is perhaps not technologically feasible, atthe same time CLCD involves complex tasks and new skills, and is important thatDWEL, and other projects that rely on volunteer educators, provide the workspaceand support that makes meeting CLCD work commitments as easy as possible.

In summary, drawing upon the observation, recording, and analysis ofcommunication within the DWEL project, the research presented in this article hasgenerated a number of implemented and potential suggestions for managing the CLCDefforts in DWEL, as well as more general design guidelines for tools to supportdistributed community-based collection development in digital libraries.

5 Conclusion

This paper has described the perceptions of the tools involved in community-ledcollections (CLCD) design amongst members and designers in the DWEL project. Wenoted how before the project commenced, there was little formal discussion of theprecise structure of project architecture. DWEL staff considered communicationneeds, while DLESE Program Center staff considered record creation andmanagement needs. We described how in practice DWEL workflow emerged as aseries of tasks that must be accomplished with a range of different tools.

In contrast to this heterogeneous CLCD architecture, we described how onoccasions DWEL members talk about CLCD as being managed in a relativelyintegrated fashion, often within just one tool (WebCT). This description was based onan analysis of how DWEL members talk amongst themselves about the tools they areusing for CLCD. The findings from this analysis were used in turn to generaterequirements for the future development of these tools, that address other emergentfindings from DWEL that indicate that some users are finding the prototype CLCDarchitecture complex or difficult to use.

What implications do these findings have for the wider digital library community?One important outcome lies in its description of CLCD itself. DLESE is a major actorin the development of digital libraries in the United States, and as the focus ofbuilding DLESE expands from the contributions of individual records to include thesupport of numerous sub-collections for wholesale accessioning, it is becoming

202 M. Khoo, H. Devaul, and T. Sumner

increasingly important to understand how to technologically support these newcommunities. As has been emphasised, as the process of CLCD is prototypical, designand management issues will only emerge in implementation and interaction. As thefoundation of DWEL and DLESE growth is based on users as contributors,understanding how to facilitate the development of user-developed thematic sub-collections via CLCD is central to other digital library communities engaged in similarprocesses.

A second important outcome lies in reporting the desirability of providingongoing monitoring and analysis of large scale digital library implementations, in thiscase CLCD. While the analysis has drawn upon theoretical roots (such astechnological frames theory), and has been implemented through inductiveethnographic methodologies that are also theoretically informed, its fundamental aimhas been to generate practical recommendations for implementation. Its utility liestherefore as much in interventions in messy local practice, as in the generation of tidytheory.

Acknowledgements. The authors thank Ed Geary, Bryan Aivazian, and other DWELmembers for cooperating with this research. This research made possible by NSFGrant # DUE-0121724.

References

1. Anderson, R.: Work, ethnography and system design. Rank Xerox Research Centre,Cambridge (1996)

2. Barley, S.: Technology as an occasion for structuring. Admin. Sci. Quarterly 31 (1986)78-108

3. Bijker, W.: Of bicycles, bakelites, and bulbs. Toward a theory of sociotechnical change.Cambridge, MA: The MIT Press (1995)

4. DeSanctis, G., Poole, M.: Capturing the complexity in advanced technology use: Adaptivestructuration theory. Organization Science 5 (1994) 121-147

5. Ellis, C. Gibbs, S., Rein, G.: Groupware. Some issues and experiences. Communicationsof the ACM 34 (1991) 38-58

6. Fischer, G., and Scharff, E.: Meta-design: Design for beginners. Presented at DIS 2000,New York, 2000

7. Fjermestad J., Hiltz, S.: An assessment of Group Support Systems experimental research.Jnl Management Info. Systems 15 (1998-99) 7-149

8. Grudin, J.: CSCW: History and focus. IEEE Computer 27 (1994) 19-269. Harper, R., Sellen, A.: Collaborative tools and the practicalities of professional work at the

IMF. Procs CHI,1995 122-12910. Harper, R.: The ethnographic turn. Why is has come about and how to do it. Rank Xerox

Research Centre, Cambridge (1996)11. Heraclous, L., Barrett, M. Organizational change as discourse: Communicative actions and

deep structures in the context of IT implementation. Academy of Management Jnl. (2001)44 755-778

12. Jackson, M.: The meaning of „communication technology“: The technology-contextscheme. Communication Yearbook 19 1996 229-268

13. Khoo, M.: Community Design of DLESE's Collections Review Policy: A TechnologicalFrames Analysis. In Procs. ACM/IEEE JCDL (2001) 157-164.

Functional Requirements for Online Tools 203

14. Khoo, M.: Ethnography, evaluation, and design as integrated strategies. In:Constantopoulos, P., & I. Sølvberg (eds.)., Procs. ECDL 2001, LNCS 2163 Springer-Verlag, Berlin (2001) 263-274

15. Kling, R.: Social analyses of computing. Computing surveys 12 (1980) 61-11016. Malone, T.: How do people organise their desks? Implications for design of office

information systems. ACM Trans. Office Info Systems 1 (1983) 99-11217. Mandviwalla, M., Olfman, L.: What do groups need? A proposed set of generic groupware

requirements. ACM Trans. Office Info Systems 1 (1994) 245-26818. Orlikowski, W., Gash, D.: Technological frames: Making sense of information technology

in organizations. ACM Trans Info Systems 12 (1994) 174-20719. Sellen, A., Harper, R.: Paper as an analytic resource for the design of new technologies.

Procs CHI 1997 319-32620. Star, S., Griesmer, J.: Institutional ecology, ‘translations’ and boundary objects. Social

Studies of Science. 19 (1989) 387-42021. Star, S., Ruhleder, K.: Steps towards an ecology of infrastructure. Presented at CSCW 94,

Chapel Hill, NC22. Walther, J.: Computer-mediated communication: impersonal, interpersonal, and hyper-

personal interaction. Communication Research 23 (1996) 3-4323. Weatherley, J. Sumner, T., Khoo, M., Wright, M., Hoffman, M.: Partnership reviewing. In

Procs. ACM/IEEE JCDL (2002). In press.24. WebCT. Corporate website: http://www.webct.com25. WebCT. Leveraging technology to transform the educational experience. White paper.

(2001)26. Weedman, J.: The Structure of Incentive: Design and Client Roles in Application-Oriented

Research. Science, Technology, and Human Values 23(3) (1998) 315–345