A Collaborative On-line Digital Data Tool for Creating Living Narrativesin Organizational Knowledge Systems

Ricki Goldman-SegallMERLin, University of British Columbia

Vancouver, BCricki.goldman-segall@ubc.ca

http://www.merlin.ubc.ca

Srinivasan V. RaoUniversity of Texas, San Antonio, TX

srao@lonestar.utsa.edua

John WillinskySeattle University, Seattle, WA 98122

willinsk@unixg.ubc.ca

AbstractThe universities have long been organizationalknowledge systems (OKS) in ways that now are underpressure to change. In addressing the challenge, theneed is to use information technologies (IT) in newways to capture, preserve, sort, and interpretorganizational memory (OM). Unfortunately theparadigm for OM tools has followed a transfermodel-how to transfer specific content or stories toothers using the most efficient tools possible. Thetechnology acts as a corridor through which themessage travels. A constructionist paradigm stronglycounters this approach. Technologies are used asmemory makers; these knowledge building tools helpcultures grow, lessening the geographical distance thatseparates organizational members by enabling them tocontribute to what Goldman-Segall calls livingnarratives, digital documents of various media formsthat change over time and in various spaces.Geographically dispersed organizations particularlyneed networked digital construction tools (1) toencourage users to understand each other’s ever-changing contextual knowledge and (2) to fostercollaboration while capturing, sorting, representingand, most important, interpreting the histories and thememories of their organizations. Web Constellationshas been designed as an on-line analysis tool thatencourages dispersed users to work on shared digitalvideo datasets and build patterns or layers ofinterpretations leading to a greater level ofunderstanding for those who were unable to participatein a given event or a series of events. In this article, weexplain the need for these collaborative on-line toolsthat build multiple meanings, and then we discuss howWeb Constellations can be used as a tool fororganizational memory system (OMS) andorganizational knowledge systems.

1060-3425/98 $1

1. Introduction: Organizational KnowledgeSystems

The academy stands at a critical juncture as a virtualOrganizational Knowledge System. The university hasalways stood as a system devoted to organizingknowledge, but when it comes to the production andorganizing of that knowledge it is only virtually asystem. For it depends on the autonomy andindependence of its knowledge producers, which suggestsa system against systems. While this worked well fornearly a millennium changes in the current state ofknowledge systems and economies pose a particularchallenge to the university’s approach to data andknowledge in an area such as the social science, callingfor innovative new ways of constructing knowledge andcorresponding tools, such as Web Constellations, toenable scholars to create OKS’s in new ways (Goldman-Segall, 1997).

To understand the change, one can look at theparallels from the textual perspective offered by thelibrary. The books in a library form a system, organizedby a catalogue although the parts are dedicated tostanding alone and often against other works. Theencyclopedia, on the other hand represents a textual OKSof a different order, with its uniform data entry and cross-referencing, dedicated to forming a coherent andcoordinated whole. This became a point of philosophicalimport with Gottfried Whihelm Leibniz, who becamelibrarian in 1690 of Augustus the Younger’s library atWolfenbüttel. The library, for Leibniz, was not a palaceof books, as it is for many of us, but a magasin de science , a knowledge storehouse or warehouse ofpropositions, all of which were stored in a mostunsystematic way in books. The old adage is that “thelibrary is a catalog.” Yet how were readers to track downthese propositions, some of which scattered among a

0.00 (c) 1998 IEEE

culture that has been committed to finding ways oforganizing knowledge into systems that would betterserve the public and public forms of understanding(Willinsky, in press).

Today, the Leibniz Lab in Grenoble, France,“studies the general properties of automata networks aswell as their use in Artificial Intelligence domains and forcognitive modeling” (1997). One decided direction instriving to achieve an OKS that warrants its name hasbeen for the university to pursue concerns of automatadata in the form of datamining and datamation. One ofthe authors of the present paper is pursuing this topic andits implications for the social science in some detail(Willinsky, 1997). However, in this paper anotheralternative for the organization of knowledge systems inthe social sciences is pursued, based on the mutabilityand distributed potential of digital data, and thepossibility of collaborative knowledge construction. Thisdirection is pursued out of a marriage of newtechnologies and a far older wisdom of living narratives.

2. The Mutability of Digital Data

Digital tools to represent organizational memorieshave a serious role to play for the organizational sciencesin the age of information technology. The question ofwhat counts as memory or history in relation to what isemerging within a global networked body of data is notthe least important of the many concerns. What happensto definitions of representation, authenticity, and thevalidity of data, information, and knowledge as it ismediated using digital data? What becomes of theprocess of maintaining and selecting moments fromorganizational histories when one considers the endlessversions that any story can take? Will a user ever knowthe best story, the real story, when digital data are soeasily reconstituted and made to look authentic? It is thecollective responsibility of organizational members,whose efforts to record and make meaning of events arecritical to the long-term well-being of the organization,to explore these questions. It is the collectiveresponsibility of social scientists to underscore thedangers without limiting the use of advancedtechnologies that potentially could contribute toknowledge building in organizational settings, whetherthese settings be in businesses, schools, or on-linecommunity multiple user domains.

The mutability of digital data has obvious advantagesand disadvantages. The most obvious advantage, well-documented in any number of fields over the last decade,is that users can augment and enrich their print noteswith video documents, annotations, and sound data tobuild what Clifford Geertz (1973) would call the thickdescription of an event (Goldman-Segall, 1989;Davenport, 1989). Sharing these living documents ondistributed networks provides members with anopportunity to become part of the interpretation process

1060-3425/98 $1

and therefore to contribute to the organization’smemories. As Ackerman and Mandel (1995) point out:

One of the most interesting memorycomponents is what astrophysicists have termedthe “living cookbook.” Scientists can providetheir data analysis recipes ... for others or fortheir own personal use. ... Sharing the recipesallows a common method of acquiring andretrieving data analysis methods, something thecommunity did not have before the ASSIST (p.329).

Clearly, the notion of fixed interpretations by onerecorder is part of the pre-electronic past. Yet, the needfor valid accounts of what transpired in any given event,a meeting for example, is even more critical given themutable nature of digital media. Although minutes takenat a meeting may never provide users with a full accountof what was said and meant, there is accountability in thefact others will read them and make appropriate changeswhich acknowledge their various perspectives. Digitalminutes, perhaps in the form of video and text chunks inan electronic environment, when used with appropriatetools for collective construction, could bring about amore layered and perhaps more valid accounting becausediverse members can bring their points of viewing thoseevents to the common electronic platform. The variousviews can build the “story,” the “memory,” or whatGoldman-Segall prefers to call the living narrative whereusers not only find out what happened but they areintricately involved in the construction of thoserepresentations.

The mutabilty of digital data underscores a potentialdisadvantage—ambiguity. Lim (1996) claims that richlanguage offered by the textured descriptions ofmultimedia lessens ambiguity. Research by theCognitive Technology Group (1993) contends that,because video data are dynamic and spatial, they allowviewers to form clear mental pictures of complexsituations—pictures that are closely related to realobjects and events “in the real.” To use their example, achild seeing a bird put stones in a glass to raise the levelof water in the glass understands this action more clearlywhen seeing it than she does when reading about it. Inrelatively straight forward cases, where seeing an imageaids understanding, this is true. But according to theresearch conducted by Gaskell, et al., it seems that whatis retained and understood differs, but is not intrinsicallymore accurate, when using diverse media forms. In theirstudy, those presented with a socioscientific issue inprinted news format were more likely to frame it in termsof a conflict between different points of view than werethose presented with the same issue in a video newsformat. Watching the video seem to lead to a singlepoint of view (theirs), downplaying the controversy. Atthe same time, many of those presented with the printformat disengaged more quickly from the story, whereas

0.00 (c) 1998 IEEE

those who were presented with the video format keptwatching and often used a dramatic image as a basis fordiscussion. In general, people engage differently with andconstruct different meanings from various media formats.

What most users of media tools seem to agree on isthat digital data can provide flexibility for the user.Digital data enable viewers to “get the picture,” onemight say, of what was occurring when the video wasshot. However, for all the blanks that are filled by seeingthe image, other ambiguities arise. “The picture is wortha thousand words” because interpretations abound wheninterpreting visual data. While in certain circumstances,video lessens ambiguity by offering both visual andauditory modes on which to base analyses, we believethat this argument is based on the notion that commoncodes for interpreting signs, gestures, and intonationsexist. As anthropologists have taught us, individualsfrom various cultures vary greatly in their use of gesturallanguage, as do individuals within the same culture. Andthey also vary in how they interpret the gesturallanguages of others. Did the person’s eyes dart back andforth while speaking. How does one understand themeaning of an event: Is the poking of the pencil on thetable aggressive, assertive, or a nervous twitch? How iswhat she says informed by what she does when she saysit? And whose interpretation is the best one? In short,although the aim is for contextual data to aidunderstanding, it could very well be the case that thegreater context provided by digital media, the greater thecomplexity in interpretation. With rich data there ismuch more to understand.

Some would ask: “Why use digital data if it makesanalysis more complex than it already is?” After all,isn’t the purpose of analysis to find the bestinterpretation with the least interference of extraneousinformation? One could argue that digital media changeshow users understand how memories, histories, andcultures in organizations are constructed. By becomingparticipants in the creation of that memory, they are nolonger recipients of historical knowledge of theirorganizations.

3 On-line Digital Video Analysis as anOrganizational Knowledge Tool

Over the past year, an on-line template called WebConstellations has been designed that demonstrates theprocess of transforming data into clusters bound bycommon themes, thereby facilitating the construction of agreater variety of interpretations (Goldman-Segall, 1997).This template with linked digital events becomes, whatPapert (1980) calls representations, “objects-to-think-with,” which take full advantage of collaborativepossibilities afforded by the Web. The purpose of thecreation of clustering data thematically for collaborativereconstruction produces a forum for exploring alternativeviewpoints, a platform for multi-loguing where multipleusers can both access and annotate multimedia data

1060-3425/98 $1

(Goldman-Segall, 1995). At issue are the ways in whichinformation is assembled as a definition of how to engagewith the world. Rather than succumbing to the forms oftechnological determinism found in the work of thosewho, like Neil Postman (1992) and Ursula Franklin(1990), would warn of us of the dangers ahead, we feelthe important task is to build prototypes and models ofdifferent ways being with technology, of using it torealize what we want and do not want in the world ofknowledge construction.

Web Constellations was designed as a research toolfor digital video analysis. The objectives of an on-linedigital video analysis tool and that of an OrganizationalMemory System (OMS) or an Organizational KnowledgeSystem (OKS) may be slightly different, but theunderlying activities are very similar. In analyzingdigital video data as in analyzing text data, the researcherseeks to understand the phenomenon under study with aview to contributing to and creating knowledge. Theinformation being captured is a sample of a naturalphenomenon or of a controlled study. The primitives foridentifying chunks are, as with text analysis, dependenton the research question(s). Multiple annotations aredesirable to enrich the interpretation of data, i.e., to yieldthe thick description (Geertz, 1973). Moreover, thelinking and integration of selected chunks highlightsunderlying themes. In using an OMS or an OKS, usersseek historical information that may bear on a currentproblem but they also create it through adding theirviews. Multiple annotations are necessary to ensure thatloss in context is minimal. Linking and integratingselected chunks is an attempt to discover or uncoverrelationships that may be applicable to a problem onhand. The use of an on-line digital video data analysistool as an OMS takes full advantage of the immediacy ofthe networked media; users can communicate with eachother to check out stories or versions of what has beenstored as memory. There can and probably will be morethan one story and these multiple stories build theplatform for multi-loguing, a place for constructingviewpoints. For example, there may be a story with thedominant theme of designing a robot. The scenario forusing Web Constellations to retrieve stories would be asfollows: a user of the system would first check if a clusterof data, called a constellation, exists that reflects a storywhich is applicable for the purpose at hand. If yes, theuser can review the constellation, and also explore theremaining chunks to construct new stories. If no relevantconstellation exists, then the user will either explore thedatabase of chunks to create the necessary constellationsor add his or her own data chunks for others to view.

4 Filtering and Framing Views

Data have recently become available to users inseemingly less mediated forms through the use of Web-based technologies. Some would say that these raw andless filtered data are more accessible and thus more

0.00 (c) 1998 IEEE

“truthful” than the highly constructed final versions oftexts that appear in current media—books, journalarticles, films, and video reports. However, the filmethnography community has taught us that even the rawdata not yet molded into products are shaped by theauthor's choice, framing, or handling, and, that "thevalue of an ethnography or a film ... must be judged onthe appropriateness of what has been included and how ithas been handled" (Heider, 1976, 17). Framing andhandling data are mediated actions that add filters. Andalthough putting data on the Net for multiple users toview is perhaps less mediated than the publication ofonly the analyses, researchers are still framing their datawith constructs that reflect their own reference points,selecting which data are most representative for what theywant to demonstrate (Minh-Ha, 1992). In short, framingis the optimal word here. How data are framed consists ofmore than the categories within which they are placed foranalyses. Analyses are taking place at every juncture asviews, points of viewing, are added (Goldman-Segall,1997). Networked technologies can change basic searchand retrieval methods by enabling the context of thestory to accompany the less descriptive data markers orkey words—the qualitative to accompany thequantitative—without forfeiting the robustness of thecategorical. To start this form of multivoiced interactivestorytelling, one needs to start with annotation tools,simple spaces to comment upon what was seen, heardand read by users and authors alike, and build clusters ofsegments, scenes and scenarios, and finally stories thatconvey interpretations of events. In text-basedapproaches, field notes become constructed into textstories or case studies according to well-known traditionsof storytelling and story reading. But digital video "fieldnotes" on the net can be manipulated into a range ofconfigurations by multiple "authors" and thenreconstituted into unrecognizable video bits and bites.They can be retold as portraits. At each point, new layersof interpretation are added by selecting and cutting awayfrom a whole or adding new materials to build an artifact.With the addition of intelligent agents, they can begin tofunction for their authors' purposes (Sheeth and Maes,1993; Davenport, 1996; Negroponte, 1996) as stories arebuilt by multiple authors.

In this milieu of competing positions and definitions,we face the dilemma of representation. Texts, to berepresentative of those whose lives are being described,need to include the participation of those who have astake in the issues—a notion that might disturbadministrators of some organizational structures.Representations can no longer be thought of asindividually constructed works by solitary authors. Norare they limited to the final products. They live. Theychange. Some die. Users can frame what they see forothers to annotate, comment on, and use within theirown frames of reference. The process of making arepresentation is itself a representation. And several kindsof representations emerge until a decision is made to put

1060-3425/98 $1

closure to a work, theory, or investigation and move onto another. As Geertz (1973) once told reminded usabout the layered texture of events:

There is an Indian story—at least I heard it asan Indian story—about an Englishman who,having been told that the world rested on aplatform which rested on the back of an elephantwhich rested in turn on the back of a turtle,asked (perhaps he was an ethnographer; it is theway they behave), what did the turtle rest on?Another turtle. And that turtle? “Ah, Sahib,after that it is turtles all the way down” (p. 28).

Geertz uses this story to explain the incompletenature of any form of analysis. This applies equally wellto how we make meanings and construct organizationalknowledge from the digital stories we collect. Each formis derivative of another form until it reaches down to theaction we thought we were describing; that is, until werealize that this turtle rests on the back of yet anotherturtle.

5 The Template

Web Constellations has been designed for users toadd their views and interpretations using a customizedversion. The software builds upon and extends anexisting software tool, Learning Constellations (1989)and then Constellations (1993), designed by Goldman-Segall, and bringing it into a Web-based environment.This prototype aims at stretching the Web's ability tofacilitate collaborative, Web-based knowledgeconstruction as well as a critical assessment of thepotential impact these technologies can have on thesocial sciences.

6 Overview of MethodologicalConsiderations

Methodological paradigms are changing and tools arenow available for a more expressive form of constructingmemory and knowledge. For example, research portraitsor stories can use a variety of media forms to convey andcreate meaning in the minds of readers and viewers.Previous research in the area of digital video analysisindicates that research on the Web is already acollaborative platform where those studied can becomepart of the research process and those who read the resultsof studies can enter into a conversation with researchersand with those studied. By analogy, the creation oforganizational memory on the Web can also becollaborative, where each observer of an event can presenthis viewpoint and where each user can add hisinterpretation of the event. The inquiry that we areundertaking attempts to move thinking (1) backwardsthrough a history of literacies that considers the

0.00 (c) 1998 IEEE

knowledge building capacities of the text, illustration,table, book, library, encyclopedia, and database, to putdevelopments on the Web into proper perspective, and(2) forward through speculative forms of computerscience. This work has been designed to inform and formpart of the content of the Web development that thisproject represents. Moreover, it looks at OM from severaldisciplinary perspectives including computer science,education an anthropology (Davenport, 1996; Harrison &Baecker, 1991; Suchman, 1987; Duffy & Jonesson,1992).

7 Web Constellations<http://www.merlin.ubc.ca/tools>

A tool in its own right, Web Constellations enablesusers to create chunks or stars [Fig. 1] from any piece ofinformation which can be described by a URL, such as achunk of video on a web server, tag those stars withkeywords, add annotations to their stars and other users'stars. This coded database of material becomes anexpanding body of material for collaborative research asusers locate stars based using a simple search mechanism[Fig. 2], cluster stars found through searchers intoconstellations, and continue to annotate stars and theconstellations built from them.

Figure 1: A Star in Web Constellations

1060-3425/98 $10.0

Figure 2: Web Constellations Search Results

Web Constellations system runs on a Web server,which allows researchers to access this system from anymachine in any location provided hay have a reasonablyfast connection to the Internet (depending on the kind ofdata being analyzed) and a browser which supportsJavaScript 1.1 or better. Researchers do not have toworry about saving their work to their own system orsending updated files to colleagues. While the actualdatabase program runs in one server location, the datareferred to by the stars can exist anywhere on the Internet.This means that distributed teams can make their dataand analysis available for perusal as quickly as it can bemade available as a file on their web server; and heavydata loads can be distributed among several locations. As

a system to enable collaborative story reading, making,and telling, the customized version of WebConstellations will allow a more limited and directed setof features, which will focus on data interpretation andpresentation. From within this environment, they canread other peoples annotations of that video clip and thenadd their own. They can use the search function or an on-line index to locate other video clips and gather theminto constellations, building their own narratives. Thesenarratives themselves can then be further annotated bytheir authors and other users who are entering thisenvironment and participating in the processes ofreading, making, and telling. This digital environmentfacilitates the development of a global community ofusers.

Text documents can be used in concert with WebConstellations. Users can view source video as part oftheir reading of a document; make their own story byannotating video pieces and building meaningfulconstellations of video pieces; and tell their story bymaking their annotations and constellations available toother users, entering into a conversation with adistributed community. Web Constellations enablesusers to view a chunk of video, a star, for example, andadd annotations, and cluster the (star) chunks intoconstellations. In this way, the coded database ofmaterial becomes an expanding body of material forcollaborative research, a living narrative.

8 Limiting Factor: Complexity

Using the Web for accumulating multiple viewpointsof organizational events, there is the problem of addingcomplexity that will need to be untangled over thecoming years. How will the future users be able to makemeaning of these digital events? Moreover, each time therepresentation media form changes, e.g., text to video orvideo to networked video on the Web, another layer of

0 (c) 1998 IEEE

interpretation and complexity is added. In themultimedia ethnographies Goldman-Segall hasconducted in two school sites (Goldman-Segall, 1997),layers of interpretations have been added by multipleusers, including those persons who are being videotaped.As we watch others view these portraits, interpretationschange further. The process of interpreting digital data isendless, recursive and often exhausting in its conceptualand operational open-endedness. However, with toolsthat enable greater meaning making, the frustration canbe minimized, the integrity of the data maintained andthe flexibility maximized—the promise of emergingtechnologies.

The new technologies for organizing knowledgesystems will not dictate of themselves the future ofknowledge or the nature of our understanding. Thatdirection will be set by people making choices about thefuture they wish to envision, and by pursing newpractices, supported by new tools, to enable it to happen.Leibniz’s dream of a single calculus of ideas, anetworked database easily mined, is one vision. Theorganization of our work around the creation of livingnarratives is another, and the least we can ask for, in thespirit of this collaborative ethos for knowledge building,is the pursuit of a bilateral future.

The main advantages of using tools such as WebConstellations is that digital-data-capture facilitates on-going collaborative access and analysis, building a rich,multifaceted view of an event. Networked digital videoportraits enable users to interact with other users’ stories.Is this the beginning of a form of "being with" others thatencourages users to become members in the on-goinghistory of their organizations, or is this opening up yetanother Pandora's Box?

Acknowledgments

Web Constellations was designed in MERLin, theMultimedia Ethnographic Research Lab at the Universityof British Columbia under the direction of Goldman-Segall. We thank Lawrence Halff for the development ofthe Web Constellations application. We also thank theSocial Sciences and Humanities Research Council ofCanada for development funds. Collaborative Researchon the Web, grant #410-97-1368, was awarded toGoldman-Segall and John Willinsky in 1997.

References

Ackerman, M. S. & Mandel, E. (1995). Memory in thesmall: An application to provide task-basedOrganizational Memory for a scientific community.Proceedings of the Twenty-Eighth Hawaii InternationalConference on Systems Sciences (HICSS'95), pp. 323-341.

Cognitive Technology Group, (1993). Examining thecognitive Challenges and pedagogical opportunities of

1060-3425/98 $1

integrated media systems: Toward a research agenda.Journal of Special Education Technology 12 pp. 118–124.

Davenport, D. & Brondmo, H. P. (1989). Creating andviewing the elastic Charles hypermedia journal.Hypertext II Conference Proceedings, pp. 1-6.

Davenport, D. (1996). Smarter tools for storytelling: Arethey just around the corner? IEEE Multimedia 3(1), pp.10-14.

Duffy, T. & Jonassen, D. (1992). Constructivism andthe technology of instruction: A conversation. Hillsdale,NJ: Lawrence Erlbaum Associates.

Franklin, U. (1990). The real world of technology.Montreal: Canadian Broadcasting Corporation.

Gaskell, P. J., Fleming, R., Fountain, R., & Ojelel, A.(1991). Socioscientific Issues. British ColumbiaAssessment of Science Technical Report III (Victoria:Ministry of Education).

Geertz, C. (1973). The interpretation of cultures. NewYork: Basic Books.

Geertz, C. (1988). Works and lives: The anthropologistas author . Stanford, CA.: Stanford University Press.

Goldman-Segall, R. (1989). Thick Description: A toolfor designing ethnographic interactive videodisks.SIGCHI Bulletin 21(2), pp. 118–22.

Goldman-Segall, R. (1993). Interpreting video data.Journal for Educational Multimedia and Hypermedia2(3), pp. 261-282.

Goldman-Segall, R. (1995). Configurational validity: Aproposal for analyzing ethnographic multimedianarratives. Journal for Educational Multimedia andHypermedia 4(2/3), pp. 163-183.Goldman-Segall, R. (1997). Points of viewing children'sthinking: A digital ethnographer's journey. Mahwah,NJ: Lawrence Erlbaum Associates.

Harrison, B. & Baecker, R. (1991). Designing videoannotation and analysis systems. Proceedings of CHI'91.

Heider, K. G. (1976). "Truth" in film and ethnography.Ethnographic film , pp. 16-18.

Leibniz Laboratory. (1997, Feb). http://www-leibniz.imag.fr/RESEAUX/reseaux-en.html.

0.00 (c) 1998 IEEE

Leibniz, G. W. (1951a). Precepts for advancing thesciences and arts. In P. P Weiner (Ed.) Leibniz:Selections (pp. 29-46). (Orig. 1680). New York:Scribner.

Lim, K. (1996). Multimedia as an enabling technologyfor enhancing organizational memory to supportmemory systems. PhD diss. University of BritishColumbia..

Minh-Ha, T. (1992). Framer framed. New York:Routledge.

Neill, M. (1995). Computers, thinking, and schools inthe "New World Economic Order." In J. Brook & I. A.Boal, eds., Resisting the virtual life: The culture andpolitics of information. (pp. 181-194). San Francisco:City Lights.

Papert, S. (1980). Mindstorms: Children, computers andpowerful ideas . New York: Basic Books.

Papert, S. (1996). The connected family. New York:Longstreet Publishers.

Postman, N. (1992). Technopoly: The surrender ofculture to technology. New York: Vintage.

Rao, V. S. & Goldman-Segall, R. (1994). Capturingstories in organizational memory systems: the role ofmultimedia. Proceedings of the Twenty-Eighth HawaiiInternational Conference on Systems Sciences(HICSS'95), pp. 315-325.

Schloss, G. A. & Wynblatt, M. J. (1995). Using alayered paradigm to model multimedia. Proceedings ofthe Twenty-Eighth Hawaii International Conference onSystems Sciences (HICSS'95), pp. 777-786.

Thomas Sowell, (980). Knowledge and decisions. NewYork: Basic.

Suchman, L. (1987). Plans and situated actions. NewYork: Cambridge University Press.

Willinsky, J. (1997). Automata data and the future ofknowledge. Unpublished manuscript, Seattle University.

Willinsky, J. (in press). The politics of theory in practice:The social sciences. In P. Trifonas & D. Aoki (Eds.). Thepolitics of theory and the construction of disciplinarity.Cambridge, UK: Blackwells.

1060-3425/98 $10.00 (c) 1998 IEEE