Pergamon 0360-131505)0t)1)66-6

Computers Educ. Vol. 25. No. 4, pp. 215-225, 1995 Copyright © 1996 Elsevier Science Ltd

Printed in Great Britain. All rights reserved 0360-1315/95 $9.50 + 0.00

E S T A B L I S H I N G A N E L E C T R O N I C C O L L A B O R A T I V E L E A R N I N G E N V I R O N M E N T I N A U N I V E R S I T Y

C O N S O R T I U M : T H E C I R C L E P R O J E C T

MARC M. SEBRECHTS, l BARRY G. SILVERMAN, 2 DEBORAH A. BOEHM-DAVIS 3 and KENT L. NORMAN 4

Department of Psychology, The Catholic University of America, 4001 Harewood Road NE, Washington, DC 20064, U.S.A., 2 Office of the Provost, George Mason University, Mail Stop 3A2, Fairfax, VA 22030 4~A4, U.S.A., 3 Institute for Artificial Intelligence, Staughton Hall, Rm 206, The George Washington University, Washington, DC 20052, U.S.A. and 4 Department of Psychology,

University of Maryland, College Park, MD 20742, U.S.A.

(Received 1 October 1994; accepted 1 September 1995)

A~tract--The Consortium Interactive Research on Collaborative Learning Environments (or CIRCLE) project, was designed to examine (a) how the distance among remote universities can be bridged electronically; and (b) how this bridge can be used to develop truly collaborative learning with shared, distributed student and faculty responsibilities. Although the problem of "distance learning" based on the model of instructional television has been well-studied, the problem of collaborative distance learning, based on a design team model, poses new technological and psychological issues. The CIRCLE project was designed to develop a model of four-way instructional collaboration that places relatively small demands on existing university technological resources. Thus, this project focused on an examination of the human resources, the nature of video and audio interaction, and simple schemes for synchronous computer-based communication. The results of a number of collaborative trials indicated that interaction over high bandwidth video channels worked well with little training. However, the trials also indicated a need for better synchronous computer-based media to facilitate interactions. Further, the trials raised a number of technical, social, and task constraints that influence the effectiveness of the interactions.

I N T R O D U C T I O N

Learning is often constrained by limited departmental and institutional resources. What students learn depends to some degree on the expertise that is available locally and the methods defined by disciplinary boundaries. This has been a particularly problematic issue in the development of advanced technology which requires expertise in several disciplines. For example, development of an information retrieval system can require expertise in such diverse areas as visual perception, information management, and artificial intelligence. More generally, gaps between technical specialists, typically from computer science or engineering departments, and human factors specialists, typically from psychology, have made adequate preparation of students difficult.

The need for wide-ranging expertise and cross-disciplinary problem solving in education poses a challenge to curriculum development in general [1]. And this challenge is particularly significant for the field of our proposed collaboration teaching, human-computer interface design. With increased technical and instructional information and decreasing resources for faculty and cur- riculum development, it is difficult to have adequate expertise at any single location. By collaborating across institutions, however, it is possible to develop a wider base of expertise, which we will term a "virtual faculty". In this approach it is the "access" to expertise rather than its "ownership" that is essential.

For many institutions, one of the biggest barriers to such inter-university instruction is the administrative overhead. In our case, the four universities with which the authors are affiliated are part of the Consortium of Universities of the Washington Metropolitan Area. Because of this consortial arrangement, registration at any of the universities can be completed through the student's home institution with no direct incremental cost to the student. Despite this arrangement, the need to spend as much as 45 min for travel to another campus can serve as a significant impediment to student exchange and to guest lectures by faculty at other institutions in the Consortium. This project explored how to eliminate that barrier and to take advantage of alternative models of learning through the use of existing technologies.

215

216 MARC M. SEBRECHTS et al.

Our specific goal within this framework was to take full advantage of our Consortium relationship to develop a virtual faculty model for human-computer interaction at our four universities. To that end, we developed a collaborative effort, the Consortium Interactive Research on Collaborative Learning Environments (CIRCLE) project, under the auspices of the Consortium of Universities of the Washington Metropolitan Area. This project addressed two major issues. First, how can we electronically bridge the distance among the institutions? In addressing this question, we considered only "realistic" options that would place relatively small incremental demands on existing resources. Thus, for example, although videoconferencing rooms linked through cable TV provide one potentially compelling solution that has been implemented elsewhere, it is not addressed here. Not only is this technology unavailable at our institutions, but it tends to be cost ineffective for most other universities [2]. Because we do have functioning microwave and Internet links, those options were evaluated.

Second, how can we use that bridge to develop truly collaborative learning with shared responsibilities? The problem of "distance learning" on the model of television, where a central facility broadcasts the information beyond the walls of the classroom or institution to individual students, has been well studied [3]. As a result, many of the technologies are designed to work on this model of dissemination of information from a central point with asynchronous response, usually via electronic mail. Even most desktop videoconferencing tools tend to be point-to-point rather than multipoint. In contrast, the model we are interested in pursuing is more that of a design team, with different members having varying expertise and contributing by collaborating. Such multipoint interaction poses new technological and psychological issues (for a discussion of some of these issues, see [4,5]).

This paper describes a number of factors that need to be addressed as the basis for establishing a virtual faculty consistent with this alternative educational model. To that end, we examined the human resources, the nature of video and audio interaction, and simple schemes for synchronous computer-based communication.

ASSESSMENT OF RESOURCES

In a collaborative venture such as the one described here, it is not possible to simply "implement" the solution. There are a substantial number of factors that need to be considered. As a consequence, our efforts proceeded on several fronts. First, it was necessary to assess the available faculty expertise in the content area. Second, it was necessary both to determine the available resources and to involve the relevant technical and decision making members from each institution. Cooperation among the institutions was necessary both to identify those resources and to devise a concrete plan for interaction. Third, once agreement was reached, each option had to be tested and evaluated. The following sections describe aspects of each of these approaches in roughly the order in which they occurred.

Workshop on faculty resources

The first step in determining a plan for collaboration was to identify the expertise of the Consortium faculty in a given topic area. In our case, the focus of this interest was human-computer interaction. Although a number of researchers from our institutions had some formal or informal interaction, there was no clear record of relevant expertise. To obtain this information, a survey was conducted of all institutions in the Consortium, and a follow-up workshop was held. It was surprising to find the extent and degree of interest in the topic across institutions. Another surprising result was that despite our relative proximity and shared interests, there had been very limited contact among institutions. A document was produced listing major resources in the Consortium in human-computer interaction, including courses, research activities, areas of ex- pertise, and contact information, including electronic mail addresses [6]. In addition, a listserv was set up at The University of Maryland (MALACHI@umdd.umd.edu), which provides a forum for announcements and information.

The workshop made it clear that the resources available to us were not being fully utilized. Collaboration among a subset of the participants was continued through electronic mail and

CIRCLE project 217

CIRCLE Video and Computer Network

Internet

Microwave Broadcast

Video Monitors

Microwave Broadcast

AudioOnly , , , , t i"

©WU

Fig. 1. Schematic diagram of the microwave video network and Internet connections among the four participating universities.

teleconferencing. Our goal, however, was to share a common classroom electronically. To that end, we conducted a study of the use of video and other computer-based communication tools.

Workshop on video resources

Our first step was to assess the relative importance and ease-of-use of video interaction. The institutions do not currently have the facilities for high quality inter-university videoconferencing or desktop video. The sites are, however, linked by microwave. Three of the institutions have microwave broadcast facilities while the fourth has the ability to receive but not to broadcast over microwave. As a consequence, the microwave link was used as the basis for assessing the video interaction, while mainframe and micro-computer links were used for non-video interaction.

Although the links were already established, making sure that the four institutions could actually communicate still required substantial planning and cooperation. Most broadcast facilities are concerned with one-way video communication, sending signals out to a viewer. In our case, the goal was to have interactive dialogue among four sites. To determine if and how that could be done, a workshop was held involving technical experts and administrators from communications and television facilities at each site. At this workshop, a plan was devised and implemented to route signals through one institution which would then rebroadcast those signals. One of the institutions needed to replace an antenna because its current placement with respect to the local geography did not provide adequate reception. The institution without broadcast facilities needed to establish a phone link to the primary broadcast site; once this was accomplished, that signal was rebroadcast with the T.V. signals. Figure 1 shows a schematic diagram of the microwave video network and Internet connections used for the video trials among the four participating universities.

218 MARC M. SEBRECHTS et al.

Video trials

Several of our media consultants had expressed concern that it would be extremely difficult to manage the four-way interaction. Many studios find it difficult to manage even phone-in audio responses to programs. The first evaluation of the link was therefore initially conducted among the four principals of this project. Our finding was that the interaction was surprisingly easy. Although we had devised several schemes for turn-taking, there was little need for any formal protocol. This may have resulted, however, from the fact that we had been interacting via teleconference on a regular basis for some months.

A second trial consisted of the four faculty and approximately 16 students. In this situation, interaction became substantially more difficult. Students expressed some reluctance to interact, but that is typical of many initial meetings, even in a common physical space. In addition, however, there were substantially more difficulties with turn-taking. On occasion, several individuals wanted to speak at once or did not know if they could join in. This difficulty was increased somewhat by the absence of video from one site. Participants at that location had no means to visually signal their interest in contributing to the discussion.

The video trials also indicated several interesting difficulties. First, video broadcasting in studio settings introduced constraints on the speakers. People had to remain physically located in a way that they would be constantly "on camera". In addition, participants appeared to be more restricted in their interaction because of the presence of the camera. The group that was not on camera was able to move more freely and could communicate among themselves without interrupting the ongoing discussion. This mimics a finding of some years ago of early videophones; many people rejected the video component because they preferred to have a certain degree of "privacy" in their communication.

M U S H trial

During the video trials, it became clear that in order to conduct a joint course effectively, the video links would need to be supplemented or replaced by other forms of computer-based interaction. In order to continue the theme of synchronous interaction, two of the sites tried using a Multi-User SHared environment (MUSH) in Colorado, which allows a number of users to log on and talk simultaneously [7]. In the MUSH we used, a classroom was set aside for the interaction. Each participant could "speak" as in a real classroom, although "speaking" was achieved by typing on the screen. The system allowed all participants to speak simultaneously in this way, with the text being sent to all other participants each time a carriage return was pressed. In addition, there was a "whisper" facility which allowed users to have private communications with other participants, much like whispering to someone in a classroom.

The classes used the MUSH to conduct a discussion about the relevance of video for interaction. In this environment, each person was able to type messages concurrently. Each carriage return then served to send the line to all other participants. However, that approach proved very ineffective as messages became interwoven from different participants depending on the sequencing of their typing. As a result, a moderator was used to assign turn-taking, with appropriate two-way (whisper) links between each participant and the moderator being used to indicate interest in participating and to make comments. Although the moderator did provide for a more orderly discussion, the overhead in monitoring and sequencing "speakers" was so high that it kept the moderator from participating substantively in the discussion. In addition, the rate of communication was limited to typing speed, making the conversation very stilted; and the procedure of establishing a queue for "speaking" made the interaction seem asynchronous [8]. In a number of cases, by the time a person was given the opportunity to speak, the issue in which they were interested was no longer central to the conversation.

Workshop on computer networking resources

In order to evaluate other forms of interaction that would be reasonable within the constraints of our institutions, a workshop was conducted involving administrative and technical staff from each of the four institution's computing centers. One issue was the feasibility of replacing the microwave link with desktop video. Although the institutions could, in principle, support reasonable

CIRCLE project 219

quality video over the Internet with appropriate machines at each institution, the cost of doing that was extremely high. In general, such video would use substantially more of the capacity of a standard T1 connection than any of the institutions could reasonably justify committing to a single course.

A number of options were explored, including the use of an enhanced MUSH, with decreased need for moderating discussions, a common server at one institution with logon permission for all participants, and a shared FTP site for rapidly uploading and downloading files created at each site. It was generally agreed that although there were many resources at the institutions, there was no inexpensive way to provide the electronic, shared "whiteboard" facility that we wanted. Nevertheless, all four universities were linked via the Internet, and we wanted to use existing computer and networking facilities to provide another channel of communication for information exchange (see Fig. 1).

File Transfer Protocol (FTP) trials

Because it required the fewest issues in terms of security and programming effort, an FTP site was chosen as the communication strategy for our group. The four faculty tested this facility in conjunction with a teleconference. Exchanging files in this way was very quick and easy. However, it was necessary to develop protocols about file naming, since it quickly became difficult to determine easily which files were most recent, or had already incorporated others' modifications.

The video link over microwave was then combined with the FTP link. In this context, about 16 students, together with the four faculty, had a common session evaluating HypcrCourseware, a hypermedia environment used for teaching in the University of Maryland's computer class- room [9-11]. Because of differences in the computer systems at each site, and the limited screen image quality over the microwave link, each site ran local copies of the software being evaluated. The presentations and discussion of the system were done via the microwave channel.

Each site then constructed its own list of pros and cons of the system asynchronously. These lists were then posted to the FTP site. During a second session, each site retrieved the master list of pros and cons, prioritized them, and posted them back to the FTP site. These prioritizations were then discussed over the video and audio channels.

An alternative system, Silverman's Trekkie [12], was presented and contrasted with Hyper- CourseWare. A discussion was conducted concerning the technologies used for each system and their relative effectiveness.

The FTP link worked well, but there seemed to be little connection between the video/audio interaction and the computer-based prioritizing of lists. Each group tended to act independently in their initial list construction and prioritization, making it difficult to recognize the updated, prioritized files and upload them. As a result, modification of the prioritized list required a moderator with participants commenting remotely on each item. Consensus was then reached primarily by sequentially polling each institution for its choices. In the absence of a common space, this type of work is perhaps best done asynchronously.

PictureTel trial

In order to test improved freedom of movement and greater privacy in the video domain, two sites conducted an additional video trial using PictureTel units with ISDN line connections. One site had one professor and 4 students while the other site had two professors and 19 students. The PictureTel units had a local control unit that could be both (l) preset to zoom to different spots in a given classroom and (2) passed around to participants to personalize the zoom and focus control. This approach increased the number of students who could be included in the collaboration, and it permitted students the flexibility to move around the classroom and engage in other activities. The extent of this freedom is illustrated by one student located at a workstation in the back of the electronic classroom who was "caught" downloading aduit-rated images from the World Wide Web when the camera zoomed over to him for an unexpected response.

This trial suggested that although PictureTel was a good system, it would not solve our problem of multipoint communication unless multiple units were purchased and a number of ISDN lines were available. Further, although the system is designed to be used by individuals, it requires

CAE 25:4-D

220 MARC M. SEBRECHTS et al.

someone other than the speaker (at the local site) or the listener (at the remote site) to position the camera. This was in contrast to our early microwave trials where the requirement for technicians was known and anticipated ahead of time.

Integrated media trials

Finally, we conducted two trials to examine the possible use of an integrated voice, video, and whiteboard capability over the Internet. In these trials, we tried out the CUSeeMe software from Cornell University through a "reflector" at one of the sites, which coordinates the signals. CUSeeMe integrates video, voice, and data into a single desktop environment [13]. It can support up to eight locations sending video to each other, although other sites can also pick up the video streams and be observers. In order to send and receive 8 video streams, however, the reflector must have the capacity to receive 8 streams of 56 kbps each and simultaneously reflect 56 more streams. This totals up to about half of a T1 line. To avoid this constraint, we limited our trial to four video/ audio generating streams, and even then our reflector crashed once during the first trial (although connections were quickly reestablished).

The first trial consisted of about 8 faculty and 20 students distributed physically over two sites and 4 workstations. Each workstation was an audiovisual Macintosh with a videocamera and microphone. Each workstation had four video windows, one for the local view, and one showing each of the remote sites. We tested the whiteboard capability ahead of time and found that it only displayed a line of text at a time, so we dispensed with it during the trial.

Since we were confronted at our workstations by low quality, tiny video windows and tinny, hard-to-hear audio, one of the professors posed the following question for group discussion: "To what uses can systems like this be applied?" The student participants came up with several suggestions such as home security (watching your home from work), identification at building entrances, and an inexpensive, novel replacement for long distance calls from school to home.

The second trial consisted of four faculty members and about 6 students distributed physically over four sites and four workstations. As with the first trial, each workstation was an audiovisual Macintosh with a videocamera and microphone. During the course of the trial, one site lost their connection to the others and continued the conference through a telephone line. The audio connection during this trial was also poorer than on the first trial. One site eventually used the one line of text displayed at the bottom of their view as their preferred mode of communication.

Of interest from these two trials were some of the dynamics that this form of collaboration afforded. Unlike the MUSH trial, there was little difficulty with turn taking, and no significant moderator role was required. Also, unlike the T.V. studio-based trials, the students did not seem hesitant or shy. We suspect that this is due to the desktop orientation of the video which gives the students a sense of ownership they did not feel in the T.V. studios. Further, the participants could control the cameras themselves, rather than being subjects of a T.V. studio crew. All in all, despite offering the poorest quality video, audio, and text of anything else we tested, CUSeeMe provided some unique and exciting dynamics.

EVALUATION OF THE TRIALS

The evaluation of each trial discussed thus far was based on a combination of faculty and student observations and discussions among ourselves. Although these informal observations derived from the individual sessions provided useful insights into the difficulties involved in a four- way instructional collaboration, we felt that a more quantitative approach to evaluating the sessions might provide us with additional information. To provide this more quantitative assessment of the microwave video and CuSeeMe trials, a set of numeric rating scales were developed. The Questionnaire for User Interaction Satisfaction [14] was used as a model for the current Ques- tionnaire for Using Interactive Teleconferencing (QUIT). The questionnaire is composed of eight sections pertaining to different aspects of video conferencing. The first section allowed the respondent to indicate the type of equipment used for the session. The remaining sections included a number of indicator items that allowed the respondent to rate specific features of the session and/or the system used. The seven sections pertained to the difficulty of setting up the video

CIRCLE project 221

session, the quality and content of the video images, the quality of the audio signal, the interaction of participants, the transmission of data, the appropriateness of the conferencing rooms, and the teleconferencing software.

Microwave video trials

Setting up the equipment was rated in the satisfactory range across all locations due to the number of technicians who took care of this aspect; however, other aspects, such as ironing out technical problems and establishing connections, were rated unsatisfactory at two of the sites. One of the sites that did not report difficulties had little technology available to cause problems. The individuals at the other site reporting no problems were experienced in using this type of technology; their greater experience in encountering and correcting problems of this type may have made them less sensitive to the problems arising in this specific trial.

Video images used during the sessions tended to be rated in the satisfactory to ideal range. The average across all video indicators for sites that both transmitted and received video was near ideal. The only low ratings came from one institution where there was a problem with viewing one's own group. In addition, the ratings for the use of window insets (picture-in-picture) varied with sessions and locations depending on when they were used.

When using multiple sites with multiple individuals, the potential number of audio sources, and problems with those sources, are substantial. The problems in controlling the quality of audio was evident in the ratings; in contrast with the highly-rated video, audio ratings tended to be in the satisfactory range across all sessions and locations. The two major problems were that the audio tended to be too soft across all locations and that the presence of extraneous noises in the rooms led to difficulties in following the discussion.

The conferencing rooms were different at the four locations. At one site, the ratings were satisfactory during the sessions when only the faculty were present but decreased to unsatisfactory for the sessions with the students present. This was primarily due to the problem of not being able to move around easily and the close physical arrangement of participants in the room. At a second site, the ratings did not depend on session. For the third site, ratings were just satisfactory when the session was held in a classroom used for instructional television with small monitors for the remote sites; the ratings increased to ideal when moved to an electronic classroom with a computer workstation at each desk and large 4 x 6 ft screens displaying the remote sites. For the university with no video transmission, the overall room rating was satisfactory. The different ratings are probably due to the differences in room layout at each location. Those individuals at locations which restricted freedom of movement tended to provide lower ratings than individuals at locations where movement was relatively unrestricted.

Ratings of participant interactions tended to be quite satisfactory when only the faculty were present and fell to unsatisfactory in sessions that included students. This reflected, in part, the complexity with greater numbers of participants. It also may result from less familiarity with each other among the students. Ratings for data exchange were unsatisfactory across all locations. Ratings also indicated the need for tools to assist in managing the interaction when groups are present and to improve the procedures for transmitting data.

CuSeeMe session

Overall, the CuSeeMe trials received substantially lower ratings than the microwave trials. Setting up the conference was perceived as being fairly difficult, perhaps because this was an approach that had not been previously tried and there was little local expertise on how to use the software at each site. In contrast, although the microwave links were substantially more complex in many respects, their usage was well established and there was substantial expertise in handling new situations.

Participants felt that the number of connections and the arrangement of the windows was satisfactory, but the quality of the interaction was not. The video images were considered to be unacceptably choppy and the focus was not sharp enough. In addition, in order to maintain a reasonable refresh rate, images had to be kept smaller than was considered acceptable. The audio was also unacceptable. Because of the combined demands of video and audio channels, the audio would often be out of synchrony with the video.

222 MARC M. SEBRECHTS et al.

One of the sites had conducted an earlier experiment with other sites and had reasonable video and audio with a few other sites when using approximately one-third of a T1 line. That capacity requirement, however, was considered too high. In the current study, a 56 kB line was tried. This proved completely unacceptable and the link had to be supplemented with a standard telephone link.

CuSeeMe as we used it did not provide tools that were considered adequate for data exchange. Picture images were useful for establishing contact, but overall the interaction was less successful than in our telephone conference calls.

Usefulness o f the quantitative tool

In general, the QUIT confirmed our informal observations. It did help to make more specific the concerns about the interaction, and to differentiate the issues that were important at each site. In general, we found the tool moderately useful in quantifying our observations. However, because we were using rather different setups at each site, it was difficult to combine data for all sites in any meaningful way. At the same time, the QUIT serves as a preliminary tool that may help provide a common mechanism for evaluating teleconferencing.

CONCLUSIONS

This project demonstrated how" technology currently available at our institutions could be used to facilitate collaborative teaching across institutions. In general, we found that interaction over high bandwidth video channels worked well with little training, although students showed the same inhibitions in interaction that occur in the classroom. Interaction over video conferencing systems such as PictureTel also proved successful but could not at present provide multipoint capability. A number of uses of the Internet were tested including email, FTP, and MUSH, and each of these added to the channels of communication. Although there are many alternative technologies, the experiences from our collaborative project provide potentially useful information for others operating under similar resource constraints, and also point to a number of issues that need to be considered in the development of multipoint collaborative efforts. In particular, the results that we have described help to highlight three important types of constraints: technical, social, and task.

Technical constra&ts

The use of microwave transmission as the primary medium for our trials lead to a number of technical issues. Although this means of transmission works well in normal operation with only a single source, it proved far more cumbersome in the kind of interaction we envisaged. As with any resource, there were significant constraints on availability. That normal constraint became much more severe when four institutions (and three T.V. studios) had to be involved simultaneously. In addition, staffing was required at each location.

The video was less successful in conveying computer screen information, primarily because of poor image quality in using a camera to transmit a computer screen image. It became evident that computer screen display is probably best accomplished by shared screens over the network. Separate programs at each site or projected video from one site do not allow easy reference by pointing or modification of a screen. In addition, we operated with a constraint that the availability of computers and computer links in the T.V. studios was quite limited. Only one institution had a fully electronic classroom equipped with video cameras and a computer workstation for each student. An additional problem was that, at least in our institutions, the telecommunications and television groups have not yet become fully conversant with each other.

The presence of one site that had only audio transmission resulted in some interesting discoveries. The absence of video output had some advantages because it imposed fewer constraints on that group. They could move around freely or converse with each other independently of the other sites. In contrast, those in the broadcast studios were physically constrained by camera position and by microphones that captured and broadcast almost everything that was discussed.

Computer collaborations were limited by the fact that three of the sites had only one computer

CIRCLE project 223

available. This meant that only one individual at each site had access to information presented using that medium. On the other hand, the site having multiple computers had some difficulty in determining who should control the interaction sent out to the other locations. In both of these situations, it was clear that the introduction of the computers had also introduced management overhead into the progress of the trial.

Technical~social constraints

The use of technology also introduced some social issues into the classroom setting. For example, at each site, the television screens showing the participants at other locations were arranged differently. At one site, the screen showing George Washington University was positioned higher than the screens showing the other two sites (due to differences in the height of the carts upon which the televisions sat). This lead to the perception at this location that the George Washington University participants were "in charge" of the trial [15], although the trial had been designed to have multiple leads at different times during the session.

At another site, the monitors showing the other locations were originally arranged such that they were on either side of the faculty member. Although the individual at this site liked having spatial separation of the other sites, he found it awkward to have to constantly turn from one side to the other in order to monitor the person speaking. In addition, he was unable to observe body language cues from one remote site while he was focused on a speaker from another remote site.

The fact that each group was physically separated from the others did lead to the perception that the course consisted of four separate groups. In the limited time allocated to our trials, we were not able to develop a sense of "one group" across the sites. This may have been hampered by the technological limits imposed by the size and location of our monitors. The fact that we used standard-sized monitors and placed them five to ten feet away from us lead to the images of people from the other locations being very small. This made it very difficult to identify who was speaking at any given time. The problem was partially alleviated when the technicians brought up the speakers in a picture within a picture (PIP), but even these pictures were often too small to see facial cues that would have been helpful in interpreting the person's comments. On the final trial, two locations did use large-screen projectors to display the images of the people at other locations, and this reduced the magnitude of the problem significantly.

The video images served in many ways as a form of social introduction; using the common T.V. practice of displaying a speaker's name on the screen when that person was talking helped in this introductory phase. Our collective sense was that the video was especially important in establishing identities. This was highlighted by the fact that the participants had no clear sense of the identify for those people at the site that could not broadcast video. The power of the video component was also shown through the experience of one of the participants. He found the video sufficiently compelling that he planned to get a ride from one of his colleagues to a meeting later in the day; it was only later that he realized that this colleague was only virtually present through the video!

It also became clear that the new media require a certain amount of management. In the case of the MUSH, a moderator was required for turn taking, something that was achieved far more naturally through video and audio channels. Interestingly, it is perhaps the audio channel that has been most important through all the trials, allowing rapid interchange and immediate commentary [16]. As the size of the group expands, issues of turn-taking are more important, and video may provide a means to help moderate this interaction. Some of these issues are merely problems of numbers. As the number of participants increases, the ease with which they can interact is reduced and the need for preplanned and structured turn-taking increases.

Finally, the different activities we tried suggested that the specific tool used influences the role assumed by the participants in their interactions. For example, T.V. studios tended to cause a certain depersonalization which was overcome by the desktop CuSeeMe and control of the cameras. Likewise, when participants shared a single machine at one of the sites, the person using the machine tended to be "in charge" whereas the others tended to be more "remote" from the activity. Individual terminals in the MUSH trials generated more individual involvement, although the lag in participation tended to generate side conversations rather than focus on the central task.

224 MARC M. SEBRECHTS et al.

Task constraints

The trials suggested that the task to be accomplished significantly affected the usefulness of the particular technology. Although the auditory and visual nature of the television image was quite good for conveying lecture-type materials, it was less useful for group discussions that cut across locations and for conveying computer-based materials. Likewise, using the computer to rank alternatives in a group exercise rendered the television medium of less value, as the television was only able to show individuals huddled around a computer monitor and keyboard.

Somewhat surprisingly, the use of synchronous communication turned out to be high cost, in terms of video, audio, and shared computer resources. Although the common interaction was among the most interesting aspects of the project, its added value remains questionable. Activities that were easily conducted among the four faculty became substantially more difficult in a context with 16 students. Establishing true synchrony with the larger group is perhaps one of the key issues. Our observations on this suggest that neither the MUSH model nor FTP provided an adequate sense of synchrony. It appears difficult to achieve the kind of perceived simultaneity of conversation through text interaction. Part of this is due to the overhead of typing and its potential visual confusability. The key here may be that in most instances, synchronous work requires a common space which is primarily visual in nature. Thus, seeing a screen and pointing to it is a task that is collaborative. Substituting textual interaction for discussion seems to be of limited use. We are therefore exploring ways of having shared whiteboards together with audio and video.

In summary, this project helped develop a better understanding of the role of technology in influencing the interaction among faculty and students as a first step in proposing a new model of learning. Although we focused on technologies accessible to our institutions, many similar facilities are widely available, so that the findings should generalize beyond our own institutions.

At the same time, it is important to realize that developments in both hardware and software, as well as greater attention to human-computer interaction issues in design are changing the available tools rapidly. A number of alternative MUSH environments are emerging, as well as multipoint videoconferencing tools. Those changes will influence some of the specific conclusions arrived at here.

Nonetheless, many of the central issues raised are conceptual and not linked to a particular technology. For example, the audio channel seemed to be far more essential than anticipated whereas the relative import of video depended both on the prior familiarity of the participants with one another and on the specific task. In addition, this project suggests some of the general issues that need to be raised in any assessment of electronic collaborative learning, and it proposed some fairly general methods for such evaluation.

Based on initial results, it is evident that fully exploiting this new approach to learning will require a more detailed analysis of emerging computer collaboration tools. In addition, as in most technological innovations, detailed plans will need to be developed for the structure of courses, including assigning tasks across universities and determining how to separate activities into joint and independent activities. This project has provided some of the background on which to build such detailed plans.

Acknowledgements--This work was supported by the Consortium of Universities of the Washington Metropolitan Area. We would also like to acknowledge the support of the television studios and computing centers at The Catholic University of America, George Mason University, The George Washington University, and The University of Maryland in making this project a reality. Finally, we would like to thank all of the workshop participants who gave freely of their time and insights.

R E F E R E N C E S

1. Johnson D. and Johnson R., Learning Together and Alone, 3rd edn. Prentice-Hall. New York (1991). 2. Jones J. I. and Simonson M., Distance Education." A Cost Analysis. Convention of the Association for Educational

Communications and Technology, New Orleans, La (1993). 3. Walsh J. and Reese B., Distance learning's growing reach. T.H.E.J. 22, No. 11, 58 62 (1995). 4. Pea R., Distributed multimedia learning environments: why and how? Interact. Learn. Envir 2, 73-109 (1992). 5. Privateer P. M. and MacCracken C., Odyssey project: a search for new learning solutions. T.H.E J. 20, No. 3, 76-80

(1992).

CIRCLE project 225

6. Sebrechts M. M., Boehm-Davis D. A., Davison S. A. and Ehret B. D., Resources in Human-Computer Interaction. Consortium of Universities of the Washington Metropolitan Area, Washington, D.C. (1993).

7. Savetz K. M., MUDs, MUDs, glorious MUDs. Internet Worm 46-51 (1994). 8. Krauss R. M. and Bricker P. D., Effects of transmission delay and access delay on the efficiency of verbal communication.

J. acoust. Soc. Am. 41,286-292 (1966). 9. Norman K. L., Defining and assessing usability in emerging systems: a case study of the electronic classroom.

Proceedings of Usability Concepts and Procedures: Third Conference on Quality in Documentation, pp. 83 95. The Centre for Professional Writing, The University of Waterloo, (1992).

10. Norman K. L., HyperCourseware for interactive instruction in the electronic classroom. Behav. Res. ~4eth. Instrum. Comput. 26, 255-259 (1994).

11. Norman K. L., Navigating the educational space with HyperCoursewarc. Hypermedia 6, 35-60 (1994). 12. Silverman B. G., Motta C. and Rosenberg M., Computer supported collaborativc learning, computer supported

cooperative work. GWU/IAI Technical Report 94-3, Washington, D.C. (1994). 13. Anon., CUSeeMe: video conferencing over the lnternet. CACM, 3, No. 7, 22 (1994). 14. Chin J. P., Diehl V. A. and Norman K. L., Development of an instrument for measuring user satisfaction of the

human~:omputer interface. Proceedings of CHI '88, pp. 213-218 (1988). 15. Taylor S. E. and Fiske S. T., Salience, attention, and attribution: top of the head phenomena. In Advances in

Experimental Social Psychology (Edited by Berkowitz L.), Vol. 11, pp. 249 288. Academic Press, New York (1978). 16. Krauss R. M., Garlock C. M., Bricker P. D. and McMahon L. E., The role of audible and visible back channel

responses in interpersonal communication. J. Pers. soc. Psychol. 35, 523 529 (1977).