Experimenting with a computer‐mediated collaborative interaction model to support engineering...

Experimenting With aComputer-MediatedCollaborative InteractionModel to SupportEngineering Courses

DAVID A. FULLER, ANDRES F. MORENO

Computer Science Department, Pontificia Universidad Catolica de Chile

Received 14 March 2003; accepted 8 March 2004

ABSTRACT: Many of the engineering education lecture courses are taught only with the

support of a board or transparencies. In both cases, the students have to copy the material

passed in class, including additional annotations and comments. We performed a controlled

experiment to measure the impact of the insertion of a computer mediated collaborative

interaction model to support the teaching/learning process in such scenarios, using a Web-

based computer application. Our experiment was done during two consecutive semesters of a

First Year Programming Engineering course, with 447 enrolled students where 234 students

were surveyed. In this paper, we describe the design and execution of the experiment, and

show the obtained results. Based on our results, we conclude that there are advantages of

using a collaborative interaction model supported by a collaborative software tool in an En-

gineering course such as the experimented.�2004 Wiley Periodicals, Inc. Comput Appl Eng Educ 12:

175�188, 2004; Published online in Wiley InterScience (www.interscience.wiley.com); DOI 10.1002/cae.20012

Keywords: computer-mediated collaborative interaction model; web based learning; CSCL

INTRODUCTION

Many of the engineering education lecture courses are

taught using a traditional method, i.e., a lecturer

explains the lesson while he is writing on the board,

and students copy it onto their notebooks. In some

cases, the lecturer does the innovation of preparing his

or her course material using transparencies. This

traditional method of teaching has several disadvan-

tages that can be avoided using a computer-mediated

collaborative teaching model.

One major disadvantage of traditional teaching

methods is that students usually have to choose

between taking notes and listening to the lecturer,

especially when the lecturer is explaining the material

at the same time [1]. The problem remains if theCorrespondence to D. A. Fuller ([email protected]).

� 2004 Wiley Periodicals Inc.

175

lecturer uses transparencies since the students have to

copy the contents of the lesson. Additionally, students

can make mistakes while copying down the informa-

tion on the board, as they are distracted between

listening and writing. As a result, the time used in

lecture is not optimized, for students rarely can fully

concentrate on the subject matter if they also have to

take notes.

Another important disadvantage is that most of

the learning in traditional teaching is individual, since

the students in the classrooms are taking notes instead

of participating. In courses like Computer Program-

ming, the level of students understanding is greatly

affected by the quality of their notes. If these notes

contain transcription errors, the student is hampered

from properly learning and achieving proficiency in

the subject, wasting time and effort. Furthermore,

most of the information passed in the classrooms is

unidirectional and poor, having slight and occasional

feedback from the students. As a result, students are

not stimulated to participate in the classrooms.

Finally, notes in a blackboard tend to be sketchier

probably because they are going to be erased soon

after. Moreover, class lessons have to be finished in

the period assigned to the class, which makes it dif-

ficult to teach more complex subjects.

Based on the above observations we decided

to experiment in the education area, in order to

‘‘improve’’ the teaching/learning process introducing

a collaborative education model supported by tech-

nology [2,3]. Nevertheless, we did not know exactly

what to improve, or what meant to improve this pro-

cess. For that reason, we decided to do a controlled

experiment, measuring the greater amount of possible

parameters, in order to determine which of them were

hit in the course of our experiment.

After evaluating our possibilities, we decided to

experiment in an engineering course for first year

students from our University. We chose this course

due to the following: (1) students and professors were

motivated to be part of an experiment like this, and

they did not require much training to use the software.

(2) The course is taken by students of all 21 different

engineering majors, and hence, we had different

backgrounds and interests. (3) There are more than

400 students in this class, and therefore, we could do

our experiments with an interesting number of people.

For this purpose, we designed our experiment based

on a collaborative interaction model called SISCO [4],

and we built a Web-based prototype system, which we

called CollaboratiWeb to support the interaction

model. Through this system, we aimed to experiment

and evaluate the benefits of applying a collaborative

model through the Web in teaching a lecture course.

CollaboratiWeb faces the problem of transcrip-

tion and temporality described before by allowing

users, students as well as lecturers, to access a persis-

tent version of the lectures’ content through internet.

This content comprises the course material of every

lecture, as well as comments, exercises, questions, and

answers from the students. Sketchiness is avoided by

organizing this content in a hypermedia structure

where contents are associated semantically, allowing

students to constructs and discover the knowledge, as

they require.

One of the characteristics of CollaboratiWeb is

to allow interaction between class members, sup-

porting the entire teaching/learning process of a

course. Through this platform, we wanted to achieve

‘‘collaborative learning,’’ instead of individual learn-

ing, where students with varying levels of proficiency,

work together to achieve common goals [5]. In order

to reach these goals, the students are responsible not

only for their own learning, but also for that of

the entire group. Therefore, the performance of an

individual within the group affects the group as a

whole [6].

As each individual in the course is different and

has particular requirements, we constructed Collabor-

atiWeb for individual access of class contents. We

identified three roles: the lecturer, the teaching assis-

tants, and the students.

In contrast to the majority of educational sites,

which are static, ours would allow us to have different

views and services to a specific user. Static sites have

the disadvantage of displaying the same information

to all users, in which case a textbook might be more

useful. CollaboratiWeb allowed us to have the flexi-

bility to design and modify the teaching/learning

environment by individualizing each group member,

and allowing us to support each person independently.

Furthermore, we did not want to use the coll-

aborative packages offered in today’s market as

WebCT [7], Blackboard [8], Virtual-U [9], etc.,

because these would have predisposed to a specific

form of teaching and collaboration tools [7,10,11].

During the years, we have gained considerable ex-

perience to develop our own system [12]. Creating our

own collaborative platform, gave us the flexibility to

support a specific collaborative interaction model, and

allowed us to adjust the system rapidly to our own

experiment.

Prior to the experiment, we did not know what

parameters were adequate to measure. Hence, we

proposed ourselves to identify the independent para-

meters that have incidence over students, in an effort

to improve the learning/teaching process. Since this is

a relatively new area of research, it is not possible to

176 FULLER AND MORENO

study the impact of only one variable, making it

necessary to monitor as many effects as possible.

THE SISCO COLLABORATIVEINTERACTION MODEL

We based our experiment and our prototype in a

simple, but general collaborative model SISCO,

developed as a tool to improve meetings productivity

[4]. This model, because of its generality and simpli-

city, allowed us to model the different instances of

collaboration inside a course, including also the group

memory, essential in a course.

This model supports face-to-face and asynchro-

nous remote collaboration. Figure 1 shows the SISCO

model.

For our research, we did a reinterpretation of

the SISCO model. In SISCO, there are participants

collaborating in a remote and asynchronous mode.

Whenever it is necessary, they gather in a face-to-face

meeting, i.e., synchronous collaboration, especially to

carry out a decision. All participants interact with a

knowledge repository corresponding to the group

memory, placing and retrieving information.

For our case, each of the SISCO model phases

were redefined in the context of the teaching/learning

process, as explained below:

* The participants in the collaboration process are

students, teaching assistants, and lecturers.* The group memory represents all the electro-

nically gathered information of a course. It

consists of practical information such as lec-

tures, assistance classes, proposed and solved

exercises, course syllabus, course calendar, tests,

homework, grades, etc. For this reason and

thanks to this reinterpretation, we defined the

group memory as the course memory.* Face-to-face collaboration occurs inside the

classroom between the lecturer and students,

since all of them are interacting in the room.* Outside the classroom, the students interact in an

asynchronous and distributive way with the

course memory, specifically to understand in

detail the contents by studying or making pro-

jects and homework. Lecturers and teaching

assistants also work in an asynchronous and

distributive way, preparing the course contents.

In our model, there are several possible interac-

tions in a course: between students, students and

lecturer, lecturer and teaching assistants, and students

and teaching assistants. The improvement of the in-

teractions between class members is of extreme

importance to us, since they help students to better

understand the subjects. For these reasons, we utilize

different forms of collaboration in the course [13],

helping the students to communicate among class

members in different ways, supporting them in the

learning process.

There are various instances where we require

distinct collaborative interactions, and the system was

flexible enough to allow us to implement and design

all of these forms of collaboration:

* The face-to-face collaborative interaction that

occurs inside the classroom, between the lecturer

and students, was supported by the system. In

this case, the class memory was displayed,

allowing the lecturer to have full mobility

throughout the class. In addition, the infrastruc-

ture in the classroom allowed us also to show the

functionality of the exercises and computer

programs in the classroom, which greatly helped

the students to visualize the results and problems

of the programs. At the same time, a teaching

assistant observed the interaction inside the

class. In various cases, the lecturer, following

the advice of the students, modified the contents

in class to improve the course memory.* The face-to-face collaborative interaction be-

tween the lecturer and the teaching assistant

resulted in a meeting after each class, in which the

course memory was revised. In this meeting,

the course memory was analyzed with respect to

the feedback obtained from the class interaction,

discussions, and questions. This resulted in the

modification of the course content, by improving

Figure 1 SISCO model. [Color figures can be viewed

in the online issue, which is available at www.

interscience.wiley.com.]

EXPERIMENTING WITH A COMPUTER-MEDIATED COLLABORATIVE INTERACTION MODEL 177

different explanations, and the addition of more

exercises that better explained the materials.* Outside the classroom, the students interact in an

asynchronous and distributive way using the

system to study the course memory, which

helped them to see the materials that had been

changed and the materials that they had already

studied.* In addition, we utilized the existing university e-

mail system to allow an asynchronous and

distributive interaction in order to improve the

relationship between students, lecturers, and the

teaching assistants.

EXPERIMENT’S DESIGN

The purpose of the project was to measure the

incidence of collaborative learning using the SISCO

model vs. traditional teaching methods, over two

consecutive semesters. We chose to implement our

model on a University course in Engineering, with the

assistance of a computer-mediated system to support

course collaboration.

For this purpose, we chose a class called First

Year Programming, which starting the semester of our

evaluation had an entirely new syllabus. By having a

new syllabus, we avoided external influences of our

evaluations, for students could not rely on prior years’

materials to prepare for this class. In addition, all

students were in their first year at the University,

which levels the experience the students had with

respect to University courses.

Participants

Five different sections were evaluated for the first

semester, each taught by a different lecturer and

different teaching assistants, but all assigned with

identical tests and projects. This allowed us to use one

section for experimentation, the experimental section,

and four as control groups. Mainly due to lack of

resources, we used only one experimental section.

Unfortunately, the following semester the tests and

projects differed, and therefore, the data gathered on

the second semester came only from the experimental

section, which was used to validate the past results

with the previous experiment’s section.

We did the experiment in both semesters. The

experimental section was chosen randomly. In the first

semester, the control group was taught using tradi-

tional teaching methods, while in the two semesters,

the experimental section was supported by using

CollaboratiWeb as a collaborative learning computer

mediated system.

Students could choose among five possible

sections, registering in the section that was most

convenient to their schedules. If more students than

the ones allowed registered for a particular section, the

University administration gives priority to those of

higher grade point average (GPA). The size of every

section was limited by the capacity of the classrooms

assigned to each one. However, we found out that the

average GPA in some sections was different enough to

be statistically significant. The experimental section

was composed of students who had a higher cumu-

lative GPA than their counterparts in the control

sections. Our analysis discusses this fact in the last

section.

Support Provided

Our purpose was to support the experimental section

in various forms:

* In the classroom with:

@ Computer support to properly display the

class content. In this case, only the lecturer

and/or the teaching assistant were connected

to the system. Students participate and

collaborate with their comments and sug-

gestions in class in a face-to-face way.

@ Computer support to allow mobility of the

lecturer to better help the interaction and

participation of the students.

@ Record class participation by means of

video, audio, and notes to feedback on the

class material.

* Outside the classroom with:

@ The course memory of the course displayed,

revised and enhanced by the class com-

ments.

@ A computer web system to assist students

with their revision and study of the course.

@ Web interaction of the students and the

lecturer via email.

Part of the objective was to change the passive-

ness of the lecturers in class to a much more active

role. Our vision was to change the presenter’s role of

the lecturer to a mediator that would help students to

interact in class and with this new form of interaction

to stimulate the students’ learning. With this new

perspective, we also changed the students’ passive


role of taking notes to a more dynamic position of

interaction, participation, and collaboration, in which

the students would benefit from the lecturers’ knowl-

edge and experience.

We divided the collaborative interactions into two

areas, as in the SISCO model, that we call syn-

chronous and asynchronous collaboration. In the

synchronous collaboration we assisted the lecturer

with the lectures, and in the asynchronous collabora-

tion we support the students outside the classrooms,

with a computer supported system and emails.

Measurements

In order to measure the incidence of collaborative

learning in students in the experimental group, we

designed different instruments focused on the results

and variations resulting from the used teaching

method. Surveys [14] to measure motivation, student

satisfaction, and course comprehension were devel-

oped. These surveys were carried on in the control

group during the first semester and in the experi-

mental group during the two semesters.

We also used ‘‘Question Analysis,’’ an informal

tool to help us to perceive how much information the

students were learning in the class itself, by measuring

the quantity of questions made by students, and the

depth of knowledge evidenced in their questions.

Every question made by students in the classroom was

given a value from 1 to 4 to reflect its complexity. The

value 1 was assigned to trivial questions, and 4 to

complex questions. Every time that a student did a

question in class, a teaching assistant registered the

question for further classification. This was done in

every class of all participating sections during the two

semesters.

Finally, we used grade correlation to properly

compare the grade improvement by the students in

this course, with respect to their grade history.

Although their grade history was usually of only one

semester, the correlation index was almost one, imply-

ing that students most probably were going to have a

grade in a course similar to their GPAs, e.g., a student

with a Bþ GPA, will probably get a Bþ in the course.

CollaboratiWeb

The platform was implemented using a client/server

architecture and Web technology, designed as an

asynchronous distributed collaborative tool, with the

following characteristics:

* It has a page entrance that allows only properly

authorized users to enter the system. The pass-

word is encrypted, using an authentication

system that gives the system great security.* The system has different environments, depend-

ing on the role of the participating user. This

allows us to store the access and navigation

history of each user in the system. The system

also uses this information to produce feedback to

the user, showing him which material has been

already visited and has not been changed.* The program handles a concept of roles, which

allows the program to act differently depending

on the identity of the user, i.e., lecturer, teaching

assistant, or student. For instance, each student

can access the grades individually, but the

lecturer is allowed to visualize the grades of the

entire class.* Through individual access, the system allows

monitoring student interaction with the course

material on the Web, and correlating the data to

grades and achievements.* The system’s navigation scheme is based on

hypertext and multimedia, which allows the

users, lecturer, and teaching assistants to access

the information in a fast and easy manner. This

allows the lecturers and assistants to complete

the material with graphics, sounds and videos, as

shown in Figure 2.* An index at the beginning of the course pages

was needed, in order to help the students find the

required information. This index was carefully

designed to help finding the information faster

and guide the students, assistants, and the

lecturer through the information. We show the

general index in Figure 3.

Because of the considerable size of the course

contents, a need arose to allow each student to keep

track of what information had been studied and what

remained to be studied. For this reason, it was neces-

sary for the program to have a visualization aid that

allows the student a way of keeping track of his or her

progress and that is independent from the navigational

visualization provided by commercial browsers [15].

Note that browsers keep track of navigation

history using a cache type of memory stored in the

computer’s hard drive. For that simple reason,

students cannot rely on the browser’s cache to keep

track of their progress, since they usually access the

system from any of the computers available at the

University. Furthermore, if different students use the

same computer to access the system, a distorted

notion of which material has been reviewed can arise

if the only indication available, of what has been

reviewed, is based on the browser’s cache contents. In


addition, cache memory from the navigation is also

temporary, since it is cleared at specific intervals.

The visualization support implemented in Colla-

boratiWeb is based on the use of graphic semaphores

[15] to give individual support to navigation history.

This can be observed in Figure 3, where large green

dots and smaller gray dots to the left of the hypertext

links are displayed to each user to signal previous

visits. Note that this visualization mechanism has to

signal not only previous visits, but also changes in the

contents since the last visit.

To review course material, students only require

the use of a computer with access to the Internet, and a

browser such as MS Internet Explorer or Netscape

Navigator, thus granting the students great flexibility.

In the classroom, the lecturer projects the lesson using

his interface to the system, which also allows him to

add and modify existing contents. This gave the

students a sense of security to know that the class

content was completely available in the system.

Course Memory

It was essential to create a system that could display

all possible course contents, so that students would

avoid taking notes during classes. This provided

students with virtually unlimited access to the class

contents from the University or from their homes.

During the face-to-face class, we accessed the course

memory and projected the current lesson. Comments,

questions, and any other important knowledge was

captured and included instantly to the course memory.

Professors and TAs had an online editing tool to

access and modify the course memory. During the

class, the students assumed the task of criticizing,

correcting, and completing the course memory by

means of their comments and questions.

Figure 2 Example of a class page.

Figure 3 Navigational index. Figure 4 Lecturer explanation in class.


The first thing needed to construct a course

memory was to introduce the course syllabus’ con-

tents into it. The syllabus was presented as an index

with the subjects to be discussed, as to serve the

students as a learning guide. This index also had the

subjects organized in chronological order, as to give a

guide to students.

We required preparing all the necessary lessons

and contents to form the new course material, which is

the class memory [4]. This approach was very flexible

because the class memory was structured with

hypermedia and the use of Internet. The use of

hypermedia to organize the class memory had various

advantages:

* A dynamic repository of information that can

grow depending on the need of its members.* The hypermedia organization of the informa-

tion allowed us to relate the information of

the course contextually, helping the process of

learning. This transforms the course material into

a course memory in which there is knowledge,

not data.* Because the structure is based on hypermedia,

the users, students, and lecturers, can view the

information in their own order, thus having a

special and unique form of navigating through

the course memory.* The contents of the course memory can be

various: texts, images, emails, and chat.* The use of the Internet and Web technologies

allow us to view the system from different loca-

tions and at different times, performing asyn-

chronous and distributive collaboration.

THE EXPERIMENT’S EXECUTION

The total number of students registered during the

first semester to take First Year Programming was

399, divided among five different sections. The

average number of students per section was 80 and

the experimental section had 64 students.

All the sections met twice a week for a 1.5 h

lecture each time. In the experimental section, each

lecture was taught with the support of a laptop com-

puter connected to the Internet, a computer projector,

a wireless mouse, and a laser pointer. This allowed the

lecturer, during class, to review the class content using

CollaboratiWeb, having mobility inside the class-

room, allowing a better interaction with the students.

In the control group, the other four sections, lecturers

used blackboards. This means that the students in the

control group were all taught through the conventional

teaching method.

The flexibility of CollaboratiWeb allowed us to

constantly improve the course content as would

become evident from the classroom interaction with

students and lecturer.

We always registered in writing the students be-

havior in class, which allowed the lecturer to visualize

needed changes in the course contents. This behavior

was evidenced by the lecturer and the teaching assis-

tant in the classroom, and the changes in the course

contents were made directly in class or afterwards.

These changes and improvements in the course

contents helped students to achieve a more efficient

understanding of the course. During the second

semester, the class memory was improved due to the

previous semester’s acquired experience.

Figure 5 Lecture encourages class participation.

Figure 6 Quantity of exercises.

}

Figure 7 Manner in which the contents were dis-

played.

Figure 8 Use of technological elements.


We usually complemented or introduced infor-

mation to the course content after each lecture,

whenever it was necessary to improve the course

comprehension. This made necessary for the system

to have a special visualization aid, allowing the

students to perceive changes on the class information

displayed on the web. We rapidly realized that the

system was required to have a visualization aid to

inform the users, of any change that occurred in

previous revised materials.

In the two semesters of experimentation, we

utilized various forms of measurements. These are:

* Surveys, in which we measured several things as:

class satisfaction, students interest in collabora-

tion, interest in new teaching methods, class

explanation, and usage of the new system (in the

experimental section only).* Grade analysis helped us to determine the

students’ learning improvement, with and with-

out the new methods of computer assisted

collaboration.* Measurement of class attendance helped us to

quantify the acceptance of the course teaching by

the students.* Class participation, since we recorded student’s

participation and classified later in one of four

categories, according to its complexity, thus

recording the quality of participation occurring

in various sessions of the experimental and

control sections.

Furthermore, during the class period, we also

registered all the questions asked and their respective

answers. This recollection of information later helped

us to improve the course contents.

We evaluated all sections during the first sem-

ester, while for the second semester we only measured

the experimental section. The use of the other sections

as a control group was invalidated, because in the

second semester, the tests and projects were different

for all the sections.

RESULTS

One of the advantages of choosing the course First

Year Programming for experimentation was that there

were several sections of the course, with the same

syllabus, projects and tests required for the course.

There is no class attendance required to pass the

course, and all of the lessons and exercises needed for

the course were stored in the course memory, accessed

via the Web through CollaboratiWeb. Thus, we had the

initial thought that by handing over to the students

the course memory at the beginning of the semester;

there was a chance that they would not attend classes.

However, class attendance in the experimental section

was almost complete throughout the semester, some-

thing that never happened on the control sections.

From the beginning of the experimentation, it was

desired to quantify the changes that were taking place

within the experimental section in comparison to the

Figure 9 Interest in attending class.

Figure 10 Students recommending the course.

Figure 11 Simplicity of navigation.

Figure 12 Main index functionality.


control group. As mentioned before, we registered

and classified the number of questions that students

made in each section. The result was astonishing,

showing the different levels of interaction happening

in both environments. In the experimental section,

there were in average, 41 questions made by students

in each session. In contrast, in the control sections,

there was an average of 8.5 questions. Furthermore,

not only there were five times more questions in the

experimental section, but also the questions made in

the experimental section were of a higher level of

understanding than the ones made in the control

sections.

First Semester

During the first semester of evaluation, the sections

were surveyed twice, at the beginning and at the end

of the semester, to better record the difference in

perception, by the students, of the course and its

methodology. Furthermore, in that semester, the

students’ achievements were recorded, with their be-

havior with respect to class attendance.

In all of the surveys, the students had to answer

several questions with the following available

answers: totally agrees (TA), agrees (A), disagrees

(D), and totally disagrees (TD). The same questions

were asked in the three surveys.

The results of the survey are very significant,

because of the large number of students inquired from

all the sections. The survey was carried out during

class in all the five sections, and 48% of all register-

ed students took the survey, as 191 students were

inquired out of 399 registered in that semester. In

detail, 78% of the experimental section was inquired

(50 students of a total 64 registered students), while

42% of the students in the control sections were

inquired (141 students of a total 335 students in these

sections as a whole). Those percentages also show the

high participation that the experimental section had,

78%, compared to the 42% of the control sections.

In this inquiry, 92% of the students surveyed in

the experimental section consider that the lecturer

explains clearly the contents of the lessons, compared

to 69% of the students in the control sections (see

Fig. 4).

In the experimental section, 80% of the students

estimated that the lecturer encourages class participa-

tion, a percentage that increased in the second

evaluation to a 98%. In the control sections, 64% of

the students estimated that the lecturer encourages

class participation (Fig. 5).

When the students were asked if the quantity of

exercises shown in class was sufficient to understand

the material, in the control sections a 35% were satis-

fied (Fig. 6). In this respect, CollaboratiWeb served to

achieve a 58% of student satisfaction at the beginning

of the semester and it increased to a 69% at the end of

the semester.

At the beginning of the semester, 77% and later

95% of the students in the experimental section agreed

that the manner in which the contents were displayed

was more appropriate (Fig. 7). In the control sections,

only 29% of the students were satisfied with the way

in which the lessons were displayed.

In the experimental section, at the beginning of

the semester 98% and later 93% of the students, at the

end of the semester, agreed that the use of technolo-

gical elements helped them to better understand the

contents of the course (Fig. 8). This shows a high level

of satisfaction by the students with respect to the

use and quantity of technology used throughout

the course. In contrast, in the control sections 91%

of the students agreed that the course lacked tech-

nological elements to support the class.

The next question to the students is ‘‘Are most

of the students interested in attending class?’’ This

question was designed to measure whether the classes

and the used methods are considered useful by the

students. At the beginning of the semester, 82% and

Figure 13 Finding information effortlessly.

Figure 14 Useful solutions to the project.


later in the second survey, 100% of the students in the

experimental section agreed that they were interested

in attending class (Fig. 9). This is a great contrast to

the 29% of students in the control section that agreed

they where interested in attending classes.

The students were asked if they would recom-

mend other students to take the same section the

following semester. In the experimental section, at the

beginning of the experimentation, 96% of the students

and later 98% in the second evaluation agreed that

they would recommend the course to other students.

Only 51% of the students in the control sections con-

sidered recommending their sections (Fig. 10).

Second Semester

The following semester, we made another inquiry to

verify the previous results and to identify the para-

meters that had the most incidences over the processes

of teaching and learning. In the second semester, 43

students out of 48 students were inquired, represent-

ing 90% of the class.

In this second semester of experimentation,

the students were asked if the navigation of the

system was simple, and 95% of the students agreed

that the navigation throughout the system was easy

(Fig. 11).

All of the students (100%) in this second semester

agreed that the main index was very useful. This index

was separated in classes, exercises, assistant classes,

projects, etc. (Fig. 12).

Furthermore, 98% of the students in the second

semester of experimentation agreed that the form in

which the information was classified allows to

effortlessly finding the information (Fig. 13).

Finally, 88% of the students concluded that it was

useful to have the solution to the projects they had

finished, which allowed them to learn from their own

mistakes (Fig. 14). The solutions from the projects

usually consist of the best previous semester projects

made by students, properly fixed by the class assis-

tants, eliminating defects.

RESULTS ANALYSIS

From our experiment, we were able to conclude

several things, leading to the feasibility of improving

the learning and teaching process in an engineering

course through a computer mediated collaborative

interaction model.

The first thing that was obvious was that course

attendance in the experimental section was nearly

100%, while in the control sections, on average, only

50% of the students went to class, which is considered

the norm in this career.

Another significant variation was the comparison

of the questions asked in class, having about 41 ques-

tions per class in the experimental section, compared

with an average of 8.5 questions in the control

sections. The apparent lower level of understanding in

the control sections, reflected by the reduced com-

plexity and number of questions, was evident by a

higher background noise, attributed to students asking

each other questions. This is not very beneficial for the

student comprehension, since while talking to each

other, students miss the explanation given by the lec-

turer to other questions made by students.

From our observations, we can conclude that

students in the experimental section not only were

more motivated to attend class, but also that they

achieved a higher level of knowledge in the course as

evidenced from our analysis. By comparing the amount

of questions asked by students in class, we see that

students in the experimental section were more moti-

vated to achieve a profound knowledge of the course.

In the first semester of experimentation, we had

various positive results with the surveys of the ex-

perimental section and the four control sections.

In this survey, 92% of the students in the experi-

mental section, compared to 69% of the students in

the control sections, consider that the lecturer explains

clearly the contents of the lessons. We believe that the

system used, CollaboratiWeb, and the work done in

the course memory achieved this big difference. This

shows that:

* The students could concentrate specifically in the

contents in class, not wasting time taking notes in

class.* The system was a support for the lecturer to

present the contents of the course more properly,

clearly and efficiently.

At the beginning of the first semester, in the

experimental section, 80% of the students and later

98% in the second evaluation, found that the new

method of teaching encourages class participation.

This increase in the students’ participation was also

reflected in the quantity and complexity of the

questions in the experimental section. On the other

hand, in the control sections, 64% of the students

estimated that the lecturer encourages class participa-

tion. This difference occurred, because in the experi-

mental section, the lecturer was completely dedicated

to explaining the class contents, which did not occur

in the other sections, were the lecturers and the

students had to dedicate class time to writing the class


contents in the blackboard and in the notebooks

respectively.

Barely 35% of the students in the control sections

and only 58% of students in the experimental section

were satisfied with the quantity of exercises shown in

class. This showed at the beginning of the semester

that students needed more quality exercises to practice

and understand the material. For this reason, during

the first semester we produced additional exercises

and published them in the course memory, achieving

at the end of the semester a 69% satisfaction with

respect to the quantity of exercises. This effort was

continued for the second semester, in order to have a

minimum of three examples per lesson in the course

memory.

At the beginning of the first semester, 77% of the

students in the experimental section agreed that the

manner in which the contents were displayed was

more appropriate, but only 29% of the students in the

control sections were satisfied with the way in which

the lessons were displayed. We worked with the

system to improve the presentation and display ability

of information in this semester, and in the second

survey this same semester, 95% of the students ap-

proved the form in which the course memory was

displayed. This is a great accomplishment of a colla-

borative computer mediated system, since most

students are used to the traditional method of teach-

ing, a method learned and acquired by students,

throughout their lives.

In the experimental section, at the beginning of

the semester, 98% of the students thought helpful the

use of technological elements to better understand the

contents of the course. This shows a high level of

satisfaction by the students with respect to the use and

quantity of technology used throughout the course. In

contrast, in the control sections 91% of the students

agreed that the course lacked technological elements

to support the class. It is clear from these results, that

students are not satisfied with the traditional form of

teaching, although the students are very used to the

traditional form of teaching and learning, approxi-

mately 12 years. By this, students in the University

expect the use of technology to support the learning

process. In the second survey in this semester in the

experimental section, 93% of the students agreed that

the technological elements were helpful to understand

the material; we attribute this decrease in satisfaction

by the students to the continuity of the technology

used and that there was no enhancement through the

end of the semester.

One of the most important results that showed the

students’ satisfaction toward the new method of

collaboration was when 82% of the students in the

first semester in the experimental section said they

were interested in attending class. This is in great

contrast to the 29% of students in the control section

that agreed they where interested in attending classes.

At the end of the semester, 100% of the students in the

experimental section agreed that they were interested

in attending class.

This shows that the students in the experimental

section were highly motivated with the course; how-

ever, the students in the control sections were dis-

pleased with the course. This is a serious problem in

most of the classes in Engineering, because students

are not well motivated, not profiting from the knowl-

edge and studies presented by the lecturers. This had

a close relation with the class attendance we also

registered.

The students in the experimental section were

highly motivated, since 96% of the students and later

98% in the second evaluation, in the experimental

section in the first semester, said they would recom-

mend the sections to other friends. A great success if

considered that in the control sections, 51% of the

students would recommend their section, this result in

the control sections shows the students’ indifference

to which section they were in.

In the second semester, we made another inquiry

to verify the previous results and to identify the

parameters that had the most incidences over the pro-

cesses of teaching and learning. In the second sem-

ester, 43 students out of 48 students were inquired,

representing 90% of the class. In this semester, only

the experimental section was studied, because the

other sections had different tests and projects. This did

not permit us to use the other sections as control

group, because different evaluations made the groups

less comparable.

In this second semester of experimentation, we

focused our attention on the group memory. We

wanted to improve the information displayed in

the system, in order to increase the collaboration

and the achievement by the students. In this experi-

mental section, obviously with different students than

the previous semester, 95% of them agreed that the

navigation throughout the system was easy. This is a

great improvement achieved by the system, and the

gathering form of the course memory. This character-

istic is essential for a system supporting students to

learn, facilitating the access to the course material.

Most of the system is based on index forms, but it is

also possible to go from one lesson to the next or from

a lesson to an exercise, utilizing the hypertext features

of the system. This helped the students and users, to

revise the material as they pleased, in a linear or non-

linear form. This explains why students can go from


the index to all the exercises of the course, or they can

go from each class to its own exercises. The main

index of the course memory helped us achieve this

success, and the totality of the students (100%) ac-

cepted that the main index was very useful. The well-

structured index definitely helped the students’

navigation through the course content. This response

helps to develop new collaborative implementations

with this form of index.

Additionally, 98% of the students in the second

semester agreed that the form in which the informa-

tion was classified allows to effortlessly finding the

information. This result revalidates the navigational

structure, the index, and the information gathered.

This is a vital success for the system, because it is not

only easy to use, but also it is simple to find in-

formation. This advantage in the navigation and the

sequence, in which the information is presented,

validates the classification of the course material by

chapters, with the chapters subdivided into classes and

their respective subtitles.

In this second semester, 88% of the students

agreed the usefulness of having the solution to the

projects they had finished, which allowed them to

learn from their own mistakes. This is crucial for the

completion of the course memory, establishing that

included solutions of the work done help them under-

stand the class contents, and learn from their own

mistakes. This procedure allows students to under-

stand the proper solutions and to comprehend their

own mistakes, because in the traditional method they

are not confronted with the right solution.

CONCLUSIONS

The results allowed us to confirm the hypothesis that

our interaction model helps to ‘‘improve’’ the process

of teaching and learning at a high standard Engineer-

ing course. We found that when using our model

supported by CollaboratiWeb, there were several

parameters in the experimental group that were modi-

fied with respect to the control group.

Not only was the learning enhanced, as the data

suggests, but the teaching methods were also changed.

For instance, the process of teaching was changed

drastically from its base, the lectures, by modifying

the entire process of each class. This was achieved by

projecting the lessons on the wall, while the lecturer

walked between the students explaining the contents

and interacting with the entire class. This changed

the lecturer’s traditional role to that of a cognitive

mediator [16], which helped the interaction with the

students. The process of teaching was also modified in

relation to the books and the class notebook, sub-

stituting it with the course memory in the system,

which contains all the material gathered for the

course. Constructing the course memory is a big effort

by the lecturer, but in time, he or she benefits from the

work done, and needing less time to maintain the

course memory in future semesters. Because of this,

the creation and perfection of the course memory

improved the process of teaching and learning in the

second semester.

Viewed from the perspective of the lecturer, the

system has important advantages to store the course

memory from past courses, in order to help them

display the contents of the course in class and for

students’ future revision. These allow new lecturers to

spend less time repeating investigation for the course,

because all of the previous work of preceding courses

is gathered. This is also a good start for a new lecturer

of course, since the old course memory is up to date,

helping to achieve more continuity to the course

through the years, something that usually does not

occur in University courses.

Thanks to the experimentation throughout the

two consecutive semesters, in which we measured the

students outcome through surveys, question analysis,

class attendance and grade improvement, we can con-

clude that a collaborative system can ‘‘improve’’ the

process of teaching and learning within an Engineer-

ing students’ environment. This was achieved because

the students were more satisfied and stimulated

throughout the semester, allowing them to understand

and comprehend the subjects more easily.

Different parameters affected the course: the

class interaction, the grades, and the class assistance.

An improvement on class interaction was discovered

by comparing the number of questions made in the

experimental section with those of the control sec-

tions. The achievements between the new methodol-

ogy and the traditional interaction reflected an

improvement in communication between students

and the lecturer. It is important to remember, that in

previous years, we detected a high correlation be-

tween the students GPA and the grade obtained in the

course by the students. In the control section, students

improved their course grades by 0.7% with respect to

their GPAs, which is almost zero, supporting our

previous study of the substantial correlation between

the GPA and the course grade. This greatly contrasts

with the experimental section where the students

improved their grades by 10.5%. This result was

important to confirm the achievement by the students

in the course, because in the first semester that the

experiment was conducted, the students came into the

class with a better average. For this reason it was


important to validate the results by repeating the

experiment the following semester. Not only was the

hypothesis confirmed during the second semester of

experimentation, but also we came to the conclusion

that the students with lower average grades can have

a better performance in a course using a colla-

borative system such as the presented here. In this

respect, student’s performance improved by 17.0%

the second semester (Fig. 15). Finally, class atten-

dance was also improved with almost a full attendance

by the students. This system helped students under-

stand with a greater depth the lessons, improving their

performance in the course.

It is important to state that, because the system

does not have fixed parameters, the system can also be

used in any collaborative group. However, there are

various restrictions with respect to the implementation

of a system as the one used, such as:

* The participants need to be familiar with the use

of computers and the Internet.* Availability of enough computers with access to

the Internet, and to a server housing the

collaborative system.* A computer in the classroom is needed to display

the material during classes or meetings.* The projection area has to be large enough for the

entire class to see it properly.* The use of a wireless pointing device is sug-

gested to provide the lecturer with sufficient

mobility while he or she is walking in the class.* A laser pointer is important to permit the lecturer

point at the lessons, not having to loose his

mobility inside the classroom.

The use of this system by the students not only

helps them to improve their learning and achieve-

ment by collaborating, but also, they learn the

importance of teamwork preparing them for real

world situations.

The profits from this research have proved to be

diverse, and they begin in the First Year Programming

course:

* The generation of the course memory for the

First Year Programming course, for future

courses in this area.* A great improvement in student satisfaction in

the experimental groups, which leads to a greater

performance by the entire course.* The development of a method to create new

group memories, which helps us understand the

life cycle of the information gathering process.* The creation of a prototype for a collaborative

tool that assists university courses, which con-

sists of a program that supports the recollection

and display of course memory, and different

forms of collaboration.

After this experiment, we learned what para-

meters can be affected in the teaching/learning

process of an engineering course when using a

collaborative interaction model supported by a web-

based system. The experience gathered here will allow

us to experiment with other engineering courses.

REFERENCES

[1] D. Johnson, Student�student interaction: The neg-

lected variable in education, Educ Res 10 (1981),

5�10.

[2] B. Collis and M. Gore, The collaborative model for

instructional software development, Educ Tech (1987),

40�44.

[3] E. Green, P. Cook, and L. Bolt, Fitting new techno-

logies into traditional classrooms: Two case studies in

the design of improved learning facilities, Educ Tech

(1996), 27�38.

[4] M. Borges, J. A. Pino, D. A. Fuller, and A. C. Salgado,

Key issues in the design of an asynchronous system to

support meeting preparation, Decision Support Syst 27

(1999), 271�289.

[5] J. Klein and D. Pridemore, Effects of cooperative

learning and need for affiliation on performance,

time on task, and satisfaction, Educ Tech Res Dev 40

(1992), 39�47.

[6] A. Gokhale, Collaborative learning enhances critical

thinking, J Technol Educ 7 (1995).

[7] S. Siekmann, Which web course management system

is right for me? A Comparison of WebCT 3.1 and

Blackboard 5.0. In CALICO Software Report. http://

calico.org/index.html. 2000.

Figure 15 Grade comparison.


[8] P. Matthew, The wired tower: Perspectives on the

impact of the internet on higher education, Prentice

Hall, New York, 2002.

[9] L. Harasim, Shift happens: Online education as a

new paradigm in learning. The Internet and Higher

Education, Elsevier, New York, 2001.

[10] C. Ho, A. Jannasch-Pennell, and S. DiGangi, Oppor-

tunities and options for web-enabled databases: Com-

paring in choosing the right software for virtual courses

and communities. Int J Educ Technol 1 (1999), 1.

[11] K. Oliver and J. Moore, Gap analysis of educational

web tools in engineering and technology. AERA

2002, American Educational Research Association

annual meeting, New Orleans 2002, 2002, http://

www.aera.net.

[12] D. A. Fuller and L. Guerrero, A pattern system for

the development of collaborative applications, infor-

mation and software technology, Elsevier, New York,

2001.

[13] H. Antillanca and D. Fuller, Refining temporal criteria

to classify collaborative systems, Int J Hum Comput

Stud 50 (1999), 1�40.

[14] C. Loo, Diseno de una Estrategia para el Aprendizaje

Colaborativo Apoyada por Tecnologıa Computacional,

Tesis para Optar al Grado de Magıster en Educacion,

Facultad de Educacion, Pontificia Universidad Cato-

lica de Chile, 1998.

[15] J. C. Lauwers and K. Lantz, Collaboration awareness

in support of collaboration transparency: Requirements

of the next generation of shared window systems,

Proceedings of ACM CHI 90 Conference, April, 1990.

[16] C. Coll, Estructura grupal, interaccion entre alumnos

y aprendizaje escolar, Infancia y Aprendizaje 27/28

(1984), 119�138.

BIOGRAPHIES

David A. Fuller received his BS degree in

electrical engineering from the Pontificia

Universidad Catolica de Chile in 1982, the

MS degree in computer science from the

University of California at Los Angeles in

1984, and the PhD degree in computer

science from Imperial College of Science

and Technology in 1989. Currently he is an

Associate Professor of Computer Science at

the Pontificia Universidad Catolica de Chile, where he directs a

research lab on computer-supported cooperative work (CSCW) and

the Center for Education and Technology. His research interests

include computer-supported collaborative work, artificial intelli-

gence, and technology in education.

Andres F. Moreno received his BS degree

in industrial engineering from the Pontificia

Universidad Catolica de Chile in 1997 and

the MS degree in computer science from the

Pontificia Universidad Catolica de Chile in

1997, and he is currently a student in the

PhD program in computer science from the

Pontificia Universidad Catolica de Chile. He

has worked in the conceptualization and

creation of www.ChileDepot.com, a system that supports computer-

supported cooperative work (CSCW). Currently he is an associate

instructor at the Pontificia Universidad Catolica de Chile. His re-

search interests include collaborative learning, collaborative knowl-

edge building, educational groupware, knowledge negotiation,

system management learning (SML), computer-supported coopera-

tive learning (CSCL), CSCW, and e-learning.


Experimenting with a computer‐mediated collaborative interaction model to support engineering...

Documents

Transcript of Experimenting with a computer‐mediated collaborative interaction model to support engineering...