Using remote experimentation in a large undergraduate course: Initial findings

7
Session S4G 978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD 41 st ASEE/IEEE Frontiers in Education Conference S4G-1 Using Remote Experimentation in a Large Undergraduate Course: Initial Findings M. C. Costa Lobo, Gustavo R. Alves, M. A. Marques, C. Viegas, R. G. Barral, R. J. Couto, F. L. Jacob, C. A. Ramos, G. M. Vilão, D. S. Covita, Joaquim Alves, P. S. Guimarães, Ingvar Gustavsson [email protected], [email protected] Abstract - The use of remote labs in undergraduate courses has been reported in literature several times since the mid 90’s. Nevertheless, very few articles present results about the correspondent learning gains obtained by students, and in what conditions those systems can be more efficient, thus suggesting a lack of data concerning their pedagogical effectiveness. This paper addresses such a gap by presenting some initial findings concerning the use of a remote lab (VISIR), in a large undergraduate course on Physics, with over 550 students enrolled. Index Terms - Remote laboratories, weblabs, learning assessment, engineering education. INTRODUCTION The importance of laboratory experience in engineering education curricula has been emphasized in a large number of science and engineering education articles [1]. Laboratory pedagogy has been recently reported to be a fertile arena of research for the coming years [2], especially in the context of the increasing need to make more use of the new developments in information and communication technology for enhancing the laboratory education. When technology- intensive teaching tools become widely available, the traditional roles of the university lecturers will change from pure classroom-based teaching to one of consultation, advice and direction giving. However, it is believed that the technology-based course will not eliminate the educators; instead it will change the type of activities the educators do. In the technology-based teaching/learning practice, the major activities of the lecturers may include preparation of the software packages, adopting new concepts and new teaching practices, modifying existing materials to suit the changes introduced by the latest version of the multimedia tools, and above all these they can spend time to continuously evaluating the teaching/learning outcomes. The roles of teachers and students are changing, and there are undoubtedly ways of learning not yet discovered. However, the computer and software technology may provide a significant help to identify the problems, to present solutions and life-long learning. It is clear that the computer-based educational technology has reached the point where many major improvements can be made, and significant cost reductions can be achieved, specifically in the area of engineering education. The proper use of techniques and methodologies is critical in any technology intensive teaching/learning development system. In addition, it should be ensured that the designed work keeps up with the curriculum review and update, and the laboratory work should be relevant to the material taught in lectures. One reason for rethinking the role of the laboratory in engineering and science education is the recent shift towards constructivist pedagogy in which the importance of knowledge gained via experience is emphasized. Furthermore, constructivist pedagogy places a larger role on student autonomy in the learning process. This is particularly important in light of the recent increase in the needs of industry for engineering graduates who are autonomous and equipped with good hands-on skills. In their literature review [1], Ma and Nickerson found that 100% of the articles concerning hands-on laboratories considered that labs should be platforms for facilitating conceptual understanding, and 65% considered that laboratories should also facilitate the design skills. However, constructing knowledge is a complex process which is often out of the timeframe of the planned laboratory sessions. Since 2000, a huge number of remote laboratories have been designed, implemented and set up over the world. [1]-[4]. Papers and books about remote labs focusing on their advantages/disadvantages, state of art, technologies, and didactic have been published. Most of these works are focused on the technology and only few articles are focused on the didactic utility of remote labs as a didactic tool. Understanding how different technologies affect the laboratory learning experience is an endeavor which will need the participation of many researchers over many years. In this paper we seek to contribute toward this larger project. We address the role of the laboratory environment while conducting experiments for perceiving some fundamental concepts of electrical circuits, namely the behavior of simple circuits built with resistors, coils and capacitors, when powered with DC and AC sources. We have used a set of tools for assessing the learning gains obtained by students enrolled in a large Physics course of an undergraduate degree on Computing Engineering. Students were divided into two natural groups: one attending both real and remote lab classes and another simply using the remote lab. Preliminary results indicate that both students groups presented learning gains, which suggests the remote lab used to be a good learning tool for developing experimental skills in the taught topic. The rest of this paper is organized as follows: section II describes the assessment scenario, namely

Transcript of Using remote experimentation in a large undergraduate course: Initial findings

Session S4G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S4G-1

Using Remote Experimentation in a Large

Undergraduate Course: Initial Findings

M. C. Costa Lobo, Gustavo R. Alves, M. A. Marques, C. Viegas, R. G. Barral, R. J. Couto, F. L. Jacob, C. A.

Ramos, G. M. Vilão, D. S. Covita, Joaquim Alves, P. S. Guimarães, Ingvar Gustavsson [email protected], [email protected]

Abstract - The use of remote labs in undergraduate

courses has been reported in literature several times

since the mid 90’s. Nevertheless, very few articles present

results about the correspondent learning gains obtained

by students, and in what conditions those systems can be

more efficient, thus suggesting a lack of data concerning

their pedagogical effectiveness. This paper addresses

such a gap by presenting some initial findings concerning

the use of a remote lab (VISIR), in a large undergraduate

course on Physics, with over 550 students enrolled.

Index Terms - Remote laboratories, weblabs, learning

assessment, engineering education.

INTRODUCTION

The importance of laboratory experience in engineering

education curricula has been emphasized in a large number

of science and engineering education articles [1]. Laboratory

pedagogy has been recently reported to be a fertile arena of

research for the coming years [2], especially in the context

of the increasing need to make more use of the new

developments in information and communication technology

for enhancing the laboratory education. When technology-

intensive teaching tools become widely available, the

traditional roles of the university lecturers will change from

pure classroom-based teaching to one of consultation, advice

and direction giving. However, it is believed that the

technology-based course will not eliminate the educators;

instead it will change the type of activities the educators do.

In the technology-based teaching/learning practice, the

major activities of the lecturers may include preparation of

the software packages, adopting new concepts and new

teaching practices, modifying existing materials to suit the

changes introduced by the latest version of the multimedia

tools, and above all these they can spend time to

continuously evaluating the teaching/learning outcomes. The

roles of teachers and students are changing, and there are

undoubtedly ways of learning not yet discovered. However,

the computer and software technology may provide a

significant help to identify the problems, to present solutions

and life-long learning. It is clear that the computer-based

educational technology has reached the point where many

major improvements can be made, and significant cost

reductions can be achieved, specifically in the area of

engineering education. The proper use of techniques and

methodologies is critical in any technology intensive

teaching/learning development system. In addition, it should

be ensured that the designed work keeps up with the

curriculum review and update, and the laboratory work

should be relevant to the material taught in lectures. One

reason for rethinking the role of the laboratory in

engineering and science education is the recent shift towards

constructivist pedagogy in which the importance of

knowledge gained via experience is emphasized.

Furthermore, constructivist pedagogy places a larger role on

student autonomy in the learning process. This is particularly

important in light of the recent increase in the needs of

industry for engineering graduates who are autonomous and

equipped with good hands-on skills.

In their literature review [1], Ma and Nickerson found that

100% of the articles concerning hands-on laboratories

considered that labs should be platforms for facilitating

conceptual understanding, and 65% considered that

laboratories should also facilitate the design skills. However,

constructing knowledge is a complex process which is often

out of the timeframe of the planned laboratory sessions.

Since 2000, a huge number of remote laboratories have been

designed, implemented and set up over the world. [1]-[4].

Papers and books about remote labs focusing on their

advantages/disadvantages, state of art, technologies, and

didactic have been published. Most of these works are

focused on the technology and only few articles are focused

on the didactic utility of remote labs as a didactic tool.

Understanding how different technologies affect the

laboratory learning experience is an endeavor which will

need the participation of many researchers over many years.

In this paper we seek to contribute toward this larger project.

We address the role of the laboratory environment while

conducting experiments for perceiving some fundamental

concepts of electrical circuits, namely the behavior of simple

circuits built with resistors, coils and capacitors, when

powered with DC and AC sources. We have used a set of

tools for assessing the learning gains obtained by students

enrolled in a large Physics course of an undergraduate

degree on Computing Engineering. Students were divided

into two natural groups: one attending both real and remote

lab classes and another simply using the remote lab.

Preliminary results indicate that both students groups

presented learning gains, which suggests the remote lab used

to be a good learning tool for developing experimental skills

in the taught topic. The rest of this paper is organized as

follows: section II describes the assessment scenario, namely

Session S4G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S4G-2

the remote lab environment, i.e. VISIR (Virtual Instrument

Systems in Reality), the course curricula and structure, and

the assessment methodology; section III presents the

assessment results, organized in two categories, namely the

integration of VISIR in the target course and the students’

results; finally, section IV concludes the paper.

SCENARIO AND METHODOLOGY

I. The VISIR system

VISIR is an open remote lab dedicated to experiments on

electrical and electronic circuits, acquired by and installed at

the School of Engineering – Polytechnic of Porto (ISEP),

Portugal, in July 2010. Its architecture and characteristics are

well described in several articles [5]-[6]. In this paper, we

concentrate on describing the actions done by persons using

this system under three different roles (administrator,

teacher, and student), to emphasize its simplicity, minimal

learning curve, and reduced institutional impact, namely on

integration and maintenance aspects.

Figure 1 illustrates the three main components used at

ISEP to store and manage institutional data (Portal),

pedagogic contents (Moodle), and, most recently, to provide

remote experiments (VISIR). When starting a new semester,

teachers have access to structured information on both Portal

and Moodle, i.e. they will have access to the course(s) they

will be lecturing, and all the students enrolled in those

courses will also have access to teaching & learning

materials made available by teachers. Adding VISIR implied

some organizational rearrangement, namely on how to

manage teachers and students access to the remote lab, and

on how to manage the courses created on it. The next

paragraphs will therefore describe the actions done by

administrators, teachers, and students, in VISIR, following a

sequential timeline.

FIGURE 1

SITES USED FOR INSTITUTIONAL AND TEACHING & LEARNING PURPOSES AT

ISEP AND ACTIONS DONE BY ADMINISTRATORS AND TEACHERS IN VISIR

Administrators start by creating a given course. They

then add (i) the responsible for the course (a given teacher),

(ii) other teachers / instructors (typically, courses with

hundreds of students have a team of several teachers), and

finally (iii) the students enrolled on that course. Presently,

this last action simply implies copying & pasting the list of

e-mails of those students enrolled in the same course, into

VISIR, as also depicted in Figure 1. In conclusion, although

VISIR is a fairly new component, it implied a reduced extra

effort into current ISEP administrators’ tasks. This fact is

thought to be crucial to convince institutional managers that

remote labs do not represent an additional cost, in terms of

workload, to the Communications & Informatics Service [7].

Once the course is created, the head-teacher is able to

add new experiments according to the course practical

component. This follows a procedure described in [8], where

the teacher needs to be aware of the components physically

available in the VISIR matrix, or, if not available, indicate

his/her needs to the lab technician, responsible for

configuring and updating the matrix. Assuming that the

components are available in the matrix and that their images

are also available in the list of components depicted by the

VISIR interface, in teacher mode, the teacher may then add a

new experiment to its course and then add a direct link to it,

into the corresponding Moodle course page. Again, these

actions are illustrated in Figure 1.

Students access VISIR either through the link provided

in the Moodle course page, where in each case they will be

required to enter their login credentials and after that will be

directed to the related experiment or, through the VISIR

URL. In this last situation, students will also have to indicate

their login credentials and then will be directed to a page that

contains a navigation menu on the left side, where students

can see the courses they are enrolled in. Selecting a

particular course will open a page displaying the list of

experiments currently available on that course. Students may

then select one experiment, ending up on the same page they

would get if following the direct link provided in Moodle. In

both situations, VISIR will track the students’ access by

incrementing a counter indicating the number of sessions

done by each student. Additionally, the VISIR log system

tracks the number of experiments done, i.e. the number of

times the button “Perform experiment” is clicked, and hence

and experiment is done in the remote lab. This log system

does not, however, record who has done which experiment,

thus not allowing tracking the experiments (and their setup)

done by each individual user (either students or teachers).

II. Environment description

ISEP is an engineering school offering a large number of

bachelor and Master Degrees and Post-graduate courses in

various engineering areas, ranging from Electronics,

Instrumentation and Computing to Chemistry, Mechanics,

and Metrology. In our study, the choice of an adequate case

study environment relied on the assumption that students

from Computing Engineering degree would have a natural

tendency to use the kind of resources a remote lab

Session S4G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S4G-3

represents. Also, Electricity is a topic covered in Applied

Physics which is a 2nd

year course, where 2nd

year students

are normally more aware of their academic responsibilities.

Like all engineering degrees at ISEP, this is also a 3-year

degree, where 1st year students acquire mathematical,

programming and lab work skills. Fundamental science

subjects, like Physics, is only taught in the 1st semester of the

2nd

year, being Applied Physics course the sole opportunity

students have contact with physics at university in this this

degree. Our study was developed over this semester, with

561 students enrolled in the course. Applied Physics was

organized in 12 weeks, from which 6 weeks were dedicated

to electricity - from September, 27th to November, 1

st - and

the other 6 weeks to waves and heat transfer. Since students

perform lab work, their assessment is divided in two parts:

(i) lab work assessment with an overall weight of 40% and

(ii) final examination with an overall weight of 60%.

Students were divided in two groups according to their

enrollment requisites. Group A, accounting for 288 students,

is the group attending hands-on lab work classes. In those

same weekly classes, group B, with 273 students, analyses

results taken with VISIR in the previous weekly class,

comparing them with a comparative theoretical circuit

analysis. This happened during weeks 2 and 4, as depicted in

TABLE I. This group already has a lab work mark from

previous years. The number of students in each classroom of

both groups was quite different. The fifteen classrooms from

group A had a maximum of 24 students each, organized in

working groups of 3-4 students. The four group B

classrooms had almost the double number of students.

TABLE I

LABORATORY CLASSES PLAN FOR TOPIC ON ELECTRICITY.

Weeks Organization of topic electricity for lab classes

Group A Group B

Week 1

27/9

Introduction to VISIR. Taking measurements in the remote

system for a DC experiment described in the lab guide.

Week 2

4/10

Real DC circuit mounting and performing measurements.

24h to hand-over the group lab

report.

Comparative DC analysis with values from previous

class and components

nominal values.

Week 3

11/10

Introduction to VISIR. Taking measurements in the remote

system for an AC experiment described in the lab guide.

Week 4

18/10

Real AC circuit mounting and performing measurements. 24h

to hand-over the group lab

report.

Comparative AC analysis with values from previous

class and components

nominal values.

Week 5

25/10

Individual lab report related to one of the two experiments

done.

Teachers use VISIR to

demonstrate the previous

AC circuit as a passive filter.

Week 6

1/11 1st partial examination related to this topic. (a)

(a) For group A the 1st partial examination is done in the problem solving classes.

As shown in TABLE I, in the first week, all students

were introduced to electricity in theoretical classes, where

concepts like resistors associations and Ohm’s and

Kirchhoff’s laws for DC circuits were taught. At the same

time, and also for all students (both groups), VISIR was

introduced by teachers in the first laboratory class, starting

with a small presentation of the VISIR interface and

explanation about its usage. After this introduction, teachers

asked their students to try for themselves, in order to

accomplish the designed circuit of the laboratory task. Every

student was provided with a lab guide describing the

experimental procedure for a DC circuit, which apart from

being implemented, they had to take measurements from. As

stated before, in the week after only group A had hands-on

lab classes to mount the same circuit. During the following

two weeks, this procedure was repeated for an AC circuit,

with the help of the corresponding lab guide. 24 hrs after

each real-circuit implementation class, every group had to

hand-over a lab report where they had to include hands-on

circuit measurements and VISIR measurements. During

week 5, every student, individually, had to write an extended

report on one of the experiments done with the

measurements previously taken, being applied only to group

A. In week 6, every student performed a partial assessment.

III. Assessment Methodology

The analysis is based on a case study investigation [9], since

it is grounded on an empirical study of a real discipline, its

problems characterization and potentials encountered during

the integration of the remote laboratory (VISIR) in an

Introductory Physics course. The two natural students’

groups created (A and B, already explained in the previous

section), allowed us to asses VISIR effectiveness in different

types of students.

These collected data not only regards students’

knowledge and competency gain in this subject, but also

students’ usage of VISIR and students and teachers

perceptions along the course.

VISIR Usage: Teachers and students usage of the

system was registered and allowed us to analyze the total

amount of sessions run by each student and the global long

term (during the 6 weeks) frequency of usage.

Teachers Perceptions: All teachers involved in this

course were interviewed and shared their experience about

their real practice with VISIR. Some of them, who had past

experience lecturing this course, compared some of the

characteristics.

Knowledge and Competence Assessment: The

knowledge and competence questionnaire was constructed

by the course head-teacher together with other

teachers/researchers and meant to assess some of the most

significant issues of the taught subject. Some of the

questions were addressed to directly infer the common

knowledge of the students (questions 1 and 2); other

questions meant to infer some laboratory competences

development (questions 3 and 4); we identified a last set of

questions (questions 5, 7 and 8) which was more specific to

the characterization of DC and AC circuits. The other two

questions (6, 9) were procedural, about parallel and series

components in a circuit. All teachers delivered those

questionnaires in class, on paper, before and after the

lectured subject. In order to allow us to compare results for

each student, these questionnaires were not anonymous. An

Session S4G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S4G-4

explanation of its goals (simply for research purposes) was

made by every teacher. Even though the questions were the

same in the pre test and post test, their order was changed to

avoid memorization.

Students Perceptions: All students were subject to a

second questionnaire (based on some literature with similar

purpose [10]-[12]) which meant to assess students’

satisfaction and their perception of learning outcomes using

VISIR. This questionnaire was done anonymously,

accessing a Google page at the end of the course. In addition

to this large inquiry, some students of each teacher’s class

were interviewed in order to reach a more personal view of

the class implementation of VISIR and to better understand

students’ autonomous usage of the system. These students

were chosen because of their higher number of run sessions

at VISIR and a more significant gain on the competency test,

or exactly the opposite.

Course Results: These results included the

laboratory reports assessment, frequency of the course and

exams’ classification.

ASSESSMENT RESULTS

The results were organized into two categories: i)

implementation of the system in a large course; ii) students’

learning results and perceptions while using this system,

both focusing on the differences encountered in the two

analyzed groups of students.

I. On VISIR Course Integration

VISIR Usage: Using the information available in the

VISIR log system, we first analyzed the total number of

accesses (i), the number of activated accounts (ii), the

number of activated accounts with zero accesses (iii), and

the number of daily accesses (iv) of both students and

teachers. This initial analysis was already presented in [13].

In this paper we focus on the analysis of students’ accesses,

from groups A and B, according to their daily classes,

throughout the six-weeks period of the module on electricity.

Therefore, we present the distribution of daily classes, per

week, for groups A and B, in Table II. Figure 2 presents the

charts with the daily number of accesses, of students’ groups

A and B (figure top and bottom, respectively), versus the

number of daily classes. A careful analysis of the depicted

charts, considering the laboratory classes plan for the topic

on Electricity (Table I), allows to extract the following

conclusions:

TABLE II

CLASSES DISTRIBUTION FOR STUDENTS’ GROUPS A & B

Hour/day Mon. Tue. Wed. Thu. Fri. Sat.

10:00 11:00 A A A B A A B

11:00 12:00

13:00 14:00 A

14:00 15:00 A

A

15:00 16:00 B B

16:00 17:00 A A

A A

17:00 18:00

21:30 22:30 A A

A

22:30 23:20

FIGURE 2

ACCESSES VS. DAILY CLASSES FOR GROUP A STUDENTS (TOP) AND FOR

GROUP B STUDENTS (BOTTOM)

Group A students have used VISIR following what was required from them, either in week terms (see top of figure 2) – access patterns in weeks 1 and 3 following the pattern of daily classes and access patterns in weeks 2 and 4 denoting the 24-hrs period to hand-over the group lab report. As using VISIR was not compulsory (but just complementary) in week 5, the number of accesses from group A students is clearly residual. In week 6, where the use of VISIR was not even complementary, the number of accesses drops to almost zero.

Group B students denote a more autonomous usage, evidenced by the lack of an access pattern close to the pattern of daily classes. Furthermore, the number of accesses in week 6 (in opposition to the number of accesses from group A students, in week 6) is also in line with this conclusion. Basically, one may say that these students take advantage of the complementary nature of remote labs.

Teachers Perceptions: All teachers were

interviewed and shared their class experience and their

perception about students learning concerning VISIR usage.

This analysis is presented in Table III.

Session S4G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S4G-5

All teachers agreed that students of group A showed, in

general, more laboratory competences in assembling the

circuits’ components and indeed a much lower percentage of

burning multimeter fuses was found (nearly half of the one

encountered in previous years). As to students’ knowledge

of the subjects, teachers agree that even though this is more

difficult to infer, it is unlikely that a greater benefit has been

produced. However, some teachers referred that higher gains

can be achieved in students who closely follow class

developments or who have already mastered part of it

(especially group B), once they better realize the potential of

the system in order to help them in their difficulties.

TABLE III

TEACHERS’ PERCEPTION ANALYZES.

VISIR Usage Group A Group B

students’

visible interest

students brought their

computers (1 or 2 per group)

fewer computers than it

was desirable (0 or 1 per group)

work in class

teacher shows how it is done (more detailed in 1st class)

students work depends on

teachers explanations; more autonomous work in 2nd

class

students little interact in 1st

class and in the following classes work with

teachers’ help

autonomous

work outside

class

some students worked beyond the expectations,

bringing printouts of the

VISIR circuits to help them in the hands-on lab

some students use VISIR in order to successively

improve their learning

laboratory

competences

development

circuits assembling

improvement; better

performance of data acquisition; greater security

while handling circuits

management

greater security while

handling circuits

management

epistemic

development

not evident a few students showed

some improvement

Greater

value

Training possibility allows students to gain confidence

Possibility of working lab competences outside class Students’ motivation

Future

suggestions

Give a meaningful feedback, in order for students to

improve their work and teachers to be able to use it to reach out their difficulties

VISIR has limited possibilities, if possible this should be

improved (or at least give information of these kind of error as a “non error” to students)

More experiments (different situations) in order to do the

linkage to theoretical models

II. On Students’ Results

Knowledge and Competence Assessment: Even

though these tests were delivered in class, allowing everyone

to participate, we only got 178 results from group A and 49

from group B, which represents nearly 41% of the total

amount of students. The students’ results were treated

according to Marx & Cummings gain lines [14], which

means positive gain lines of 20%, 40%, 60% and 80%,

successively, in the upper left triangle and similar negative

gain lines in the lower right triangle. Group A (Figure 3)

shows a large percentage of students with positive gains –

the majority has gains between 0-40%. Group B (Figure 4)

presents similar results, slightly improved: the majority has a

gain between 20% and 40%.

FIGURE 3 GAIN RESULTS IN THE TOTAL QUESTIONNAIRE, FOR GROUP A STUDENTS

FIGURE 4

GAIN RESULTS IN THE TOTAL QUESTIONNAIRE, FOR GROUP B STUDENTS

As explained, questions addressed different objectives. In

order to be able to recognize and more clearly identify those

gains, we analyzed these data separately, referring only to

questions 3 & 4, which addressed directly the development

of laboratory competences, respectively for both students

groups, A and B. The general gains in these two questions

are the greatest observed in all sets of questions delivered.

Once again it is perceived a similar distribution, with Group

B showing fewer problems in handling those questions

before and after working with VISIR, not even showing

cases of 0% in the post test.

Students Perceptions: A group of students who were

identified as enthusiastic users were invited to an interview.

Students largely reported doing access from home, outside

of school, and described the access sessions to last about 15

to 30 min. Noteworthy is the fact that most of these students

Session S4G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S4G-6

say they have experienced the VISIR beyond what was

required in the laboratory classes and have also used VISIR

to check elements of the lecture. Regarding the perceived

advantages for using VISIR, the interviewed students

expressed the view that VISIR helped to better understand

concepts, enhance training and clarity of procedures,

increasing motivation. Some students suggest the existence

of tutorials to support the use of VISIR. They all recognized

VISIR as a good instrument to improve their laboratory

skills. Some of them compared with their previous

experience at the course and stated that it had improved and

that this year they understood the subject better due to it. At

the end of the course, students were asked to complete a

questionnaire about their perception of the utility VISIR had

to their learning and motivation. This was not mandatory,

yet 185 students choose to share their opinion, which

represents a sample of 33 %. As can be seen in TABLE IV,

students agree that VISIR was most helpful in developing

laboratory competences. Even though they did not feel it

helped with their motivation, a great percentage would like

to extend its usage to other subjects, showing their

palatability towards the system. These results also indicate

that the system has not shown many irregularities, but

students felt that a formal tutorial could have been helpful,

as some of them stressed out in the interviews.

TABLE IV

STUDENTS’ PERCEPTION QUESTIONNAIRE RESULTS

Questions (using a Likert scale: 1 - minimum and 5

- maximum

Results

Average Standard

deviation

Q1

Did VISIR help you on:

a) concepts and problem solving?

b) handling the equipment? c) Mounting the circuits?

3.40

3.57 3.84

1.07

1.22 1.20

Q2 Did VISIR motivate you to the learning

objectives? 2.77 1.26

Q3 a) Provided usage information was sufficient?

b) Was VISIR operating in stable conditions?

2.61

3.66

1.10

1.26

Q4 Would you like to have similar systems for

other course subjects? 3.34 1.44

Course Results: Although it was not possible to

compare directly between laboratory evaluation (concerning

VISIR implementation) with previous years, in general, the

teachers noticed a slight increase in the reports’

classification, based in a more complete data collection (this

analysis was made only for students in group A, since the

group B maintained the frequency of laboratory grade).

Although this can not be directly attributed to VISIR usage,

it shows that students were not harmed in their knowledge

construction, by using VISIR.

The exams’ results, after analyzing the issues directly related

to the subject worked with VISIR, strongly enhance the

differences between the two groups of students. Figures 5

and 6 show the grades obtained in the two exam questions

related to this theme (P1 - DC, P2 - AC, and Additional - an

optional question which allowed students to improve their

thematic assessment evaluation). The final grade distribution

is also presented, for information purposes.

Group A

0%

20%

40%

60%

80%

0 10 20 30 40 50 60 70 80 90 100

G ra de (0-100)

Stu

de

nts

' P

erc

en

tag

e F inal Grade

Additional

P 1 (D C )

P 2 (AC )

FIGURE 5 GRADES DISTRIBUTION IN EXAM QUESTIONS, FOR GROUP A STUDENTS

Group B

0%

20%

40%

60%

80%

0 10 20 30 40 50 60 70 80 90 100

G ra de (0-100)

Stu

de

nts

' P

erc

en

tag

e F inal G rade

Additional

P 1 (DC )

P 2 (AC )

FIGURE 6

GRADES DISTRIBUTION IN EXAM QUESTIONS, FOR GROUP B STUDENTS

Although the overall results are very poor - which is usual

for this course - group B students show a relatively higher

ratings distribution. Comparing the results of the students

who used VISIR with those who did not (e.g. for the P1

exam question, as shown in Figure 7 and 8), it appears that

not only students who used VISIR can achieve better overall

results, but also that group B students have a slight

supremacy.

0%

10%

20%

30%

40%

0 20 40 60 80 100

Stu

den

ts' P

erc

en

tag

e

Grade (0-100)

Group A

Students who used VISIR

Students with no VISIR access

FIGURE 7

GRADES DISTRIBUTION ON P1 EXAM QUESTION, FOR GROUP A STUDENTS

Session S4G

978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD

41st ASEE/IEEE Frontiers in Education Conference

S4G-7

0%

10%

20%

30%

40%

0 20 40 60 80 100

Stu

den

ts' P

erc

en

tag

e

Grade (0-100)

Group B

Students who used VISIR

Students with no VISIR access

FIGURE 8

GRADES DISTRIBUTION ON P1 EXAM QUESTION, FOR GROUP B STUDENTS

CONCLUSIONS

Currently there is a consensus in the remote lab literature

that a combination of remote and hands-on laboratories is

useful for students' learning be better sustained [1]. In our

study, we found that students, who already attended the

hands-on lab classes, could easily accompany the course

using VISIR. In general, comparing VISIR usage and

students’ results, students of group B (who already attended

lab classes) seems to be able to make more use of the

system, despite their initial reluctance (seen for instance in

the number of computers brought to class) and they show a

more continuous usage of the system throughout the 6

weeks, which can mean they use it in order to study for their

class assessment on this material. In fact all our results point

to similar or better results in these students.

Our findings are also aligned with literature concerning

the motivation and enthusiasm raised on students [3]. Since

the same tool was available to all students, we were able to

compare the way it leveraged students to use it. This

approach enabled to perceive a more active role dependent

on how students felt its usefulness, which is readily achieved

in group B students, who already understand their

difficulties more easily and are able to potentiate their

learning with VISIR.

REFERENCES

[1] Ma, J & Nickerson, J. V. Hands-On, Simulated, and Remote

Laboratories: A Comparative Literature Review. ACM Computing

Surveys, vol. 38, no. 3, article 7. 2006.

[2] Feisel, L. D.,Rosa, A. J. The role of the laboratory in undergraduate engineering education. J.Eng. Education, 121–130. 2005.

[3] Lindsay, E. D., & Good, M. C. Effects of laboratory access modes upon learning outcomes. .IEEE transactions on education, vol. 48, no. 4, pp.619-631. 2005.

[4] Nickerson, J V; Corter, J E; Esche, S. K. & Chassapis, C. “A model for evaluating the effectiveness of remote engineering laboratories and simulations in education”. Computers & Education, Volume 49, no.3, pp. 708-725. 2007.

[5] I Gustavsson, J. Zackrisson, K. Nilsson, J. Garcia-Zubia, L. Håkansson, I. Claesson, and T. Lagö, “A Flexible Electronics Laboratory with Local and Remote Workbenches in a Grid,” International Journal of Online Engineering (iJOE), vol. 4, n.º 2, pp. 12-16, 2008.

[6] I. Gustavsson et al., “On Objectives of Instructional Laboratories, Individual Assessment, and Use of Collaborative Remote Laboratories,” IEEE Trans. on Learning Tech., vol. 2, pp. 263-274, Oct.-Dec. 2009.

[7] N. Hine et al., “Institutional Factors Governing the Deployment of Remote Experiments: Lessons from the REXNET Project,” 4th International Conference on Remote Engineering and Virtual Instrumentation (REV'07), Porto, Portugal, June 2007.

[8] I. Gustavsson, “Teacher manual for VISIR,” https://openlabs.bth.se/electronics/index.php?page=AboutPage#, accessed 8th December, 2010.

[9] Cohen, L., Manion, L., & Morrison, K. Research Methods in Education (6th ed.). London and New York: Routdlege Falmer, 2007

[10] I. Gustavsson et al., “On objectives of Instructional Laboratories, Individual Assessment, and Use Collaborative Remote Laboratories”, IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, vol.2, nº4, October-December 2009, pp.263-274.

[11] J. Garcia-Zubia, Hernández, I. Angulo, P. Orduña and J. Irurzun, “Acceptance, Usability and Usefulness of WebLab-Deusto from the Students Point of View”, Acceptance, Usability and Usefulness of WebLab-Deusto from Students Point of View, 2009.

[12] A. Hyder, S. K. Choi and D. Schaefer, “Remotely Controlled Laboratory Experiments: Creation and Examples”, Proceedings of 2010 IEEE Systems and Information Engineering Design Symposium, University of Virginia, Charlottesville, VA, USA, April 23, 2010.

[13] Gustavo R. Alves et al., “Using remote experimentation in a large undergraduate course: initial findings”, IEEE Engineering Education Conference (EDUCON), pp., Amman, Jordania, April 2011.

[14] J. D. Marx and K. Cummings, “Normalized Change”, American Journal of Physics, 75(1), pp.87-91, 2007.

AUTHOR INFORMATION

M. C. Costa Lobo, Psychologist, ISEP - School of Engineering - Polytechnic of Porto, [email protected] Gustavo R. Alves, Adjunct Professor, ISEP - School of Engineering - Polytechnic of Porto, [email protected] M. A. Marques, Adjunct Professor, ISEP - School of Engineering - Polytechnic of Porto, [email protected] C. Viegas, Adjunct Professor, ISEP - School of Engineering - Polytechnic of Porto, [email protected] R. G. Barral, Adjunct Professor, ISEP - School of Engineering - Polytechnic of Porto, [email protected] R. J. Couto, School of Engineering – ISEP - School of Engineering - Polytechnic of Porto, [email protected] F. L. Jacob, ISEP - School of Engineering - Polytechnic of Porto, [email protected] C. A. Ramos, Adjunct Professor, ISEP - School of Engineering - Polytechnic of Porto, [email protected] G. Vilão, Junior Professor, ISEP - School of Engineering - Polytechnic of Porto, [email protected] D. S. Covita, Invited Professor, ISEP - School of Engineering - Polytechnic of Porto, [email protected] Joaquim Alves, Adjunct Professor, ISEP - School of Engineering - Polytechnic of Porto, [email protected] P. S. Guimarães, Junior Professor, ISEP - School of Engineering - Polytechnic of Porto, [email protected] Ingvar Gustavsson, Senior Professor, Blekinge Institute of Technology, [email protected]