Post on 06-Mar-2023
Session F3C
978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD
41st ASEE/IEEE Frontiers in Education Conference
F3C-1
A Video-Message Evaluation Tool Integrated in the
UNED E-learning Platform
Rafael Pastor, Roberto Hernández, Salvador Ros, Antonio Robles-Gómez, Agustín Caminero, Manuel Castro, and
Rocael Hernández Spanish University for Distance Education: rpastor@scc.uned.es, roberto@scc.uned.es, sros@scc.uned.es,
arobles@scc.uned.es, accamineros@scc.uned.es, mcastro@ieec.uned.es
Galileo University: roc@galileo.edu
Abstract – Use of multimedia services has become very
important in learning process. This is due to the
traditional approach of document based assignments not
provide any feature which allows to evaluate some basic
competences (oral communications skill, for example).
In this paper, the UNED (Spanish University for
Distance Education) solution will be presented focusing
on user’s interaction: lecturers and students. In the first
case, the lecturer has to define a task and propose a
solution (available only at the end of task availability).
This solution is also video/audio based, so the student
can learn about it and compare it with his/her solution.
The student has several attempts, in order to try better
responses and decides which of these attempts will be
selected as final response. The whole interaction schema
will be presented and how is integrated in the e-learning
platform as a usual task (like document based tasks).
This integration permits the use of all features of the
platform evaluation tool, simplifying the evaluation
process and tasks grading (no special procedures are
required).
Index Terms – Video assignments, e-learning, evaluation
process, service integration.
INTRODUCCTION
The oral and communications competences and skills are
features not so much supported in the lecturer planning of
subjects and its evaluation. This is due to, normally, e-
learning platforms have several tools to include recordings
of video/audio but not integrated with assignment or tasks.
This issue makes difficult the inclusion of multimedia based
activities, basically because the relation between tools and
evaluation is not clear and also, non ICT management
procedures must be used in order to add the grading to
evaluation tools of platforms (some much extra work for
lecturers, depending of student number).
Nowadays, Internet access is an usual facility that
students has incorporated in their learning process, basically
by means of virtual learning environments, social networks
(Face Book, Twitter, Linked it, etc.) , multimedia services
(like YouTube) or video conferences systems. So it is
expected that e-learning platforms will be capable not only
of incorporate these technologies in a basic way (link based)
but adding it as evaluation tools transparently to lecturers
and students.
UNED [1] has its own conferencing software: AVIP
(Audio-Video over Internet Protocol) [2]. This tool was
developed to permit a wide range of online video-
conferencing techniques to be combined with smart-board
equipped classes and digital repositories to provide
interactive taught classes to a large number of students
(those actually in the class, those connecting online, and
those who access recordings of the sessions).
UNED CONFERENCING TOOL
As Internet access has improved, it has become easier to
access audio and video recordings online While this is an
effective way to expand our knowledge, albeit somewhat
“passively”, it does nothing for the development of
production and interaction competences and skills, and the
communicative functions involved (defence, argumentation,
persuasion, reasoning, justification, etc.) in real-time and
real-world scenarios. In order to fulfill these specifications,
the AVIP tool provides several services available to UNED
users (lecturers, tutors, students, etc.):
AVIP Classrooms. Physical classroom with
incorporated videoconferencing (MCU based and IP
based) systems and digital boards. These classrooms
provide full service (board, chat, video, audio,
document presentation, desktop sharing, etc.) for tutors,
connecting several AVIP classrooms in order to get
“virtual classroom” between students connected over
Internet or located in the same/another AVIP
Classroom.
Campus Broadcast. Full service for real time
broadcasting of UNED events (seminars, tutors activity,
etc.), including the storage facilities and Internet access
(IP classrooms).
E-learning Platform Integration Service. The UNED
e-learning platform aLF/.LRN [3], [4], provides a full
integration with AVIP. This integration allows to
courses administrator managing of “virtual classes”
based on AVIP classrooms (access control for users,
date, time assigned, password protected, etc.). The
interface for a virtual classroom is shown in Figure 1.
Session F3C
978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD
41st ASEE/IEEE Frontiers in Education Conference
F3C-2
The basic features of AVIP Classrooms (IP based) are
similar to other commercial tools like Elluminate [5], and
Adobe Connect [6] or open source as Kaltura [7] or
BigBlueButton [8]: real time video/audio, whiteboard, chat,
desktop sharing, etc. These features are useful in order to
provide virtual classes or virtual training [9], replacing the
traditional concept of class. But really, these tools only
provides a type of activity to be used as collaborative
systems in real time, with lecturers/administrators doing
teleconferences or moderating the audio/video access.
FIGURE 1
AVIP CLASROOM
Really, the uses of these tools are focusing in video/audio
recording of sessions but not other transparent integration is
provided (as it is shown in this paper). For example,
Elluminate’s Moodle integration provides the concept of
live sessions [10], and also it is possible to get live/recorded
sessions, integrated as course content objects inside Moodle.
But there is no a straight way to include this content objects
as response for assignments, only using hyperlinks as
responses. Of course, the lecturers must define a “forever
live session” in order to let student record responses and
“remember” the hyperlink to include as a response for
assignment. This procedure have several problems
associated: shared use of session (students cant not use the
same live session at the same time), complex managed
procedure for lecturers (defining session, instruct students
about tool use and adding response, and moderating and
coordination of session use) and technological competence
of students (not all students have skills to manage
hyperlinks in a simple way).
In the case of Adobe Connect [11], Kaltura and
BigBlueButton, the e-learning platforms integration
modules are similar (activity/session based) and they
provides simple integration with platforms as Moodle, Sakai
and Blackboard (and many others). Kaltura have
development plans to add video extensions: add/edit video,
assignment type integration at course level and addition of
grading in video assignment uploads, but for now, it is only
an announce [12].
No one of tools mentioned before allows a full integration
with other specific tool like evaluation/grading of video
based assignment, very useful to get asynchronous video
recordings associated to course activities. The basic
difference between the last tools and AVIP is the
evaluation/grading tool integration, so for aLF users
becomes transparent the using of video responses for
assignments (in the same way as usual documents
responses) and students/lecturers have not to manage
copy&paste operations of recoding URLs in order to
integrate video/audio recordings. It is very important not to
force to lecturers to learn new tools outside of virtual
courses, so the impact of using could be negative. In this
case, the integration is transparent and the evaluation
process uses the same mechanics that traditional
assignment, so the learning curve is very short.
AVIP VIRTUAL CLASSROOM.
Based on AVIP and to face the problem mention before,
new features were added (AVIP-PRO) to enable video
recordings to be made online without having to be in a
moderated AVIP session (external tool) and is integrated in
the UNED e-Learning platform. It has been designed as a
didactic tool for multiple functions, namely the training of
specialized oral discourse (of particular relevance in
disciplines like Law and Business), real-time oral evaluation
(fairly common in face-to-face institutions with a reasonable
student-teacher ratio) and training and evaluating second
language oral competences online. Language learning tasks
can be prepared where students undertake any kind of online
oral communication and training, combining the flexibility
of the AVIP framework together with the task-based
structure of aLF. Institutionally, the tool is scalable and can
be managed from within aLF as a standard task type
enabling large-scale training and evaluation to be
undertaken. In next sections, basic properties of task-based
structure will be presented focusing in the video task
features, showing the lecturers/students point of view and
how responses are managed in the same way as it is done
with “normal” responses (document/hyperlink based).
DEFINING VIDEO TASKS FEATURES
The assignments tool of aLF/.LRN allows delivering some
common formats (document based) but in this case, a
specific type of assignment has been included to support
multimedia integration in deliverables. An assignment in
aLF has several general properties, but the more important
are:
Task Name. It defines the task name that will appear in
the assignment dashboard and the grading dashboard.
How will the task be answered?. Type of format
supported by assignment. There are four types of
formats: Submit online (document based), Associate
with Forums (to grade forums participation in courses),
No submission (to support not deliverables tasks) and
Multimedia recording (AVIP-Pro).
Session F3C
978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD
41st ASEE/IEEE Frontiers in Education Conference
F3C-3
Time assigned. Only applicable for AVIP-PRO tasks.
It defines the maximum recording time for
lecturers/students solutions
Description. HTML formatted text to describe the
objectives of assignment. It can be complemented with
one attach (document) to include other relevant
information.
Start and due date. Availability times for assignment
Perfect score. Maximum grade for assignment.
Attempts number. Maximum number of recordings
for lecturers/students.
The defining process of a video assignment is the same as
the rest of supported formats, so, in this way, is fully
integrated with the assignment/grading (evaluation) tool in a
transparent way. In the case of a multimedia task, the two
more important properties that must be defined are Time
assigned and Attempts number. These two parameters
must be clearly set before start date of assignment, so the
students have a clear knowledgement of time required and
attempts available. In fact, these parameters cannot be
changed after task start date.
Once the assignment is created, it will be available in the
assignment dashboard (portlet of assignment tool). In Figure
2, part of an assignment dashboard it is presented. These
assignments are included in a subject of a master degree,
developed by the Communications and Control Systems
Department of UNED. In this case, the video task is
oriented to be complementary to the final practice in the
subject (Web standards design). The main objective is to do
a brief presentation (ten minutes) in which the students have
to explain clearly: development methods, standards used,
usability issues and professional usage (if it is applicable).
As it is shown in Figure 2, a camera icon indicates the video
assignment, in order to clearly distinguishing it of the rest of
assignments. For every assignment, in the corresponding
dashboard, it s possible to do some basic operations:
(View assignment properties), (Edit assignment
properties) and (Delete assignment). Finally, a global
option called Upload solution can be used to add lecturer’s
solutions to tasks. In case of deliverable document based
tasks, a solution document can be included to provide
lecturer feedback to students.
RECORDING TASK SOLUTIONS: LECTURER POINT OF
VIEW
In order to provide an assignment solution, lecturers have
the option of recording a video scene using their own
camera (usually, it is incorporated to the same
laptop/computer used by lecturer). To do this, there is an
option in the assignments dashboard called
that will show the recording dashboard for lecturers, as
shown in Figure 3.
FIGURE 2
TASKS LIST, VIDEO TASK SELECTED.
FIGURE 3
SOLUTIONS DASHBOARD WITH RECORDINGS
The solutions dashboard shows all the recordings have been
made by lecturers. There are two main actions associated, in
order to work with the recordings dashboard:
. This button allows starting a solution’s
recording, presenting the record window (see Figure
4).
. When the solution board has one or
more recording sessions, it is possible for lecturers to
select the assignment solution. This solution will be
visible for students at the end of task availability (due
date), so students can compare their solutions with the
lecturer proposal.
For all recordings, there are two sub-actions to manage each
recording: View recording in the recorded sessions player
( icon) and delete recording ( icon).
The recording window (see Figure 4) has all the information
needed to work with the solution and it has two panels:
A) Information panel (left and bottom space of recording
window). It provides the indications about video
recording (time assigned and attempts available), task
name and description (HTML based and document
attached to the task)
B) Recording panel (upper right space of recording
windows). It provides the video/audio feedback of
recording and options available (recording modes and
recording publish options).
VIDEO TASK RECORDING WINDOW.
From this window, the lecturer can record a solution simply,
using the record button located in the recording panel, see
Figure 5 (a). Once the record button is clicked, a new
window, see Figure 5 (b), it is shown in order to write the
name of solution (name used in the recording dashboards)
and also to select the type of recording (video/audio, only
audio). See Table 1 for icons meaning. Once the name of
Session F3C
978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD
41st ASEE/IEEE Frontiers in Education Conference
F3C-4
solution is selected, the lecturer only has to select the type
clicking the corresponding icon and the recording begins. In
order to stop the recording, the record button (a square while
it is recording) must be clicked and the solution recording is
added to the lecturer’s recordings dashboard.
FIGURE 4
RECORD WINDOW WITH STUDY MATERIAL
(a) (b)
FIGURE 5
SOLUTIONS DASHBOARD WITH NO RECORDINGS
In Table I, another icon has been included in order to
manage the public settings associated with recordings.
These recordings can be produced using AVIP-PRO or
AVIP conferencing tool integrated in aLF. In Figure 6, a full
list of recordings (some omitted in order to clarify options)
is presented (when the lecturer clicks the third icon
presented in Table I). This list contains all the recordings
(not only the recordings associated with video task) in order
to manage them. There are several options: public
publishing (to all the users of UNED space), edit recording
or delete recording, and provides a full dashboard to manage
all the recording information provide for every user
(lecturers, students, tutor, etc.) of AVIP system. TABLE I
RECORDING MODES AND OPTIONS
Icon Mode/Option
Provides a full recording supporting audio and video
modes defined in the preferences/options panel
Provides only audio recording
Hiperlink to recordings sessions management site
Finally, the lecturer can change basic parameters or
recording using the tools panel located in the recording
panel (left of image panel):
. Switch audio off/on.
. Freeze/Resume video (only in full recording
mode).
. Video/Audio options. They are very useful to get
a good user experience because it is possible to change
size of image, frames per seconds and image quality.
Also it is possible to modify the band width to adapt the
net connection to the current internet profile. In Figure
7 it is shown the options panel.
FIGURE 6
AVIP RECORDINGS DASHBOARDS
FIGURE 7
CONFIGURABLE RECORDING SETTINGS.
When the lecturer has finished a recording, he/she can
preview the video/audio file using the integrated player
provided (see Figure 8). For every recording, at the start of
previewing, basic information is presented to lecturer:
assignment associated, course in which the assignment is
defined, author, name of recording and start date. This
information is very relevant for lecturers in order to grade
students, as it will be seen later.
Finally, the lecturer only has to select the correct solution in
the recordings dashboards and click the Assign Solution
button.
RECORDING TASK SOLUTIONS: STUDENT POINT OF VIEW
The student actions to provide answers (solutions) to the
video assignment are very similar to the lecturer’s process.
In fact, the recording window and actions are the same, so
the previous section describes all the features which a
student has in his/her recording sessions.
Session F3C
978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD
41st ASEE/IEEE Frontiers in Education Conference
F3C-5
FIGURE 8
VIDEO/AUDIO SOLUTION.
In the case of a student, the assignment dashboard is lightly
different that lecturer’s one but it is very similar. All the
administrative actions are not available for a student
(logically) but the Submit answer option is located exactly
in the same place that the “Upload solution” option provided
for a lecturer. In Figure 9, the video assignment of the
master degree’s subject is shown.
FIGURE 9
ASSIGMENTS DASHBOARD FOR A STUDENT
The student has only to click the submit answer option to
access to his/her recordings dashboard (very similar to
lecturer’s one). In this case, the options available for student
are Record (the same features which are provide to a
lecturer) and “Assign record to task” to select the
corresponding recording as student solution to the
assignment. In Figure 10 (a), a selected recording is
assigned as answer. Also, once an answer is selected, a
student can change the answer election or view current
answer, as it is presented in Figure 10 (b). These actions are
only possible before the due date.
EVALUATION OF STUDENTS
The final stage that a lecturer will have to do it is the
grading of answer’s assignment. The grading process is the
same as the rest of types of tasks, so the same tool is used to
evaluate the performance of students. The answer is located
in the Submission column in Figure 11, and clicking the
View submission shows the recordings player with the
student’s video (similar to Figure 8, depending of recording
type: video/audio or only audio). Once the students answer
is played, lecturers must assign a grade to the submission
and the process of evaluation will finish.
(a)
(b)
FIGURE 10
MANAGE OF ANSWERS OF A VIDEO ASSIGMENT
FIGURE 11
EVALUATION OF STUDENTS ANSWERS FOR A VIDEO ASSIGMENT.
Another role which participates in the evaluation process is
tutor. The tutor participation [13] is very important in
UNED degree/master degree subjects because the number of
students (some subjects with near of 14000 students). Tutors
can evaluate to her/his students assigned (usually, by
geographical aggregations) using the same tool used by
lecturers. The only differences between roles corresponds to
the number of students in the different list/tabs in the
grading tool (graded, not yet graded, not yet submitted) due
to, usually, a tutor has assigned a student’s group of 50
students (maximum).
CONCLUSIONS
Web meetings tools provide useful features in order to have
virtual training or virtual classes. But the LMS integration
modules available, like in Elluminate, Connect, Kaltura or
BigBlueButton, only have features about manage sessions
and do recordings by lecturers and students. This
management procedure is complex for lecturers, especially
in courses with a big number of students (UNED has a
medium number of students per subject about 2000), forcing
lecturers to do extra work in order to coordinate the
evaluation/grading of task. Also, from the student point of
view, they are force to manage recording URLs, in order to
get their responses as video messages, and including it in
the grading tool of LMS. This can be a big problem with a
big number of students, like UNED.
To solve this issues, AVIP PRO provides a simple full
integration with evaluation/grading tool of UNED learning
platform (not only for recording sessions in available
rooms), allowing lectures manage video assignment in the
same way as document based assignment. The task
properties are the same and only two new properties are
applicable to these types of assignments (recording
Session F3C
978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD
41st ASEE/IEEE Frontiers in Education Conference
F3C-6
maximum time, attempts available). This transparent
integration facilitates the use of this tool, avoiding the
problems mentioned before.
In the paper, the grading/evaluation process has been
presented to demonstrate: 1) Task management process is
the same as document based task: defining of task and
delivering of responses by students; 2) There is no negative
impact on lecturer traditional works. This is due to the
evaluation process is not modified and only the specific
recording interface details will have to be learned.
Using multimedia tools provide a new way of
evaluation/grading in order to have a good approach to the
face to face exams (oral presentations, very useful on
language learning). Some competences which are not very
easy to take in account in document’s assignment can be
evaluated using AVIP PRO. In this case, oral competences
are applied in the context of final projects in a master degree
subject, so it can be evaluated competences like project
understanding, synthesis capacity or oral presentation skill.
ACKNOWLEDGMENT
The authors wish to thank the Spanish Ministry of Science
and Innovation and the National Plan R&D TIN2008-
06083-C01/TSI and TIN2008-06083-C03/TSI “s-Labs:
Integration of open services for remote and distributed
virtual laboratories, reusable and safe”.
Also authors would like to acknowledge the support of the
Project 142788-2008-BG-LEONARDO-LMP mPSS –
mobile Performance Support for Vocational Education and
Training Project and IPLECS Project – Internet-based
Performance-centered Learning Environment for Curricula
Support Project ERASMUS 141944-LLP-2008-1-ES-
ERASMUS-ECDSP.
Finally, the authors want to acknowledge the support
provided by e-Madrid Project, S2009/TIC-1650,
“Investigación y Desarrollo de tecnologías para el e-learning
en la Comunidad de Madrid”.
REFERENCES
[1] National Distance Learning University of Spain,
http://www.uned.es
[2] C. Rodrigo, J. L. Delgado, J. Vega, "Using Interactive
Videoconference to Promote Active Learning in a
Blended Learning Environment," ICALT, pp.658-662,
2010 10th IEEE International Conference on Advanced
Learning Technologies, 2010.
[3] R. Pastor, R. Hernández, S. Ros, T. Read, M. Castro,
and Rocael Hernández, "An enterprise e-learning
solution: The UNED practical case in the EHEA".
EDUCON 2010, IEEE Engineering Education 2010 –
The Future of Global Learning in Engineering
Education, Madrid, Spain, 2010
[4] R. Pastor, T. Read, S. Ros, R. Hernández and R.
Hernández, “Virtual communities adapted to the EHEA
in an enterprise distance e-learing based environment”.
HCI-2009 International Conference on Human
Computer Interaction, San Diego, California, USA,
2009.
[5] Elluminate, http://www.elluminate.com/
[6] Adobe Connect, http://www.adobe.com/products/adobeconnect.html
[7] Kaltura, http://www.kaltura.org/
[8] BigBlueButton, http://www.bigbluebutton.org/
[9] B. Fallery, S. Gerbaix and R. Ologeanu, 'Videotraining:
A comparison between “virtual class” and “remote
class”'. CTS 2008, International Symposium on
Collaborative Technologies and Systems, Irvine,
Canada, 2008.
[10] Ellumnite Integration Module:
http://www.elluminate.com/Products/Elluminate_Learn
ing_Suite/Elluminate_Bridges/Moodle_/?id=178
[11] Adobe Documentation about Connect Integration in
LMS:
http://www.adobe.com/education/elearning/pdfs/connec
t_lms_intgrtn_sb_ue.pdf
[12] Kaltura development plan announce:
http://www.remote-learner.net/kaltura_story
[13] R. Pastor, R. Hernández, S. Ros, T. Read, M. Castro,
and Rocael Hernández, "A Complex Tutoring System
for e-Learning: The New Evaluation Model", FIE 2010,
Frontiers in Education Annual Conference, Washintong
D.C., USA, 2010
AUTHOR INFORMATION
Rafael Pastor, senior lecturer at the Spanish University for
Distance Education. IEEE Member, rpastor@scc.uned.es
Roberto Hernández, senior lecturer at the Spanish
University for Distance Education. IEEE Senior Member.
roberto@scc.uned.es
Salvador Ros, senior lecturer at the Spanish University for
Distance Education. IEEE Senior Member.
sros@scc.uned.es
Antonio Robles-Gómez, Assistant Lecturer at the Spanish
University for Distance Education. arobles@scc.uned.es.
Agustín Caminero, Assistant Lecturer at the Spanish
University for Distance Education,
accamineros@scc.uned.es.
Manuel Castro, Professor at the Spanish University for
Distance Education, IEEE Senior Member.
mcastro@ieec.uned.es.
Rocael Hernández, General Manager GES, Galileo
University, roc@galileo.edu.