1

The quality-interactivity relationship in distance education

Draft of:

Trentin G. (2000). The Quality-Interactivity Relationship in Distance Education,

Educational Technology, 40(1), 17-27.

This paper seeks to discuss the complex weave of elements that can help define the term

“quality” in distance education. The basic assumption will be that quality is not

synonymous with excellence but rather indicates the management of a continuous process

aimed at bridging the gap between the expected effect (e.g. what ought to be learnt) and the

actual effect (what has been learnt).

Achieving such a result demands frequent interaction between all the components in the

process. Interaction is imperative if the quality of the process is to be raised: interaction

with materials, between students and their tutors/teachers, as well as between all the

participants.

Introduction

Dealing with the issue of quality in distance education is undoubtedly an exacting task.

The reasons are many, the prime one being that an agreement has still to be reached on

what is generally meant by quality when referring to an education process and all that

this involves (educational effectiveness, social and professional impact, return on

investment, etc).

Although this topic has been addressed a number of times before, and from different

perspectives, the answers have almost always proved to be a sort of "short blanket",

meaning that they have managed to cover just some of the aspects that make the

problem so intricate. Generally, the aspects addressed have been those of greatest

relevance to the specific context in which the education process was designed and

conducted. For example let’s consider the differences between company training and

teachers' in-service training, especially the different way of interpreting the cost/benefit

ratio and the subsequent return on investment - a parameter which is inextricably linked

to the quality level the education process should reach.

The problem is therefore to be tackled in a systemic way, centring on the integration of

two complementary levels: that of the elements that contribute to define a distance

education effort (learning considerations, logistic and technological aspects, expected

benefits and so on), and that of the specific features characterising the contexts in which

education efforts are based.

Admittedly, the great number of elements at play means that there is no codified recipe

that will permit a univocal specification of quality factors in distance education.

2

Acknowledging this fact, this paper seeks to discuss the complex weave of elements that

can help define the term “quality”. The basic assumption will be that quality is not

synonymous with excellence but rather indicates the management of a continuous

process aimed at bridging the gap between the expected effect (e.g. what ought to be

learnt) and the actual effect (what has been learnt).

In this sense the article does not seek to define a model/framework that may be adopted

for analysing the quality of a distance education process, but rather offers the

opportunity for careful consideration of the main elements effecting quality.

Brief examination of the three generations of distance education

systems

To clarify the discussion that follows, it is worth recalling some of the key concepts

related to the various types of distance education systems, with special reference to so-

called second and third generation systems (Nipper, 1989)1.

Second generation systems (or multimedia systems) sprang to light in the fifties and are

based on widespread distribution of instruction through various media: printed matter,

television programmes, audio or video tapes, educational software, etc.

Personal interaction is limited to one-to-one exchange between teachers/tutors and the

students, while communication among participants is only sporadic. This is hardly

surprising seeing that these systems were created mainly to solve problems linked to the

coverage of wide geographical areas and/or large user communities, hurdles that can

only be overcome through effective methods for mass distribution of learning material.

In this context, learning can only be achieved as an individual process, based on the

participant's interaction with educational material.

More recently, the increasingly widespread use of network technologies has made it

possible to define new models for distance education that can add value to the social

component of the learning process and provide strong interaction opportunities for all

participants in the educational effort (students, tutors, teachers and experts). Known as

third generation distance education systems, these models are also referred to as online

education. They permit the creation of a fully-fledged learning community where

individuals can overcome their isolation and draw benefits from interacting with the

group. The method in question has yielded extremely positive results in adult education

(Briano, Midoro & Trentin, 1997) (Trentin, 1997a), a context where the sharing of

personal experience in the subject area may indeed play a significant role in collective

betterment.

The difference, then, between traditional (first and second generation) distance

education and online (third generation) education lies in their different ways of

organising the learning process: the former is an extensive process addressing a large

community, while the latter is an intensive process targeting fewer people but involving

strong interaction between them.

This means that where the goal is to reach a mass usership, the second generation

approach remains the most effective. If, however, the quality of the process is to be

raised, there is often a need to increase participant interaction, so third generation

models are more desirable.

3

Quality in distance education

Today, it is widely accepted (Barchechath, 1996) that successful analysis of distance

education systems must take account of four key dimensions: pedagogy, educational

engineering, economics and technology. Pedagogical approaches involve a series of

tools, procedures and sequences, the combination of which is of special interest to

educational engineers, who aim to develop distance education systems that are

increasingly responsive to individual learning needs.

Drawing on studies performed by the sociologist Parsons (1979), Barchechath (1996)

states that any teaching service should allow the beneficiaries (or, if you will, the end

user) to co-produce the service together with the supplier. This means that, where

quality in an education effort is concerned, end users must be encouraged as much as

possible to take an active part in setting the objectives and defining the contents, as well

as in capitalising on their professional experience so that the learning process may be

enriched.

The logical consequence would therefore seem to be that of hinging the learning process

on strong interaction between all the participants involved. This is a point made by

Herman (1995), who, when dealing with the subject of quality in training, highlights the

importance of involving all the potential beneficiaries of the process, be they direct (end

users) or indirect (their workplace, society, etc).

All this would appear to suggest a strict link between quality and the capability to

manage a learning process based on the active participation of all its beneficiaries. In

the words of Branson and Buckner (1995), quality (is) not so much a question of

‘excellence’ as a process intended to bring the actual effect as close as possible to the

expected effect.

Quality as process management

Keegan (1990) suggests analysing distance education processes using methods similar

to those applied in industry. These have been drawn from control systems theory and

offer a coherent set of strategies for reaching a prefixed objective. In this sense, it is

possible to identify two types of distance education system: open-ring and closed-ring

systems. Open-ring systems (see figure 1) are based on evaluation of the correctness of

the system’s input and of the soundness of the system’s overall structure.

result obtainedprocessexpectations

Figure 1. Open-ring system.

This definition covers first and second generation systems, where the main focus is on

the learning material to be studied individually (input) and the logistic support offered

by the course provider (system structure).

However, the benchmark for distance education that Keegan provides is represented by

closed-ring systems (figure 2), i.e. those that have a capacity for timely self-regulation

while in progress, thanks to the measurement of what is expected of the learning process

and what is actually obtained. The key to this sort of systematic realignment is constant

4

monitoring (represented in Figure 2 by the measurement block) aimed at formative

evaluation of both the users and the whole process.

process

measurement

result obtainedexpectations

Figure 2. Closed-ring system.

Third generation systems fully encompass this suggestion. Indeed, they are based on

intense interaction among all the components of the process, giving the tutors and, more

generally, the training provider the opportunity for in-progress monitoring of the

process. In this way they can perform any necessary adjustments to bring the course

more closely in line with the stated objectives, in other words with the expected effect.

By the same token, it must be recognised that the latest applications of second

generation distance education do feature some retroaction in a sort of closed-ring

system, albeit to a somewhat limited degree. Consider for example multimedia

courseware with a high level of interactivity: the program responds to the user’s actions

by reinforcing the given contents or by dealing with them at a higher level of

complexity. Another example is where participants in a distance course are provided

with assistance delivered via computer network. Nevertheless, the retroaction featured

in closed-ring third generation systems leads to far greater quality and effectiveness.

So we might well draw an initial conclusion in saying that the strong interaction among

system components that is a typical feature of third generation distance education in

particular does indeed create the retroaction ring essential for matching the actual effect

with the expected effect. At the end of the day, this in effect means enhancing quality in

education.

Quality and interaction within the process

Moore (1989) proposes a quality analysis model based on three types of distance

interaction 2:

interaction between participant and learning material;

interaction between participant and tutors/experts;

interaction among participants.

Interaction between participants and learning material – the quality of the learning

material has an enormous effect on the overall quality of an online course, so the

material offered to students should not only deal with topic contents in a well structured

way but also be pleasant to use.

In addition, the course kit provided to participants should contain a guide explaining the

program of course modules. This serves to introduce participants to the course, helps

5

them interpret the assignments set at various stages, and assists in associating the

learning material to the modules comprising the course. This guide should also contain a

substantial bibliography to support the search for further material, for project

development, and so on.

Interaction between participants and their tutors – this is perhaps the most dynamic

and crucial aspect of the online course. The tutor should be seen as an advisor, leader,

facilitator, and sometimes as an expert in the topic contents. There are three

fundamental moments in participant/tutor interaction:

the initial approach, aimed at breaking the ice between the students and those

responsible for leading and assisting them throughout the course. This preliminary

“didactic conversation” usually takes the form of a “learning contract” where the

programme in discussed, the contents, objectives and methods are brought into

sharper focus, and the students’ expectations are clarified.

The outcome of this process is that the participants’ sense of isolation is reduced,

and this helps considerable in steering the process towards what was earlier defined

as the desired effect, both for the course managers and for students.

the act of distance tutoring, which involves a much wider spectrum of functions

than face-to-face tutoring does. In a way, the tutor’s presence online must be made

apparent, as indeed must that of the participants. When using Information and

Communication Technology (ICT) and strategies typical of distance interaction,

tutors need to made manifest both their presence and their readiness to help the

students; this can be done by answering queries promptly, making suggestions and

offering support throughout the entire learning process.

Where the technology is available, tutor assistance may be reinforced by periodic

“face-to-face” videoconferences. While these are not nearly as effective as actual

face-to-face meetings, they nevertheless offer a useful surrogate for the social

presence of the two interlocutors (Short, Williams & Christie, 1976; Trentin &

Benigno, 1997);

reflection about the process underway is a need that many participants demonstrate,

especially adult learners. In other words, they feel the need to understand not only

the contents being studied, but also the reason behind certain methodological

choices in the management of the course.

This “meta-communicative” space, which in courses based on computer

conferencing takes the form of a permanent conference for tutors and participants,

has the undoubted advantage of bridging the gap between course providers and

recipients.

Interaction among participants – the previous section highlighted the importance of

tutor/participant interaction in the form of two-way communication, which has a strong

impact on the quality of online courses. Further and perhaps even greater impulse comes

from the possibility to integrate “vertical“ tutor/participant communication with a more

“horizontal” form of interaction: that between peers, i.e. the course participants.

6

This is in fact the raison d’être of third generation distance education, although it must

be pointed out that a more restricted and sporadic form of inter-participant

communication was in fact introduced in second generation courses.

Horizontal communication is wide in scope. At one end of the scale there is simple free

interaction, introduced largely to strengthen social ties between participants: this may

cover topics like hobbies, gossip, professional problems and so on. Then there are

discussion groups set up to examine some of the issues touched upon in the course. At

the other extreme we have fully-fledged computer conferencing systems structured to

suit the specific aims of the course.

One of the factors that has an impact on peer interaction, and therefore on the quality of

the course as a whole, is group size. In a nutshell, the more the communication is

directed towards socialisation and sharing of ideas and experiences, the larger the

discussion group may be. Conversely, the more the communication is directed towards

collaborative study, the more limited group numbers need to be. In either case, the

tutors must be capable of guaranteeing that the whole mechanism for participant

interaction runs smoothly.

In this light, distance education systems, especially those based on ICT (or, rather, those

that seek to make the most of the potential that networking offers for interactivity)

should not be restricted to the kind of individualised instruction typical of

correspondence courses. Instead, they should be used for managing highly interactive

groups, where greater emphasis is placed on the social aspect of learning to the benefit

of the entire learning process (Barker, Frisbie & Patrick, 1993).

It is nevertheless worth making a point about the interactivity typical of third generation

systems. One of the key ingredients for raising the quality of an online course is strong

interaction between the players in the process; organised in fully fledged virtual classes,

the participants must obviously respect schedules and deadlines if a collaborative

working strategy is to be successful. However, strong interaction may in some cases

actually hinder flexibility in course participation. Those participants who for personal or

professional reasons are unable to interpose themselves in the collaborative learning

mechanism find it very difficult to exploit the full potential of interpersonal

communication; in these cases their communication ends up being restricted to

interaction with the tutor or course provider, mainly for the purposes of seeking

personal support.

The relationship between technology, interactivity and quality

While it is true that Information and ICT has the potential to raise the quality level of

distance education processes (Kirkwood, 1998), clearly it is not the only factor to

consider when seeking to improve quality. There are many others, beginning with the

clear definition of learning goals and subsequently of the learning activities to be

undertaken in order to reach those objectives. Only when this has been carried out can

we move on to choosing the most suitable medium, i.e. the one that will contribute

towards the achievement of a good quality level in the distance education process

(Trentin, 1997b). That said, we should not lose sight of the fact that the choice often

depends more on economic than on methodological considerations. What’s more, in

many cases course providers opt not for a single medium but a suite of media, each of

which offers a different level of interactivity.

7

In order to analyse the technology/interactivity/quality relationship, we first need to

clarify some of the aspects that characterise the direct relationships between (a)

technology and interactivity, and (b) interactivity and quality.

The technology-interactivity relationship – technological tools can be divided into two

categories: those that does not foresee any kind of interactivity at all with the user; and

those that, at least potentially, guarantee it to some degree or other.

Tools in the former category provide what is known as one-way communication. Digital

technology is mainly used here as a means for wide-scale dissemination of course

material to participants (TV programs, audio-video recordings, etc), though without

permitting any kind of interactivity. Tools in the latter category do allow interactivity,

although this cannot be said to be an intrinsic characteristic of the technology, but rather

of the way that technology is used. For example, where network technology is used to

provide access to databases of learning material, the level of interactivity is no greater

than when one “interacts” with the shelves of a library. By contrast, when the network is

used to manage personal interaction, its potential for interactivity is exploited more

fully.

Actually, when talking about interactivity we should refer not so much to interaction

with learning material that someone has made available on the network or on CD-ROM,

but rather to human-machine interaction governed by software, or, better still, personal

interaction mediated by network technology.

In any case it is clear that, even when using the technology’s potential to the full, there

is no guarantee about the quality that the interaction will bring to the learning process.

In some cases the interaction may not have been used in an effective manner for

reaching the learning objectives that have been set.

Interactivity-quality relationship – distance education has traditionally been based on

the production of material to be used individually. The development of new technology,

especially over the past decade, has led to the production of multimedia material of high

quality, at least in aesthetic terms.

Looking at these products from the communication viewpoint, we can identify two

extremes. At one end, there are products with a low level of interactivity, which tend to

reproduce the traditional lecture format. At the other end we have the so-called adaptive

products, which are based on more or less sophisticated schemes of dialogue with the

user. These seek to tailor the user’s learning path to his/her knowledge requirements,

learning style, and so on (Trentin & Midoro, 1996).

As stated earlier, there is another type of interactivity, namely direct interactivity

between individuals, where the technology does not “drive” but acts as a passive

mediator. This is the case with computer networks.

While interactivity is no guarantee of quality, social interaction with peers or with one’s

trainers plays a key role in the learning process, both from the psycho-social and

psycho-affective viewpoints (Barchechath, 1996). In this sense, ICT services can make

an enormous contribution.

8

Despite this, there is still a general trend in distance education towards one-way (or

frontal) schemes. In other words, the participants are offered a multiplicity of material

based on different technologies: lessons that are videotaped or distributed online using

real-video technology, audio cassettes, and printed learning material, guides, etc. that

are sent by post or delivered via computer network.

This kind of technology-supported distance learning not only undervalues the potential

of that technology, it is also based on the assumption that frontal-type teaching is the

standard to which distance learning should conform. This may well prove satisfactory

for “transmissive” teaching (e.g. a geography lesson) but less so when discussion,

sharing of experience or ongoing readjustment of the learning process is called for. For

example, consider a methodology-based course (learning strategies, sales techniques,

etc) where each participant clearly needs to master new knowledge and learn during the

course how it can be applied in his/her professional field.

No matter how sophisticated and interactive a multimedia package designed for

individual use may be, it will never allow the learner to obtain the quality of instruction

that can be achieved by direct interaction with experts/specialists in the field or with

other course participants.

It is worth remembering here that one way of gauging the quality of a multimedia

product is to measure the number of different paths it envisages for the interaction with

the end user. Clearly, no matter how well it has been designed, such a package can only

cover a fraction of the possible directions that this dialogue may follow.

On this point, Diana Laurillard (1993) shows that stand alone multimedia packages

designed for individual use will no longer be capable of meeting the needs of remote

learners because they are unable to support discursive interaction with the end user; as

we have seen, this is the key element in raising the quality of distance courses.

Furthermore, efforts to produce multimedia material of the very highest standards do

not in themselves guarantee overall improvement in learning quality. In fact, investment

in the design and production of material could shift resources away from student

support, which would be counterproductive especially where interaction and discussion

among all the participants (students, tutors, and experts) are central to achieving the

learning objective.

By contrast, those who seek to meet learners’ needs by fostering interaction in fully-

fledged co-operative communities are trying not so much to replicate traditional one-

way teaching but rather to create the conditions for a highly stimulating and fruitful

learning process based on the creation of innovative learning environments.

Cost/quality ratio

A decade or so ago, Garrison and Shale (1987) wrote:

“As a result of advances in telecommunication and microprocessor technologies,

distance education methods exist that are ‘de-massifying’ delivery. The future task

for education at a distance is to design and direct personalised information to

specific target audiences. The industrialised form of distance education, with its

9

mass-consumption ideology and approach, will become less dominant in the

future.”

This is clearly in line with the hypothesis that there is a strict correlation between

quality and interactivity. But how should this stand be seen from the costs viewpoint. If,

as Garrison and Shale claim, there should be a shift towards the “demassification “ of

the distribution of learning, this would remove one of the longstanding cornerstones of

distance education: economy of scale.

Economy of scale means that courses must attract as many potential beneficiaries as

possible in order to offset the costs of investing in the production of high-quality

learning material (Kirkwood, 1998). So the material must be adaptable to a wide

spectrum of possible users and have a long shelf life.

By contrast, courses based on the third generation model, which feature a high degree of

interactivity, entail virtual classes of necessarily limited size and the presence of tutors

for the entire length of the learning process. This means shifting funds away from the

production of ad hoc learning material in order to cover tutoring and experts’ fees. The

problem, however, it is not just one of re-allocating funds; the cost per student of third

generation courses is usually higher than that of second generation courses. Either way,

raising quality means higher costs of one kind or another.

So the core of the problem lies in evaluating to what extent the promotion of

interactivity results in a higher quality course and what effect it has on overall costs. In

other words, if we accept that interactivity has a strong impact on course quality, how

much more are we willing to invest with respect to a traditional distance course in

exchange for a given return on our investment?

Quality and return on investment

How to go about measuring return on investment (ROI) in education is still open to

question, especially because the issue can be approached from many different

viewpoints. Some see the question from the standpoint of schooling, others from that of

company training, and so on. Some consider that ROI should be measured in the long-

term (e.g. schooling), ruling out any short-term evaluation. By contrast, others who have

a direct stake in training (e.g. companies) want a quicker return, and see the training

process as something that ought to bear fruit (especially in economic terms) in the short

to medium term – preferably the short term!

Phillips (1998) considers ROI to be the fifth in the list of items that Kirkpatrick (1975)

proposes for evaluating course success. Here is a brief summary of those items.

Reaction and planned action – this means gauging the participants’ satisfaction and

studying how they intend to apply what has been learnt during the course. However

important satisfaction levels may be, they are no guarantee that the knowledge and/or

skills that the course was designed to impart have actually been learnt.

10

Learning – this reveals what the participant has learnt from the course, which can be

measured in a variety of ways: for example, using tests, practical activities (e.g.

designing), role plays, simulations, evaluation etc (Thorpe, 1993). Nevertheless,

positive outcomes at this point still do not guarantee that the course participants are

actually capable of applying correctly what they have learnt. The literature is full of

examples of just how wide the gap is between knowledge and know-how (Broad &

Newstrom, 1992).

Job applications – there is a variety of follow-up to determine whether participants

apply what they have learned on the job. These are usually based on observation of the

frequency and way in which the newly acquired knowledge and skills are applied.

Even if the evaluation yields positive results, there is still no guarantee that the

organisation to which the course participant belongs will benefit from an equally

positive impact.

Business results – the benefit gained by the organisation can be seen from different

angles: in economic terms; from the viewpoint of customer satisfaction with the service

offered (by a company, institution, training body, etc); in terms of production costs; and

so on.

In this case, it is worth remembering that “customer satisfaction” with the quality of a

product or service provided does not necessarily lead to a positive impact on the

provider. For instance, the production cost may have been excessive and the outlay may

be impossible, or at least difficult, to redeem; or the organisation may not be ready to

take the innovation on board. And this is precisely where Phillips suggests adding a fifth

level, namely ROI.

Return on investiment (ROI) – there are many different ways of expressing ROI but the

most common is in terms of the cost/benefit ratio. Analysis of this aspect clearly cannot

be done without careful evaluation at the previous four levels. This means that to

estimate ROI, we must first evaluate how the knowledge and skills acquired in the

training course (level 2) are applied in the workplace (level 3), resulting in a positive

impact on participant’s organisation (level 4). This kind of approach is in line with the

quality testing adopted by the American Association for Training and Development

(Kimmerling, 1993).

Unless these measures are carried out, it is extremely difficult to claim that the results

really are the fruit of the training course in question, and that they actually represent the

ROI.

In reality, evaluation in the training field rarely goes beyond the fourth level. The reason

is that, apart from being difficult and complicated, ROI is also a long and expensive

process because it relies on study and analysis being carried out at the previous four

levels. So ROI analysis itself may end up adding to overall course costs, which in turn

affects the ROI estimate. This is especially true when ROI is expressed, as is often the

case, as a cost/benefit ratio, an important indicator that is closely related to the sought-

after quality level.

11

Cost/benefits ratio

Evaluating the cost/benefit ratio in distance education, and in the education field in

general, has always been an arduous task. It is even tougher where third-generation

distance education is concerned, because as yet there is no consolidated tradition in

tackling the problem.

There are at least two kinds of organisations interested in cost/benefit analysis for

distance education: public institutions that provide open and distance courses, and the

private sector (small companies, corporations, training bodies).

In the public sector (i.e. public education), costs analysis is usually based on comparison

with tradition training, which is carried out with the aid of fairly well-defined

parameters. By contrast, the comparison used in the private sector is with the cost of

hiring a classroom trainer for a fixed period. Although difficult to perform, cost analysis

proves to be more objective than benefits analysis in both cases, and is therefore easier

to determine.

Indeed, calculating the benefits of distance education is always a subjective matter,

especially where there is no reference to immediate economic benefits. It is no

coincidence that, as a number of studies have demonstrated, when different definitions

of “benefit” are applied to the same course, different ROI estimates are obtained.

What’s more, the weight of ROI varies according to the context in which it is measured.

There is a distinct difference between companies, who need to get a return on their

investment in the short to medium term, and the public education system, where

investment in teaching staff has an initial impact on direct users (students) and a long-

term knock-on effect on the country’s society and economy. In any case, the two cannot

be measured with the same yardstick.

Let’s now see how costs and benefits may be evaluated separately, so that the ratio

between them can be estimated with a view to arriving at the ROI.

Costs

Many of the studies dealing with distance education have focused on comparison with

traditional face-to-face courses (Cukier, 1997). The conclusion generally reached has

been that distance education may proved to be more or less expensive than traditional

courses depending on how course planning, development and delivery are approached.

Thus, the common belief that

distance education cost < onsite education cost

is not always true; there may be cases where distance education turns out to be more

costly than onsite courses (a difference that, naturally, can only be justified by the

higher quality delivered).

There are several ways of quantifying costs, ranging from cost per unit (or average

cost) to the total cost of a course. The various cost items themselves may also vary in

several ways. To provide an example, the cost of the course may be derived dividing the

12

total cost either by the number of participants who enrolled or by the number of

participants who actually completed the course (hence the importance of drop-out rates,

which heavily affect the ROI). Then again the total cost itself may relate to a specific

period (a month or a year) or to a longer spell of time equivalent to the "average life" of

the course, which would make it possible to take its depreciation into account.

Another distinction that can be made is between fixed and variable costs. Fixed costs

include all expenses that are independent of the number of students enrolled, while

variable costs are those that vary in response to this factor. This applies both to the use

of infrastructure (rooms, technology, services, etc) and personnel involved (teachers,

course directors, administrative and technical staff, etc). The production of a TV

programme or of multimedia courseware, for instance, represents a fixed cost, as it is

totally independent of the number of eventual beneficiaries. By contrast, the payment of

fees for online tutors is considered to be a variable cost, in that the number of tutors and

their degree of involvement are factors related to the number of students enrolled.

A further costs classification may be direct and indirect costs, i.e. those that are related

to the production of given material or the delivery of a certain service and those which

are incurred in any case by the educational institution and have no direct

correspondence with the production of given material or the delivery of a given service.

Direct costs include the production of purposely-designed educational material

regarding a given content area or the hiring of an expert to work within the learning

network. Indirect costs may include the purchase of a computer conferencing system to

be adopted in the course in question but which will also be used for other online events.

This is by no means the end of the story as far as cost classification is concerned but we

shall pursue the matter no further as it has already been examined extensively elsewhere

(Rumble, 1997) (Keegan, 1990). The purpose here is to look for parallels between face-

to-face education, traditional distance education (first and second generation) and online

education (third generation). The intention is to show that when it comes to costs

analysis, third generation distance education has many aspects in common with face-to-

face teaching. But let's proceed one step at a time.

Costs comparison between face-to-face teaching and traditional distance education –

in one of his works, Rumble (1997) compares the costs incurred for traditional students

and distance-education students. He expresses the total cost (TC) as the sum of the fixed

costs (F) (including the semi-fixed costs) plus the variable cost per unit (V) times the

number of units (N):

TC = F + VN

The outcome of this study is summarised in figure 3: fixed costs for face-to-face

education are slightly lower than distance education costs (Rumble refers here to first

and second generation distance education). The reason is straightforward: in distance

education, the economic impact of generating purposely-designed material is quite

significant, while this is not normally the case with face-to-face education.

When it comes to variable costs, however, the situation is different, since in face-to-face

education these are closely linked to the number of students attending the course: the

greater the attendance, the greater the number of teachers involved, thus the system’s

variable costs rise. Variable costs may be incurred in distance education as well

13

(providing copies of teaching material, distributing it, etc). However, the cost vs.

attendance ratio does not rise at the same rate as that for a face-to-face teaching

situation (figure 3).

number of students

tota

l co

st

face-to-face fixed costs

DE fixed costs

face

-to-

face

var

iabl

e co

sts

DE variable costs

Figure 3. Fixed and variable costs in face-to-face teaching and traditional distance

education.

Discussing the average cost per student in distance education, Rumble notes that this

value tends to diminish quite markedly as the number of students in the course grows,

as expressed by the equation:

AC = TC / N

His conclusions are clearly reflected in figure 4: the average cost per distance student

falls (gradually) and the curve flattens out when the number of students in the course

grows significantly; from there on, it stays parallel to the face-to-face curve, although at

a considerably lower level.

number of students

aver

age

cost

per

stu

den

t

face-to-face

DE

14

Figure 4. Average cost per student in face-to-face teaching and traditional distance

education.

Comparing costs of traditional distance education and online courses – the gradual

introduction of communication technology in distance education and the development

of innovative third-generation approaches make it imperative to revise the studies

carried out by Rumble. In particular, when calculating the cost of third-generation

distance education systems, we need to reintroduce some items that were previously

typical of face-to-face teaching (e.g. tutor and subject experts’ fees).

In practice, this has radically affected the economy of scale idea that has always

underpinned large-scale distance education systems. The curve traced by the new

courses closely resembles the cost graph typical of face-to-face education (see figure 3):

low material production costs; high variable costs related to the presence of tutors

(whose number grows with the number of students). Tutor costs remain high

irrespective of whether the strategy implemented is based on synchronous

communication (e.g. videoconferencing and one-to-many interaction) or asynchronous

communication (e.g. computer conferencing and many-to-many interaction), though in

the latter case they are somewhat higher, on average 3.

If we go back to per capita costs, once again we find a very similar situation to that in

Figure 4: the curve for online courses is more similar to face-to-face teaching than to

traditional distance education. This means lower initial costs (thanks to ready-to-use

teaching material), but higher costs along the way resulting from online tutoring (which

makes online courses generally more expensive than traditional distance education).

Benefits

As mentioned earlier, defining and assessing the benefits of any education effort is a

difficult task, irrespective of whether it is conducted face-to-face or at a distance. One

way of tackling the problem is to make assumptions on the basis of the three

fundamental needs that education systems generally seek to address (Barchechath,

1996):

professionalisation, i.e. meeting the needs of production (training the workforce);

socialisation, i.e. meeting the needs of society;

individualisation, i.e. meeting the needs of individuals in the areas of development

and self-fulfilment.

Having said that, we can classify the benefits of education efforts into expected,

intangible and unforeseen benefits.

Expected benefits – all those benefits that explicitly constitute the final goal of a course.

In defining these, special attention must be paid to the context in which the learning will

be set. Let’s take teacher training, for instance: suppose we are launching a typical

course in the educational use of computer network technology. Among the expected

benefits, we might list:

15

familiarisation with ICT services;

familiarisation with collaborative working methods;

acquisition of skills in planning and managing innovative courses based on

educational use of network technology;

transfer of some of these skills to school students.

The idea here is that the main benefit to be gained lies in cultural growth of the

participants, which flows down from them to the students and beyond. For this very

reason, the benefits should be seen in the long rather than the short term.

However, when we shift the focus to the business world we see that benefits are usually

gauged in terms of profit, improved image, etc. In this context, it is felt that the

knowledge gained through training must be cashed in on as soon as possible in order to

recoup the time and money invested.

Intangible Benefits – the benefits gained from online training are not necessarily

tangible ones, like economic return. In fact, as described in (Phillips, 1998), this

category covers:

increased work satisfaction;

improved company image;

improved group work within the organisation;

better customer services;

fewer customer complaints;

less conflict inside the organisation and with the outside.

Clearly, these intangible benefits do not translate directly into increased income, but

nevertheless they can pave the way for this to happen.

Unforeseen Benefits – in some cases these represent the paradox of training, in that

they occasionally outnumber the expected benefits. In a way, unforeseen benefits do not

fall entirely into one of the two above-mentioned categories; there may be unexpected

tangible benefits, just as there may be intangible benefits there are unforeseen.

An example can be seen in the first course delivered within the Polaris pilot project, a

joint undertaking of the Italian Ministry of Education and the National Research

Council’s Institute for Educational Technology in Genoa. This project aimed to test the

intensive use of ICT within in-service teacher training in accordance with online

training approaches (Trentin, 1997a). It was seen that in Polaris the use of online

techniques had a positive effect on the participants’ acquisition of new cognitive skills.

Apart from gaining the knowledge that the course explicitly set out to impart, the

16

participants also acquired noticeable awareness of how to use the network

autonomously to cater for their own information and knowledge needs.

Essentially, the awareness that was gained during the course about the potential of ICT

was translated into equivalent awareness that the same means could be exploited to

cater for individual training on a continuous basis, not just through access to the

information and knowledge available on the web but especially by using opportunities

for interpersonal communication with colleagues, experts and others. In truth, during the

design phase of the course something of this kind was identified as an expected benefit.

What confounded expectations, however, was the degree to which this phenomenon was

developed.

Besides consideration of how to tackle costs and benefits separately, the problem still

remains of judging how much to invest in the expectation of a certain benefit (supposing

that this can be defined in a fairly precise manner). The problem gets even trickier when

the main benefit is not exclusively one of an economic nature, and is thus harder to

compare with investment costs.

Referring once again to Polaris, it may be said that the per capita cost was undoubtedly

higher than that typically incurred in more traditional teacher training. This is somewhat

paradoxical when we recall the common notion that distance education is supposed to

be cheaper than face-to-face training. Nevertheless, the high quality results that

participants achieved (both in terms of learning and capacity for application) coupled

with the relatively low dropout rate (14% out of around 150) suggest that in the final

analysis the outcome of Polaris was probably positive. We have to say “probably” here

because systematic study of the cost/benefit ratio of online education has only been

carried out in a very limited number of cases, too few in fact to permit certainty – if any

such thing as certainty exists in this field!

But the question can also be approached from a different angle. We might ask ourselves

questions like: What price would be charged for a traditional course of the same quality

as the Polaris courses? How many seminars and workshops would have been necessary

to achieve the same geographic coverage? How could such an intense level of

collaboration between all the participants have been managed, at the same time

guaranteeing continuity?

Actually, a comparison of this kind might well be out of the question, in that the

adoption of strategies for highly intensive collaboration usually occurs in two distinct

scenarios: local groups with the opportunity for frequent face-to-face interaction, or

remote participants in close contact via computer network. Despite the common context

(teacher training) and learning approach (collaborative learning), the many differences

in the surrounding conditions mean that the comparison cannot in fact be made.

Looking at traditional training, the choice lies between a low-cost course addressing a

small group of participants located in the same geographical area, or, where the

objective is more widespread coverage, the more costly option of repeating the course a

number of times in different locations. In the latter case, the logistical cost incurred

(transport, accommodation, etc) will be high if we want to provide frequent

opportunities for face-to-face interaction within a large group spread over a wide area.

17

In the case of online courses, costs are not affected by distances but rather by the

number of tutors involved, which varies in relation to the size of the learner population.

Nevertheless, online courses have a big advantage over the face-to-face variety when it

comes to the geographical coverage of a single training event.

As we can see, the “short blanket effect” mentioned at the outset continues to make

itself felt, even within a single context (in our case teacher training).

Conclusions

This paper has examined the complex mix of elements that together determine the

quality level of a distance education process. There are actually many different ways of

combining the same elements, depending on factors like the operational context, the

different meaning attributed to the cost/benefit ratio and investment return, and, more

basically, the very definition of the concept of quality in distance education.

So, in order not to overburden the discussion with too many elements, our analysis has

utilised the term quality not as a synonym of excellence but as the effort to bridge the

gap between the actual effect of an education process and what is expected by the

course provider as well as the recipients.

Achieving such a result demands frequent interaction between all the components in the

process. Interaction is imperative if the quality of the process is to be raised: interaction

with materials, between students and their tutors/teachers, as well as between all the

participants.

Information technology and, more importantly, computer communication technology

can play a vital role in fostering this interaction, even if use of these means is not in

itself a guarantee of higher quality. As is usually the case, a fundamental role is played

by course designers and co-ordinators, who have the means to steer the process towards

the predefined goals.

However, raising interactivity in the process either involves investing in multimedia

products which are very (excessively) complex and expensive, or entails increasing the

amount of online tutoring. Thus, higher interactivity is undoubtedly synonymous with

higher production and management costs.

At this point, the central question is how much should be invested in the project in order

to reach a given quality level, or, seen from another perspective, how much we are

prepared to spend to guarantee a given return on the investment made.

The answers obviously vary depending on our definition of expected benefit - whether

or not it is seen as an economic value. More particularly, there may be cases where

investment return is taken to mean something different from income (e.g. the acquisition

of knowledge, say, in teaching geography). In this light, it would be extremely difficult

to evaluate a ratio (between cost and benefit) when one side of the equation is a

quantifiable value (cost) and the other is something economically intangible.

In this case, there is no way of reaching a quantitative evaluation of the cost/benefit

ratio, and all that can be done is to optimise this relationship in either of two possible

ways. One is to define how close the real effect ought to be to the expected effect, and

18

then to invest resources in this direction. The other option is taking care to set objectives

that are affordable and which guarantee a satisfactory degree of quality.

No mean feat, in either case!

References

Barchecharth, E. (1996). La progettazione dei sistemi formativi a distanza dal punto di

vista economico, pedagogico e organizzativo. In M.A. Garrito (Ed.), La

multimedialità nell’insegnamento a distanza. Roma: Garamond.

Barker, B., Frisbie, A. & Patrick, K. (1993). Broadening the definition of distance

education in light of the new telecommunications technologies. In K. Harry, M. John

& D. Keegan (Eds.), Distance Education: New Perspectives. London: Routledge.

Branson, R.K. & Buckner, T. (1995). Quality applications to the classroom of

tomorrow. Educational Technology, 35(3), 19-22.

Briano, R., Midoro, V. & Trentin, G. (1997). Computer mediated communication and

online teacher training in environmental education. Journal of Information

Technology for Teacher Education, 6(2), 127-145.

Broad, M.L. & Newstrom, J.W. (1992). Transfer of Training. Reading, Addison-

Wesley.

Cukier, J. (1997). Cost-benefit analysis of telelearning: developing a methodology

framework. Distance Education, 18(1), 137-152.

Garrison, D.R. & Shale, D. (1987). Mapping the boundaries of distance education:

problems in defining the field. American Journal of Distance Education, 1(1), 7-13.

Herman, J.L. & Herman, J.J. (1995). Total quality management (TQM) for education.

Educational Technology, 35(3), 14-18.

Keegan, D. (1990). Foundations of Distance Education. London: Routledge.

Kimmerling, G. (1993). Gathering best practices. Training and Development, 47(3), 28-

36.

Kirkpatrick, D.L. (1975). Techniques for evaluating training programs. Evaluating

Training Programs, 1-17, Alexandria, ASTD:VA.

Kirkwood, A. (1998). New media mania: can information and communication

technologies enhance the quality of open and distance learning?. Distance Education,

19(2), 228-241.

Laurillard, D. (1993). Rethinking University Teaching. London: Routledge.

Moore, M. (1989). Three types of interaction. American Journal of Distance Education,

3(2), 1-6.

Nipper, S. (1989). Third generation distance learning and computer conferencing. In

R.D. Mason & A.R. Kaye (Eds.), Mindweave: Communication, Computers and

Distance Education. Oxford: Pergamon Press.

Parsons, T. (1979). Organisation and Clients. Merril Publications.

19

Phillips, J.J. (1998). The return-on-investment (ROI) process: issue and trends.

Educational Technology, 38(4), 7-14.

Rumble, G. (1982). The cost analysis of learning at a distance. Distance Education,

4(2), 101-131.

Rumble, G. (1997). The Costs and Economics of Open and Distance Learning. London:

Kogan Page.

Short, J., Williams, E. & Christie B. (1976). The Social Psychology of

Telecommunications. London: John Wiley & Sons.

Thorpe, M. (1993). Evaluating Open and Distance Education. Longman Group.

Trentin, G. (1997a). Telematics and on-line teacher training: the Polaris project.

International Journal of Computer Assisted Learning, 13(4), 261-270.

Trentin, G. (1997b). Logical communication structures for network-based education and

tele-teaching. Educational Technology, 37(4), 19-25.

Trentin, G. (1998). Network-based collaborative education. International Journal of

Instructional Media, 26(2), 145-157.

Trentin, G. & Midoro, V. (1996). A learning-lab where artificial intelligence meets

hypermedia. Proceedings of the World Conference Ed-media/Ed-telecom ’96,

AACE, 276-282.

Notes

1 For argument’s sake, first generation systems, also known as learning by

correspondence, may be considered as part of second generation systems.

2 A number of different models may be considered when discussing the quality-

interaction relationship. This paper adopts Moor’s model because it is the one most

similar to the model the author applied in his experiences evaluating the quality of

distance education processes.

3 In online education the tutor-participant ratio is somewhere between 1/10 to 1/15

(depending on the type of learning activities involved) (Trentin, 1998).