Conceptual framework for Europe's future knowledge services

IST-2001-37440eLearnTN

eLearning Thematic NetworkIST 2002-III.5.2Preparing for Future Research Activities

Document Name: Conceptual framework for Europe’s future knowledge services (draft)

Document Date: 2003-01-31Document Owner: Open Universiteit NederlandDocument Author/s: WP-1 project teamDocument approved by:Deliverable Number: D1Work Package contributing to the Deliverable: WP-1

Deliverable Type: [Report]Nature of the Deliverable: [Int.]Version 0.10

Abstract:

Keyword List:

eLearnTNeLearning Thematic Network

IST 2002-III.5.2Preparing for Future Research Activities

Annexed Documents

Document Name Date

Identified ‘top-10’ problems

Contributors

IAll rights reserved by the eLearnTN Consortium



Versions Control

Version Date Changes

0.10 2003-01-31 First draft:

IIAll rights reserved by the eLearnTN Consortium



Contents Table

1 SUMMARY ................................................................................................................................ 3

2 INTRODUCTION ..................................................................................................................... 3

2.1 PURPOSE AND SCOPE...............................................................................................................32.2 CONTRIBUTIONS......................................................................................................................42.3 DESCRIPTION OF WORK..........................................................................................................52.4 NATURE OF WP-1 DELIVERABLE...........................................................................................52.5 WORKING PROCEDURE...........................................................................................................62.6 SYSTEM APPROACH.................................................................................................................6

3 THE EUROPEAN KNOWLEDGE SOCIETY ...................................................................... 7

3.1 CONTEXT..................................................................................................................................73.2 THEMES....................................................................................................................................73.3 KNOWLEDGE SERVICES...........................................................................................................93.4 LEARNING IN THE KNOWLEDGE SOCIETY............................................................................10

4 ELEARNING IN HIGHER DISTANCE EDUCATION ..................................................... 11

4.1 INTRODUCTION......................................................................................................................114.2 ELEARNING SYSTEMS............................................................................................................11

5 DIMENSIONS IN THE ELEARNING DOMAIN ............................................................... 14

5.1 THE FUNCTIONAL DIMENSION..............................................................................................145.2 THE ORGANIZATIONAL DIMENSION......................................................................................165.3 THE TECHNICAL DIMENSION................................................................................................16

6 ISSUES FOR THE IMPROVEMENT OF ELEARNING SYSTEMS .............................. 17

6.1 DEVELOPMENT ISSUES..........................................................................................................176.2 SHARING & REUSE ISSUES.....................................................................................................196.3 DIFFERENTIATED DELIVERY ISSUES.....................................................................................21

1All rights reserved by the eLearnTN Consortium



6.4 ASSESSMENT ISSUES..............................................................................................................22

7 AN ELEARNING NETWORK .............................................................................................. 23

7.1 NETWORK-LEVEL ANALYSIS.................................................................................................237.2 IMPLICATIONS FOR THE OTHER WORK PACKAGES.............................................................24

8 IN CONCLUSION .................................................................................................................. 25

References .................................................................................................................................... 25




1 SummaryThis document is the first deliverable of the eLearnTN project: the conceptual framework for Europe’s future knowledge services in the form of a domain description, where possible expressed in UML notation. This document was developed under Work Package 1 (WP-1), coordinated by the Educational Technology Expertise Centre (OTEC) of Open University of the Netherlands (OUNL).

Domains are defined to prevent ambiguity in communication and cooperation between a community of practitioners. A domain is an area of knowledge or activity characterised by a set of concepts and terminology understood by practitioners in that area1. The domain covers all relevant aspects of the real-world situation in which an envisaged future system will operate, but excludes implementation aspects/specific instantiations.

In principle the domain defines for a specific community what is in scope and what is out of scope. The process of domain definition should thus be driven by the problem at hand. The ‘problem at hand’ for eLearnTN is the use of e-learning in the context of higher distance teaching institutions, in support of the European knowledge society.

The relation of the present deliverable to the other work packages, the approach taken and other contextual information is contained in chapter 2. In developing the domain definition we start from the perspective of the European knowledge society, as this is the central focus in the Information Society Technologies programme under which eLearnTN is carried out. Its central themes, and the demands and implications for learning and eLearning are covered in chapter 3.

The specific focus of eLearnTN – the domain of eLearning in higher distance education – is elaborated in chapter 4 in the form of a network of distance teaching institutions and their interfaces. Three sub-domains – also called aspects or dimensions – of this domain are further specified: the functional domain, the technological domain and the organizational domain.

Finally, chapter 5 concludes with discussing the implications and recommendations for the further elaboration of the three sub-domains in WP-2, WP-3 and WP-4, and for the application of the domain descriptions in WP-5 in developing Europe’s future RTD roadmap in the area of eLearning for higher education.

2 Introduction

2.1 Purpose and scopeThis document describes the first deliverable of the eLearnTN project as developed under Work Package 1 (WP-1) of that project: A Domain Definition for e-learning in the context of higher distance teaching institutions, in support of the European knowledge society.

1 Grady Booch, James Rumbaugh and Ivar Jacobson, The Unified Modeling Language User Guide, ISBN 0-201-57168-4; also available at http://cgi.omg.org/docs/formal/01-09-79.pdf


http://cgi.omg.org/docs/formal/01-09-79.pdf



The objective of WP-1 is to develop such a domain description to be used by WP-2, WP-3, and WP-4 for:- further specification of the sub-domains ‘pedagogy’, ‘technology’ and ‘organization’ (the

output of WP-1 will be used as input for the ‘state of the art’ workshop preparation documents in WP-2, WP-3, and WP-4)

- identification and mapping of present e-learning RTD initiatives and key-actors on these sub-domains (benchmarking)

- specification of themes and prospective participants and audiences to be included in the RTD roadmap for the domain.

As such WP-1 is a prerequisite for the other WP’s to start, conceptually as well as sequentially.

Figure 1 relationship between the eLearnTN work packages

In developing the domain definition, OUNL also establishes, supports, and chairs a Special Interest Group comprising of external experts, who will comment and validate the domain definition.

2.2 ContributionsThis report was compiled under the responsibility of the WP-1 coordinator, the Educational Technology Expertise Centre (OTEC) of the Open University of the Netherlands (OUNL). A first version of the domain definition was drafted by the OUNL. This was then validated by the eLearnTN Special Interest Group (SIG) - comprising representatives from project partners and external experts – during a three-day conference. The partners provided problem descriptions for the three learning sub-domains: the pedagogical sub-domain, the technological sub-domain, and the organisational sub-domain (see annex). These problems will be used to start the process of further refining (in WP-2, WP-3 and WP-4) the three sub-domains.

After receiving feed-back on the initial version of the report by the SIG and the Scientific Committee, the final editing was done by OUNL. For a detailed overview of the individual contributors to this process (see annex)




2.3 Description of workThe work for WP-1 was divided into four subtasks (T1.1 through T1.4, quoted from the eLearnTN project document):

T1.1 Establish, support and chair the Special Interest Group (SIG). This will involve detailing the mandate and modus operandus of the SIG; the identification, invitation and enlisting of SIG-members; communication and secretarial support; and chairing the meetings. To be carried out by the WP-1 leader.

T1.2 Outline a conceptual framework for Europe’s future knowledge services. This will mainly be carried out through desk studies by the WP-1 leader, to be validated against representatives from interest groups through the SIG. Additional services of an UML expert will be required.

T1.3 Elaborate and refine the conceptual framework for Europe’s future knowledge services. This will be done during a 3-day work conference by a small group of experts on the basis of the outline resulting from T1.2. Participating experts will be selected from the Special Interest Group, members of the Scientific Committee, and the leaders of WP-2, WP-3, and WP-4. The WP-1 leader will organise the conference.

T1.4 Elaborate and refine the conceptual framework for the specific areas of ‘Technology and Infrastructure’, ‘Business and Law’ and ‘Pedagogy’ through (an) electronic work group(s)2. This is in preparation of the workshops planned under WP-2, WP-3, and WP-4. For each of the three themes a desk study will be carried out under the respective work packages in preparation of the work groups. The outcomes will be used by WP-2, WP-3 and WP-4 as input to their Preparation Documents.

2.4 Nature of WP-1 deliverableWP-1 has one deliverable (D1), being this document, which is the domain description for eLearning in the context of higher distance teaching institutions, in support of the European knowledge society. Where possible this domain description is expressed in UML notation.

A domain is an area of knowledge or activity characterised by a set of concepts and terminology understood by practitioners in that area3. The domain covers all relevant aspects of the real-world situation in which the envisaged future system will operate, but excludes implementation aspects/specific instantiations (the latter are covered in an implementation model).

A domain is not:- a theory or model (though these can be part of a domain, a domain should be generic

enough to allow mapping of various – sometimes even competing – theories and models)- a standard (though standardisation can be a long-term aim in domain definition)- a blueprint for implementation (though a technical implementation plan may be derived from

it, a domain description should be as free of implementation decisions as possible)- a flow chart (though flow charts may be used to express behaviour or change within a

domain)- a scenario (though a domain may be future-proof by excluding implementation

requirements).

2 This activity falls under the responsibility of the respective WP-2, WP-3 and WP-4 leaders.3 Grady Booch, James Rumbaugh and Ivar Jacobson, The Unified Modeling Language User Guide, ISBN 0-201-57168-4; also available at http://cgi.omg.org/docs/formal/01-09-79.pdf


http://cgi.omg.org/docs/formal/01-09-79.pdf



Domains are defined to prevent ambiguity in communication and cooperation between a community of practitioners. In principle the domain defines for a specific community what is in scope and what is out of scope. The process of domain definition should thus be driven by the problem at hand. As such there may be no ‘best’ domain definition for example for eLearning; only for eLearning within a specific context.

The ‘problem at hand’ for eLearnTN is the use of eLearning in the context of higher distance teaching institutions, in support of the European knowledge society.

2.5 Working procedureIn developing the domain description, the vision of the European knowledge society was matched with the latest insights and developments in the area of learning technologies. The central concepts of the knowledge society – actors requiring various sorts of knowledge to become competent workers and citizens – were linked to the central concepts of eLearning pedagogy, technology and organization.

For the vision of the European knowledge society various documents of the European Commission were studied. The focus here was on the meaning of ‘knowledge’ in its different manifestations, and on related processes of knowledge creation, knowledge management, and knowledge transfer (learning). This seems in line with recent trends where knowledge management and learning management – at various levels of analysis –converge with the support of ICT.

From there the concept of learning as a means of producing knowledge was applied to the ‘problem at hand’: eLearning in higher distance teaching institutions. This was further elaborated in terms of central processes, inputs and outputs for individual higher distance teaching institutions, as well as for a network of such institutions cooperating through defined interfaces. The three sub-domains of ‘pedagogy’, ‘technology’ and ‘organization’ are positioned as aspects of this system (aspect system).

2.6 System approachIn the work procedure as described above, the soft-systems approach was used to describe concepts in terms of system boundaries, inputs, processes and outputs, and the Unified Modelling Language was used to model these concepts.

A system is a complex network of highly interconnected components and their relationships, where the components and their relationships have attributes. A system transforms inputs into outputs: these can be physical or abstract. The process in the system is characterized by feedback where the behaviour of one element may feed back either directly from another element or indirectly via a series of connected elements. A central idea is that ‘the whole is greater than the sum of its parts’. One of the most important characteristics of system thinking is the concept of the environment. The environment imposes requirements on the system, and the system tries to respond to in an adequate way.

The soft system approach in general has been promoted as a legitimate approach for ill-defined and ill-structured problems in dynamic domains and situations, involving people with their behaviours, attitudes and relations. In the soft-system approach the components can be considered as sub-systems.

In eLearnTN the domain components and their relationships will be presented as much as possible as UML diagrams. UML notation is based on object orientation. Object-Oriented




Methodology is a representative of the system thinking approach, with notions such as classes, objects and attributes.

3 The European knowledge society

3.1 ContextIn March 2000 the Lisbon European Council adopted the development of the knowledge society as a key priority in its strategy to make Europe ‘the most competitive knowledge-based society in the world, capable of sustained economic growth providing more and better jobs and greater social cohesion’. This took practical shape in the eEurope 2002 initiative, launched by the Commission to promote ‘the information society for all’. The initiative was further elaborated in June 2002 in an action plan known as eEurope 2005.

Where previously the emphasis was on improving Europe’s (internet) connectivity and developing its human (ICT) resources, the latter action plan emphasizes the generation of advanced services: e-business, e-government, e-learning, e-health, etc. Everyone in the European Union - every citizen, school, company, administration – should have access to the new ICT’s and exploits them as fully as possible in everyday activities, services and products such as education, government, health, culture and entertainment.

The European society is now defined, instead of an “Industrial society” as an "Information Society", a society in which low-cost information and ICT are in general use, or as the "Knowledge(-based) Society", to stress the fact that the most valuable asset is investment in less tangible human and social capital, and that the key competitive factors are knowledge and creativity 4. This European Knowledge society is a society where creating, sharing and using knowledge are determining prosperity and well-being of its people. The prospects are significant: new employment possibilities, more fulfilling jobs, new tools for education and training, easier access to public services, increased inclusion of disadvantaged people or regions, etc..

This results in a number of significant changes for people and organisations in Europe as knowledge and information become major sources of creating value,5. The state of the art in technologies for gaining, sharing and applying knowledge are changing rapidly under the influence of research and technology development, and knowledge tends to grow at exceptional rates. The effective and efficient management of this transformation processes makes heavy demands on European and national institutions, and on individual citizens, workers and students.

3.2 ThemesThe transformation towards a European knowledge society is structured around a number of themes. The themes and a summary of the main concepts and activities are given below 6:

E-inclusion - The more the Information Society advances, the more social and economic opportunities depend on the usage of Information and Communication Technologies (ICT). By giving attention to the theme of E-Inclusion, the European commission aims to prevent risks of

4 http://europa.eu.int/comm/employment_social/knowledge_society/index_en.htm5 http://www.morst.govt.nz/guide/knowledge.html#what6 http://europa.eu.int/comm/employment_social/knowledge_society/index_en.htm




'digital exclusion', that is to ensure that disadvantaged people are not left behind and to avoid new forms of exclusion due to lack of digital literacy or of Internet access. At the same time e-Inclusion means also tapping new 'digital opportunities' for the inclusion of socially disadvantaged people and less-favoured areas. The Information Society has the potential to distribute more equally knowledge resources and to offer new job opportunities, also by overcoming the traditional barriers to mobility and geographic distance.

E-working - The diffusion of Information and Communication technologies (ICT) induces deep organisational changes in jobs, in production and delivery of products and services. The spread of ICT changes the way we work. New forms of work and work organisation emerge as they are critical in the transition to a knowledge-based economy. In this respect digital training and skills for workers for reducing a persistent digital skills gap in the knowledge-based economy are seen as priority areas.

Quality of life - The spread of information technology impacts on employment and as such on job-related health and safety. There is a need to improve health and safety standards. In this way, improved quality at work forms part of a virtuous circle of increasing productivity, rising living standards and sustainable economic growth.

EU-regions - Information technology can be used to renew urban and regional development while learning and training opportunities can be offered amongst others through the development of local learning centres. Active employment policies should provide employment services with a European-wide database on jobs and learning opportunities. Information and communication technologies (ICT) are to play a main role in this respect facilitating networking, exchange of experience and good practices.

Education and training- Europe's education and training systems need to adapt both to the demands of the knowledge society and to the need for an improved level and quality of employment. Digital literacy is essential in the knowledge-based economy and society. By investing in this theme the European commission aims not only to develop physical infrastructure but also to boost 'human investment', human and social capital. The knowledge-based economy demands that more Community resources than ever should be dedicated to job creation and particularly to lifelong learning. Policies in the fields of electronic communications, education and training are of fundamental importance in preparing the way for the knowledge society.

The Sixth Framework Programme, as an RTD programme, has as its main aim the creation, demonstration, and dissemination of research outcomes and technologies, and concentrates on seven thematic areas. Two of these areas – Information Society Technologies and Citizens and governance in a knowledge-based society – approach knowledge as the object of R&D itself.

In the Information Society Technologies (IST) thematic area two priority areas deal with knowledge specifically. The first priority area is ‘Integrating research into technological areas of priority interest for citizens and business’, which covers topics that have knowledge as its main focus like ‘offering access to the information society for all’, ‘interactive and intelligent systems for health, mobility, security, leisure, ….’, ‘new tools and new methods for work’, ‘technologies for learning’, and ‘systems for corporate knowledge management’. The second priority area where knowledge itself is a major focus is ‘Information management and interfaces’, which covers topics like ‘knowledge representation and management systems’ and ‘tools form creating, organizing, navigating, retrieving, sharing, preserving and disseminating digital content’.

In the ‘Citizens and governance in a knowledge-based society’ thematic area one priority area deals with knowledge specifically: ‘Knowledge-based society and social cohesion’. Topics covered under this priority area that have knowledge as its main focus are ‘systematic analysis of best methods for improving the production, transmission and utilization of knowledge in




Europe’ and ‘life-long learning …. with due consideration for the various social models in Europe’.

3.3 Knowledge servicesThe concept of knowledge gives rise to many interpretations. In the context of the ‘knowledge society’ as applied in various EC policy documents, the term is generally used in a popular but rather indistinct way. This is evidenced by the fact that ‘knowledge’ and ‘information’ (as for example in the term ‘information society’!) are often used interchangeable. What is clear however, is that knowledge used in this specific context is perceived as an immensely valuable asset at the individual, organizational, national and even Community level. Realisation of the European knowledge society as outlined above requires the development, operation and maintenance of special services: services for the creation, explication, dissemination and application of knowledge.

Knowledge creation services typically comprise (national and international) research instruments (activities, projects and programmes). The Framework Programmes form a good example of such a service at the international level. Knowledge explication services – making knowledge accessible by validating, categorising, and indexing - are typically the realm of clearing houses, libraries and more recently web-services (taxonomies and meta-data tagging). Knowledge dissemination takes place through media and through direct knowledge transfer from one person to another. Relevant services for publishing, storing, searching and delivery of media-stored knowledge are provided by publishers, libraries, etc. Facilitating the transfer of new knowledge to - and incorporation by - a learner are performed by education and training services. Knowledge application covers the use of knowledge in realising the aims of the knowledge society: in business, government, health, education, etc.

These four services can be linked into a knowledge value chain, with RTD at the front and knowledge application at the end. In between there is a field that has ‘claimed’ knowledge as its core business: knowledge management. Knowledge management has become a field of study in its own right since the early ‘90’s, with the realization that knowledge - just as more traditional assets like physical infrastructure, capital and people - is a major asset for any organization. The importance of knowledge management is increasing under the influence of concepts like the knowledge society, knowledge workers, etc. Knowledge management borrows from various disciplines, like organization- and management theory, motivational theory and learning psychology, and ICT. Initially knowledge management focused mainly on the organization and its workers. Only recently the focus seems to be broadened to include on the one hand the individual - as being responsible for his/her own knowledge management, which is closely related to the concept of life-long learning - and on the other hand to encompass the level of the individual organization and to include all phases of the knowledge value chain.

A far older and more established field which always has had knowledge at its heart, is of course education – or learning as it is coined today. These learning services will be further elaborated in the next section.

As knowledge becomes more valuable, grows faster, gets more complex, there is a growing need to manage it effectively to capture its full benefit. In the European Knowledge Management Forum, knowledge management is described as a theoretically founded management system aimed at providing and organising the increasing 1quantity of knowledge in such a way that it can be searched and reused by whoever has the permission to access its archiving systems. To achieve mission success, existing organizations have to transform to learning organizations: flexible, adaptive, information-rich organizations that rely on the power of individuals to learn




and to change the behaviour of the organization as a whole. Here is where the concepts of knowledge management and learning meet.

3.4 Learning in the knowledge societyThe challenge for Europe is to embrace the digital age and become a truly knowledge-based economy. The way in which the European Union manages this transition will help determine quality of life, working conditions and the overall competitiveness of the industries and services in Europe.

This knowledge-based economy requires new competencies such as learning to learn, active citizenship, cultural and social competences, and overall access and participation. Informed citizens and competent workers are thus the key to the European knowledge society. This requires training and education for people of all ages. Every citizen must be equipped with the skills needed to live and work in this new information society. Competitiveness depends on how much you invest in people: businesses and citizens must have access to an inexpensive, world-class communications infrastructure and a wide range of services. In this perspective life long learning serves as an instrument to meet these requirements and provide sustainability for the future, and should become a basic component of the European social model.

Life long learning opportunities7 should be available to all citizens on an ongoing basis. In practice this should mean that citizens each have individual learning pathways, suitable to their needs and interests at all stages of their lives. The content of learning, the way learning is accessed, and where it takes place may vary depending on the learner and their learning requirements.

Europe’s higher distance teaching institutions have a key role to play in facilitating this new society. Not only through offering learning opportunities to knowledge workers, citizens, and disadvantaged students, but also through further developing and disseminating their experience with eLearning to other actors and institutions who are key to the realisation of the knowledge society.

The following chapters 4, 5, 6 and 7 were accepted to be published as chapter 6 in: Koper, E.J.R. (2003). Learning technologies: an integrated domain model. In W. Jochems, J. van Merrienboer, E.J.R. Koper, Integrated eLearning (pp. xx-xx). London: Kogan Page.

7 http://europa.eu.int/comm/education/life/communication/staff_en.pdf




4 eLearning in higher distance education

4.1 IntroductioneLearning is a fact nowadays. There is no discussion about whether or not eLearning should be applied or not, the discussion is about how and when it should be applied. Most of the basic technologies are available, the pedagogical principles are worked out to a certain extend, there are different business models available for eLearning, the content can be transformed to be delivered electronically. This sounds really promising, but … in fact there is little knowledge available in the field on how to integrate all these different approaches and technologies in order to create the best possible eLearning solution for different needs and situations. There is a lack of integration and harmonization in the eLearning field and even very basic theories and models about eLearning are missing. There is a lot of conceptual confusion. This hinders implementation and the further development of the field.

In this chapter the different dimensions of eLearning are brought together by introducing an integrated conceptual framework that enables researchers, developers, implementers, managers and others to understand, organize, classify, plan and approach the issues in eLearning. The framework is presented as a domain model. It defines the field of eLearning and its basic structure, vocabulary and issues.

4.2 eLearning SystemsIn eLearning most people tend to focus on the computer systems, e.g. network infrastructure, learning management systems, collaborative learning systems, content management systems and student administration systems. However, various decisions are to be made at other levels than the technological one, before an eLearning system has an optimal degree of effectiveness, efficiency, attractiveness and accessibility. These decisions can only be made and justified from a wider, systemic perspective (also see Banathy, 1996). Hence, the scope of the system under consideration in eLearning is not the computer system only, but the complete organization of the educational system. In this chapter we will analyse the eLearning system at this organizational level. An 'eLearning system' is defined here as an educational system that uses network-based learning technologies to support its educational functions. Figure 2 provides a basic model for eLearning systems.




Figure 2. The organisational structure of an eLearning system

The primary inputs of an eLearning system are the learners with their entry characteristics. In the knowledge society these learners are often typified as ‘students’ (those who participate in the formal educational system, usually financed from public funds), ‘citizens’ (those who learn in their own time, often self-financed), and ‘workers’ (who learn within the context of a job, and are financed by their employer). The primary outputs are the 'transformed' learners with additional knowledge (declarative and procedural with a certain competency level) and proofs of that knowledge (e.g. certificates).

Another important input are the staff members (teachers, mentors, assessors, etc). Staff members bring their knowledge into the learning process and this knowledge changes during their work. The learner and staff characteristics that are changed by the system, are termed the properties of the learners and staff.

The system operates on so called learning artefacts, i.e. all the physical products produced before, during or after learning, like courses, programmes, learning designs, activity descriptions, books, reports, tests, remarks and comments. The learning artefacts are the containers of explicit knowledge and are ordered according to a variety of factors like knowledge domain and application contexts. Learning artefacts have an aggregation relationship: a course consists of a learning design that uses lower level learning resources like books, websites and communication services. Most eLearning systems add value to the learning artefacts that are provided at the input side, e.g. the output comprises courses built from lower level learning artefacts like articles and services.

Several providers can be identified behind the inputs. For instance, a learning artefact can be provided by another system, like a publisher; learners can be provided by, for instance an association or a company; certificates can be provided by other educational institutions; financial means by the government or a company.

Finally, the eLearning system needs financial means and infrastructure (network facilities, buildings, etc.) in order to operate.

Because an eLearning system is a kind of educational system, which in turn is a kind of organization, it inherits characteristics of organizations and educational systems, e.g. its




classification of subsystems (see Daft, 2000). In eLearning systems we can distinguish the following subsystems:1. The production subsystem is responsible for the transformation process of the input to the

output. In eLearning systems the primary means of the production system are the so-called (network-based) learning technologies. Learning technologies are the specific means that establish the functionality of the eLearning system, i.e. to develop, deliver and evaluate learning opportunities for learners in certain contexts and knowledge domains.

2. The adaptation subsystem is responsible for the innovation and change of the production system in order to better fit the environmental constraints and opportunities. A major instrument for the innovation and change of educational systems is to change the learning technologies within the production subsystem.

3. The maintenance subsystem is responsible for the status quo of the system, including the training and support of staff.

4. The input boundary spanning subsystem communicates directly with the input-side of the environment to attract and enrol students, to buy resources, to communicate with the financial bodies, etcetera.

5. The output boundary spanning subsystem communicates directly to the outside world to deliver the products and services like certificates, graduates, and learning artefacts.

The most significant difference of eLearning systems, as compared to regular educational systems, is that they use a different kind of production subsystem. Instead of classrooms there are computer networks that connect learners, teachers and learning artefacts. This provides a complete new set of possibilities and constraints in the relationship with the outside world, e.g. freedom of place and time, the possibilities to automate parts of the teaching-learning process, the possibility to simulate parts of the learning environment and the possibility to renew the pedagogical models in a complete different way (see Koper, 2001).

For institutions wanting to change from a traditional educational organization to an organization that includes eLearning the adaptation subsystem is important as well, as it should solve the questions of how to change the technology, management, collaborations, etcetera. Although these are interesting questions, they are not specific to eLearning systems: every organization has to deal with them.

eLearning system can be studied at different levels of analysis: at the level of individuals, groups, organizations, and of networks (of organizations and individuals). Because of the inherent network characteristics of the ICT technologies used, we tend to prefer the higher-level analysis: at the level of organizations or even better, at the network level. Furthermore, each of these levels can be studied from a micro, meso, or macro perspective. At the micro level one looks at the function of the smaller parts within the system, at the macro level one looks at the overall functionality of the system in relationship with the environment, e.g. the effectiveness, efficiency, attractiveness, accessibility and adaptability. We have a preference for the meso level of analysis that combines both perspectives by looking at the macro phenomena as emergent behaviours that comes from the activities of the subsystems at the micro level (see e.g. Prietula, Carley & Gasser, 1998). This stance is elaborated in the complexity theory (see Waldrop, 1992), the study of emergence (e.g. Johnson, 2001), self-organization or autopoietic theory (Varela, Thompson and Rosch, 1991; Maturana and Varela, 1992), pattern analysis (Gamma et al, 1995; Fowler, 1997; Larman, 2002) and technological approaches as peer-to-peer systems (Barkai, 2002), multi-agent approaches (e.g. Axelrod, 1997; Ferber, 1998) and the Grid (Foster, Kesselman and Tuecke, 2001).




5 Dimensions in the eLearning Domain

In the previous chapter we introduced the structure and dynamics of the eLearning system; now we will focus on the different characteristics of an eLearning system in order to allow further understanding, comparison and design of eLearning systems. Three dimensions can be distinguished:1. The functional dimension, that deals with the pedagogical and knowledge issues. The

functional dimension is highly related to the production subsystem: the transformation process of input into output.

2. The organizational dimension, that deals with the structural characteristics, the contextual characteristics, the economical (or business) characteristics and legal issues (intellectual property rights, licences for learning artefacts, software and documentation, etc.).

3. The technical dimension, that deals with the architectural aspects, the interoperability protocols and standards, the network infrastructure, the servers and applications, and the user interfaces, etc.

These three dimensions are highly interrelated: certain functional requirements implicate specific organizational and technological solutions. The three dimensions apply to the system as a whole, and to its subsystems (that are also systems, seen from another level of analysis). Of course, the functional dimension is the most important and leading one. The others offer possibilities and constraints for the implementation of the functionality.

First we will further elaborate the dimensions independently, and subsequently will focus on some issues related to all of them.

5.1 The functional dimensionThe production subsystem of eLearning systems differs most from traditional educational systems as is stated before. We will now look deeper into this production system to identify what different subsystems it comprises (figure 3). For the modelling of systems and subsystems we will use the Unified Modeling Language (UML), an open standard for modelling (OMG; Booch, Rumbaugh, & Jacobson, 1999; Warmer, 2001).




Figure 3. The subsystems of the production system.

In the production subsystem we distinguish three core processes and two data stores:1. The development process. In this process learning artefacts are created and adapted. It

retrieves existing learning artefacts (e.g. learning objects, activities or units of learning) from the repository. The process can be set and called from outside (e.g. from the management subsystem).

2. The repository is a data store for learning artefacts. It can import and export learning artefacts from other systems.

3. The learning process (including learning support). In this process the actual learning and teaching process takes place in runtime. It can be set and called from outside (e.g. before or during learning by a learner, teacher, or management subsystem). It retrieves existing learning artefacts (e.g. activities or units of learning) from the repository and can set and call the development process during learning to create and store new learning artefacts. Besides access to the repository, the learning sub process also has access to the dossier to retrieve and store properties.

4. The dossier is a data store for the properties of individual users, groups and roles that are local or global in scope. The dossier can import/export properties from outside.

5. The assessment process. This process can be set and called from the learning subsystem at different moments: before learning, during learning and after learning. It uses properties from the dossier and learning artefacts from the repository. It stores new (or aggregated) properties in the dossier, representing e.g. the learners position, grades, etc.

For each of these subsystems and combinations of them, different issues can be identified, again perceived from the functional, organizational and technological dimension. More complex issues are related to the relationships between the subsystems.




5.2 The organizational dimensionAny organization has structural characteristics and contextual characteristics. Table I sums them up (from Daft, 2000). The structural characteristics are the internal characteristics that allow the measurement and comparison of different organizations (see scale of Marcic, 1996). The contextual characteristics describe the complete organization in its relationship with the environment.

Structural Characteristics Contextual Characteristics1. formalization 1. targets & strategy2. specialization 2. environment3. standardisation of work process 3. size4. hiërarchy 4. technology type (= education)5. complexity 5. culture6. centralization7. professionalism8. personnel ratios

Table 1. Structural and contextual characteristics of an organization.

In organization theory several patterns have been derived from the comparison of the characteristics of organizations. For instance, certain contextual characteristics are correlated to certain structural characteristics, e.g. a large size, routine technology and a stable environment is correlated to high formalization, standardization, specialization and centralization. How these characteristics are related in eLearning systems - what the optimal characteristics are for the design of an eLearning system and how systems can change from one to the other - is still to be answered for the eLearning field.

Besides the structural and contextual characteristics, other issues are of importance in this field: The economical and business characteristics of an organization. What business model is

suitable for eLearning? What economical models can be used to invoke successful use, collaboration, sharing and reuse within eLearning systems?

The legal issues like intellectual property rights and licences. Copyright issues introduce a lot of unwanted constraints for eLearning systems (see e.g. Nayyer, 2002). In the context of licences for software and documentation a debate is going on about the openness of the licences and the related business models (open source software, (e.g. Wheeler, 2002). Specifically in eLearning there are issues with use, reuse, adaptation, aggregation and redistribution of learning artefacts; issues of open source for software and licences for documentation.

5.3 The technical dimensionAlso in the technical dimension we can identify a large number of issues, of which a selected list of more important issues is the following: Reference architectures for eLearning. There are several initiatives worldwide to define

reference architectures for eLearning. E.g. the IEEE LTSC (LTSC WG1) defined a preliminary architecture. More recent initiatives are the MIT (OKI, see Livingston Vale, and Long, 2003); IMS will start working on architectures (imsglobal.org), commercial parties like SUN (Sim, 2002), the eLearning Industry Group in Europe and the Valkenburg Group are defining architectures. There is evidently a big need for interoperable reference architectures in eLearning. At the moment however, there are too many; they are still incomplete; and a lot of underlying protocols and standards are still missing. We cannot expect any concrete worked out




architecture to be standardized in the near future. However it is good to sketch and update the architecture at this moment in time to see how the lower-level issues fit together.

The interoperability specification, protocols and standards. Besides specifications and protocols from the general technology and network field, there are specific open specification and standards under development for the eLearning field. Interoperability specifications are crucial for the establishment of larger interconnected networks and collaborations like learning networks and the GRID. There are several successful initiatives releasing specifications at a regular basis in the eLearning field. Examples are the AICC, IMS, IEEE LTSC and ISO SC36. The most important initiative up to date is IMS, a consortium of eLearning vendors, universities and training departments. The current status however is that there are still lots of specifications missing in this field and that others have to be harmonized.

The network infrastructures, servers and applications. Having reference architectures is one thing, having the actual implementations in terms of network facilities, servers and applications is another. Most of this work, except from several specific services and applications, is not driven by eLearning, but are more generic facilities for all types of application fields. The major problems in this domain are the lack of standardisation and the lack of valid and accepted ideas about eLearning requirements and specifications.

The user interfaces for eLearning applications. The user-interfaces are of specific importance in eLearning. The requirements are different from most business applications (like word processors), because these interfaces are the primary means for the realisation of learning.

6 Issues for the improvement of eLearning systems

In this chapter we will identify the key issues that relate to the functionality, organization and technologies of the production subsystem of eLearning systems. As stated above, the functional dimension is leading and will provide for the organization of the issues. The following key issues can be identified to support and improve eLearning systems in a situation of permanent change where the production subsystem and underlying learning technologies must support:1. The development of learning artefacts, by providers, teachers and learners, within or

between organizations, in the most effective, efficient, attractive, accessible and adaptable way. This issue is related to the development subsystem.

2. The sharing and reuse of learning artefacts between providers, teachers and learners, within or between organizations, in the most effective, efficient, attractive and accessible and adaptable way. This issue is related to the repository.

3. The differentiated delivery and use of learning artefacts by different target groups and individuals in the most effective, efficient, attractive and accessible and adaptable way. This issue is related to the learning process subsystem and the dossier.

4. The assessment of the position and accomplishments of learners and others in the most effective, efficient, attractive and accessible and adaptable way. This issue is related to the assessment subsystem and the dossier.

Each of these four key issues will now be elaborated in the following sections.

6.1 Development issuesIn eLearning systems, one has to develop digital learning artefacts such as units of learning (courses, programmes) and learning objects. A major problem for organizations when introducing eLearning is the question how to deal with this development process of digital




learning artefacts. The development process is complex and expensive. Technology provides a lot of new possibilities and constraints and there are several major copyright and interoperability issues. Possibly the most challenging issue is, that in eLearning the expected quality of the units of learning cannot easily be provided by the one person who is traditionally responsible for this job: the teacher. Multidisciplinary teamwork is one approach to create units of learning at an acceptable level of quality, taking into account issues of interactivity, personalization, use of multimedia, granularisation of the units, and coding of content in standard formats (e.g. xhtml, jpeg, mathml, smile, etc.).

Another problem is the question of what to develop. What must be developed, in what format and in what level of detail, in order to provide for a (re-) usable unit of learning that can be delivered through a computer facility? What is needed to answer these questions is an integrative model that describes the semantic components and the relationships between these components. In recent years we have worked on this model, which has been published as the 'Educational Modelling Language' (Koper et al, 2000). EML has provided the basis for the IMS Learning Design specification (IMSLD, 2002). This model describes the semantic structure of a unit of learning in an abstract and generic way. It is based on a pedagogical meta-model or better, an abstraction of instructional design theories and models (see Koper, 2001). This abstraction can be summarized as follows: in a designed learning situation, learners and staff members are engaged in one or more activities that are dynamically related in a learning design method. Every activity is performed in an environment that consists of a collection of one or more learning objects and services. Specific instructional designs (c.q. learning designs) are specialisations of this abstract model. These specific models prescribe the preferred type of activities, the preferred method and the preferred environment (see e.g. Duffy & Cunningham, 1996, p. 171). An actual unit of learning uses an instance of such a specialized model.

Figure 4 presents a summary of the abstract model. Besides the relationship discussed before, it states that a unit of learning contains a ‘learning design’ that organizes physical resources in a semantic instructional framework. The learning design method connects the learners and staff members to activities and environments.




Figure 4. The abstract structure of a unit of learning, expressed as an UML class diagram.

This model can be used in a variety of ways: it defines the semantic components and the relationships between the components that have to be developed in the development process. It serves as a framework to identify the validity of a unit of learning in terms of its structure and completeness. It can be used to compare specific instructional design approaches in research settings. Furthermore the model can be used to create XML bindings to design and express concrete courses and programmes in a format that can be interpreted with a computer programme. Example applications of this model are EML and the IMS Learning Design. The last problem to be discussed in this paragraph related to development, is the issue of the tools needed to support the development process. In a complete development process one needs tools to design, edit, search, store, manage and test the units of learning and the underlying learning resources. In eLearning there are still many practical problems with the development tools, at the level of availability, usability and interoperability. Simply stated: the real interoperable tools are still to be developed. All current solutions are suboptimal. A group of R&D experts, users and vendors from all around the world have recently identified this problem as a major issue and have set up a group, called the Valkenburg Group, to discuss and solve this issues. One of the basic activities is to provide an open, generic architecture for eLearning development tools, based on open standards.

6.2 Sharing & reuse issuesMaybe the most promising, but also one of the most complex, advantages of eLearning is that it provides the possibility of the sharing and reuse of learning artefacts, like learning objects,




activities and learning designs. This will increase the economic benefit of eLearning, like it is expressed in the so-called 'learning object economy' principles (e.g. Campbell, 2003). However, there are several issues to be resolved before sharing and reuse will be feasible.

The first issue addresses the question of which types of learning artefacts are reusable and which types aren't? The current idea is that the units of learning themselves are the least reusable, while the smaller underlying non-purposed resources are the most reusable (e.g. Downes, 2000). Besides the resources, we expect that also the activities, the environments and the learning designs can be reused in different units of learning.

The second issue is about the granularisation of learning artefacts (e.g. Wiley, 2002; Duncan, 2003). How small or how large must a learning artefact be, in order to make it suitable for reuse? Is it necessary to fix the granularity in a learning object economy (e.g. number of study hours), or can it be variable. For what purpose and which actors are the learning artefacts available: for authors to develop new units of learning or for students to be studied directly or both? For students to use, the smallest meaningful unit is a learning activity that includes the necessary environment and the connected support activities. Also developers can easily build new units of learning from these activities, by repurposing and sequencing them into an instructional method. However at the lower level, within the environment, are the learning objects that can be reused in different learning activities, but only after they have been repurposed by a developer. They are not reusable as such.

The third issue is the problem of aggregation and repurposing. How can lower level resources and artefacts be aggregated to activities and activities to units of learning? Are there rules and principles that can support the automation of this process, so that the task is made easier?

The fourth issue is the problem of disaggregation of existing units of learning, to prepare lower level reusable learning artefacts. In most institutes there is a large amount of existing materials that are not prepared or suitable for eLearning or for reuse. How to deal with these materials? From an economical point of view it is not suitable to develop everything new and again for eLearning. Connected problems are the granularity, formats, conversion, and copyrights.

Last but not least there is the issue of finding and sharing the learning artefact for reuse. Presumed is a shared, large, distributed repository where users can search for learning artefacts, get them, adapt them, store new ones and where legal and economic principles are supported in a workable way. Such a repository functions in the context of what is called a learning object economy (Campbell, 2003). The principles for success for such an economy are not well known yet. Campbell identifies the following issues in a learning object economy: the granularity, interoperability, resource description and discovery, incentives, quality control and peer review, intellectual property rights and digital rights management, pedagogical frameworks and cultural barriers. There are already some sharing initiatives put into practice, like Ariadne (Forte et al, 1997), Cuber (Krämer, 2000) and Merlot (merlot.org), but overall evaluation data of the success and failure factors for the approaches are still missing.

In order to facilitate the reuse issue, it is necessary to identify different layers of aggregation of learning artefacts. Every layer has its own reusability, but with different restrictions. We propose the following layers in the context of IMS Learning Design (Table 2).

6.2.1.1.1 Unit of learning6 learning design method5 activity structure (sequence, selection)4 activity3 environment (organization of learning objects and services)2 learning object/service 1 physical resources




Table 2. Aggregation layers within a unit of learning in the perspective of sharing and reuse (1 is lowest level of aggregation and 6 highest level)

6.3 Differentiated delivery issuesOne of the possible advantages of eLearning is the ability to provide for differentiation in the delivery and use of learning artefacts. The usage can be varied according to factors like:1. Delivery medium, like the web, print, dvd, …2. Characteristics and preferences of learners, like the pre-knowledge, learning style, needs,

personal circumstances, disabilities, …3. Context characteristics, e.g. integrated into the work environment, or in classroom teaching,

or completely through distance teaching.4. The quantity and quality of the educational services offered. E.g. the amount of tutoring and

assessment services can vary per situation.

At this moment smooth differentiation is still one of the future promises of eLearning. The basic mechanisms are available, but the integrated models, tools, and guidelines for use are still missing. It is also unknown what the cost-benefit balance will look like. Does differentiated use and delivery increase the effectiveness and efficiency of the educational system? Does it attract new target groups? Does it make education more accessible to certain groups? Some further complications in differentiation are the following:1. Who should control the differentiation under which conditions: the learner, the teacher, an

intelligent agent, the developer or a mixture of these? Another approach would be to look into self-organization mechanisms.

2. There is a need for a tight integration of development and delivery mechanisms. For example, students should be able to use the development environment to add or adapt learning artefacts or to create personalized learning routes. Also designers must be able to adapt learning artefacts and it must be possible to easily integrate the learning artefacts that are created in runtime (e.g. during collaborative learning) in new units of learning.

3. From the behavioural point of view there is a principal difference between the design of learning activities and the actual learning activities that are performed by the students in runtime. A design always provides an abstracted description of the expected learning activities, but different learners can interpret the activity descriptions differently; circumstances can force the learner to act differently; or learners can decide themselves to follow other ways than the ones prescribed. In the EU 5th framework RTD project Alfanet, some of the eLearnTN partners are looking at ways how to close the gap between the learning design and runtime activities by using agent technologies.

4. Differentiation implies several things that are not very well known yet. It implies for instance the availability of global dossiers for learners, where the preferences and characteristics are stored independent of the units of learning. These dossiers must be accessible from a variety of different implementations. The IMS Learner Information Package (IMSLIP, 2002) in combination with the IMS Learning Design property mechanisms provide the first mechanisms for this, but are not well integrated or tested yet. This also implies the problem of the positioning of learners in a kind of global competency grid.

5. The current user interfaces of computers put a lot of constraints on the representation design of learning artefacts.

Differentiation, or personalisation, seems to be an important improvement, but the work has to be taken up further in research and development initiatives before it can be put into real practice.




6.4 Assessment issuesAssessment in eLearning has the regular educational purposes: feedback, monitoring, tracking, quality rating and overall evaluation of the effectiveness of the educational system. In eLearning there are still various problems in this area, not only because of security issues, but more in general because of the repositioning of the function of assessment in modern education. The field is evolving rather rapidly. New, alternative assessment methods are developed and applied (see e.g. Hambilton, 1996; Sluijsmans, 2002) and also new problems occur in eLearning environments, specifically the problem of learner positioning in learning networks: what is the current state of knowledge of the learner relative to the learning opportunities provided. This is needed to allow for differentiated delivery (personalization).

In most current eLearning systems the traditional tests are implemented, but the newer once aren’t. Also the only open specification available in the assessment field, the IMS Question and Test Interoperability (IMSQTI, 2002), restricts itself to traditional tests. It specifies the interoperability format at a rather technical level, but doesn’t connect this to the functional use of a test within the context of a unit of learning. What is missing is an integrated model specifying what assessment method should be applied under what conditions in a very specific but semantic way. In order to provide for such a model OUNL did some preliminary work (Vermetten, Daniëls, Ruijs, Schlusmans & Koper, in press) in the form of defining a matrix of assessment methods, considering five basic questions: what is assessed, how is it assessed, why is it assessed, when is it assessed, and who assesses (e.g. Brown & Glasner, 1999; Van der Vleuten & Driessen, 2000). These questions can be applied to any assessment form and map all components in a systematic way. Secondly, a semantic domain model was created, specifying all types of interactions with the learners within the context of a unit of learning (Hermans, Van den Berg, Vogten, Brouns & Verhooren, 2002). Such integrative frameworks are needed to specify the assessment requirements for eLearning systems in a later stage (figure 5).




Figure 5. (part of the) semantic domain model specifying interactions types

A next step will be to tackle the problem of learner positioning in learning networks.

7 An eLearning network

7.1 Network-level analysisThe problem at hand as defined for eLearnTN is ‘the use of e-learning in the context of higher distance teaching institutions, in support ofn the European knowledge society’. Under the 6 th Framework Programme, tackling this problem from the perspective of the eLearnTN partners implies cooperation at a European level.

Cooperation between institutions for higher distance education can take various forms. In terms of the eLearning system as described in chapter 4, these forms differ in the organization of the sub-systems. At the one extreme we find a cooperation-form in which the partner institutions together establish a completely new organization with its own sub-systems for the production process, input boundary spanning process, output-boundary spanning process, management process, maintenance process, and adaptation process. The other extreme cooperation format comprises a ‘virtual organization’ of networked institutions where only the interfaces between the virtual organization and the separate institutions is defined.




In such a collaborative network, the eLearning network is a system in its own right, with all the participating institutions as outside actors. These institutions will provide inputs (staff, students, learning material and potentially more) to the eLearning network.

This leads to figure 6, where the eLearnTN network is shown as the system, with the individual partner institutions as outside entities.

Figure 6. eLearning network

Note that, from the point of view of the eLearning network, the institutions are black boxes, with a well-defined interface to the eLearning network. How individual institutes are organized internally is not relevant, as long as they provide the defined interfaces to the eLearning network.

7.2 Implications for the other work packages

For practical, political and economical reasons the preferred format for cooperation between the eLearnTN partners will tend towards the ‘network’ approach, as this allows for maximum autonomy within the partner institutions, but at the same time allows sharing and exchange of resources: as long as the latter conform to the defined interface requirements!

In further refining this domain description under WP-2, WP-3 and WP-4, it is therefore necessary to be more specific about the required functionality of the future eLearnTN system: what is in scope and what is out of scope at the network level; what is in scope and what is out of scope at the institutional level; and what are the interfacing requirements between these two levels in terms of the sub-systems, dimensions and issues described in the previous chapters.

A first effort in defining relevant issues was made with the submission of ‘top-10 problems’ by the partners for each of the three dimensions: functional, technological and organizational (see annex 1). Once the functionality and the required interfaces of the eLearnTN network are defined, it will be possible to validate, further expand, detail etc. these top-10 problems and to




translate them into more detailed sub-domain architectures. These will then function as the basis for clarifying the state-of-the-art and deriving the eLearnTN RTD roadmap.

8 In conclusionIn this deliverable we presented an integrative domain model for eLearning in a network of European higher distance teaching institutions. The model basically states that eLearning issues must be approached at both the organizational and network level of analysis. eLearning systems are approached as complex, dynamic, non-linear, evolving and adaptive systems. The model deals with two aspects of eLearning systems: the subsystems (adaptation, production, management, maintenance, boundary spanning) and the dimensions (functional, organizational and technical).

The most critical part for the effectiveness of an eLearning system is the production subsystem. This subsystem typically uses network-based learning technologies to function and contains five interrelated sub-subsystems (development, repository, learning process, dossier and evaluation). For each of the sub-subsystems of the production system several issues for future improvement where addressed from the functional, organizational and technical perspective. It should be noted that there are also issues at the level of the other subsystems, e.g. how to change from the current state into a future eLearning organization and the consequences for management, maintenance, and boundary spanning subsystems. These issues where not dealt with here.

The framework presented here provides the basic terminology, definitions and some of the models. It can be used in a variety of ways, e.g.: To identify future research and development questions in the field. To build better eLearning systems by integrating requirements from the issues discussed in

the model. To be able to identify and plan the implementation issues when trying to use new learning

technologies to change, adapt or innovate an educational system. To be able to identify the different issues a eLearning experts should be acquainted with, for

example to build eLearning courses and curricula.

For the eLearnTN consortium, all four purposes are relevant. Deciding on the functionality at the level of individual partner institutions and the eLearnTN network level, and the interfacing requirements for cooperation between these two levels, provides a more focused approach to establishing the state-of-the-art and the future RTD roadmap. The major challenge for the next work packages will be to acknowledge the integrative relationships that are responsible for the overall effectiveness, efficiency, attractiveness, accessibility and adaptability of such an eLearning network.

ReferencesAxelrod, R. (1997). The Complexity of Cooperation: Agent-Based Models of

Competition and Collaboration. Princeton, NJ: Princeton Press. Banathy, B.H. (1996). Systems Inquiry and its application in education. In D.H.

Jonassen, Handbook of Research for Educational Communications and Technology (pp. 74-92). New York: Macmillan.

Barkai, D. (2002). Peer-to-Peer Computing: technologies for sharing and collaborating on the net. Santa Clara: Intel Press.




Booch, G., Rumbaugh, J, & Jacobson, I. (1999). The Unified Modeling Language User Guide. Reading: Addison-Wesley.

Brown, S. & Glasner, A.(1999). Assessment Matters in Higher Education (Suffolk, St Edmindsbury Press, SRHE & OU).

Campbell, L.M. (2003). Engaging with the Learning Object Economy. In A. Littlejohn (Ed.), Reusing Online Resources: A Sustainable Approach to eLearning (pp. xx-xx.). London: Kogan Page.

Daft, R.L. (2000). Organization theory and design (7th ed). Mason: South-Western College Publishing.

Downes, S. (2000). Learning Objects. Alberta: Academic Technologies for Learning. [avaibable online: http://www.atl.ualberta.ca/downes/naweb/Learning_Objects.doc]

Duffy, Th.M., & Cunningham, D.J. (1996). Constructivism: implications for the design and delivery of instruction. In D.H. Jonassen, Handbook of Research for Educational Communications and Technology (pp. 170-198). New York: Macmillan.

Duncan, C. (2003). Granularisation. In A. Littlejohn (Ed.), Reusing Online Resources: A Sustainable Approach to eLearning (pp. xx-xx.). London: Kogan Page.

Ferber, J. (1998). Multi-agent Systems. Reading: Addison-Wesley.Forte, E., Wentland-Forte, M., & Duval, E. (1997). The ARIADNE project (part I and

II): Knowledge Pools for Computer Based & Telematics Supported Classical, Open & Distance Education. European Journal of Engineering Education, 22 (1/2), 61-74 (part I) en 153-166 (part II).

Foster, I., Kesselman, C., & Tuecke, S. (2001). The Anatomy of the Grid: enabling scalable virtual organizations. To be published in International Journal of Supercomputer Applications. [available online: http://www.globus.org/research/papers/anatomy.pdf]

Fowler, M. (1997). Analysis Patterns: reusable object models. Boston: Addison-Wesley.

Gamma, E., Helm, R. Johnson, R. & Vlissides, J. (1995). Design Patterns: elements of reusable object-oriented software. Boston: Addison-Wesley.

Hambleton, R.K. (1996) Advances in assessment models, methods and practices. In D.C. Berliner en R.C. Calfee (Eds.), Handbook of educational psychology (pp. 899-925) (New York, NJ: MacMillan).

Hermans, H., Van den Berg, B., Vogten, H., Brouns, F, & Verhooren, M. (2002). Modelling test-interactions. Research Report. Heerlen: Open University of the Netherlands.

Hoogveld, A.W.M., Paas, F., Jochems, W.M.G., & Van Merriënboer, J.J.G. (2001). The effects of a web-based training in an instructional systems desing approach on teachers' instructional design behavior. Computers in Human Behavior, 17, 363-371.

IMSLD (2002). IMS Learning Design. Information Model, Best Practice and Implementation Guide, Binding document, Schemas. [available online at: http://imsglobal.org].

IMSLIP (2002). IMS Learner Information Package. Information Model, Best Practice and Implementation Guide, Binding document, Schemas. [available online at: http://imsglobal.org].

IMSQTI (2002). IMS Question and Test Interoperability. Information Model, Best Practice and Implementation Guide, Binding document, Schemas. [available online at: http://imsglobal.org].

Johnson, S. (2001). Emergence. New York: Scribner.


http://imsglobal.org/





Koper et al (2000). Educational Modelling Language Reference Manual. Heerlen: Open University of the Netherlands. [available online at: http://eml.ou.nl]

Koper, E.J.R. (2001). Modelling Units of Study from a Pedagogical Perspective: the pedagogical metamodel behind EML. Heerlen: Open Universiteit Nederland. [available online: http://eml.ou.nl/introduction/docs/ped-metamodel.pdf]

Koper, E.J.R. (2001). Van verandering naar vernieuwing [From change to renewal]. In P. Schramade (Ed.). Handboek Effectief Opleiden, 26 (pp. 45-86). Den Haag: Elsevier. [available online at http://eml.ou.nl]

Koper, E.J.R. (2003). Combining reusable learning resources and services to pedagogical purposeful units of learning. In A. Littlejohn (Ed.), Reusing Online Resources: A Sustainable Approach to eLearning (pp. xx-xx.). London: Kogan Page.

Krämer, B.J. (2000). Forming a Federated Virtual University through Course Broker Middleware. Paper presented at LearnTech2000. Karlsruhe. [available online: http://www.cuber.net/web/html/publications.html]

Larman, C. (2002). Applying UML and Patterns sec.ed. Upper Saddle River: Prentice Hall.

Liber,O., Olivier,B., & Britain,S. (2000). The TOOMOL Project: supporting a personalised and conversational approach to learning. Computers and Education, 34, 327-333.

Livingston Vale, K. & Long, P.D. (2003). Models for Open Learning. In A. Littlejohn (Ed.), Reusing Online Resources: A Sustainable Approach to eLearning (pp. xx-xx.). London: Kogan Page

LTSC WG1. IEEE Architecture and Reference Model Working Group P1484.1: http://ltsc.ieee.org/wg1/

Marcic, D. (1996). Scale for the measurement of the dimensions of organizations. Refered in R.L. Daft (2000). Organization theory and design (7th ed). Mason: South-Western College Publishing.

Maturana, H., & Varela, F.J. (1992). The Tree of Knowledge: The Biological Roots of Human Understanding. Rev.Ed. Boston: Shambhala/New Science Press.

Mayer, R.E. (1992). Thinking, problem solving, cognition, 2d ed. New York: Freeman.Nayyer, K. (2002). Globalization of Information: Intellectual Property Law

Implications. First Monday (online journal). [available online: http://www.firstmonday.dk/issues/issue7_1/nayyer/index.html]

OMG. UML Specification, version 1.4. [available online: http://www.omg.org/technology/documents/formal/uml.htm]

Prietula, M.J., Carley, K.M., & Gasser, L. (1998). A computational approach to organizations and organizing. In M.J. Prietula, K.M. Carley, & L. Gasser, Simulating Organizations, (pp. 13-19).Cambridge: MIT Press.

Sharp, H.et al (1996-1999). Pedagogical Patterns – successes in teaching object technology. [see: http://www-lifia.info.unlp.edu.ar/ppp/public.htm]

Shuell, Th.J. (1988). The role of the student in learning from instruction, Contemporary Educational Psychology, 13, 276-295.

Shuell, Th.J. (1993). Towards an integrated theory of teaching and learning. Educational Psychologist, 28, 291-311.

Sim, S. (2002). eLearning Reference Architecture. SUN.Sluijsmans, D. (2002). Student involvement in assessment: the training of peer

assessment skills [thesis]. Heerlen: Open Universiteit Nederland.


http://eml.ou.nl/



Stolovitch, H.D., & Keeps, E.J. (Eds.) (1999). Handbook of Human Performance Technology. San Francisco: Jossey-Bass Publishers.

Van der Vleuten, C.P.M., & Driessen, E.W. (2000). Toetsen in probleemgestuurd onderwijs [Assessment in problem-based education]. Groningen: Wolters Noordhoff.

Varela, F.J., Thompson, E., & Rosch, E. (1991). The Embodied Mind: Cognitive Science and Human Experience. Cambridge: MIT Press.

Vermetten, Y., Daniëls, J., Ruijs, L., Schlusmans, K., & Koper, E.J.R. (submitted journal article). Justifiable choices for adequate assessment in higher education: analysis in terms of basic questions and quality criteria.

Waldrop, M. (1992). Complexity: The Emerging Science at the Edge of Chaos. New York: Simon & Schuster.

Warmer, J. & Kleppe, A. (2001) Praktisch UML [practical UML]. Amsterdam: Addison-Wesley.

Wheeler, D.A. (2002). Why Open Source Software / Free Software (OSS/FS)? Look at the numbers! Available online at: http://www.dwheeler.com/oss_fs_why.html

Wiley, D. (Ed.) (2002). The Instructional Use of Learning Objects. [available online: http://www.reusability.org/read/].


Conceptual framework for Europe's future knowledge services

Documents

Transcript of Conceptual framework for Europe's future knowledge services