Coalescing individual and collaborative learning to model user linguistic competences

User Model User-Adap Inter (2006) 16:349–376DOI 10.1007/s11257-006-9014-5

O R I G I NA L PA P E R

Coalescing individual and collaborative learningto model user linguistic competences

Timothy Read · Beatriz Barros · Elena Bárcena ·Jesús Pancorbo

Received: 4 October 2005 / Accepted: 9 May 2006 / Published online: 9 September 2006© Springer Science+Business Media B.V. 2006

Abstract A linguistic, pedagogic and technological framework for an ICALL systemcalled COPPER is presented here, where individual and collaborative learning arecombined within a constructivist approach to facilitate second language learning.Based upon the Common European Framework of Reference for Languages, theability to use language is viewed as one of several cognitive competences that are mo-bilised and modified when individuals communicate. To combine the different types oflearning underlying the European Framework, a student model has been developedfor COPPER that represents linguistic competences in a detailed way, combining highgranularity expert-centric Bayesian networks with multidimensional stereotypes, andis updated following student activities semi-automatically. Instances of this modelare used by an adaptive group formation algorithm that dynamically generates com-municative groups based upon the linguistic capabilities of available students, and acollection of collaborative activity templates. As well as the student model, which is

T. Read (B) · B. BarrosDpto. de Lenguajes y Sistemas Informáticos,Escuela Técnica Superior de Ingeniería Informática,Universidad National de Educatión a Distancia (UNED), C. /Juan de Rosales 16,28040 Madrid, Spaine-mail: [email protected]

B. Barrose-mail: [email protected]

E. BárcenaDpto. Fac. de Filología Extranjeras y sus Lingüísticas,Universidad National de Educación a Distancia (UNED), Paseo Serda del Rey 7,28040 Madrid, Spaine-mail: [email protected]

J. PancorboDpto. Ingeniería Informática,Universidad Antonio de Nebrija,Madrid, Spaine-mail: [email protected]

350 User Model User-Adap Inter (2006) 16:349–376

a representation of individual linguistic knowledge, preferences, etc., there is a groupmodel, which is a representation of how a set of students works together. The resultsof a student’s activity within a group are evaluated by a student monitor, with moreadvanced linguistic competences, thereby sidestepping the difficulties present whenusing NLP techniques to automatically analyse non-restricted linguistic production.The monitor role empowers students and further consolidates what has been previ-ously learnt. Students therefore initially work individually in this framework on certainlinguistic concepts, and subsequently participate in authentic collaborative commu-nicative activities, where their linguistic competences can develop approximately asthey would in ‘real foreign language immersion experiences’.

Keywords ICALL · Bayesian networks · Adaptive group formation · Collaborativeactivity templates · European framework for languages

1 Introduction

In our globalised ever more interconnected world, the need to be able to understandand communicate in languages other than our own native tongue is essential. A smallnumber of people fit naturally here since they were lucky enough to be brought up inbi- or polylingual environments from an early age. The majority of us, however, startto learn a second language (L2 hereafter) later on in life when the demands of ourprofessional lives impose this necessity upon us. It is not therefore surprising that alarge number of adults identify L2 learning as one of their principal intellectual goals.Such an undertaking, unfortunately, even for students in arguably ‘perfect’ languagelearning conditions (immersed in an authentic L2 environment with unlimited accessto native speakers who can exemplify, correct, and provide endless practice), requiresconsiderable time and effort. Furthermore, this type of scenario is ideal and arguablyidealistic. The reality of most people’s L2 development is radically different, since theycannot leave their own community, and the demands of modern life limit the time andopportunities available to attend classes or make use of related L2 resources, such aslistening to radio programmes, watching TV stations, reading, writing, and speaking,all in different modalities.

Another issue regarding L2 learning is that people who have spent similar periodsof time ‘trying to learn a language’ will have different degrees of linguistic compe-tence based upon various factors. Hence, when someone’s L2 competence is exam-ined, what is found is not a unitary measure of ability but heterogeneous capabilitiesthat encompass the different modalities and features of communication, which go farbeyond the traditional formal aspects of language. While the exact underlying cogni-tive and neuropsychological processes involved in communication are still largely amatter for academic debate, certain aspects of communication are directly observable.These underlying capabilities, which are at the core of communication, can be seento involve the functioning of a wide range of general mental mechanisms (e.g., per-ception, memory, transformation), that can be argued therefore to go beyond whatcould be supplied by some ‘monolithic language module’ within our overall mentalarchitecture (cf. Fodor 1983; Ullman 2004). Hence, when language use and learningare conceptualised and their underlying functionality studied, what has been argued(Council of Europe 2001) is that such learning should be undertaken in the contextof an individual’s overall cognitive and communicative dimension. Such a perspective

User Model User-Adap Inter (2006) 16:349–376 351

has had effect on L2 teaching/learning in the language classroom, but has been largelyignored by designers of computer systems intended for L2 learning. Since computersfirst appeared, they have been presented as tools which could play an important role inL2 learning (Levy 1997). The incorporation of techniques from Artificial Intelligencehave lead to the development of a wide range of systems (henceforth, ICALL sys-tems, e.g., Bailin 1995; Chanier 1994; Gamper and Knapp 2001; Holland et al. 1999),although their general role in L2 teaching/learning is still far from clear (or generallyaccepted; Garrett 1995). Such applications have typically tried to focus on the mostformal linguistic aspects of student comprehension, attempting to replace a humannative speaker, to correct erroneous student comprehension and/or production. Fourproblems can be identified with such systems: firstly, a real analysis of free studentlinguistic output goes beyond what can be achieved with natural language process-ing (henceforth, NLP) techniques at the moment, requiring levels of representationbeyond what are available (Harrington 1996). Secondly, L2 tutoring systems oftenplace the emphasis on the formal aspects of the language constructs they are trying toconvey, ignoring the overall communicative context for which they are typically used(Suri and McCoy 1993), thereby limiting the usefulness and applicability of what isbeing learnt. Thirdly, few systems attend to pedagogic theories of how people learn orhow they might learn best (Oxford 1995). Fourthly and finally, as well as technologicalquestions related to L2 learning, there is also a range of issues related to the designand functionality of ICALL systems (Garrett 1995; Salaberry 1996).

In this article a system called COPPER (Collaborative Oral and written languageadaPtive Production EnviRonment) is presented which addresses the four problemsstated above, combining individual learning activities with collaborative group-basedones, in an effective way to facilitate learning, The Common European Frameworkof Reference for Languages (Council of Europe 2001) provides the general linguisticfoundations for COPPER, since it contains the first real thorough analysis of linguisticcompetences from functional, formal, and pedagogic perspectives. In accordance withthis work, the authors undertook the design and development of a student model thatenables linguistic competences to be represented in a fine-grained quantitative way,using a novel three-dimensional relationship to characterize L2 learning. This modelcombines high granularity expert-centric Bayesian networks with multidimensionalstereotypes, and is updated following student activities semi-automatically, includingthe results of human evaluation to complement those of the networks. A group modelis needed as a representation of how students work together in specific activities, to becontrasted with the student model, which is a representation of knowledge, learning,preferences, etc.

A student works with the system in three ways: firstly, individually (in a mechanicalexercise-based sense) to establish prototypical knowledge and rules of the linguisticconcepts for a given learning stage (Read et al. 2002a). The results of the work under-taken by a student are interpreted by Bayesian networks. These update the student’sprogress by establishing and modifying individual learning parameters within thestudent model. Secondly, students participate in ‘authentic’ (Reeves et al. 2002) col-laborative communicative activities, where their linguistic competences can developand improve in an analogous way to how they would in ‘real foreign language immer-sion experiences’. These activities are dynamically generated based upon the linguisticcapabilities of available students at the time and a collection of collaborative activitytemplates, and are examples of structured problem solving tasks including aspects ofnegotiation. The results of these activities take the form of projects, presentations,


and dialogues typically made up of both recorded oral and written material. Thirdlyand finally, in a monitoring role, where more advanced students evaluate these results(and a log of student participation represented in a group model), sidestepping thedifficulties present when using NLP techniques to automatically analyse non-restrictedlinguistic production. The monitor’s evaluation is used by the system to set collabora-tive learning parameters in the student models of the group members. This monitorrole empowers students and further consolidates what has been previously learnt bythem. Furthermore, initial results obtained from COPPER show it to be motivatingfor the students (in the sense of being aware of their own relative progress with respectto others). The overall state of learning is indicated by the combination of individualand collaborative learning parameters within the student model. Once a student hasintensively practised certain linguistic concepts individually, they are generalized andconsolidated by applying them in collaborative activities, and newer more advancedconcepts become available to be subsequently studied individually.

In the following section a brief presentation of the pedagogical framework whichunderlies the way in which individual and collaborative learning are combined inCOPPER is undertaken. Following this, this system’s architecture is presented in termsof its overall functional structure, the ways in which a student can work with it, thestudent and group models, how collaborative activities are defined and instantiated,how the adaptive group formation takes place, and how monitoring, diagnosis andevaluation of group activities are undertaken. Subsequently, COPPER is comparedand contrasted to other work presented in the literature. Finally, some conclusionsare drawn about the current state of the system, some results of early evaluations arediscussed, and directions are presented for future work.

2 Individual and collaborative learning in COPPER

The design and development of COPPER have been undertaken in a pedagogicframework where the combination of individual and collaborative learning is seento be fundamental. The essentially individualist way in which L2 teaching/learningis quite often undertaken (e.g., language academies with large numbers of studentsseated in front of computers, or teachers giving essentially non-participative classes)does not lead on its own to the long term integration of the linguistic knowledgewithin the set of general cognitive competences that are mobilised when we wish tocommunicate. If language is considered as a vehicle for communication in a socialcontext, then its very acquisition requires such contextualization in order to enablestudents ‘to effectively learn’.

Sociocultural theory presents language learning as a developmental process medi-ated by semiotic resources (books, gestures, classroom discourse, etc.; Wertsch 1991),undertaken in socially mediated activities (Lantolf 2000). Vygotsky (1986) stated thathuman psychological nature represents the aggregate of internalised social relationsthat have become functions for the individual. What the inability of L2 students to usetheir linguistic knowledge in real world activities seems to demonstrate is that suchlanguage learning cannot be undertaken alone, in a non-participative instructionalway. Vygotsky (1978) had argued that a child follows the adult’s example and gradu-ally develops the ability to do certain tasks without help or assistance. Krashen (1982)distinguished between an implicit acquisition of language based upon the presenceof comprehensible input (such as that available in social situations) and the explicit


conscious learning (making use of formal study and grammar rules). While such a viewprobably oversimplifies the nature of the mental processes involved in L2 learning, itdoes complement sociocultural theory in the importance of the role of collaborativesocial interaction in L2 learning. Since Vygotsky views individual learning as thenatural result of group activity, it could be argued (following a strict socioculturalmodel) that a student can develop his/her L2 competences directly by undertak-ing collaborative communicative tasks (without individual learning). However, whatpractical (effective) teaching experience shows is that combining individual learningactivities with collaborative group-based ones is an effective way to enhance learning(Cortright et al. 2003, 2005). What a student learns individually can be consolidatedand expanded in collaborative tasks, and what is learned in such communicative activ-ities is incorporated into the individual student model. Furthermore, recent researchon peer scaffolding shows that L2 learners reach a higher level of performance byproviding assistance to one another (Ohta 2000), where different strategies can beapplied (Bull 1994). However, if the students never actually ‘study’ the language, thenbeing able to help becomes somewhat limited and learning takes longer.

Another way to view the relation between individual and collaborative learning is inanoperationalsense,consideringtheseriesofprocessesundertakeninL2learning.Stahl(2000) presents a model of learning that relates the social aspects of this process to thedevelopmentofpersonalunderstanding.Incontrast,Fig.1presentstherelationbetweenindividual and collaborative learning in COPPER. It shows how the need to communi-cate incollaborative learninghasapositive influenceonanindividual’s linguisticknowl-edge, as a result of the need to practice and use such knowledge in a social context.Other approaches such as COMET (Suebnukarn and Haddawy 2006) and COMTEL-LA (Cheng and Vassileva 2006) also combine individual and collaborative modelling.

Initially, individual learning is undertaken between a student and the system, whichupdates the individual learning zone of the student model. Once such learning startsto take place, collaboration becomes possible, which requires a group model to spec-ify the nature of the tasks to be undertaken for the students available and theircorresponding linguistic competence levels. For collaboration to be possible, the stu-dents working together must be capable of reaching mutual understanding. Such anunderstanding requires communication between the task participants which, in turn,requires communicative strategies to be adopted. Strategies can limit as well as facil-itate learning (Faerch and Kasper 1983), for which both the activity structure and theway in which student participation is evaluated need to control them. The applica-tion of these strategies therefore permits collaboration to take place, the results ofwhich update the collaborative learning zone in the student model, reinforce previ-ous individual learning, and trigger further individual study. In the following section,

STUDENT MODEL

Individual learningzone

Collaborativelearning zone

Collaborative learning

Mutualunderstanding

Collaboration Communicationrequires requires requires

permits

Individuallearning

GROUPMODEL

updates updates

reinforces/updates

permitsStrategies (learning,

production,communication)

requires

Fig. 1 The relation between individual and collaborative learning


the student and group models in COPPER are detailed, together with the way inwhich the linguistic domain knowledge is modelled, and how the two different typesof learning take place.

3 The COPPER architecture

The system architecture underlying COPPER together with the learning process areillustrated in Fig. 2. This process is made up of six steps: the system’s didactic planneranalyses (➊ in the Figure) each student model within the student community, andbased upon the progress of each member, either assigns a student an individual activ-ity or a group activity, the selection of which is undertaken in step three. If individuallearning activities are to be undertaken (➋ in the Figure; perhaps because a studentneeds to refresh some previously ‘learned’ concepts that have not been shown to besufficiently internalised in subsequent collaborative activities), the student carries outa reception task. This type of task is managed by the I-PETER II module (see Sect. 6),in which, once the tasks are finished, a Bayesian diagnosis process is used to updatethe individual part of the student model, which represents the state of learning at thecognitive level.

The profiles of the students that have been selected to participate in collabora-tive activities are processed by the adaptive group formation algorithm (➌ in theFigure), which groups students together to participate in ‘authentic’ communicativeactivities. This algorithm, detailed below, analyzes the state of knowledge in eachindividual model, and makes use of libraries of collaborative activity templates, toolsand resources, to dynamically form the groups for each type of activity. Each group isrepresented in a group model (➍ in the Figure), which stores the collaborative learningprocess, together with general data about the group activity and the individual resultsof each member. As the activities are undertaken, they are evaluated (➎ in the Figure).This evaluation is undertaken, for each group, by students at higher knowledge stagesthan those participating in the activity, and makes use of a randomly selected set ofevaluation questions taken from an extensive bank of questions (reflecting aspects

Fig. 2 The COPPER system architecture and learning process


of both formal and functional linguistics, sociolinguistics, discourse, pragmatics, andcollaboration), developed for each knowledge stage. The data obtained from this pro-cess is used in a process of group activity diagnosis (➏ in the Figure) to infer changesin the knowledge stages of each student, updated in the collaborative learning zoneof the student’s model for each group member. This zone of the student model alsostores the activity name, a log of the learning process, the general evaluation of thecollaboration activity and the results generated. These data will be of great interestfor future work regarding the utility of the history of the learning process of eachstudent.

In the following sections, different aspects of this architecture are presented ingreater detail. The student interface is presented firstly, to show how the studentsactually work with COPPER, followed by the structure and role of the student andgroup models. Subsequently, the collaborative activity templates and adaptive groupformation are presented. Finally, the evaluation process is considered together withthe way in which group activities are diagnosed and the student and group modelsupdated.

3.1 The student interface

The exact nature of the student interface to COPPER depends upon whether a studentis working as a ‘language learner’ or a ‘group monitor’. The type of activity (individ-ual or collaborative) assigned to a student depends upon the state of his/her studentmodel. Initially a student works with the system by undertaking individual receptiveactivities, i.e., s/he uses the linguistic skills of reading and listening to learn about thelinguistic functions and related notions covered at his/her current knowledge stage,in terms of the inventories and grammar elements. Evaluation occurs using explicitquestioning (Self 1994) via closed exercises, which are automatically assessed by aBayesian diagnosis process (as can be seen in Fig. 3).

Fig. 3 Evaluation exercise following individual theory section


Fig. 4 An example of COPPER’s collaborative student task interface

As progress is made with these individual activities, newer material becomesavailable and the student is established at a particular knowledge stage. It is justthis heuristic knowledge stage assignment process that the didactic planner uses toindicate to the adaptive group formation algorithm that the students may participatein collaborative activities (appropriate to the conceptual units that have been learntfor their current knowledge stage). Once the Adaptive group formation algorithmassigns a group activity to certain students, a message is sent to each one by the sys-tem, and the corresponding activity appears in his/her group space. In general, eachactivity requires the students to solve a content-based problem in a specified timeby working collaboratively to achieve a solution, which can be subsequently analy-sed and justified by all involved. While the students are not explicitly obliged to uselanguage from a given knowledge stage, the tasks are designed so that they enablespecific constructions and vocabulary to be practised and hence consolidated. Thegeneral student task interface for an activity can be seen in Fig. 4. The screen containsa generic menu on the left hand side of the screen (used to access personal informa-tion, files and resources; use shared group resources and undertake the active tasks;monitor the work of lower level students; and finally, use the available tools), and anactivity specific part, on the right. The activity task structure appears toward the topof the screen, with the active task shown in detail. Specifically, the task descriptionis presented, together with a set of recommendations for this task, and links to theresources available for the task.


Fig. 5 Tool use in collaborative activities

In this example two students need to meet in a train station, one of which hasarrived to meet the other from a train. Subsequently they need to decide where togo to have lunch, and then actually order the food in the restaurant with a third stu-dent playing the role of the waiter. As can be seen in the figure, different tools areavailable to the students for the planning and undertaking of the tasks, the interfacesof which are presented in Fig. 5. The students initially use a threaded VoiceChat andDiscussion tool to structure the different tasks (➊ in Figure), and subsequently pre-pare voice recordings or written comments (➋ and ➌ in Figure). Once the studentsare in agreement with the preparation of each contribution for a given task, the Pre-sentationBuilder (➍ in Figure) is used to combine the interventions into a sequencewhich represents the results of a given task.

The results that the students generate can be accessed in one of two ways, eitherby activating the tool that they used to create them within the task, or by clicking onthe ‘Results generated’ link, which leads them to a summary screen, where evalua-tion information also appears. The interface is stateful, in the sense that a student’scurrent location within an activity, together with the selected user preferences, arestored when the session is finished. Hence, when the student returns to the system,s/he is returned to the same position in which s/he was last working. Furthermore, anychanges that have been produced in the group activity in which the student is currentlyworking are also presented when s/he logs in again. All four tools can be used eitherindividually or collaboratively (synchronously or asynchronously), depending uponthe availability of the students to work together and practical network bandwidthrestrictions. Once the students in a group complete all the tasks that make up anactivity and agree that the activity is complete, it is labelled as such by the system,so that the monitor assigned to assess the work undertaken by this group can inspectwhat has been produced.

3.2 The student and the group models

The student model (summarised in Fig. 6) represents the state of L2 learning togetherwith the profile and history of a student’s individual and collaborative activities.

This model can be divided into four parts:

1. The personal details and linguistic background of the student. This is importantbecause of the way (the order and emphasis of certain aspects) in which thetheoretical material is presented to a student depend to a large extent on thisbackground.


Fig. 6 A summary of the information represented in COPPER’s student model

2. The individual learning activities undertaken with the system. This informationshapes the way in which a student works with the system.

3. The collaborative activities undertaken by a student. This takes the form of ref-erences to instances of group models, which log the roles, tasks, etc., undertaken,and the results.

4. The L2 learning characterisation, specified at a high level of granularity (Readet al. 2002a). This is an instantiation of the three dimensional space presentedin Fig. 7. This characterisation represents a space that contains a set of pointsrepresenting relations between conceptual units at different knowledge stages,which includes the different linguistic levels at which the units can be represented(dimension 1), the linguistic skills that can be used to access and manipulate theconceptual units (dimension 2), and the prototypical linguistic activities, whichdefine the communicative aspects of their use, including the learning phase. Thisreflects the degree of attention being applied by the student (dimension 3).

An important part of the work that has gone into the design and development ofCOPPER, is the way in which the L2 knowledge is structured and represented. This isa problem for any kind of ITS, but is particularly complicated for the representation ofnatural language. This L2 learning characterisation is not an arbitrary simplification ofthe learning process, but is founded on the Common European Framework, the resultsof many years of pedagogic and linguistic analysis by the Council of Europe. Thisframework presents a functional perspective of language learning and use, includingthe underlying ‘traditional’ formal aspects. However, its essentially qualitative naturemakes it impossible to directly use in any language learning system. The authors havetherefore produced a quantitative model that subsumes and expands this framework,thus enabling all the necessary linguistic conceptual and functional knowledge in theL2 domain to be structured. The model takes the form of a three dimensional spacethat characterizes L2 learning (illustrated in Fig. 7). It is divided into two parts: theindividual and collaborative learning zones (shown as A and B). The intersection ofeach dimension does not define a small set of values that portrays learning, as if it were


Mastery

EOP

Vantage

Threshold

Waystage

Breakthrough

Reception / Production

MediationInteraction

Reading Listening Writing Speaking

Mastery

EOP

Vantage

Threshold

Waystage

Breakthrough

Reception /Production

MediationInteraction

Reading Listening Writing Speaking

A B

Individual learning zone Collaborative learning zone

Fig. 7 The three dimensional space (A+B) that characterizes L2 learning in COPPER

a “pure” stereotype student model (Kay 2000). This space is initially empty and thengradually populated by points corresponding to conceptual units, which are relatedacross the learning dimensions, as the students learn and apply them in differentactivities using different skills.

Some new conceptual units can be learned individually (depending upon their rela-tion to the linguistic activities and skills used), and appear as new points within theindividual zone (A in Fig. 7), following a Bayesian diagnosis of the results of evaluation(Read et al. 2002b). Others, those relating to richer collaborative activities (that implya social communication beyond what can be evaluated automatically using tests) areevaluated by other more advanced students working as monitors. Hence, each newpoint that appears in this space corresponds to a relation between conceptual unitscorresponding to the six knowledge stages and the other two dimensions that definethe learning space in three ways (illustrated in Fig. 8):

1. Relations that are modified automatically by the Bayesian networks in the system(excluding speaking and writing undertaken in Iteration and Mediation).

2. Relations that are modified following human evaluation (peer student monitor-ing) of collaborative activities (i.e., writing and speaking undertaken in Iterationand Mediation). Following system modification which results from human evalu-ation, an automatic penalisation mechanism is used to readjust related conceptualunit relations within the individual learning zone, so that the student is forced torevisit more material.

3. Relations that are modified both automatically and following human evaluationdepending on the details of the relation (essentially Production-based activities).

At any given time, a student is not completely ‘at a particular knowledge stage’, in thesense of having all the points within the three dimensional space at the same stage,

Fig. 8 Conceptual relations within the student model


Fig. 9 Attributes of the group model in COPPER

but has the points distributed over adjacent levels, depending on past experience andthe consolidation of the conceptual units using different linguistic skills in differentprototypical linguistic activities. The uniform progression within the space ensuresthat ‘knowledge holes’ do not appear and therefore prevents ‘structural collapse’ ofa student’s linguistic competence (Read et al. 2002a).

As well as the student model used in COPPER, another model is required both tostructure and to coordinate the way in which the students work together, namely thegroup model. This model stores collaborative data and encapsulates the adaptive partof the system, since the group generation is dynamic, based upon a variety of factors,and represents the details of the sets of students working on particular tasks, detailinginteractions, mistakes, etc. This model (shown in Fig. 9) is a single register that charac-terizes the collaborative activity of the group. The data in the model comes partiallyfrom the questions a student-monitor has to answer when the group is evaluated, andpartially from the activity log.

The information contained in this model is useful for two reasons. Firstly, the adap-tive group formation considers the previous group activities of each student whenforming new groups. Secondly, the log of these group models will permit the learningprocess undertaken by each student to be analysed in the future, and will also enablethe overall pedagogic properties of the system to be evaluated, for example, studyingthe collaborative activities in terms of the suitability of the roles assigned and the lin-guistic skills and knowledge stage required. In the future, it is intended that some ofthese indicators will be obtained automatically using interaction analysis algorithms(Barros and Verdejo 2000). However, before this functionality is introduced into thesystem, the authors prefer to refine this prototype with data generated by a humanmonitor. Finally, it should be noted that for evaluation it is interesting to have human


Fig. 10 Comparative structure of student and group model in COPPER

generated indicators in order to compare the results with the indicators generated byan analysis tool.

A comparison of the student and group models can be seen in Fig. 10. These modelshave different functional roles in the system. A student model is instantiated whena student enters the system for the first time, and exists until the student is removed(administratively speaking) from the system. A group model in contrast, once formed,is active and available only during a particular collaborative activity. Once the activityhas finished, the group is dispersed, the group data is logged as part of the ‘history’of the learning process’ (to be able to check if the student has interacted within thegroup), and the group model deactivated.

In the collaborative communicative activities that COPPER contains, the proto-typical linguistic activities are either Interaction or Moderation (as defined in theEuropean framework), and hence the students use all four linguistic competenceswhen they participate. However, the individual work that a student has undertakenwith the system, has established knowledge stage conceptual units, only in terms ofthe reading—and listening—reception conceptual units used. Hence, when a studentstarts to work in collaborative activities, the system automatically transfers a percent-age of these values to the speaking—and writing—production fields of the conceptualunit. Such transfer is included to emulate the way in which both L2 students andchildren learning their first language imitate what is said to them when they attemptto use a word or phrase at the beginning.

3.3 A definition of collaborative activities based on templates and tools

The work undertaken in the groups is organized using collaborative templates (thegeneral structure of which can be seen in Fig. 11), that are stored together in a repos-itory. Such templates define a group activity in terms of a set of tasks, each of which isundertaken by students playing different roles. The templates are instantiated usingthe group formation algorithm (each instance represents a separate group), basedupon the roles that the students will undertake and the related knowledge stage andlinguistic skills required.

An activity is defined in terms of the following characteristics: its system reference,its type and description, the goals to be achieved, the roles to be used, the resourcesand tools available to achieve the goals, the set of tasks to be undertaken (that gofrom Reception to Mediation), and the moderation/evaluation details. All activitieshave an associated sub-language or thematic areas, as identified in the Europeanframework, for example: personal identification, house and home, environment, daily


Fig. 11 Collaborative activity template definition, focusing on the activity_template, tasks and roles

life, free time, entertainment. Figure 13 shows the structure that the template for thepreviously presented example activity of meeting a friend from a train and havingluch. The types of activities possible have been refined from Barkley et al. (2005)and Willis (1996). The goals (and sub-goals) are defined in terms of the roles of thestudents who will try to achieve them, and in general, each goal-role(s) combinationdefines a task. The roles that the students undertake in an activity are defined in termsof both general and activity-specific system criteria. The role type is important forgroup formation, both to establish what a student will do, ensuring different aspectsof a task are addressed, and to prevent a student from repeating the same role in thefuture, if desired. Figure 12 shows examples of roles in COPPER: facilitator, recorder,reporter, etc.

As can be seen in this Figure, each role requires certain linguistic skills to be atspecific knowledge stages (or in a range). It is quite common for a particular activity(in an L2 classroom) to be repeated by students as they change competence levels,since it enables them to implicitly compare and contrast the communicative richnessof each level. The tasks that make up an activity are designed to facilitate differ-ent linguistic skills in order to carry out a set of purposeful actions with a clearlydefined goal and a specific expected outcome. Each task is defined in terms of itsinput, output, its type (including establishing contact, debating, etc.), the social cul-tural knowledge needed to undertake the task (knowledge of the society and cultureof the community in which a language is spoken). Two task examples are provided inFig. 13.


Fig. 12 Example activity template definition

Fig. 13 Two example task definitions

3.4 Adaptive group formation

Once a student has worked with the part of COPPER that facilitates individual learn-ing, the student becomes an active member of the virtual learning community (Wenger1998). While the student is free to carry on with individual learning using the system,what such community membership implies is that the student can now be selected bythe system to work with other students, in collaborative communicative activities. Suchcollaboration both reinforces what s/he has previously learnt, and more significantly,facilitates the general mobilisation of the cognitive mechanisms underlying commu-nication in authentic social contexts, giving rise to the collaborative development ofthe full range of competences used in real communication (which produces changes inthe collaborative learning zone of the student model). However, care has to be takenwhen forming the groups because, as Alfonseca et al. (2006) note, in collaborative


Level_rolen = Level SMk Studentk = Rolen

Level_role1 = Level SM1 Student1 = Role1

ACTIVITY REF, ACTIVITY TYPE, DESCRIPTION, GOALS

ROLE1

Level1

Type_Rolen

TASKa TASKb TASKz

Level_role2 = Level SMn Studentn = Role2

Virtual learningcommunity

Student model SM1

Student model SM2

Student model SMn

Student model SMk

GROUP MODEL

U1UN

UK

Level2 Level3 Leveln

ROLE2 ROLE3 ROLEn

Type_Role3Type_Role2Type_Role1

→

→

→

Fig. 14 Adaptive group formation based upon collaborative templates

learning, the way in which students are grouped together may affect the results of thelearning experience.

The group formation process can be seen in Fig. 14. It is adaptive in the sensethat the algorithm is used to dynamically match the students available at a particularmoment in the virtual community (using the data in each individual student model),with collaborative templates that define group activities, in terms of the tasks thatmake up an activity, the roles to be assigned to the students, and the criteria by whichthe monitor can evaluate both the collaboration process and the results produced. Thealgorithm generates a set of groups and corresponding activities using four sources ofinformation:

(1) The set of student models available in the student community, for students taggedby the didactic planner as being able to participate in collaborative activities. Pre-vious research has demonstrated that just belonging to such a community is initself motivating for the students, and the actual interaction between the stu-dents has been shown to foster greater student participation and collaboration(Warschauer, 1999). It can be seen to be similar to the classroom in that it pro-vides an opportunity for joint work and also enables students to communicateamongst themselves.

(2) The group model log, which records all the previous activities undertaken bygroups in the system. Once a group activity finishes the group is dissolved, butlogged in the system.

(3) The set of tools and resources available in the system. The tools currently avail-able for students in COPPER include a speech recorder, a text-to-speech reader,a (a)synchronous threaded chat/voice messaging tool (that enables studentsto participate in logged threaded chats in real time, or leave recorded mes-sages if the other party is not online at the time), a multimedia document/


presentation builder (that enables students to put together pre-recorded mes-sages and dialogues, with texts and images to produce overall multimedia docu-ments and presentations), a text authoring tool (the tool used here is the VirtualAuthoring Tool [or VAT1], which is a text editor that helps a student to writecorrectly by structuring the linguistic knowledge that is supposed to be used ata particular knowledge stage, in a bottom-up fashion).

(4) A template from the collaborative activity template repository. The collabo-rative templates define the activities and tasks that the students can realise inCOPPER. In essence, each template defines an activity in terms of a set of goalsto be obtained, the roles to be adopted, and the tasks that need to be undertakento fulfil the goals.

The adaptive group formation algorithm is used to check the pool of available stu-dents within the student community, as indicated by the didactic planner. As wasnoted previously, the actual state of learning of each student is not a single highlevel value indicating membership of a particular knowledge stage; hence, the adap-tive group formation algorithm needs to approximate the set of conceptual linguisticunits contained in the student model to produce a generalised knowledge stage value.Therefore, when no match is obtained for a particular role, more specific details of thecurrent state of learning can be consulted to achieve a match. Finally, if no candidatesare available, a student with a higher knowledge stage value for the linguistic skill willbe proposed for the role, provided all other factors remain the same. As well as anactivity to undertake, each group is also assigned a specific period of time in order tocarry out the necessary tasks. Failure to participate causes re-allocation of students,and in the worst case, group dissolution. Before any group assignments are made, theparticipation of each student in previous group activities is retrieved from the groupmodel log, so as to check what types of activities a student has undertaken, and withinthese, what roles have been adopted previously. As can be seen in Fig. 11, a collabo-rative activity template includes information about the type of activities and the rolesthat a student can be undertaken within it. This enables the repetition of activity typeand role to be able to be controlled by the system. Once this data is processed, a shortlist of matches between students and activities are made, and depending upon othersystem configuration data, the groups are instantiated.

3.5 Monitoring, diagnosis, and evaluation of group activities

The monitoring and diagnosis of a group’s work by other students in the learning com-munity is a fundamental part of the approach adopted in COPPER, in that it enablesstudents with higher level knowledge stages to evaluate the work being generatedby lower level students, thereby avoiding the NLP problems present when attempt-ing to automatically analyse such work. Any student with a knowledge stage beyondBreakthrough (i.e., not a beginner) will have three roles in the system, one as a stu-dent working individually with the system, another collaborating in group activities,and a third as a student monitor. The combination of these roles is a crucial peda-gogic principle in this system. Here the evaluation process provides a student with anopportunity to apply what has previously been learnt (and in theory consolidated bydistinct individual and collaborative activities making use of all four linguistic skills),through the evaluation of what other students have done. As such, it really acts as

1 Partially developed with a grant from the Vicerrectorado de Investigación of the UNED.


the ‘litmus test’ for the strength of the representation and integration of the differenttypes of linguistic knowledge present in the knowledge stages of the student model,and any errors present would indicate problems that require the system to weaken theactivation of conceptual units in the student model, thereby forcing the student to goback and repeat some of the previous individual studies and participate in additionalgroup activities to ensure that the underlying linguistic concepts are consolidated. Theevaluation undertaken by a student monitor of a group’s work is in itself evaluatedby a higher level student (or teacher), as a meta-monitor.

In the context of this system, a student is not just encouraged to participate as amonitor but is obliged to do so. Examples of such evaluation can be seen in Fig. 15;firstly, there is a standard evaluation between S1 (student number 1) and G1 (groupnumber 1), where the student evaluates a linguistic production of the group. Thenature of the carefully designed activities at each level together with the classificationof the students makes such evaluation feasible in the majority of cases. Secondly, anexample of a meta-evaluation can be seen between S2, G2 and S3. Automatic collab-oration analysis tools are planned to be incorporated into this process (Jermann et al.2001). Such tools are based upon the previous experience of the authors (Barros andVerdejo 2000) in this area. However, before such analysis in undertaken, it is impor-tant to study the way in which humans undertake this role, in order to completelycharacterise how the analysis can be undertaken.

It is not realistic to expect a student monitor to undertake a long detailed eval-uation of the work of the members of a group, focussing on all types of criteria.Furthermore, not all errors need to be identified, or should be identified, since it hasbeen argued (and is generally accepted by language teachers), that such correctionwill bring a student’s production to a halt and lead to motivation problems (Willis1996). Many students do not wish to participate in public communication for fear ofmaking such mistakes. Hence, evaluation is limited to identifying a few key errors ineach group member’s work. If an error is encountered, the relevant activation levelsof the conceptual units in the student model are lowered; if no error is found (i.e.,there is explicit evidence of the correct application of a specific linguistic concept),

KNOWLEDGE STAGE

Mastery

Effective OperationalProficiency

Vantage

Threshold

Waystage

Breakthrough G1 G2

G3 G4

T1

S1 S2

S3 S4

S5

G5

S6 S7

MONITORING INTERACTIONS BETWEEN STUDENTS, GROUPS AND

TEACHERS

G6

S8 S9

Key:

S – Student G – GroupT – Teacher or tutor

– Evaluation – Meta-evaluation

Fig. 15 Monitoring and evaluation relations within COPPER


the activation values are increased. If no evidence is present, i.e., the monitor has notevaluated that aspect, then no change in activation values occurs. Hence, gradually,as students participate in group activities (at a particular knowledge stage), then evi-dence of correct language usage will be obtained by the evaluation process, and theactivation values will be increased, triggering material at higher knowledge stages tobe studied individually.

In order to undertake the evaluation process, COPPER contains an extensiveknowledge base of questions that are organised into knowledge stages. The questionsare organised into two groups, where some questions are relevant to individual mem-ber work and others to the group as a whole. The interface for the student monitorrole can be seen in Fig. 16. The student monitor has to evaluate a group in both aqualitative and quantitative way. Furthermore, a free text window is provided wherespecific examples of errors can be included to enable additional feedback to be pro-vided to the students working in the group. Hence, when a monitor needs to evaluatea group member’s performance, a sub-set of five questions (the exact number can beconfigured, but some initial tests seem to demonstrate that five is a good number)are randomly selected from the knowledge base and incorporated in the monitor-ing/evaluation interface (as can be seen in Fig. 16 above). Such a questionansweringevaluation process is simple, quick, and requires only a reasonable level of effort on astudent’s part. The evaluation is undertaken at the task level. When needed, specificlinguistic information is included in the question, hence facilitating the student moni-tor’s task (e.g., question 2 for the student Victoria Lopez in the interface actually liststhe possessive adjectives/pronouns). Furthermore, a link to additional explanationsand relevant definitions is included in each question, so that a student can be sure

Fig. 16 Student-monitor evaluation interface


of what is being asked. The results of the evaluation process, i.e., the answers to thequestions presented in the student monitor interface (together with any specific errorexamples that the student wishes to include, e.g., “mine shoes”—no, it should be “myshoes”), is passed on to the diagnostic process to be logged, and to produce changes inthe relevant group and individual student models. A similar interface is provided forthe meta-evaluation process, and does not depend upon whether the meta-monitoris a student or a teacher. The result of each question produces a direct change in theactivation value of the conceptual unit(s) within the student model. The transferenceeffect between linguistic skills means that any changes that result from the spokenor written use of a particular conceptual unit will be propagated to the equivalentreading–listening-knowledge stage relations. Hence, if a student has demonstratedthat s/he has successfully learnt a particular conceptual unit, by the individual workundertaken with the system (by studying theory and relevant examples and undertak-ing closed-evaluation exercises), and then makes mistakes when s/he tries to use it ina collaborative activity, not only will the relevant written- or spoken-knowledge stagerelation have its activation value lowered, but also the reading–listening-knowledgestage relations as well. In which case, depending on the actual values in question, aswell as having to demonstrate the successful use of the conceptual unit again in groupactivities, the student may have to go back and repeat some of the individual studyactivities related to it.

4 COPPER and ICALL

COPPER is an ICALL system with a student model that represents L2 learning interms of a three dimensional characterisation that contains the set of points thatcorrespond to conceptual units, related across the learning dimensions highlightedby the European framework. COPPER combines Bayesian diagnosis (with networksthat have a high granularity level) for points that can be modified automatically,with a human monitoring process for other linguistic aspects that cannot currently beprocessed automatically using NLP, to give rise to a kind of multidimensional stereo-types. These are subsequently used for group formation, so that the students can worktogether on relevent collaborative activities. There are three aspects in COPPERthat need to be contrasted with other ICALL systems: the role of collaboration, theBayesian diagnosis process, and the way stereotypes are used. COPPER combinesindividual and collaborative learning in a constructivist framework. Initially, studentswork on their own with the system to undertake simple repetitive theory-based read-ing and listening tasks that enable them to begin to establish prototypical linguisticknowledge. This phase enables the system to establish a detailed model of their knowl-edge and difficulties. Once the students have reached a certain stage in this process,they are identified by the system as members of a virtual learning community wherethey can be selected to form part of groups that use collaborative activities in socialcontexts to enable them to use what they have previously learnt in writing and speak-ing activities. Other systems in the literature such as GEROLINE (Heift and Schulze2003) or GRACILE (Ayala and Yano 1996) are collaborative from the beginning.The former focuses on collaboration based upon the interchange of learning objectsbetween students and an instructor. Evaluation is undertaken using automated tests.This system is interesting since its history is similar to that of COPPER, representingthe evolution from another system, German Tutor, to include collaboration.


The latter, GRACILE, is an agent-based system which assigns an agent to eachstudent that stores a set of beliefs as a schema for the learner model. This model repre-sents collaborative processes, and not cognitive ones. Here, the role of the agent is toenhance awareness and effective collaboration between learners. This approach is sim-ilar to that of COPPER in the way in which a community of students work together tolearn a second language, but differs in the defintion of groups. In GRACILE groupsare predefined in advance, and in COPPER they are created dynamically depend-ing on the level of competence of the students when the tasks begin. Furthermore,GRACILE has been designed to enable students to practice only written language,while COPPER enables spoken language to be used as well. Regarding the studentmodel, systems such as CASTLE (Murphy and McTear 1997), WEBPVT (Virvou andTsiriga 2001) and ICICLE (Michaud et al. 2001), use models based upon stereotypes,with some features in common with COPPER and others that are different. CASTLEuses a simpler user model where the stereotype is used to classify students in termsof their knowledge level. WEBPVT is a more complicated system that uses a multi-dimensional stereotype-based approach to represent a student’s knowledge level, a‘carefulness estimation’, together with prior knowledge of other languages. Finally,ICICLE uses a student model that captures the stereotypical stages of L2 acquisitionand the status of the grammatical constructions. Student progress is detected and thestereotypes are used to classify the student dynamically. In this sense COPPER usesstereotypes in a similar way to ICICLE.

Bayesian diagnosis is used in CAPIT (Mayo and Mitrovic 2001), where the lan-guage learning model takes the form of a data-centred network (i.e., the values used inthe network are learnt from domain data processed by the network). COPPER usesexpert-centric networks since it directly models expert knowledge of the teachingprocess. Furthermore, it is the experts (language teachers) who help to fix the valueswithin the networks and to refine their functionality. Finally, due to the amplitude ofthe L2 domain, it should be noted that Bayesian diagnosis needs to be complementedin COPPER with user evaluation, since natural language analysis problems limit theway in which automatic error detection can be undertaken.

None of the systems identified here, which have features in common with COPPER,combine the three aspects mentioned above, in such a way as to approach L2 learningas conceptualised in the European framework. The systems reviewed here appearto be appropriate for teaching the formal aspects of language: grammar, vocabulary,pronunciation, related to Reception and Production. However, the technology (andpedagogic framework) employed is currently not sufficiently advanced for generalInteraction activities between a student and the computer, the very type of activitiesthat really enhance L2 learning. The Artificial Intelligence techniques applied in thesesystems are far from what is needed to facilitate the real communicative competencesthat students require. While the NLP techniques are used here to analyse text atmainly morphosyntactic and lexical-semantic levels, the analysis is typically under-taken in restricted linguistic domains, and these techniques are currently unable toperform the types of pragmatic analysis that any person needs for any real communi-cative activity. While continued research in this area is essential, the current state ofNLP techniques is not yet ready for the kind of system argued to be necessary in thisarticle.

For both technical and pedagogical reasons, studies such as Barr et al. (2005), Gar-rett (1995), and Bailin (1995), amongst others, question whether the use of ICALLsystems can really help to improve the communicative abilities of students in any


significant way. They suggest that new pedagogic frameworks are needed that caneffectively apply technological advances, not only to help students improve somespecific aspect of language use, such as pronunciation, but more importantly, to helpstudents improve the L2 communicative competences required for real world L2 use.No system analysed here combines individual and collaborative learning in the waythat COPPER does, even when collaboration can be seen to be a good example of anauthentic activity for language learning. When students collaborate, they share andmutually reinforce what has been previously learnt by one or more members of thegroup. Such collaboration mobilises the cognitive mechanisms underlying communi-cation, giving rise to the collaborative development of the full range of competencesused in real communication. As will be seen in what follows in this article, COP-PER is a system that has been designed and developed based upon the technological,pedagogical and linguistic framework presented here, to address these needs.

5 Discussion

The work presented in this article was motivated originally by connecting the view oflanguage use as one of several cognitive competences that are mobilised when peoplecommunicate, with the difficulties present with using NLP techniques to automati-cally analyse such communication. A linguistic, pedagogical and technical frameworkhas been developed for an ICALL system, COPPER, which attempts to move awayfrom the standard goal of developing a system that can correct individual studentlinguistic activities, to tackle the problem from a different angle. Here, a constructivistapproach is adopted to combine individual and collaborative learning in order to facil-itate L2 learning, by enabling students to work individually and then participate inauthentic collaborative communicative activities where their linguistic competencescan develop, as they do in real ‘immersion experiences’. Participation in such commu-nicative tasks (where the communication may be interrupted to focus on the languagebeing used; although the learners do not lose sight of what they are attempting toaccomplish; Chapelle 1995), especially those designed to promote language learning,result in the further development of the learner’s competences.

COPPER has evolved over a four-year period following a formative evaluationprocess (Read et al. 2005; Scriven 1967; Seels and Richey 1994). It started as a toolfor individual learning, I-PETER, which was tested with a group of students in adistance-learning context. Work continued both to refine the functionality of the firstversion of the system for L2 learning (Bárcena and Read 2004), and to develop thesecond version, I-PETER II (developed with a grant from the Spanish Ministry ofEducation). The results of this work indicated that aspects of collaborative learningneeded to be incorporated to extend and consolidate L2 learning in a broader, lessformal and more communicative context. The tools that are included in COPPERfor the collaborative activities were developed separately and tested with real usersin specific scenarios. This lead to the refinement of the tool interfaces and, in somecases, functional modifications were incorporated. The adaptive templates and theirmanagement mechanism were also tested in practical laboratory sessions in order tocheck that the tast definition formalism was sufficiently flexible to represent the typesof scenarios that were needed, and that results generated by the students could beincorporated into COPPER together with relations to the tools that had been usedto generate them. Subsequently, pilot evaluation was undertaken of the system with


a small number of students divided into groups of two and three. Initially, problemsarose when the students tried to work together with the system, because some ofthem did not understand how to use the tools to undertake the scenario activities,and others did not even understand the idea of ‘role playing’. What followed wasthat the students started to work on their own, and did not use the available toolsto collaborate or coordinate the way in which work was being done. Explicit tutorialmaterial showing how to use the system together with example scenarios were pro-vided so that the students understood what was expected. What proved to be moreimportant than mechanical descriptions of how to use the tools were motivationaldescriptions explaining why they had to work together and how that would help themimprove their L2 capabilities. Furthermore, since the use of the collaborative toolrequired students to communicate amongst themselves, this also meant that they hadto interact linguistically, and hence were practicing their L2 use before they actuallystarted to undertake the actual tasks they had been assigned by the system.

The general COPPER student interface was also modified to incorporate aware-ness mechanisms enabling the overall state of an activity to be seen by the students ina group, together with the work being undertaken by each group member. Since thestudents were working in a distance-learning context, the majority of the collabora-tion undertaken was asynchronous, so an additional history mechanism was requiredto indicate what had happened since the last time a student had connected to thesystem. A calendar was also added to enable students to schedule synchronous tasks.Once these problems had been overcome, the students reported that the tools helpedthem in the tasks and that the problems they had to solve were adequate for their L2knowledge levels. The students also liked being made to change group when under-taking new activities. The students acting as monitors were also observed to show alot of interest in their respective tasks and made use of available opportunities (e.g.,using the chat) to help and motivate their colleagues. The study of the optimum wayin which roles can be changed within a community of students, when undertakingdifferent tasks in new groups, could well prove to be a fruitful line for future researchin the field of collaborative learning.

The actual activities that the students using COPPER are assigned have beendeveloped by the linguists in the group and have been refined following practicalface-to-face sessions where they were tried out (without using any computer tools).This enabled the activities to be modified, and furthermore, data obtained about theway in which they were undertaken, that can be contrasted with that obtained fromfuture trials with the system.

The system COPPER, presented in this article, makes eight contributions:

1. A student model that combines Bayesian diagnosis (using expert-centric networkswith a high granularity level) with multidimensional stereotypes, and is updatedfollowing student activities semi-automatically, including the results of humanevaluation with those of the networks.

2. The establishment of a relationship between the three dimensions that are arguedto characterize L2 competence, and which appears to provide a way to modelhow individual and collaborative learning can be combined to reinforce theassimilation of linguistic knowledge and the skills associated with its use.

3. The way in which the currently intractable NLP problems, present when attempt-ing the fully automatic analysis of such production, are avoided by letting moreadvanced students evaluate the oral and written production of less advanced ones.


4. The adaptive formation of groups based upon individual student progress, thenature of the tasks and roles to be performed in each activity, and previous expe-rience of each student in similar activities.

5. The quantitative measure of L2 competences used here offers a very flexiblemechanism for combining students of different abilities in the most suitable col-laborative activities for their current linguistic development.

6. Students are able to undertake L2 learning for all six levels defined by the Coun-cil of Europe, from Breakthrough to Mastery, for all four prototypical linguisticactivity types, and not some conventional restricted aspects of certain levels oflinguistic competence. Furthermore, any standard L2 materials that have beendeveloped in accordance to the European framework could be incorporated intosystems based upon this framework, like COPPER, with relative ease.

7. The nature of the collaboration undertaken by the students here, i.e., both betweenstudents in a group (as peers) and as student-monitors evaluating the progress ofgroups of lower level linguistic competence (as monitors), is argued to consolidatethe activities undertaken in the individual learning phase of the system.

8. The adoption of an approach that is innovative for the following four reasons: thecontextualized nature of the activities in COPPER empowers students to assumeownership of their knowledge and skills, and establishes the conditions for sharedunderstanding via collaboration, thereby reinforcing learning. The type of col-laborative constructivism presented here, namely task-based ‘learning by doing’(using authentic activities) appears to be particularly effective for L2 learning(Chapelle 1995), and has been argued to aid interaction and hence, facilitatelearning in a constructivist sense (Brooks and Brooks 1993). Finally, collabora-tion is facilitated here, something important for the intrinsically communicativenature of language production.

The effectiveness of this framework is founded on three pillars: firstly, the classifi-cation of student linguistic competences (if a student is assigned a level that does notcorrespond to reality, then his/her learning will be limited and any evaluation under-taken will be potentially disastrous). Secondly, the nature of the activities that thegroups have to undertake (what may constitute an authentic activity for one studentmay not for others, independently of the 10 characteristics presented by Reeves et al.2002). Thirdly and finally, a rigorous control of group participation and productionis required by monitors and meta-monitors to prevent errors going undetected andensure that groups are unable to progress, either due to the lack of participation ofone member or because their work has not been evaluated. In principle these issueshave been taken into consideration when the framework was designed, and appear tobe working in current applications. However, future analysis is needed to be sure.

The next step in the evaluation of COPPER is a large-scale test to actually see howstudents working with the system do actually improve their overall L2 capabilities.Students that have already worked individually with the system will participate inthis process since their user models already exist, making group formation immedi-ate. This work is important because as well as enabling linguistic improvement to betested, it will also enable the robustness and scalability of the system to be tested, andwill enable us to explore the degree in which collaboration actually enables studentsto improve learning. In our group full scale evaluations of other systems such as theActiveDocument system (Verdejo et al. 2003) and DEGREE (Barros and Verdejo2000) have been undertaken, so the problems that may appear when undertaking this


endeavour are familiar. Future work is also intended to focus on how to model boththe way in which students improve their L2 linguistic competences by working witha system like COPPER, and the way in which collaboration leads to improvementsin individual learning. If particular combinations of both individual and collabora-tive activities are found to be effective, then the adaptive group formation algorithmcould take them into account when forming groups. Finally, It is also planned thatautomatic evaluation and diagnosis of student collaboration be introduced into thesystem. Such analysis mechanisms would prove particularly interesting because theresults could be compared with the way in which humans evaluate group activities(Merrill et al. 1992). It is also the intention of the authors to include some of the sug-gestions made by students that have worked with the system, regarding how sub-tasksappear in the interface, and how preferences can be used to let the students pro-duce their own task list for a given activity dynamically, before the activity is actuallyundertaken.

Acknowledgements The work in this project has been undertaken (and partially financed) in thecontext of two projects funded by the Spanish Ministry of Education and Science: I-PETER II (Unsistema de tutorización inteligente de inglés de negocios con modelado adaptativo para diagnosis yselección de materiales personalizadas [An inelligent tutoring system for business English with anadaptive model for personalised diagnosis and material selection]; CICYT HUM2004-05758/FILO)and ENLACE (Un entorno basado en agentes para comunidades de aprendizaje; escuela y natu-raleza, lugares para aprender, colaborar y experimentar [An agent-based environment for learningcommunities; school and wildlife, places to learn, collaborate and experiment]; CICYT TIN 2004-04232). The authors would also like to thank the reviewers of the original version of this article forthe thoroughness of their work. Their comments have greatly helped improve the quality and focusof the final version.

References

Alfonseca, E., Carro, R.M., Martín, E., Ortigosa, A.: The impact of learning styles on student groupingfor collaborative learning: a case study. User Model User-Adapt Interact, this issue (2006)

Ayala, G., Yano, Y.: Learner models for supporting awareness and collaboration in a CSCL environ-ment, intelligent tutoring systems. In: Frasson, C., Gauthier, G., Lesgold, A. (eds.) ITS’1996 (LNCS1086), pp.158–167. Springer Verlag, Heidelberg (1973)

Bailin, A.: AI and language learning: theory and evaluation. In: Holland, V.M., Kaplan, J.D., Sams,M.R. (eds.) Intelligent Language Tutors: theory Shaping Technology, pp. 327–343. Lawrence Erl-baum Associates, Mahwah, NJ (1995)

Bárcena, E., Read, T.: The role of scaffolding in a learner-centered tutoring system for business englishat a distance. Proceedings of the Third EDEN Research Workshop, University of Oldenburg, Ger-many (2004)

Barkley, E.F., Cross, K.P., Major, C.H.: Collaborative learning techniques: a handbook for collegefaculty. Jossey-Bass, San Francisco (2005)

Barr, D., Leakey, J., Ranchoux, A.: Told like it is! An evaluation of an integrated oral developmentpilot project. Language Learn. Technol. 9(3), 55–78 (2005)

Barros, B., Verdejo, M.F.: Analysing student interaction processes in order to improve collaboration:the DEGREE approach. Int. J. Artif. Intell. Edu. 11, 221–241 (2000)

Brooks, J.G., Brooks, M.G.: Becoming a constructivist teacher. In search of understanding: the case forconstructivist classrooms, pp. 101–118. Association for Supervision and Curriculum Development,Alexandria, VA (1993)

Bull, S.: Student modelling for second language acquisition. Computers and Education 23(1/2), 13–20(1994)

Chanier, T. (ed.): Language Learning. A special issue of Journal of Artificial Intelligence in Education5(4), (1994)

Chapelle, C.A.: A framework for the investigation of CALL as a context for SLA. CFLL J. 6(3), 2–8(1995)


Cortright, R.N., Collins, H.L., Rodenbaugh, D.W., DiCarlo, S.E.: Peer instruction enhanced meaning-ful learning: ability to solve novel problems. Adv. Physiol. Edu. 29, 107–111 (2005)

Cortright, R.N., Collins, H.L., Rodenbaugh, D.W., DiCarlo, S.E.: Student retention of course contentis improved by collaborative-group testing. Adv. Physiol. Educ. 27, 102–108 (2003)

Council of Europe: Common European Framework of Reference for Languages: Learning, Teaching,Assessment. CUP, Cambridge (2001)

Cheng, R., Vassileva, J.: Design and evaluation of an adaptive incentive mechanism for sustainededucational online communities. User Model. User-Adapt. Interact., this issue (2006)

Faerch, C., Kasper, G.: Strategies in Interlanguage Communication. Longman, New York (1983)Fodor, J.A.: The Modularity of Mind. An Essay on Faculty Psychology. MIT Press, Cambridge, Mass

(1983)Gamper, J., Knapp, J.: A review of intelligent CALL systems. Comput. Assist. Lang. Learn. 15(4),

329–342 (2001)Garrett, N.: ICALL and second language acquisition. In: Holland, V.M., Kaplan, J.D., Sams, M.R.

(eds.) Intelligent Language Tutors: Theory Shaping Technology, pp. 345–358. Lawrence ErlbaumAssociates, Mahwah, NJ (1995)

Harrington, M.: Intelligent computer-assisted language learning. On-CALL 10(3), http://www.cltr.uq.edu.au/oncall/harrington103.html (1996)

Heift, T., Schulze, M.: Student modeling and ab initio language learning system. The InternationalJournal of Educational Technology and Language Learning Systems. 31(4), 519–535 (2003)

Holland, M., Kaplan, J., Sabol, M.: Preliminary tests of language learning in a speech-interactivegraphics microworld. CALICO J. 16(3), 339–359 (1999)

Jermann, P., Soller, A., Mühlenbrock, M.: From mirroring to guiding: a review of the state of arttechnology for supporting collaborative learning. In: Dillenbourg, P., Eurelings, A., Hakkarainen,K. (eds.) Proceedings EuroCSCL-2001, pp. 324–331. Maastricht (2001)

Kay, J.: Stereotypes, student models and scrutability. In: Gauthier, G., Frasson, C., VanLehn, K. (eds.)ITS2000, LNCS 1839, pp. 19–30. Springer-Verlag, Heidelberg (2000)

Krashen, S.: Principles and Practice in Second Language Acquisition. Prentice-Hall, New York (1982)Lantolf, J.P.: Sociocultural Theory and Second Language Learning OUP, Oxford (2000)Levy, M.: Computer Assisted Language Learning: Context and conceptualization, OUP, Oxford (1997)Mayo, M., Mitrovic, A.: Optimising ITS behaviour with Bayesian networks and decision theory.

International Journal of Artificial Intelligence on Education 12(3), 124–153 (2001)Merrill, D.C., Reiser, B.J., Ranney, M., Trafton, G.: Effective tutoring techniques: comparison of

human tutors and intelligent tutoring systems. J. Learn. Sci. 2(3), 277–305 (1992)Michaud, L.N., McCoy, K.F., Stark, L.A.: Modelling the acquisition of english: an intelligent CALL

approach. Proceedings of the 8th International Conference on User Modeling, Sonthofen, Germany(LNAI 2109), pp. 14–23. Heidelberg, Springer-Verlag (2001)

Murphy, M., McTear, M.: Learner modelling for intelligent CALL. In: Jameson, A., Paris, C., Tasso, C.(eds.) Proceedings of the Sixth International Conference on User Modelling, pp. 301–312. Springer-Verlag, Vienna (1997)

Ohta, A.S.: Rethinking interaction in SLA: developmentally appropriate assistance in the zone ofproximal development and the acquisition of L2 grammar. In: Lantolf, J.P. (ed.) Sociocultural The-ory and Second Language Learning, pp. 51–78. OUP, Oxford (2000)

Oxford, R.L.: Linking theories of learning with intelligent computer-assisted language learning. In:Holland, V.M., Kaplan, J.D., Sams, M.R. (eds.) Intelligent language Tutors: Theory Shaping Tech-nology, pp. 359–369. Lawrence Erlbaum Associates, Mahwah, NJ (1995)

Read, T., Bárcena, E., Barros, B., Verdejo, M.F.: I-PETER: modelling personalised diagnosis and mate-rial selection for an on-line English course. In: Garijo, F., Riquelme, J., Toro, M. (eds.) Advances inArtificial Intelligence – Iberamia 2002 (LNAI 2527), pp. 734–744. Springer-Verlag, Berlin (2002a)

Read, T., Bárcena, E., Barros, B., Verdejo, M.F.: Adaptive modelling of student diagnosis and materialselection for on-line language learning. J. Intell. Fuzzy Syst., 12(3/4), 135–150 (2002b)

Read, T., Bárcena, E., Barros, B., Varela, R., Pancorbo, J.: COPPER: modeling user linguistic produc-tion competence in an adaptive collaborative environment. In: Ardissono, L., Brna, P., Mitrovic,A. (eds.) 10th International Conference on User Modelling, Edinburgh, (LNAI 3538), pp. 144–153.Springer-Verlag, Berlin (2005)

Reeves, T.C., Herrington, J., Oliver, R.: Authentic activities and online learning. In: Goody, A., Her-rington, J., Northcote, M. (eds.) Quality Conversations: Research and Development in HigherEducation, vol. 25, pp. 562–567. HERDSA, Jamison, ACT (2002)

Salaberry, M.R.: A theoretical foundation for the development of pedagogical tasks in computermediated communication. Calico J. 14(1), 5–34 (1996)


Scriven, M.: The methodology of evaluation. In: Stake, R.E. (ed.) Perspectives of Curriculum Evalu-ation, pp. 39–83. Rand McNally, Chicago, IL (1967)

Seels, B.B., Richey, R.C.: Instructional Technology: The Definitions and Domains of the Field, Asso-ciation for Educational Communications and Technology, Washington, DC (1994)

Self, J.A.: Formal approaches to student modelling. In: Greer, J.E., McCalla, G.I. (eds.) Student Mod-elling: The Key to Individualized Knowledge-Based Instruction, NATO-ASI Series F, pp. 295–352.Springer-Verlag, Heidelberg (1994)

Stahl, G.: Collaborative information environments to support knowledge construction by communi-ties. AI & Society 14, 71–97 (2000)

Suebnukarn, S., Haddawy, P.: Modeling individual and collaborative problem-solving in medical prob-lem-based learning. User Model. User-Adapt. Interact., this issue (2006)

Suri, L.Z., McCoy, K.F.: Correcting discourse-level errors in a CALL system for second languagelearners. Computer Assisted Language Learning 6(3), 215–231 (1993)

Ullman, M.T.: Contributions of memory circuits to language: the declarative/procedural model. Cog-nition 92(1–2), 231–270 (2004)

Verdejo, M.F., Barros, B., Gómez Antón, R., Read, T.: The design and implementation of experimentalcollaborative learning in a distance learning context. ITHET 2003 Conference Proceedings, pp. 1–6(2003)

Verdejo, M.F., Barros, B., Read, T., Rodriguez-Artacho, M.: A system for the specification and develop-ment of an environment for distributed CSCL scenarios. In: Cerr, S.A., Gouardères, G., Paraguaçu,F. (eds.) Intelligent Tutoring Systems, ITS’2002 (LNCS 2363), pp. 139–148 (2002)

Virvou, M., Tsiriga, V.: Web Passive Voice Tutor: An intelligent computer assisted language learningsystem over the WWW. Proceedings of ICALT’01, pp. 131–134. IEEE Computer Society Press(2001)

Vygotsky, S.: Thought and Language. MIT Press, Cambridge (1986)Vygotsky, S.: Mind in Society: the Development of Higher Psychological Processes. Harvard Univer-

sity Press, London (1978)Warschauer, M.: Electronic literacies: Language, Culture, and Power in Online Education. Lawrence

Erlbaum Associates, Hillsdale, NJ (1999)Wenger, E.: Communities of Practice: Learning, Meaning, and Identity. CUP, Cambridge, Mass (1998)Wertsch, J.V.: Voices of the Mind: A Sociocultural Approach to Mediated Action. Harvard University

Press, Cambridge, Mass (1991)Willis, J.: A Framework for Task-Based Learning. Longman, London (1996)

Biography

Timothy Read obtained an Honours Degree in Computer Science from the University of the Westof England and a Ph.D. in Cognitive Science from the University of Birmingham. He is currently aSenior Lecturer in the Department of Computer Languages and Systems at the Universidad Nationalde Educación a Distancia (UNED). Since arriving at the UNED he has researched on systems forcollaborative and individual distance learning, specifically second language (L2) learning with twofoci: firstly, the application of User Modelling techniques from Artificial Intelligence to the design anddevelopment of learning systems that adapt to the changing needs and progress of the students. Seond-ly, the design and development of L2 learning systems that incorporate recent results from cognitive,neuropsychological and psycholinguistic research on the nature of language acquisition/learning andmemory, in order to facilitate and consolidate language learning.

Beatriz Barros received her Computer Science degree in 1994 and her Ph.D. in Computer Sciencein 1999 from the Technical University of Madrid (UPM). She has been a lecturer at the UNEDsince 1996, and has been a Senior lecturer there since 2001. She has been working in research pro-jects on group systems and collaborative learning applications during that period. Currently, she isinvolved in several European and National Projects related to collaborative and distance learning, forexperimental and computer-based second language learning.

Elena Barcena obtained an Honours Degree in English Philology from the Universidad de Deusto anda M.Sc. in Machine Translation and a Ph.D. in Computational Linguistics from UMIST (University ofMachester Institute of Science and Technology, Manchester, UK). She was a postdoctoral researcherin the Universidad de Lieja (Belgium) y has also worked in the Universities of the País Vasco, Sevillaand Granada. She is currently a Senior Lecturer in the Department of Foreign Philologies and their


Linguistics at the UNED. Her research interests have centred on the use of sublanguages for compu-tational purposes. Since her arrival at the UNED she has been researching on CALL related topics.She is the head of the research ATLAS (Artificial Intelligence for Linguistic Applications) group andcurrently leads the research project I-PETER II (Intelligent Personalised Tutoring Environment forBusiness English), funded by the Spanish Ministry of Education.

Jesus Carlos Pancorbo Lopez, obtained a B.Sc in Mathematical Sciences from the Universidad Com-plutense of Madrid and is currently writing his thesis in the Department of Computer Languages andSystems of the UNED. He is a permanent lecturer at the Universidad Antonio de Nebrija (Madrid),and collaborates in diverse research projects such as PETER II (intelligent personalised tutoringenvironment for business English) and PRODELE (creation of a didactic multimedia prototype totrain elderly people to teach Spanish as a foreign language teacher).

Coalescing individual and collaborative learning to model user linguistic competences

Documents

Transcript of Coalescing individual and collaborative learning to model user linguistic competences