Eurocall 2013, Évora, 11 September 2013

Katrien BeulsVUB Artificial Intelligence Laboratorykatrien@ai.vub.ac.be

A constructionist approachto student modelling

2

State-of-the-art language learning systems promise “immersive learning”

Rosetta Stone

“Don’t learn a language, absorb it”

“Language learning should be natural”

Terminology of learning theories(Acquisition-learning distinction, Natural approach)

LANGUAGE LEARNING THAT WORKSIt's not an overpromise.It's at the core of our method's effectiveness.Because instead of translating to and from your current language, wecreate a unique world that exposes you to new vocabulary, grammar andpronunciation in a scientific way that stimulates and engages your brain'snatural language learning ability.

The secret of our success isn't magic. It's immersion. Dive in.

From lesson one, the language you're learning isthe only one you'll use.Although it may sound intimidating, it's actually a goodthing. Without your native language for support, you'll haveno choice but to be actively engaged in learning your newlanguage. And that totally increases your chance forsuccess.

Other language learning programs have you merelymemorize words and phrases. Our method introduces yournew language systematically—to help you internalize it andnaturally develop a sense of how to use it correctly.

Every step of our method exposes you to a combination oflanguage you've learned and language you don't know yet. Itcan be challenging at times, but it's also exciting—and alittle like the experience of being in country. Think of it asanother reason our approach is so effective: We don't justteach you a new language, we're actually teaching you howto learn it.

With our approach, you'll

navigate a totally new

language environment

without the influence of

your native language.

Start learning in a way that seems totally simple andintuitive—because the experience is so well designed.We believe the best way to learn a new language is to use it. That's why our method combines listening, reading,speaking, and writing to reinforce your new skills and help them stick.

Our interactive course carefully choreographs the way you're introduced to images, words, and phrases so you'll learntheir meaning through context.

PERSONAL PROFESSIONAL SOLUTIONS EU: +44 207 158 0451Sign in Support Basket

Select your next languageTry it free

TRYTRY METHODMETHOD FEATURES TOURFEATURES TOUR REVIEWSREVIEWS PRICINGPRICING3

But they fail to accommodate for individual student differences

Input is not very interesting

- matching activities

- elimination

Every student follows same path irrespective of his answers

Similar to programmed instruction of the early days of ITS

Krashen, S. (2013). Rosetta Stone: Does not provide compelling input, research reports at best suggestive, conflicting reports on users' attitudes. International Journal of Foreign Language Education, 8(1), 1–3.

4

Academic initiatives try to counter this lack of individualization

German Tutor (Heift & Schulze, 2007)

- web-based German exercises

- linguistic analysis for feedback

TAGARELA (Amaral & Meurers, 2007)

- web-based language tutoring system for Portuguese

- annotations of learner input

5

I propose an active student model

That can predict a student’s answersby simulating the learning task

It is implemented as a student agentthat can process and learn the target language

6

A constructionist approach to student modelling

1. Design of an agent-based language tutor

2. Operationalizing processing and learning

3. Ideas about tutoring

7

A constructionist approach to student modelling

1. Design of an agent-based language tutor

A competent language user

A student model

Tutoring strategies

8

A language agent simulatesa competent language user

A construction inventory (grammar)

An engine to process constructions

Flexibility strategies to flexibly process the target language

cxninventoryi

grammarenginei

flexibilitystrategies

language agent

9

A student agent has the same architecture

Instead of flexibility strategies, the agent makes use of learning strategies that target specific acquisition problems

Also interacts with language agent(no real student required)

cxninventoryi

grammarenginei

flexibilitystrategies

language agent

cxninventoryj

grammarenginej

learningstrategies

student agent

10

Tutoring strategies mediate the tutor-student interaction

tutor agent

tutoringstrategies

studentprofile

cxninventoryi

grammarenginei

flexibilitystrategies

language agent

cxninventoryj

grammarenginej

learningstrategies

student agent

11

The tutor agent’s student model consists of a dual structure

A runnable student agent

- predict student’s answers

- align to student every interaction

A more static student profile

- store user profile, preferences

- update interaction logs and scores

12

A constructionist approach to student modelling

2. Operationalizing processing and learning

Fluid Construction Grammar (FCG)

Spanish verb conjugation

Error correction

13

FCG lends itself well for tutoring purposes

Construction-grammar formalism

Everything is a construction (phon, morph, lex, phrasal, pragmatic)

A construction consists of features

Unification-based (HPSG) but less strict

Customizable search process

Steels, L. (2011). Design Patterns in Fluid Construction Grammar. (L. Steels, Ed.). Amsterdam: John Benjamins.

14

The formalism is informative about failed constructional matches

Processing problems can be detected when parsing student input

Uninterrupted processing guaranteed with meta-level architecture

initial jugaría-past-imperfect-2/3

1/3sg-morph

15

Language agent is initialized with 600 most common verbs in Spanish

18 x 6 conjugated forms / verb

+ 2 imperatives

+ 2 gerunds

= 112 forms / verb

Fred Jehle’s database contains > 11 000 conjugated verb forms

Fred F. Jehle, University of New Mexico, Verb list taken from http://users.ipfw.edu/jehle/VERBLIST.HTM

16

Proficiency evaluated on learner errors from SPLOCC II corpus

“The emergence and development of the tense-aspect system in L2 Spanish”

408 errors made by low intermediate learners

- *cogue ➔ coge, ‘he takes’

- *leó ➔ leyó, ‘he read’

- *escuchían ➔ escuchaban, ‘they heard’

Evaluated by parsing and reproducing

Mitchell, R., Dominguez, L., Maria, A., Myles, F., & Marsden, E. (2008). A new database for Spanish second language acquisition research. EUROSLA Yearbook, 8(1), 287–304.

17

SPLOCC results!

verb class

stem change

unknown su!x

su!x change

unknown stem 43%

26%

18%

1%

12%

!

70%

80%

100%

accuracy

stem change

su!x change verb class

unknown stem

unknown su!x

18

The student agent is initialized with an empty grammar

cxninventoryi

grammarenginei

flexibilitystrategies

language agent

cxninventoryj

grammarenginej

learningstrategies

student agentEmpty construction inventory

Default grammar engine settings

Learning strategies to acquire target grammar

19

Learning strategies tackle learning problems instead of processing problems

diagnostics

D1 problem-a

D2

D3

D4

D5

problem-b

problem-c

problem-d

repairs

R1

R2

R3

R4

R5

0.5

0.30.9

0.2

1.0

0.6

0.1

20

Three types of learning strategies

Learning the basics

- unknown stem, suffix

- irregular verb

- new grammatical meaning

Learning verb stem/suffix changes

- coje < coger; pienso < pensar; ...

Learning verb classes

- hablamos, comemos, vivimos21

Learning happens in discriminative contexts

Rosetta Stone Version 3, Spanish

22

The student agent can be the speaker or the hearer in a game

t

signalsuccess/

failure

game n game n+1

produceselect topic

parse consolidatefind

topic

cxninventoryj

grammarenginej

learningstrategiesj

student agent

cxninventoryi

grammarenginei

learningstrategiesi

language agent

1 1,2,31

2

3

situation situation

produce

1

parse produce givefeedback

consolidate

1,2,3

game n+2

select topic

find topic

signalsuccess/

failure

23

Example: Three picking events

Student agent:

- topic: event 2

- utterance: “ha recojado”

Language agent:

- correction: “ha recogido”

Student agent learns verb class

ha recogido

Can you link the conjugated verb form to the situation?

Check the corresponding box on the time line

Skip this game

now

t

1 2 3

24

Communicative success reaches 100%

0

20%

40%

60%

80%

100%

0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000 0

100

200

300

400

500

600

700

com

mun

icat

ive

succ

ess

cxn

inve

ntor

y si

ze

games

Learning full conjugation of 25 verbs (mixed)25

Most learned constructions are suffixes

0

50

100

150

200

250

300

350

lex suffix irreg aux gram

num

ber o

f cxn

s

Learning full conjugation of 25 verbs (mixed)26

A constructionist approach to student modelling

3. Ideas about tutoring

The Colour Tutoring Game

Tutoring strategies for learning problems(work in progress)

27

First tutoring game prototype for colour word learning

Web interface with colour chips

User can be student or tutor (teach the system a colour lexicon)

New colour lexicon can be used for future students

Beuls, K., & Bleys, J. (2011). Game-based Language Tutoring. In B. Knox (Ed.) Proceedings of the IJCAI 2011 Workshop on Agents Learning Interactively from Human Teachers, Barcelona.

28

The colour tutoring gameis lacking tutoring strategies

Tutoring strategies serve two functions

1.Selecting a new situation

2.Providing constructive feedback

Spanish verb tutor should tackle learning problems that are diagnosed

29

Alignment is critical for a predictive student model

The tutor agent needs to alignthe linguistic knowledge of the student with the constructions known by the student agent after every game

30

Conclusions

The language agent and the student agent form the basic foundations for an adaptive tutoring system for language

Using Construction Grammar to represent linguistic knowledge is beneficial for understanding the student’s difficulties

Work in progress...

- Building a user interface

- Developing tutoring strategies and a data structure they work on

- Experiments with alignment

Possible future extensions lead to

- other game scenarios

- larger sentences

- more languages

Further reading

Beuls, K. (2012). Grammatical error diagnosis in Fluid Construction Grammar: A case study in L2 Spanish verb morphology. Computer Assisted Language Learning. doi:10.1080/09588221.2012.724426.

Beuls, K. (2012). Inflectional patterns as constructions: Spanish verb morphology in Fluid Construction Gammar. Constructions and Frames, 4(2). p 231-252.

Steels, L. (Ed.). (2011). Design Patterns in Fluid Construction Grammar. Amsterdam: John Benjamins.

Steels, L. (Ed.). (2012). Computational Issues in Fluid Construction Grammar. Berlin: Springer.

Additional slides

Flexibility strategies are active during linguistic processing

D1

problem-aR1D4

problem-c R3R5

problem-bR4R3R2

norestartrestart

restart

37

Constructions relate meaning to form via semantic and syntactic categorizations

meaning form

semanticcategorizations

syntacticcategorizations

38

Detect feature mismatch

Second merge fails due to different verb class feature in stemía = 2/3; jugar = 1

Diagnostic returns verb class feature and its correct value

initial jugaría-past-imperfect-2/3

1/3sg-morph

39

Detect unknown stem

Very frequent problem with beginning learnersjuqaba => jugaba

Repaired with closest match on stems in grammarLevenshtein distance with additional weight on first letter

40

Learning problem priority list

Front

LP1 LP3LP2 LP4 LP5 LP7LP6

Back

Front

LP8 LP2LP3 LP4 LP5 LP7LP6

Back

FI = 0.7LET = 0.41 > FI > LET

LP1

LP8

LP9

LP10

...

123

41