Oralité(s) et écriture(s) Oralité(s) et écriture(s) Are they following

Gerda Haßler, Cordula Neis (éds)

Oralité(s) et écriture(s)

Studium SprachwissenschaftBeiheft 41

G

erda

Haß

ler,

Cor

dula

Nei

s (é

ds)

Ora

lité(

s) e

t éc

ritu

re(s

)

Gerda Haßler, Cordula Neis (éds.)

Oralité(s) et écriture(s) Préface (Gerda Haßler, Cordula Neis) / Sylvie Mutet : Cultures de l’oral, cultures de l’écrit : hiérarchies, histoire et représentations / Rossana De Angelis : Entre oralité et écriture. Enjeux théoriques d’une théorie des institutions / Ilka Mindt : Talking like a book. Investigating colloquialization as a change in written English / Bernard Darbord : Réflexions autour de l’oralité du proverbe / Agathe Cormier : Écrire ou prononcer son propre nom. Emplois oraux et écrits du nom propre hors phrase / Léda Mansour : Signalisation du discours direct de paroles inaudibles. Quand dire, c’est ne pas prononcer / Khalifa Missaoui : Oralité, gestualité et intentionnalité / Maria Rosaria Compagnone : Fenomeni di oralità nel francese e nell’italiano digi-tato / Diana Balaci : Influence linguistique des conditions imposées par l’ordinateur à la situation de communication / Mokhtar Farhat : L’approche d’un genre hors norme : le one-man-show / Banafsheh Karamifar : L’oralité à l’écrit pour représen-ter les acteurs sociaux. Analyse sémantico-critique d’une publicité de BlackBerry / Laura Corcione : Analisi lessicometrica dei libri dei commenti dei visitatori dell’Ara Pacis / Xavier Pietrobon : Les limites du langage dans l’apprentissage. Zhuangzi 莊子 face aux pratiques corporelles / Frances McDonald : Are they following? A survey of teenagers’ L2 listening competences in terms of the Common European Framework of Reference / Victoria Dubrovina : Lexicalization vs. grammaticaliza-tion. The historical development of composite predicates of the pattern give+a+N in Early Modern English — a corpus-based study / José Luis Aja : Los Racconti romani de Alberto Moravia y el tratamiento del discurso oral en las traducciones españolas y francesas. Usos y estrategias traslativas de “allora” / Sandra Falbe : Ex-presividad, emocionalidad y oralidad ficcional. El recurso lingüístico de la hipérbole / Sybille Schellheimer : La traducción de la oralidad ficcional en la literatura infantil. El ejemplo de las traducciones al castellano y al catalán de Emil und die Detektive / Cordula Neis : Fingierte Mündlichkeit. Ein theoretisches Problem und seine über-setzungspraktischen Lösungen

ISBN 978–3–89323–141–6 ISSN 0721–7129

��

��

___________________________________________________________________________

Gerda Haßler, Cordula Neis (éds.) : Oralité(s) et écriture(s), 182–209 © Copyright 2012 by Nodus Publikationen, Münster. ISBN 978–3–89323–141–6

Frances McDonald

Are they following? A survey of teenagers’ L2 listening competences in terms of the Common European Framework of Reference1 1. Introduction The skills, knowledge, learning awareness and positive disposition towards learning throughout a lifespan2, which the general population of citizens need in order to play an active part in society, also include second language learn-ing. There is now acute awareness that the study of languages must be pro-moted in the student population at large and no longer be confined, as it had been in the past, to select culturally aware students oriented towards university studies. It is within such a scenario that the idea to carry out a survey on teen-agers’ competences in relation to listening comprehension took form. The ra-tionale behind the decision to carry out a survey on 15 year-old students who must continue their education for a number of years before embarking on uni-versity studies or taking up a job is that a test can give students, families, teachers and all others involved in education useful information on the learning and teaching process and can constitute the basis for guidance on future stud- 1) Delivered September 2010 Potsdam at Colloque Oralité(s) et écriture(s): approches linguis-

tiques et didactiques under the title La valutazione dell’ascolto in relazione al processo di ap-prendimento linguistico degli adolescenti. A debt of gratitude is due to Prof. Marco Svolacchia, Università Roma Tre, for his critical review of the research project, Prof. Giuseppe Bove, Uni-versità Roma Tre, for guidance on statistical processing and Ugo Peru, University of Cassino, for support in the construction of audio texts. Responsibility for the contents lies entirely with the author.

Correspondence concerning this article may be addressed to Frances McDonald, Istituto Tecni-co Commerciale “Leonardo da Vinci”, Via Piave, 03100 Frosinone, Italy. Email address: [email protected] (F. McDonald).

2) The European Commission’s Lifelong learning Programme 2007–13 furthers the aim of facilita-ting access to learning of all citizens, whether young or not so young http://ec.europa.eu/education/lifelong-learning-programme/doc78_en.htm

Are they following? ___________________________________________________________________________

___________________________________________________________________________

– 183 –

ies: it can inform about progress in relation to a starting point, in relation to all the other people taking the test and in relation to a profile of successful per-formance.

The specific scope of the survey is to look at how an unselected sample of 15 year-old students in general secondary education in Italy respond to a listen-ing comprehension test which aims at gathering evidence on their ability

to process extended samples of realistic and spoken language, automatically and in real time,

to understand the linguistic information that is unequivocally included in the text,

to make whatever inferences areunambiguouslyimplicated by the content of the passage (Buck 2001: 114).

Rivers and Temperley (1978) underline that

Listening is a complex operation integrating the distinct components of perception and linguistic knowledge […] (T)he cognitive nature of listening […] involves perception based on internalized knowledge of the rules of the language […] (essentially) an active process of constructing a message from a stream of sound with what one knows of the phonological, semantic, and syntactic potentialities of the language. (Rivers and Temperley 1978: 63)

Listening was selected as focus for the survey, on the one hand, because of the psychological receptiveness of children and teenagers to the oral dimension of language and, on the other hand, because it is an area of language learning as yet not fully explored. A particular aspect of listening comprehension dis-criminated by the test is the learners’ degree of independence. In the process of learning a language, acquiring the ability to comprehend discourse autono-mously is an important objective because, once learners have reached this stage, they can use the language for a twofold purpose: on the one hand, it is possible to understand others and begin to communicate with them and, on the other hand, they can take control of their future language studies. Hence, by the term autonomy, or independence, is meant both the ability to engage in communication and the ability to direct one’s own studies.

In recent decades, language education in Europe is being guided by A Common European Framework of Reference for Languages: Learning, Teach-ing, Assessment (Council of Europe 2001a), henceforth referred to as the CEFR, a study which, among other things, defines and describes profiles of language learners’ competences and knowledge at ascending stages of learning. At an initial level of independence in overall listening comprehension, denomi-nated B1 Level, the CEFR defines and describes the language user as someone who

can understand the main points of clear standard speech on familiar matters regularly encountered in work, school, leisure, etc., including short narratives. Can understand

��

��

Frances McDonald ___________________________________________________________________________

___________________________________________________________________________

– 184 –

straightforward factual information about common everyday or job related topics, iden-tifying both general messages and specific details, provided speech is clearly articulated in a generally familiar accent. (Council of Europe 2001b: 8)

It further defines and describes a B1 Level profile of “Communication strate-gies, i.e. ability in identifying cues & inferring”, according to which a learner

can identify unfamiliar words from the context on topics related to his/her field and in-terests. Can extrapolate the meaning of occasional unknown words from the context and deduce sentence meaning provided the topic discussed is familiar. (Council of Europe 2001b: 24)

An earlier stage in the learning process had also to be defined for the survey in the likely event that the results from the test should point to a considerable number of learners who were not able to comprehend discourse autonomously. The CEFR A2 Level language learning profile for listening comprehension, which immediately precedes the B1 Level profile, was deemed appropriate by teachers. It specifies that learners

can understand enough to be able to meet needs of a concrete type provided speech is clearly and slowly articulated. Can understand phrases and expressions related to areas of most immediate priority (e.g. very basic personal and family information, shopping, local geography, employment) provided speech is clearly and slowly articulated. (Council of Europe 2001b: 8)

and, as regards “Communication strategies, i.e. ability in identifying cues & inferring”, they

can use an idea of the overall meaning of short texts and utterances on everyday topics of a concrete type to derive the probable meaning of unknown words from the context. (Council of Europe 2001b: 24)

With the CEFR becoming a widely adopted instrument for setting learning objectives and assessing progress both in school and in external examinations and also an instrument in research on language learning, it was possible to use it as a common yardstick to describe the results of the present survey. 2. Background Second language listening has until recently attracted relatively little research but there are signs that it is now emerging as a clearly-defined field with spe-cialized areas such as word recognition, phoneme discrimination, strategy use, academic listening, negotiation of understanding and testing. Research ap-proaches are inspired by distinct theoretical standpoints such as learner vari-ables, cognitive psychology, socio-cultural linguistics and discourse analysis.

Language processing has been recognized to be extremely complex. McClelland, Rumelhart and PDP Research Group (1986) noted that there are multiple simultaneous constraints on language processing, and understanding arises through the interplay of these different sources of knowledge. Hence,


___________________________________________________________________________

– 185 –

we must conceive of listening comprehension as a multidimensional activity involving a large number of knowledge sources and skills that are simultane-ously active and are simultaneously interacting with each other. Research dis-tinguishes top-down and bottom-up processing.

By top-down processing of discourse is meant the construction of meaning through a process of hypothesis making and testing in which listeners infer the unknown from context and prior knowledge. It was studied by Markham and Latham (1987), Long (1989; 1990) and Schmidt-Rinehart (1994). Context was felt to be a predominant factor in comprehension which could compensate for perceptual errors, in the sense that the weight of schematic knowledge and fa-miliarity with situation and topic could facilitate exploitation of contextual evi-dence in efforts to extract meaning from spoken discourse. A distinction is made between filling gaps in knowledge by means of haphazard guessing and the principled employment of a series of metacognitive skills and strategies.

Bottom-up processing, meaning the way in which perception contributes to the meaning representations that second language listeners build, has been in-vestigated by Lynch (2006). Field (2004) discusses the demands made upon working memory by inefficient decoding which limits the listener’s ability to form a detailed and coherent message. On the other hand, when bottom-up processing is accurate and automatic, the listener can build complex meaning representations because the working memory capacity is not engaged by com-prehension of basic language exponents. Brown (2006) calls for attention to L2 syntax and intonation. Research into phoneme perception is investigating the cross-linguistic factors determining which phonemes a listener has difficulty in distinguishing in L2 (Flege, 1995; Best, 1995; Escudero and Boersma, 2004). De Jong et al. (2009) have analyzed consonant identification according to structured models. Studies of how lexis is processed look at the nature of vo-cabulary and how it is accessed. As connected speech does not feature consis-tent pauses between words, listeners need to determine for themselves where word boundaries fall. Both for L1 and L2, specialists assume that word recog-nition is achieved through competition between word candidates, a kind of competition which is more complex for L2 (Cutler, 1990, 1997; Meara 1997; McQueen, 2004; Broersma and Cutler, 2008). Language specific strategies to locate word boundaries have been identified (Field, 2008). They include pro-sodic features such as the role of strong syllables in English and of moras in Japanese (Cutler, 1997), word-level factors (Norris et al., 1997) and phono-tactic constraints (McQueen, 1998; Weber and Cutler, 2006; Al-Jasser, 2008). In yet another direction, studies of brain imaging are producing evidence to suggest that the perception of the mother tongue and second languages are not identical (Dehaene et al., 1997; Aglioti and Fabbro, 1999; Pallier et al., 2001).

Findings from research suggest that strategic processing appears to be a generic activity common to all areas of learning. In fact, general learning

��

��


___________________________________________________________________________

– 186 –

strategies reported by L2 learners were similar to the learning strategies re-ported in general learning tasks. Anderson (1995) identified three phases in first language listening processing. The first phase, perceptual processing/en-coding of message, involves segmenting phonemes from the continuous speech stream, retaining sounds in echoic memory and employing strategies such as selective attention and directed attention; the second phase, parsing, involves grouping word sequences into chunks according to syntactic structures or cues to meaning, processing chunks, and inferring the unknown from context using a top-down approach; when an utterance is segmented, the words are trans-formed into a mental representation of the combined meaning of these words. These segments are then recombined to generate a meaningful representation of the original sequence. In the third phase, utilization, the mental representa-tion is related to existing knowledge and stored in long-term memory as propo-sitions or schemata. Perception, parsing and utilisation represent different as-cending levels of processing and together they spiral through a listening ex-perience. They are “by necessity partially ordered in time; however, they also partly overlap. Listeners can be making inferences from the first part of a sentence while they are already perceiving a later part” (Anderson, 1995: 379). In their research of second language listening, O’Malley, Chamot and Küpper (1989) found evidence of Anderson’s three L1 processes: in the per-ceptual processing phase, involving strategies such as selective attention and directed attention, skilled listeners maintain attention or redirect it when dis-tracted; in the parsing phase, when words are transformed into a mental repre-sentation of the combined meaning of these words, skilled listeners inferred the unknown from context using a top-down approach; in the utilization phase re-quiring use of prior knowledge to assist comprehension and memory, the skilled listeners took a global approach, inferring meaning from context, en-gaging in self-questioning, relating what was heard to world knowledge and personal experience. Overall, from a qualitative perspective, skilled listeners used self-monitoring, elaboration and inferencing more than less skilled listen-ers did.

Research on comprehension and metacognitive strategies concerns itself with pedagogical outcomes. Features distinguishing the more competent from less competent listeners have been structured as profiles of behaviours and sub-sequently didactized into learning and teaching protocols to promote listening comprehension skills. Vandergrift (2003), in particular, has described and categorized metacognitive strategies while Goh (2000) has analized students’ mistakes in order to throw light on their difficulties in putting strategies to use. Vandergrift (2003) found significant differences in the use of the category of metacognitive strategies as well as in individual strategies for comprehension monitoring, questioning for elaboration, and translation in 12–13 year old school-children learning French as a foreign language. Conceding that an un-


___________________________________________________________________________

– 187 –

derstanding of the complex processes involved in listening comprehension strategies may be limited, Vandergrift (2003) notes that, although cognitive strategies such as elaboration and inferencing are used almost equally by all listeners, more skilled listeners use them as well as metacognitive strategies such as selective attention and comprehension monitoring more effectively and more frequently and can flexibly employ strategies in effective combinations. Listener think-aloud protocols have been used to identify the three phases of the listening process (perceptual processing, parsing, utilization), as well as strategies associated with each phase. In addition to a quantitative analysis, Vandergrift supports a qualitative analysis of protocols in order to acquire greater insight into the differences between more skilled and less skilled listen-ers.

Investigating the comprehension problems of second language listeners from a cognitive perspective, Goh (2000) identified ten cognitive processing problems which occurred during Anderson’s (1995) three-phase model of per-ception, parsing and utilisation. Five problems were linked to word recognition and attention failure during perceptual processing. There were also problems related to inefficient parsing and failure to utilise the mental representations of parsed input. She proposes a cognitive framework for understanding learners’ listening difficulties as it pinpoints those places in cognitive processing where comprehension can break down and allows us to trace the source of these diffi-culties in learners.

Research of real-time cognitive constraints during listening and the data obtained through learners’ introspection is being used to produce training pro-grammes for helping learners to become more effective listeners. Continuing research on metacognitive awareness raising activities has in turn led to the development of the Metacognitive Awareness Listening Questionnaire (MALQ) (Vandergrift et al., 2006), an instrument which can serve both research and learning. For research purposes, it can be used to assess the development of metacognitive awareness of the listening process: as a pre- or post-test to chart the impact of listening strategy training and to assess learners’ growing aware-ness of the processes underlying successful L2 listening; student responses can also be correlated with listening test performances to determine which behav-iors correlate best with listening achievement. Furthermore, learners can de-velop their awareness of the comprehension process: for self-assessment pur-poses to determine their current level of metacognitive awareness, and to chart the development of their strategy use/listening awareness over time. Teachers can use MALQ to diagnose students’ metacognitive awareness and self-regu-latory abilities, thereby receiving assistance in helping L2 learners to overcome listening comprehension difficulties.

In a case study to explore the difference between the strategy use of a suc-cessful listener and that of a less successful one, Graham et al. (2008) em-

��

��


___________________________________________________________________________

– 188 –

ployed think-aloud interviews (verbal reports). In a temporal dimension of two points in time, strategy use was compared to see if the performance of the weaker listener becomes more accomplished with longer experience of L2 lis-tening. Patterns in strategy development were detected: inferencing and reli-ance on prior knowledge seem to decline as learners’ linguistic base increases and the use of metacognitive strategies increases in more competent learners. Suggesting that this may be linked both to the availability of processing capac-ity and to linguistic knowledge, the researchers point to a relationship between strategy development and overall language proficiency. However, findings have been qualified by a caveat relating to problems posed by the close link between the behaviour of the subjects and the contexts in which it occurs (the listening text and event), which makes it difficult to consider a strategy inde-pendently of the circumstances which give rise to it. “(...) any discussion of strategies is necessarily linked with what gave rise to the uses of these strate-gies, i.e. the task set” (Graham et al. 2008: 66). The researchers conclude that a case study approach examines strategies within the context of a particular task while bearing in mind the demands made by different aspects of the task in relation to how learners deal with them.

Construct in the design of listening tests is another issue affected by the close link between the behaviour of the subjects and the contexts in which it occurs. The specification “process(ing) ... language ... in real time ...” (Buck 2001: 114) assumes particular relevance for students in relatively early stages of language learning. Generally interpreted in national and international lan-guage examinations as speech at natural rate, candidates, however, are not re-quired to process the language in real time in the same way that situations in-volving authentic communication would require them to because they are al-lowed to listen to the recordings twice. Although language learning and as-sessment activities differ from real-life communication in this and in many other ways, they are nonetheless effective surrogates of communication for learning and assessment purposes. Other interesting departures from strictly authentic communication in learning environments are offered by sound tech-nology. For instance, technological manipulation of speech rate is being ex-ploited, on the one hand, to gather evidence of less competent learners’ listen-ing skills and comprehension strategies and, on the other hand, to help begin-ner students to accelerate their progress in acquiring skills. Zhao (1997) ma-nipulated speech rate to facilitate learners’ comprehension of spoken discourse: when listeners had control over it, he observed that their listening comprehen-sion improved. Meinardi (2006; 2009) used the patented DITCALL digital slow-down tool to enhance word recognition in rapid authentic native speaker speech for the purpose of making it accessible to immigrant language learners, including beginners.


___________________________________________________________________________

– 189 –

Dealing as it does with learners at quite different levels of competence, the present survey avails of software to slow down speech rate so that the same selection of spoken texts can be accessed by all the listeners in the sample population. With such a population, the options available in the design of lis-tening tasks were either a large number of spoken texts at different levels of complexity or a smaller selection of texts at both natural and slowed speech rates. As well as facilitating task administration, the speech rate variation fa-cilitates investigation of various dimensions of performance relating to a com-mon series of stimulants. But, more importantly, the slowed versions of re-cordings constituted a test construct in keeping with the CEFR A2 Level speci-fication (...) provided speech is clearly and slowly articulated in that students who managed to respond correctly and/or approximately to the tasks could be classified as A2 level on the grounds that they were able to cope with content although not the speed of discourse. Slowed recordings were produced by means of the free software package Audicity3. The same content at natural speech rate also complied with the B1 specifications as regards subject matter and fluent speaker delivery.

The promotion of language learning in Europe has been driven by the Council of Europe’s A Common European Framework of Reference for Lan-guages: Learning, Teaching, Assessment CEFR (2001). Developed by a com-mittee of organisations and individual experts in language and assessment, the CEFR is essentially a guide to the communicative use of second languages. Its principal characteristic lies in the categorisation of second language use and communicative aspects of discourse according to a scale of levels of compe-tence of increasing complexity. Each level is illustrated minutely by a range of descriptors relating to language use. The approach, which is descriptive and action oriented rather than prescriptive, aims to be sufficiently flexible to per-mit users to adapt the framework to their specific needs and contexts. It grew from the realization that language learning should have the primary goal of equipping people to use language for communication. Among others, Widdow-son (1978; 1979; 1983) argued strenuously for an approach to language learn-ing which would engage students in meaningful communication and at the same time incorporate elements of structure, lexis, pronunciation, and so forth. Wilkins (1976) had already pragmatically codified communicative language use and knowledge into a workable syllabus to which he gave the name no-tional-functional syllabus; moving in a similar direction in a Council of Europe project, van Ek and Trim designed The Threshold Level in a European unit/credit system for modern language learning by adults (van Ek 1975; van Ek and Trim 1991) with the object of defining the language and communica-tive competences European citizens would need in order to live and work in-dependently in a country where they had to use their second language. The 3) Audacity software package available at http://audacity.sourceforge.net/

��

��


___________________________________________________________________________

– 190 –

project, which involved a statement of objectives in relation to functional and notional categories and linguistic exponents, aimed to quantify and certify pro-gress in adult workers and students’ learning. Over the years, the scope of the Threshold Level was broadened to include other categories of language learn-ers, and, rather than one general level, the comprehensive CEFR ascending range of competences has been defined. 3. Research design The study sought evidence on specific characteristics of teenagers’ listening competences and on how they spread through proficiency levels in a large het-erogeneous population. The evidence has been analyzed so that it might pro-vide some general indications regarding: A) the mapping of heterogeneous groups of students’ listening performances onto the CEFR, B) the consistency of independent listening competences in the sample population, C) speech rate as a distinguishing feature in A2 and B1 CEFR level profiles and D) features of students’ listening strategies. 3.1 Participants The survey was carried out on a mixed urban and rural population of 298 stu-dents in 17 classes in mainstream secondary education in central Italy. These 15 year-old non-language specialist students, who were relatively inexperi-enced L2 listeners, were recruited as intact (whole) classes4. 64 students from two classes at Istituto tecnico industriale and a further two from Istituto al-berghiero constituted the informant sample for the investigation of comprehen-sion of discourse at slowed speech rate. 3.2 Materials: listening passages and tasks The four listening passages used in the survey consist in a variety of contextu-alized exchanges/conversations and presentations, each featuring a limited number of identifiable focal points. They vary in duration from 2'30" to 3'30": sufficient to delineate a specific context while not so long that memory of initial points would fade in the effort to catch subsequent information. They include both pedagogically structured simulated discourse typical of language learning and testing materials and semi-spontaneous authentic discourse. Three of the passages are transactional, in that they have a defined communicative purpose to achieve an objective (specifically to make an enquiry, make ar-rangements, give information) and the information load is greater than in au-

4) Two classes from university oriented Liceo scientifico, one class from university oriented Liceo

classico, six classes from Liceo artistico specializing in visual arts, two classes from Istituto Tecnico Industriale Statale specializing in applied sciences, technology and industrial proce-dures, six classes from Istituto Professionale di Stato per i Servizi Alberghieri e della Ristora-zione (hotel and catering vocational school).


___________________________________________________________________________

– 191 –

thentic exchanges in an equivalent time span because such characteristics of authentic speech as circumlocution, repetition, hesitation and backtracking rarely feature in simulated interaction. There is one interactional authentic ex-change in which the participants communicate for the sake of conversation rather than for a preordained purpose.

Task 1, a purpose built simulation of a telephone conversation of 2'43" du-ration at natural speech pace and 3'02" at slowed speech rate, consists in an enquiry about a possible employment opportunity. A deliberate effort was made to ensure that topics, vocabulary, phrases, speed of delivery, etc. should be appropriate to intact classes of 15 year-old test takers in mainstream secon-dary education. Task 2 is an authentic conversation5 of 3'24" duration at natu-ral speech pace and 4'44" at slowed speech rate featuring repetition, hesitation and backtracking. Although some slurring of pronunciation, unfamiliar expres-sions and emphatic interjections might be difficult for a school-going popula-tion, the information load is not excessive. Task 36, of 2'85" duration at natu-ral speech pace and 3'49" at slowed speech rate, consists in an audio telephone recording giving information about three different restaurants in a city in Great Britain. The task requires students to understand factual information according to a set sequence. Some difficulty is presented by information embedded in extended discourse, by a logical operation which must be carried out and by the unfamiliar a.m./p.m. time modality. Task 4, also taken from London Test of English, Level 2, Task 1 (past paper, May 2007), is a simulated conversa-tion of 3'25" duration at natural speech pace and 4'09" at slowed speech rate in which three friends arranging to go out to dinner together discuss details such as place, date, time, type of dishes and who to invite. The task consists in ten multiple choice questions with three pictorial options. Speech, standard native British, is expressive but not fast, but the information load is fairly challenging. 3.3 Procedure The survey was carried out in morning school hours during regular English language class time. The experimental procedure, which was strictly identical for all classes, consisted in explaining test instructions and purpose/goal to stu-dents in Italian (L1), a first exposure to audio recording (responses and note taking allowed in L1 or L2), a four minute break for silent individual note re-view and a second listening during which further note-taking was allowed in L1 or L2.

A further administration of the tasks according to a procedure which would distinguish between comprehension of discourse at natural and reduced pace 5) Downloaded from www.elllo.org 6) From London Test of English, Level 2, Task 2. (Past paper, May 2007)

http://pearsonpte.com/Pages/Home.aspx

��

��


___________________________________________________________________________

– 192 –

was carried out in four classes, two at Istituto Tecnico Statale Industriale and two at Istituto Professionale Alberghiero. The passages had been re-recorded at a 10–12% reduction in speech rate by means of the free software Audicity package in order to render them “… clearly and slowly articulated … .” The life-like impression was maintained despite the slight reduction in pace and, for the purposes of the survey, became more accessible to students with A2 level listening comprehension competences. A group of four classes listened at both natural and slowed speech rates. On completion of the procedure outlined above, the students went on to listen a third time at slowed speech rate and wrote their responses in L1 or L2 with red pens on a new set of answer sheets. The tasks were administered according to an anchored design, by which all classes did Task 1 plus another one or two of the remaining tasks 2, 3 and 4, but not all of them.

Two students who had not taken part in the main study followed a think-aloud protocol. They listened to the passages, responded to the items, and re-flected out loud on how they arrived at their answers and why they interpreted specific utterances in the way they did. 3.4 Method The listening texts were analyzed in terms of semantic units, each one corre-sponding to a test question. Discourse characteristics such as explicit state-ment/implicit communication, information embedded in digressive discourse, etc., were annotated along with the corresponding comprehension demands on listeners such as understanding explicit/implicit information, carrying out a logical deduction, etc. Similar to the approach adopted by Guichon and Mc Lornon (2008) and for analogous reasons, the short answer response modality was preferred in most of the questions in tasks nos. 1, 2 and 3: the approach aimed at capturing what students actually understood without guidance from structured multiple choice (m/c) and True/False questions. When a student simply wrote a word or phrase that was associated with the semantic unit, the item was scored as being ‘understood’ even if grammatically inaccurate or misspelled as this did not indicate a failure to comprehend the text. In Task 4 the structured pictorial multiple choice (m/c) modality was retained.

CEFR A2 and B1 levels depict profiles of language users who, to greater (B1) and lesser (A2) degrees, manage to cater for their language needs at sur-vival level. A criterial procedure was followed in marking responses to open questions requiring short answer and single word/phrase responses. Responses were labelled as 1) correct (correct and complete information although not necessarily accurate from the point of view of language), 2) approximate (some essential information with partial indication of details), 3) vague (some detail or elusive reference to required response), 4) irrelevant (some detail mentioned but not in relation to the specific item), 5) incorrect, 6) blank. Re-


___________________________________________________________________________

– 193 –

sponses were scored dichotomously with correct/approximate responses being awarded 1 point and vague/irrelevant/incorrect/blank responses 0 points.

The cut off score for CEFR A2 level included sufficient correct/approxi-mate responses to give evidence that students were able to “… understand enough to be able to meet needs of a concrete type” and “… use an idea of the overall meaning … to derive the probable meaning of unknown words”: just under 50% correct/approximate responses in tasks with a large number of items and 50% in tasks with fewer items. As the CEFR B1 level further in-volved “… extrapolat(ing) the meaning of occasional unknown words from the context and deduc(ing) sentence meaning” and “… understand(ing) the main points …”, the cut off score was set at approximately 66% correct/approximate responses so that it would include items probing both comprehension of overall meaning and perception of detail.

CEFR level profiles indicate typical performances in language use rather than one-off achievement levels, hence a specific profile was attributed on the basis of performance on more than one task according to a five tiered scale: below A2 (fail) was attributed to scores below A2 cut off on all of the tasks undertaken, approaching A2 to scores above A2 or B1 cut offs on one task and below A2 cut off on remaining tasks, A2 to scores above A2 and below B1cut offs on two tasks, approaching B1 to scores above B1 cut off on one task and, in a second task, above A2 and below B1 cut offs, B1 to scores above B1 cut off in two tasks. Scores on Task 1 were included for all classes; both tasks were included for classes which took only two tasks; the higher score was included as the second task and the third task discarded in classes which undertook three tasks.

Task administration in an anchored pattern made it possible to expose all of the classes to varied discourse types and items which varied in focus. For example, Tasks 1 and 2 featured items which probed both overall comprehen-sion and comprehension of details whereas tasks 3 and 4 mainly focused on details. It was particularly interesting to expose students to Task 2, a semi-spontaneous piece of discourse but it was not possible to use it in the voca-tional school because it proved to be too difficult.

Statistical and linguistic analysis consisted in descriptive processing (mean, max, min, range, standard deviation, reliability — Cronbach’s Alpha), item analysis: item facility (p-values) and discrimination, cross check of discourse features against degree of resistance to comprehension (discourse features la-belled with item facility and discrimination values to investigate difficulty and facility areas), comparative analysis of performance on individual tasks: investigation of scoring pattern to ascertain task specific comprehension facility as opposed to even distribution of comprehension facility over two tasks and CEFR level grouping: numbers and percentages of students in relation to CEFR A2 and B1 profiles.

��

��


___________________________________________________________________________

– 194 –

4. Results Processing of results has served, firstly to ascertain whether the tasks have usefully served to carry out quantitative assessment of students’ competences in understanding and interpreting brief contextualized exchanges and presenta-tions, secondly, to delineate listening comprehension profiles of an unselected heterogeneous population in relation to the CEFR A2 and B1 levels and, ulti-mately, to reflect on the comprehension challenge created by specific discourse features. In particular, the more specific analytical processes provide insight on the comprehension strategies students used to interpret discourse. 4.1 Overview of descriptive statistics The range of values in Table 1 depicts a rather heterogeneous group of lan-guage learners. While the mean indicates around 50% or just below success rate in all of the tasks, heterogeneity is noticeable in the considerable distance between maximum and minimum values in tasks 1, 2 and 3. The multiple choice Task 4 produced less heterogeneous results but, in giving the chance to guess at the correct response, features a higher minimum value (0.22). Reli-ability values (Cronbach’s Alpha) are acceptable for tasks 1, 2 and 3; the small sample of students (36) engaged in Task 4 accounts for the lower value (0.371).

Table 1: Whole group 2009–10. Overview of descriptive statistics Task 1 Task 2 Task 3 Task 4

items 12 12 15 10 test takers 298 205 125 36 mean 0.5 0.5 0.490 0.436 max 0.92 0.91 0.8 0.75 min 0.14 0.12 0.02 0.22 range 0.78 0.79 0.78 0.53 SD 2.63 3.17 3.05 1.81 theoretical SD 0.78 0.73 0.88 0.52 Cronbach’s Alpha 0.748 0.824 0.738 0.371

4.2 Results in relation to educational curriculum A comparison of results produced by the various types of school in the sample, Table 2, reveals a number of differences. Having been taken by all classes in the study, Task 1 results can portray the degree of heterogeneity within classes in the specific curriculum types. Student competences at Liceo artistico and vocational school (IPSSAR) show a marked degree of heterogeneity, the for-mer making up 40% of the lower and 32.5% of the upper score band and the latter 47.5% of the lower and 15% of the upper score band. Taken together,


___________________________________________________________________________

– 195 –

Liceo classico and Liceo scientifico show relative homogeneity with 2.5% of the lower and 48.75% of the upper score band. Competences at ITIS tend to be grouped in the middle score band (not shown in Table 2) given that they only make up 10% of the lower and 3.75% of the upper score bands.

Considering that Task 2 was undertaken mainly by students following more academically oriented curricula (Liceo classico, Liceo scientifico) and mixed academic and applicative curricula (Liceo artistico, Istituto tecnico in-dustriale), the rather high maximum value (0.91) and 50% mean success rate conform to expectations generated by school based assessment; likewise, the lower mean (0.490 and 0.436) and maximum (0.8 and 0.75) values in tasks 3 and 4, which were undertaken by students attending Istituto tecnico and voca-tional school (Task 3) and vocational school only (Task 4), also conform to school based assessment results.

Table 2: Top-bottom distribution per curriculum type in Task 1

Lower score band lic cl & sci 2.5% lic art 40% ITIS 10% IPSSAR 47.5% 100% tot. students N= 80 Upper score band lic cl & sci 48.75% lic art 32.5% ITIS 3.75% IPSSAR 15% 100% tot. students N= 80

4.3 Results in terms of CEFR levels It has been possible to portray alignment of survey results to CEFR levels both in relation to overall student performance per individual task and in terms of individual student performance on two tasks.

Overall performance per task, Table 3, shows a fairly even distribution among levels for Task 1, bi-modal grouping (peaks) at B1 and below A2 (fail) for Task 2 and few B1 performers and a high failure rate (below A2) in Tasks 3 and 4 (administered to students following less academic curricula). The bi-modal B1- below A2 result for Task 2 suggests that, when students’ compe-tences can rise to the challenge, comprehension of most of the items is not a problem but, when students’ competences are unsure, comprehension of even a few items poses serious problems.

��

��


___________________________________________________________________________

– 196 –

Table 3: CEFR distribution per task whole group

Task 1 Task 2 Task 3 Task 4 PASS A2 scores and above 79% 64% 62% 44% B1 scores 36% 36% 15% 8% A2 scores 43% 28% 46% 36% FAIL below A2 21% 36% 38% 56% tot. students on task N= 298 205 125 36

In the more conservative five tiered CEFR classification of individual students’ combined performance on two tasks, Table 4, competence levels can be seen to group fairly evenly over the complete range of levels. Table 4: CEFR classification of results on 2 tasks

below A2 approaching A2 A2 approaching B1 B1 students N= 40 72 59 63 64 % students 13.4 24.2 19.8 21.1 21.5

4.4 Individual student performance on pairs of tasks A comparison of performance on pairs of tasks per individual student shows a marked variation in scores on tasks rather than an even sprinkling of points over both tasks. For the tasks 1 and 2 combination, there was a 36.3% varia-tion, for the tasks 1 and 3 combination, a 55.9% variation, for the tasks 2 and 3 combination, an 86.7% variation, and for the tasks 3 and 4 combination, a 56.3% variation. Variation in the Task 1 and Task 4 combination was less marked (26.7%). Performance variation of individuals over tasks could be ex-plained by contextal clues which aid comprehension, for example, when con-text becomes clear at the early stages of listening, the remaining pieces of in-formation might be more easily understood. 4.5 Speech pace variation Two versions of the tasks were administered to 64 students in two classes at vocational school (IPSSAR) and two at technical school (ITIS). For each of the tasks, students listened twice to the recording at natural pace and filled in their answer sheets with a blue or black pen. Answer sheets were gathered up and then, with red pens and a second answer sheet, students again carried out the tasks, this time listening to the passages at reduced speech rate. The results show a marked improvement in overall scores for the reduced speech rate ver-sions of tasks with 67% of students improving their scores on Task 1, 88% on Task 2, 77% on Task 3 and 39% on Task 4. From a pedagogical point of view, the improvement is significant at the pass-fail cut-off line, Table 4, with considerable numbers of students classified below A2 level on passages at


___________________________________________________________________________

– 197 –

natural speech rate managing to rise above it when listening at slowed speech rate.

Table 5: CEFR A2 speech rate score variation Below A2 natural slow Task 1 25% 8% Task 2 78% 25% Task 3 36% 16% Task 4 50% 33%

During the natural pace plus slow speech rate sessions, tension was lower than it had been in the classes which had listened at natural pace only. Students ap-preciated the chance to listen to the passages at reduced pace because it al-lowed them to clarify their uncertainties. The improvement in their perform-ances would suggest that reducing speed of delivery helps students recognize and make sense of what they already know.

Table 6: Natural speech rate vs. slowed speech rate scores compared

Task 1 Task 2 Task 3 Task 4

speech rate natural slow natural slow natural slow natural slow PASS

A2 and above 75% 92% 22% 75% 64% 84% 50% 67%

B1 scores 12.50% 9% 17% 11% A2 scores 62.50% 13% 47% 39%

FAIL below A2

25% 8% 78% 25% 36% 16% 50% 33%

no score change

28% 9% 16% 61%

improvement slow

67% 88% 77% 39%

disimprovement slow

5% 3% 8%

tot. students on task N=

64 64 32 32 64 64 18 18

The comparison of performances for natural pace and slowed speech rate listening is relevant to learners in the lower A2 compentence range, but not those in the B1 range: by definition, the B1 profile cannot be attributed to those unable to understand discourse at natural pace because the overall char-acteristic of the B1 profile is independence in communication. Table 5 gives a comprehensive breakdown of data.

��

��


___________________________________________________________________________

– 198 –

4.6 Discourse features vs. item difficulty While statistical analysis tells us which items are difficult, cross-checking re-sults against discourse features can help to explain why. From an analysis of data, it would appear that a combination of factors might account for the rela-tive ease or difficulty with which test takers tackled the questions in the sur-vey. While it would be rather simplistic to attempt to establish a direct inverse relationship between values for correct responses (p-values) and complexity of discourse, in that no distinct pattern emerges to justify the assumption that the more complex the discourse the greater the percentage of incorrect responses, particular features of discourse warrant attention.

Analysis of responses suggests that identifying general information and dealing with discourse featuring multiple complexities appear to be more chal-lenging than identifying details. On the other hand, culturally familiar and con-textually predictable content couched in explicit linear discourse is less chal-lenging. Discourse characteristics also seem to influence the comprehension challenge. The kind of simulated discourse typical of language learning activi-ties featured in Tasks 1, 3 and 4 tends to be linear, explicit and rational and it is also densely informative because it lacks characteristics of authentic speech such as circumlocution, repetition, hesitation, backtracking, and so forth. Tasks 1 and 4 contain both factual information and transactional give and take; Task 3, a concentrated series of facts, features a number of items requiring careful reasoning in order to extract the correct information. On the whole, students were fairly successful in tackling these tasks.

The kind of spontaneous conversation not scripted for structured language lessons or tests featured in Task 2 posed a marked challenge to comprehen-sion. Being rather digressive and inferential with a lot of talk and just a few facts, it did not carry a heavy information load, but the conversational-interac-tive mode of the exchange, in which personal reflection merged with a factual account of Marion’s experiences in Italy, authentically seasoned with laughter, emphatic interjections, hesitation and repetition, gave an impression of com-plexity. It proved to be the most challenging of the four tasks. However, the two classes which listened to both natural pace and slowed versions of it did improve performances considerably on the slowed version.

It is interesting to note that, contrary to fears expressed by language teach-ers, complex grammar does not seem to weigh heavily on comprehension. Dif-ficulties seem to lie elsewhere. For example, we can see that difficulty arose when an item required retrieval of information embedded in digressive dis-course (Task 3 no. 28), when more than one opinion was expressed, with dis-cussion revolving around multiple adjective combination (Task 4 no. 47), when there was a shift from negative to affirmative (Task 4 no. 46), when the


___________________________________________________________________________

– 199 –

question required a mental operation such as dividing a number uttered in the exchange by two (Task 3 no. 30).

Rambling exchanges with embedded information sometimes shuffle around a topic and, in alluding to information elicited by items, also shift over to other issues. This is confusing for students and, in fact, they had difficulty in re-sponding to items nos. 3, 5 and 6 in Task 1 and items nos. 20 and 22 in Task 2, despite the fact that Italian words were mentioned in item no. 3 in Task 1 and item no. 20 in Task 2. The conversational mode rather than direct ques-tion-answer sequence probably means that students do not find explicit terms of reference to identify the information they are looking for.

R: And why are you looking for work in Italy?

B: Well, I can prepare many typical Italian dishes, particularly well-known ones such as cannelloni with spinach and ricotta cheese, but I would like to become more spe-cialized in Italian regional cooking. (Item no. 3, Task 1, eliciting reasons for interest in Italy)

Marion: I had one professor for example, who would constantly say to me, in class, in Italian, “Are you following? Are you following?” so it was a bit intimidating.

Todd: Yeah, I don’t think I could do that with my students.

Marion: No, no, it was really awful, he was really singling me out, and so all the other people in the class used turn their head around and look at the strange foreigner in their class who maybe or maybe, maybe could understand or maybe didn’t, but I just felt very nervous and a bit awkward there because I was just waiting for him to pounce on me, “Mi segue? Mi segue?” It was terrible. (Item no. 20, Task 2, eliciting information about the teacher)

Listening and reckoning, requiring students simultaneously to identify informa-tion and reckon the answer to questions, is an added challenge. They involve language, when discussion revolves around a multiple adjective combination, as in

(…) and we all really liked the course. It was difficult — but enjoyable — not at all boring or slow like my last English course. What does Brigitte think about the English course?

A. slow — boring/B. difficult — boring/C. enjoyable — difficult (Item no. 47, Task 4)

and logic, when there is a shift from negative to affirmative, as in

… Unfortunately, it doesn’t open on Monday or Tuesday. …

What’s the problem with the Lebanese restaurant?

Figure 1. Opening hours, Lebanese restaurant (Task 4 no. 46)

��

��


___________________________________________________________________________

– 200 –

or when students must divide a specified number by two, as in

The average price of a meal is £30 for two people. (Task 3 no. 30).

Proper nouns such as names of persons and places, the use of the alphabet in spelling and numbers are discrete elements. Because names are arbitrary in that they cannot be inferred from context, the two items focusing on street names (nos. 26 and 36, Task 3) produced evidence of phonological discrimi-nation unaided by contextual clues. In the case of ‘Church Street’, some per-ceived ‘Shirt Street’ (no. 26), showing correct perception of vowel but blurred ability to distinguish between ch and sh. ‘Tidy Street’ (no. 36) was more trou-blesome. It was admittedly quite difficult to discern, it being possible to per-ceive almost any consonant, e.g. ‘Tiley’, ‘Timey’, ‘Tiny’, in the place of d. Only one student responded ‘Tidy Street’ and many had a guess at ‘Time Street’. Item n. 9, Task 1, requires perception of a culturally unfamiliar name (arbitrary) and interpretation of the English alphabetical code: statistically very “difficult” to understand.

Can I take a note of your name and phone number, please?

Of course. My name is Breda O’Reilly. B-R-E-D-A, O’Reilly O-apostrophe-R-E-I-L-L-Y O’Reilly.

Item n. 19, Task 2, is embedded in a flow of discourse: statistically very “difficult” to understand

(…) the first time I went, my Italian wasn’t so good. I had only studied for two years before I went the first time, so, even though my Italian improved enormously over that first year, in the beginning it was quite difficult.

4.7 Evidence of listening strategies During testing sessions, evidence of students’ comprehension and of how they arrived at their responses was gathered from short answers, remarks on answer sheets and post-test discussion; further evidence was subsequently gathered from think aloud interviews with two students. Overall, the quantitity, variety and quality of information gathered are in proportion to the degree of freedom

Figure 2. Eliciting devices vs. freedom of expression

think aloud interview

short

answer

T/F

m/c

True/False T/F; multiple choice m/c


___________________________________________________________________________

– 201 –

of expression allowed by eliciting devices (items, post-test discussions and think aloud interviews): the less structured the device, the more the informa-tion (Figure 2). However, the more structured devices do force students to fo-cus attention where the tester desires whereas the less structured ones run the risk that they will focus on what the tester considers to be less important points. In constructing meaning, it appears that students use lexical clues, interpret vocal expression, use their knowledge of the world, use their acquired lan-guage knowledge, when puzzled, suggest a meaningful interpretation rather than a nonsensical response, accommodate input to their personal narrative of the world, are indifferent to grammatical clues and use clues in answer sheet (test wiseness).

Post-test discussion and think aloud interviews have been useful in reveal-ing students’ ability to make connexions between the various utterances in the passages, an aspect which might have gone unnoticed if analysis had been lim-ited to answer sheet responses. During her think-aloud interview Clelia re-marked that it was obvious that the restaurant staff could speak English well (Item 12, Task 1) simply because the whole conservation had been carried out in English: the reassurance, “No problem, our staff speak fluent English” in response to Breda’s admission that she was still struggling to learn Italian (Item 11, Task 1) acted as confirmation.

Graham et al. (2008) remark that the more proficient students seldom need to employ comprehension strategies because a command of language is in itself a sufficient condition to understand everyday speech while, on the other hand, the state of students’ listening strategies can be investigated in less proficient students.

Item no. 3, Task 1, “Reason for interest in Italy”, gave indications on the use of comprehension strategies. Those who responded correctly looked below the surface of widely held notions on personal eating preferences and common-places on Italian cooking while a lack of comprehension strategies shows up in the frequent tentative response

perché le piace la cucina; per esempio i cannelloni

as interpretation of

I can prepare many typical Italian dishes, particularly well-known ones such as cannel-loni with spinach and ricotta cheese, but I would like to become more specialized in Italian regional cooking.

To respond correctly, the listener must understand information about compe-tences and aspirations with can as an important cue. This particular item did, however, show a tendency of less competent students to clutch at an obvious meaning without adequate verification of what has been said.

��

��


___________________________________________________________________________

– 202 –

Constructing erroneous meaning representations is something which not only the weaker students risk doing. Even fairly competent listeners can get carried away by a too personal interpretation, perhaps constructing meaning according to their own narrative of the world, as is revealed by a student’s comment

She had intimidating occuard moments with her Italian professor


… it was great, one of the best years of my life. (item no. 24, Task 2)

Another student commented

un professore la seguiva nello studio, e le urlava di seguire la lezione


Marion: … I had one professor for example, who would constantly say to me, in class, in Italian, “Are you following? Are you following?” so it was a bit intimidating.

T: Yeah, I don’t think I could do that with my students.

M: No, no, it was really awful, he was really singling me out, and so all the other peo-ple in the class used to turn their head around and look at the strange foreigner in their class who maybe or maybe, maybe could understand or maybe didn’t, but I just felt very nervous and a bit awkward there because I was just waiting for him to pounce on me, “Mi segue? Mi segue?” It was terrible.

T: “Mi segue?” is “are you following?”

M: Are you following me? literally, yeah, basically, Do you understand? (item no. 20, Task 2)

When they give reasons for their responses, we can sometimes get close to students’ thought processes, in the sense that we discover the hypotheses they have made in order to arrive at their responses. During a listening session a student became upset because, although unable to distinguish a word, he felt he had understood the answer to the question and wanted permission to write it down. During the post-test discussion, he volunteered

Ho pensato, dato che gli indiani non mangiano la mucca perché è sacra … ho pensato alla verdura.

as explanation for his response to

If you prefer Indian food, the Prince of Bengal could be a good choice for you. Vikram Kumar, the new head chef, is famous for his vegetable dishes, which have received ex-cellent reviews in local and national newspapers. (Item n. 30, Task 3)

Reflection on overall evidence of comprehension strategies, on the one hand, suggests that many students respond when they feel fairly sure that they have understood correctly but, when in doubt, leave a blank rather than give a tenta-tive response while, on the other hand, many students who take risks can often


___________________________________________________________________________

– 203 –

be seen to be guessing wildly. A smaller group shows ability to match up the pieces in the puzzle, as it were.

5. Discussion The study tackled the task of describing L2 listening competences in a school-going teenage population consisting in whole classes which were made up of students at very diverse competence levels. Two CEFR competence levels for listening comprehension and comprehension strategies were used: A2 level in-volving concrete communication needs provided speech is clearly and slowly articulated and B1 level essentially implying independent comprehension of discourse on familiar matters. The assessment tool designed for the study con-sisted in four listening passages and accompanying test items. All the students in the sample listened to two or three passages at natural (unmanipulated) speech pace and responded to the items on the answer sheet. A select group of four classes listened to the passages a third time at slowed speech rate and re-sponded to items again on a second answer sheet using a red pen.

Results on listening to two passages at natural speech pace indicated that 21.5% of students were at or above B1 level, 19.8% at A2 level, 24.2% just under but approaching A2 level and 13.4% below A2 level. The proportion of students at or above B1 level differed widely over the various types of school — from almost 50% at Liceo classico and Liceo scientifico to around 15% at IPSSAR (vocational school) — but generally confirmed school based assess-ment. On the other hand, the considerable percentage of students classified below and just under A2 level required further investigation in order to ascer-tain the precise nature of their difficulties with listening. In classes where stu-dents’ levels were more homogeneous, mainly at Liceo classico and Liceo sci-entifico, the listening passages at natural speech pace functioned adequately in gathering evidence of competences in terms of CEFR B1 and A2 levels.

The mini-survey using passages at both natural and slowed speech rate has provided insight on the nature of the low scorers’ competences. The distribu-tion over CEFR levels changed noticeably with most of the listeners improving their scores to a greater or lesser extent on the various tasks at slowed speech rate. On Task 1, which was taken by all students, only 8% remained below A2 level and an average of 15% achieved scores below A2 over the four tasks.

These results suggest that manipulating speech rate makes it possible for learners at lower levels to understand continuous discourse of various types: pedagogically constructed in Tasks 1, 3 and 4 and authentic native speaker dis-course in Task 2. The results are in line with Meinardi’s Ditcall software pro-ject (2006; 2009) to make native speaker discourse accessible to immigrants at all levels of learning, including the lower levels. By helping immigrants to be-come accustomed to native speaker discourse, it can promote their speedy in-

��

��


___________________________________________________________________________

– 204 –

tegration as active members of society. In a non-native educational setting where contact with native speakers is rare, it can act as an amalgamating agent in large mixed-level classes of learners. Listening at slowed pace can make the student group more homogeneous, with lower level students working autono-mously to improve their competences, and more proficient students tackling a wide variety of discourse. As with immigrants, assiduous use of the tool can also facilitate access to a whole range of native speaker discourse.

On the pedagogical side, the changed distribution in CEFR competence levels allows us to see competence and knowledge analytically: while students who achieve B1 at natural pace reveal both adequate competence and adequate knowledge, those who rise from below A2 on natural pace to A2 and above on slowed pace might be said to reveal adequate knowledge but inadequate com-petence in dealing with discourse in real time.

Strategy use is an important issue in the context of the present study be-cause helping such relatively young students to improve in this area will offer them considerable future benefits. The consistent numbers of students in the CEFR A2 level score range provide useful data for reflection. Generally they manage to process several items in a task on the basis of perception, i.e. in bottom-up fashion, and, when an item poses perception problems, many tend to provide a contextually plausible although rather obvious response without adequate probing of all the information available in the text. These findings are consistent with Vandergrift’s (2003; 2006), Goh’s (2000) and Goh and Taib’s (2000; 2006) studies of listening comprehension and would, as Vandergrift and Goh had previously done, suggest the need for a specific pedagogic pro-gramme for the development of listening competences.

Overall, students’ competence profile does not lag very far below the pro-file of the CEFR B1 independent language user, hence the specialized listening comprehension activities would envisage a fairly practicable educational goal. However, it is a goal not to be neglected in view of increasing necessity for second language competences in the lives of all citizens. 6. Some implications Structured models and frameworks of language learning and assessment, whether they be the CEFR or some other set of guidelines, have an important place in the learning environment. In Europe, work on the original Threshold Level, which blossomed over the years into the CEFR, has promoted aware-ness of the importance of communicative competences in language learning and has served to overcome past emphasis on knowledge and accuracy for its own sake. In an increasing number of educational systems, learning objectives and assessment are inspired by the CEFR. But there are still many issues in language learning and assessment which need attention. While everybody in


___________________________________________________________________________

– 205 –

our global society needs to learn languages, we are only slowly becoming aware of new dimensions and implications for less academic student groups with work oriented language learning needs because our conceptions are still moulded by how a culturally aware and literate minority learned languages in the past. Learning and assessment frameworks and models, including the CEFR, grew from this scenario and it is likely that future research will offer more complex insights. The findings in the present survey carry implications on assessment of heterogeneous groups of learners: by using the same tasks for all with two variations in speech rate it was possible it distinguish between various dimensions of listening skills and strategies. On the other hand, many exams, including recently launched external exams for schools, impose an a priori sorting of learners into levels after which they test them to see how high or low they come within a particular level. It is a healthy sign that doubts are being expressed about the appropriateness of the CEFR to describe the learn-ing process: in pointing out that we have designed a linear CEFR scale to con-ceptualize and guide a non-linear learning process, Alanen et al. (2010) lead us to pose the question of whether exams should condition the learning process or the learning process condition exams.

Learning activities and formative classroom assessment, being intrinsically intimate and loosely structured, can only episodically link to the CEFR. Of course, no harm is done if students’ learning does not assiduously link to CEFR levels and descriptors or international exams: what is important is that they should learn. But the moment for measuring a class’s learning in terms of an outside standard does inevitably arise and it is important for teachers and students to understand how they relate. Otherwise they risk falling into passive acceptance of the assumption that a given set of exam tasks represents, for ex-ample, CEFR B1 or A2 level, simply because a particular exam board has stated that it is so.

Set in the extremely varied learning environments characteristic of unse-lected and unstreamed classes in mainstream education, the present study has developed a procedure to map listening competences to CEFR A2 and B1 lev-els and descriptors. It lays bare task design, materials, specifications, scoring method and analytical procedures used to capture students’ competences so that those interested can see for themselves how they have been linked with the CEFR. At a basic level, the procedure could be used for the purpose of devel-oping students’ listening competences or for formative classroom assessment in any educational setting and, with some additional statistical elaboration, teach-ers could map their students’ performances onto the CEFR. Exploiting CEFR descriptors and levels, which will help to demystify examinations and the CEFR, will ultimately empower teachers to bridge the gap between their own peculiar micro-learning and assessment environments and external examina-tions. Indeed, with external examinations relentlessly moving into even the

��

��


___________________________________________________________________________

– 206 –

early stages of mainstream education, teachers are in a position to understand whether a particular exam is an effective instrument in promoting further learning (a function it should perform in interim stages of education), and ul-timately guide exam boards to design better exams. References

Aglioti, Salvatore / Fabbro, Franco. 1999 “Cervello poliglotta e apprendimento delle lingue”. Le scienze 365, 54–59.

Alanen, Riikka / Huhta, Ari / Jarvis, Scott / Martin, Maisa / Tarnanen, Mirja 2010 “Combining language testing and second language acquisition research — insights from

Project CEFLING”. Centre for Applied Language Studies & Department of Languages, University of Jyväskylä. Paper delivered at The 7th Annual EALTA Conference Col-laboration in Language Testing and Assessment. The Hague: European Association for Language Testing and Assessment.

Al-Jasser, Faisal 2008 “The effect of teaching English phonotactics on the lexical segmentation of English as a

foreign language”. System 36, 94–106.

Anderson, John R. 1995 Cognitive Psychology and its Implications. 4th Edition. New York: Freeman.

Best, Catherine T. 1995 “A direct-realist view of cross-language speech perception”. In: Winifred Strange (ed.),

Speech Perception and Linguistic Experience. York Press, Baltimore, MD, 233–272.

Broersma, Mirjam / Cutler, Anne 2008 “Phantom word activation in L2”. System 36, 22–34. Brown, Gillian 2006 “Second language listening”. In: Keith Brown (ed.), second ed., Encyclopaedia of Lan-

guage and Linguistics, vol. 11 Elsevier, Oxford, 81–88.

Buck, Gary 2001 Assessing Listening. Cambridge, UK: Cambridge University Press.

Council of Europe 2001a A Common European Framework of Reference for Languages: Learning, Teaching,

Assessment (CEFR). Cambridge, UK: Cambridge University Press. [Also on-line: http://coe.int/portfolio]

2001b Structured overview of all CEFR scales. Strasbourg: Council of Europe. [http://www.coe.int/T/DG4/Portfolio/?L=E&M=/documents_intro/Data_bank_de-scriptors.html]

Cutler, Anne 1990 “Exploiting prosodic probabilities”. In: Gerry T. M. Altmann (ed.), Cognitive Models

of Speech Processing: Psycholinguistic and Computational Perspectives. Cambridge, MA: MIT Press/Bradford Books, 105–121.

1997 “The comparative perspective on spoken-language processing”. Speech Communication 21, 3–15


___________________________________________________________________________

– 207 –

Dehaene, Stanislas/ Dupoux, Emmanuel / Mehler, Jacques / Cohen, Laurent / Paulesu, Eraldo / Pe-rani, Daniela / van de-Moortele, Pierre-François / Lehericy, Stéphane / LeBihan, Denis

1997 “Anatomical variability in the cortical representation of first and second language”. Neuroreport 17, 3809–3815.

de Jong, Kenneth J. / Silbert, Noah H. / Park, Hanyong 2009 “Generalization Across Segments in Second Language Consonant Identification”. Lan-

guage Learning 59 (1), Wiley-Blackwell,1–31.

Ek, Jan A. van 1975 The Threshold Level in a European unit/credit systemfor modern language learning by

adults. Strasbourg: Council of Europe.

Ek, Jan A. van / Trim, John 1991 The Threshold Level 1990. Strasbourg: The Council of Europe.

Escudero, Paola / Boersma, Paul 2004 “Bridging the gap between L2 speech perception research and phonological theory”.

Studies in Second Language Acquisition 26 (4), 551–585.

Field, John 2004 “An insight into listeners’ problems: too much bottom–up or too much top–down?”

System 32, 363–377. 2008 “Guest editor’s introduction. Emergent and divergent: A view of second language lis-

tening research”. System 36, 2–9.

Flege, James E. 1995 “Second language speech learning: theory, findings and problems”. In: Winifred

Strange Ed. Speech Perception and Linguistic Experience. Baltimore, MD: York Press, 233–277.

Goh, Christine M. 2000 “A cognitive perspective on language learners’ listening comprehension problems”.

System 28, 55–75. Goh, Christine M. / Taib, Yusnita 2006 “Metacognitive instruction in listening for young learners”. ELT Journal 60, 222–232.

Graham, Suzanne / Santos, Denise / Vanderplank, Robert 2008 “Listening comprehension and strategy use: A longitudinal exploration”. System 36,

52–68.

Guichon, Nicholas / McLornan, Sinead 2008 “The effects of multimodality on L2 learners: Implications for CALL resource design”.

System 36, 85–93.

Long, Donna R. 1989 “Second language listening comprehension: a schema-theoretic perspective”. Modern

Language Journal 73, 32–40. 1990 “What you don’t know can’t help you”. Studies in Second Language Acquisition 12,

65–80.

Lynch, Tony 2006 “Academic listening: marrying top and bottom”. In: Alicia Martinez-Flor, Esther Usó-

Juan (eds.), Current Trends in Learning and Teaching the Four Skills within a Commu-nicative Framework. Berlin: Mouton de Gruyter, 91–110.

Markham, Paul / Latham, Michael 1987 “The influence of religion-specific background knowledge on the listening comprehen-

sion of adult second-language students”. Language Learning 37, 157–170.

��

��


___________________________________________________________________________

– 208 –

Mc Clelland, James L. / Rumelhart, David E. and PDP Research Group 1986 Parallel distributed processing: explorations in the microstructure of cognition. Volume

2: psychological and biological models. Cambridge, MA: MIT Press.

McQueen, James 1998 “Segmentation of continuous speech using phonotactics”. Journal of Memory and Lan-

guage 39, 21–46. 2004 “Speech perception”. In: Koen Lamberts / Robert L. Goldstone (eds.), The Handbook

of Cognition. London: Sage, 255–275.

Meara, Paul 1997 “Towards a new approach to modelling vocabulary acquisition”. In: Norbert Schmidt,

Michael McCarthy (eds.), Vocabulary: Description, Acquisition and Pedagogy. Cam-bridge: Cambridge University Press, 109–121.

Meinardi, Marty 2006 The Effects of Listeners’ Control of Speech Rate on Second Language Comprehension.

Unpublished PhD thesis. Dublin Institute of Technology. 2009 “Speed bumps for authentic listening material”. ReCALL 21(3): 302–318.

Norris, Dennis G. / McQueen, James M. / Cutler, Anne / Butterfield, Sally 1997 “The possible-word constraint in the segmentation of continuous speech”. Cognitive

Psychology 34 (3), 191–243.

O’Malley, J. Michael / Chamot, Anna U. / Küpper, Lisa 1989 “Listening comprehension strategies in second language acquisition”. Applied Linguis-

tics 10, 418–437.

Pallier, Christophe / Colomé, Angels / Sebastián-Gallés, Núria 2001 “The influence of native language phonology on lexical access”. Psychological Science

12, 445–449.

Rivers, Wilga M. / Temperley, Mary S. 1978 A practical guide to the teaching of English as a second or foreign language. New

York: Oxford University Press.

Rumelhart, David E. / Mc Clelland, James L. and PDP Research Group 1986 Parallel distributed processing: explorations in the microstructure of cognition. Volume

1: foundations. Cambridge, MA: MIT Press.

Schmidt-Rinehart, Barbara C. 1994 “The effects of topic familiarity on second language listening comprehension”. Modern

Language Journal 78 (2), 179–189.

Vandergrift, Larry 2003 “Orchestrating strategy use: toward a model of the skilled second language listener”.

Language Learning 53 (3), 463–496.

Vandergrift, Larry / Goh, Christine M. / Mareschal, Catherine J. / Tafaghodtari, Marzieh H. 2006 “The Metacognitive Awareness Listening Questionnaire: Development and Validation”.

Language Learning 56 (3), 431–462.

Weber, Andrea / Cutler, Anne 2006 “First-language phonotactics in second-language listening”. Journal of the Acoustical

Society of America 119, 597–607.

Wenden, Anita 1991 Learner Strategies for Learner Autonomy. Hertfordshire, UK: Prentice Hall.

Widdowson Henry G. 1978 Teaching Language as Communication. Oxford: Oxford University Press.


___________________________________________________________________________

– 209 –

1979 Explorations in Applied Linguistics. Oxford: Oxford University Press. 1983 Learning Purpose and Language Use. Oxford: Oxford University Press.

Wilkins, David A. 1976 Notional Syllabuses. Oxford: Oxford University Press.

Zhao, Yong 1997 “The Effects of Listeners’ Control of Speech Rate on Second Language Comprehen-

sion”. Applied Linguistics 18, 44–68. Web resources

Audacity The free software package, audio applications. Available at http://www.audacity.sourceforge.net/ [retrieved 12/01/2010]

European Commission Education & Training. The Lifelong Learning Programme: education and training opportunities for all http://ec.europa.eu/education/lifelong-learning-programme /doc78_en.htm [Accessed 15/06/2010]

English Listening Lesson Library Online. www.elllo.org [Material retrieved 12/12/2009] Pearson Language Tests PLT. London Test of English, Level 2. Past paper, May 2007.

http://pearsonpte.com/Pages/Home.aspx [retrieved 13/03/09]

��

��

Oralité(s) et écriture(s) Oralité(s) et écriture(s) Are they following

Documents

Transcript of Oralité(s) et écriture(s) Oralité(s) et écriture(s) Are they following