Assessment-Mini Assignment 1_reading-writing

Damianopoulos Damianos H.O.U. Academic Year: 1999-2000 Mini-Assignment Module: ΑΓΓ 61

Hellenic Open University

MINI-ASSIGNMENT 1

MODULE ΑΓΓ 61

Assessment in Language Learning

Academic Year: 1999-2000

Tutor: Dr Nikolaos Sifakis


2TABLE OF CONTENTS

Foreword 3

1. General description of the PET 3

1.1 Identity of the test 3

1.2 Organisation-Grading-Administration 3

1.3 The identity of the Candidates 4

2. Categorisation of Paper 1 4

2.1(Reading) 4

2.1.1 Part 1 (The Signs) 4

2.1.2 Part 2 (The Short Texts) 5

2.1.3 Part 3 (The Longer Factual Text) 6

2.1.4 Part 4 (The Longer Continuous Text) 6

2.1.5 Part 5 (The Cloze Text) 7

2.2(Writing) 8

2.2.1 Part 1 (The Transformations) 8

2.2.2 Part 2 (The Form Completion) 8

2.2.3 Part 3 (The Continuous Writing) 9

3. Evaluation 10

3.1 Marking Scheme 10

3.2 Washback and Washforward 10

3.3 Application of the PET in our teaching situation 11

BIBLIOGRAPHY 12

APPENDIX I 13

APPENDIX II 15

APPENDIX III 21

An Analysis of the


3University of Cambridge Local Examinations Syndicate

Preliminary English Test (Paper 1: Reading and Writing)

Foreword

The object of this paper is a general overview of Paper 1 of the Preliminary English Test (PET) administered by the University of Cambridge Local Examina-tions Syndicate. The test comprises two more component papers, namely Paper 2: Listening and Paper 3: Speaking. Paper 1 is the longest of the three in terms of du-ration (1 hour 30 minutes) and consists of two parts: Reading and Writing.

1. General overview of the PET 1.1 Identity of the test The PET is a proficiency test and it is meant for people needing some official recognition of their ability to use the English language for professional or academic purposes. Therefore the assessment objectives of the test and the syllabus on which it is based require ability to use the language in real life situations, where effective communication is a pivotal point. However, accuracy is also a prerequisite for the successful performance in the test and sufficient care is taken for the fulfilment of this end, too. The learners are expected to be able to manipulate language in real life every day linguistic situations, in which they may come in contact with either native or non-native speakers of the language. As regards their reading proficiency they are ex-pected to be able to understand posters, forms, notices, brochures, street signs, magazines and newspapers and several other types of authentic texts. Concerning listening competence the learners are expected to be able to handle various an-nouncements in public places, radio announcements and instructions, weather forecasts etc. However, it is not only the factual nature of aural information which candidates are expected to handle, but also texts of more subjective nature like opin-ions, moods, wishes, attitudes and other material of more subtle nature. However, the candidates are not expected to be able to handle specialist language required for particular fields of knowledge or professions (PET Handbook UCLES 1999). 1.2 Organisation – Grading – Administration The Reading and Writing parts constitute two quarters of the total mark of the test each weighted to 25% of the general mark, listening and speaking conferring an-other 25% each. The final mark of the test is an aggregate of the three papers whereas the candidates are not required to achieve a minimum mark in order to pass each pa-per. The grades bestowed to candidates of the test are four, namely “Pass with Merit” requiring 85% or more of correct answers, “Pass” requiring 70-84%, “Nar-row fail” being 5% short of the “Pass” lowest grade limit and “Fail” for candidates


4who have not achieved the aforementioned requirements (PET Handbook UCLES 1999). 1.3 The identity of the Candidates As the UCLES PET Handbook (UCLES 1999) states most candidates (70%) are under the age of 20, whereas a 20% are between the ages of 21-30. The purposes PET candidates state for taking the exam are 55% personal interests, 50% in order to improve their employment prospects and a 30% wishing to continue with their study of English.

2. Categorisation of Paper 1 2.1(Reading)

2.1.1 Part 1 (The Signs) This part of the test contains a number of signs, public notices or labels, which can be encountered in real life and the candidates are required to deal with a multiple choice activity choosing the correct answer out of four, which are provided. The technique is a second generation one due to the use of multiple-choice items. However the texts are authentic, although the task is not completely so, as in real life we do not really have any multiple-choice options. The candidates have to understand the texts globally and associate them with some imaginary environment in order to achieve this global understanding. In this respect we could claim that the activity has third generation traits in the sense that it is communicative in its nature and it has some purpose, namely to understand and convey this knowledge to a third party.1 The fact that the candidates do not have to write anything, only to read and un-derstand, safeguards the purity of the test; there is no demand on the part of the can-didate as regards their writing ability.

Another advantage of this part of the test is that it is easy to find relevant au-thentic texts and it is also easy to be scored. Therefore its practicality would be high if the devising of plausible distractors were just as easily achieved. Besides, the fact that the scoring is 100% objective renders this part entirely reliable.

As regards the security of this part, we could claim that it is fairly high, but not

tight, as the number of the questions and relevant answers could be easily signalled with the use of the fingers of only one hand. 2.1.2 Part 2 (The Short Texts)

1 Cf. Appendix II, p. 15, Reading, Part 1, the rubric.


5 The second part of the test deals with five factual texts, namely the profiles of five people, which are longer than those of the previous part. The candidates are re-quired to understand these texts in detail using their scanning and skimming skills. Subsequently they have to match them with five counterparts locating them among eight texts, blurbs of books2, which fit the preferences of the people outlined in the questions. The candidates have to choose from eight texts marked A to H, as three of them are used as distractors. Again, the texts can be authentic or authentic-like in na-ture and the task can be authentic, too. However, the sample, which is being referred to in this paper, does not have any task authenticity, as we rarely, if ever, have to engage in such an activity in real life, namely matching written profiles of people to blurbs. As with the previous part, the performance of the candidates is not threatened by any demands on their writing competence, thus the purity of the test being se-cured. Regarding the practicality of this part we could contend that it is fairly high as scoring is quite straightforward, whereas there might be some difficulty as regards the preparation of the material, mainly the people’s profiles. Furthermore, due to the fact that the scoring of the activity is entirely objective we can claim that its reliability is high, as scorers would never disagree about correct answers. Lastly, the security rating of this part could be characterised as quite high, due to the large number of options the candidates have before them rendering the possi-bility of cheating entirely remote. 2.1.3 Part 3 (The Longer Factual Text) In this part the candidates have to scan a longer factual text and answer ten dual choice (True/False) questions. The text provided in the sample presented in Appendix II3 is of authentic nature; however the rubric of the activity and conse-quently the task are not authentic, as the purpose of the task posed before the candi-dates is to decide on the correctness of the statements rather than to fulfil some other contextualised objective. Therefore, we might say that the text is third generation due to its authenticity; the task, however, has a second generation trend, as it is not entirely authentic. Due to the form of the questions, which require the candidates to employ their scanning skill, the comprehension of the text can only be fragmental, as is the case in real life with similar texts. 2 Cf. Appendix II, p. 16, Reading, Part 2. 3 Cf. Appendix II, p. 17, Reading, Part 3.


6 Again as in the previous parts, the writer of the test has taken care that no de-mand is made on the writing skill of the candidates, avoiding jeopardising their per-formance by demanding no response of any continuous text. From the viewpoint of practicality this part is not very difficult to prepare or score, a fact, which renders it easy to produce. Besides, the scoring is 100% objec-tive, which adds to the reliability of the test in general. In terms of security, we believe that dual-choice questions are quite vulner-able to cheating violations on the part of the candidates, as they are easy to signal. 2.1.4 Part 4 (The Longer Continuous Text) Here the candidates are called to understand a longer continuous text glob-ally, whereas they have to display their abilities to understand people’s opinion, atti-tudes, feelings and generally more subtle content, which requires higher reading skills. While such a text can be authentic and the task can be fully contextualised, the sample which is provided in the UCLES handbook does not seem to be authentic and the task lacks authenticity entirely, as the candidates are required to read the text and answer multiple-choice items. Therefore, we believe this part has more second than third generation traits for lack of a task environment compatible with real life needs. The only exception could be item #25 where the candidates have to choose a sign, which reflects the main idea or the article. As in previous parts, reading is purely examined here without any demands on other skills, while the practicality of this part is low due to the difficulty to prepare items with plausible distractors. The scoring of this part is entirely objective; therefore it is 100% reliable. Be-sides, the security is not very high due to the multiple-choice technique, which is employed here. 2.1.5 Part 5 (The Cloze Text) This part is completely different from the foregoing ones as regards technique and purpose. The candidates here have to deal with a random cloze, in which they have to fill in choices, which range from nouns to prepositions, verbs, participles and adverbs. The focus here is obviously vocabulary and grammar within a multiple-choice environment, where there is total absence of contextualisation or authentic-ity of task, nor is there any authenticity of text, either. It is the part of the test, which we can claim is purely second-generation material.


7 Again the purity of this part is not jeopardised with demands on the candi-dates’ writing skill, while the practicality of the test is high, due to the fact that the items are quite easily prepared. Being second-generation material the scoring is totally objective; therefore the reliability of this part is of top rating. However the security of this part may suffer, as it is quite easy again for candidates to signal each other the correct answers.

2.2(Writing) 2.2.1 Part 1 (The Transformations) In this part the candidates are called to transform five sentences, which touch upon a number of grammatical structures with the purpose to test their accuracy in using the language. Therefore, as regards the technique of this part, it is purely sec-ond-generation because such an activity is not connected with linguistic needs in real life. Neither is there communicative authenticity of task nor process or product authenticity: that is we never transform sentences when we communicate in real life and the sentences, which the candidates are called to produce, are not authentic in na-ture nor do they serve any real or realistic-looking purpose. Due to the fact that this part focuses entirely on accuracy, it is natural to lack some more characteristics distinctive of communicative tasks, namely there is no task en-vironment and of course the sentences produced have no addressee other than the scorer. The activity can be characterised as totally controlled as regards the freedom of the candidates to compose the required sentences; the “starters”, namely the be-ginnings of the response sentences, restrict the candidates to employ a particular pat-tern in the sentence. Therefore, due to the nature of this activity, we deem it can prove particularly useful for diagnostic purposes, namely plumbing whether our examinees have grasped a particular pattern or usage, as well as in an attainment test, examin-ing the overall performance of a body of students at the end of a course. 2.2.2 Part 2 (The Form Completion) Here the test moves one lung higher on the ladder towards communicativeness. Starting with the rubric of the activity we see that the candidate is presented with a situation in which they assume a role and they have to perform a specific task, namely to fill in a questionnaire on their study habits. The writer of the test does not provide any more details regarding the purpose, which this questionnaire is going to serve, probably because they do not wish to jeopardise the performance of the candi-dates by providing a demanding rubric as regards the candidates’ reading skills. Simi-larly no addressee is clearly stated, as the task is not meant to be really communica-tive.


8We believe that this activity could be characterised as both a guided and con-

trolled semi-communicative task, because some items require specific information, as stipulated in the marking scheme.4 Besides, despite the contextualised character of this part, the marking scheme reveals that most items focus on spelling or correct vo-cabulary rather than the ability of the candidates to compose written discourse.5 Therefore, the activity may be realistic in format, however the objective seems to be a test of the language performance of the candidates and not their communicative skills. However, due to its format and the fact that form completion is a daily practice for most people, we may claim that is a third generation activity. The activity is totally safe as regards security as it is impossible for the candi-dates to signal any answers. Regarding the content of the responses they do not cause any imagination load on the part of the candidates, as they have to answer about their own personal situation. 2.2.3 Part 3 (The Continuous Writing) As with the previous part the last writing activity is one degree higher regard-ing its communicative orientation. The candidates are presented with a clearly stated task environment, in which they have to pass news on to an English-speaking friend regarding a day out with friends. The activity is guided as the rubric directs the candidates in terms of type and style of text (friendly letter), addressee (an English-speaking friend), purpose (to pass on personal news), as well as situation (a day out with friends) and content (where, what, why). We believe it is worth noting that the writer of the test does not allow the candidates to choose whether they liked the day or not as it would not have been possible to provide the aforementioned guidelines, which reduce the imagina-tion load on the part of the candidates. However, by reducing the imagination load the writer of the test has also restricted the prospects of creativity on the part of the candidates. Another reason, for which the letter guidelines have been provided, is that the scoring of the product becomes less subjective, as it can be derived from the marking scheme6 for this part. Up to five marks can be awarded for the task, namely inclusion of the points stipulated in the rubric and organisation of the text, and another five for the accuracy displayed. The inclusion of the stipulated points and the language of the produced text can be more or less objectively scored. The only subjectivity of the

4 Cf. Appendix I, p. 13, Writing Paper, Part 2 items nos. 6, 7, 8, 11, 15. 5 Cf. Appendix I, p. 12, Writing Paper, Part 2 items nos. 8, 9, 11. 6 Cf. Appendix I, Writing Paper, Part 3, p.13 and p. 14.


9scoring will derive from the organisation of the text and in that case the difference of opinion of various scorers cannot be extremely divergent. As regards the scoring procedure for this activity the approach is global rather than analytic, as it can be confirmed from the marking scheme employed.7 The task, process and product of the activity can be characterised as authen-tic or authentic-like, since the candidates are aware that they are not writing to a real person and that a scorer rather than a friend is going to be the actual recipient of the letter. From the aspect of security this part is not 100% safe, as the topic is a very or-dinary one and it can have been prepared in one way or other. Otherwise, no trickery can be employed on the part of the examinees towards copying from their fellows.

3. Evaluation 3.1 Marking Scheme From the above analysis of the items of the Reading and Writing parts, in con-junction with the marking scheme,8 it becomes obvious that great care has been taken to construct a test, which will be as reliable as possible. We can see that there is no room for any scorers’ disagreement on most of the items, and for the only part, which could raise some dissent - Writing Part 3, due to the subjectivity in the scoring of the produced text - detailed instructions9 have been provided as to the way this subjectiv-ity can be minimised. However, the validity of the test is not put to risk, as it tests exactly what it claims to test. 3.2 Washback and Washforward From the syllabus, which emerges from the activities of the PET, we can con-clude that the test requires ability to comprehend and manipulate language in authen-tic situations with the use of reading skills such as scanning, skimming and inten-sive reading, as well as writing proficiency in a variety of texts and writing styles. In this way the learners will be able to comprehend texts either globally or in a frag-mentary way depending on their needs and sharpen their efficiency in the synthesis of written communication. Therefore, instruction towards sitting this test will help students to learn how to utilise a linguistic environment in real life. This will have as a result their gradual independence from their instructor, which will help them be-come mature users of the foreign language bringing them benefits in their daily life, academic endeavours or further professional evolution. 7 Cf. Appendix I, Writing Paper, Part 3, p.14. 8 Cf. Appendix I, pp. 13 and 14. 9 Cf. Appendix I, Writing Paper, Part 3, p. 13 and p. 14.


10 3.3 Application of the PET in our teaching situation We have chosen to focus our attention on the PET as we thought that it com-bines excellently with the course-book, Snapshot Pre-Intermediate, which is cur-rently in use in the 3rd form of our School10. We find that this course-book adopts the communicative approach in the teaching of English and the PET could serve as an attainment test at the end of the school term. It could also be used to measure the students’ progress towards taking the PET on their way to sit for the Cambridge First Certificate in English exams. (2.982 words).

10 2nd Junior High School, Corfu.


11BIBLIOGRAPHY

Abbs B., I. Freebairn and C. Barker (1999)

Snapshot Pre-Intermediate. London: Longman.

Alderson C.J. C. Clapham and D. Wall (1996)

Language Test Construction and Evaluation. Cam-bridge: CUP.

Bachman L.F. and A. Palmer (1997)

Language Testing in Practice. Oxford: OUP.

Cambridge Examinations Certificates & Diplomas

PET Preliminary English Test Handbook. University of Cambridge Local Examinations Syndicate.

Canale M. and Swain M. (1980)

Theoretical bases of communicative approaches to second language teaching and testing. Applied Lin-guistics Vol. 1 No. 1.

Harrison A. (1983) A Language Testing Handbook. London: Macmillan Press.

Heaton J.B. (1988/1997) Writing English Language Tests. London: Longman. Heaton J.B. (1990/1997) Classroom Testing. London: Longman. Hughes A. (1989/1996) Testing for Language Teachers. Cambridge: CUP. Jordan RR (1983) Case Studies in ELT. Collins 1983. Morrow K. (1979/1981) ELT documents 111-Issues in Language Testing. The

British Council. Morrow K. (1977) Techniques of evaluation for a notional syllabus. The

Royal Society of Arts. Morrow K. (1983) The Royal Society of Arts. Examinations in the Com-

municative Use of English as a Foreign Language. In Jordan RR (1983).

Weir C. (1990) Communicative Language Testing. Prentice Hall. West R. (1991) Developments In The Testing Of Reading. In English-

A World Language Vol. 1 no. 1.


12APPENDIX I


13


14APPENDIX II


15


16


17


18


19


20APPENDIX III

Table of Skills in Reading and Writing in Snapshot Pre-Intermediate TYPES OF TEXTS UNITS

Interviews 1, 19 Letters and articles 2, 9, 11, 12 Factual information 3, 8, 12, 16 Narrative 6, 14 Wishing 13 Biography 17 Story reading and writing 18

Assessment-Mini Assignment 1_reading-writing

Documents

Transcript of Assessment-Mini Assignment 1_reading-writing