A Comparative Study of Measures of Vocabulary

Knowledge and IELTS Proficiency Scores as

Potential Predictors of Academic Performance

Sandra George (589880)

College of Arts and Humanities

Swansea University

Submitted to Swansea University in fulfilment of the

requirements for the degree of Master of Arts (MATEFL)

14th October 2014

ABSTRACT

This study was motivated by work related observations at

Swansea University, which noticed the need for improved

predictive assessments of international students’ potential

academic performance, in order to better support them in-

sessionally. Additionally, positive results from a range of

studies linking vocabulary knowledge measures with

proficiency levels and their capacity for predicting future

performance in the four skills (Laufer, 1992; Staehr, 2008,

Milton et al, 2010) directed an investigation into how

effectively vocabulary scores could predict academic

performance for L2 learners on a degree programme, and how

these results would compare with the predictive

capabilities of the International English Language Testing

System (IELTS) proficiency bands.

Data was collected from 20 students on a Swansea University

Law exchange programme. The participants’ vocabulary scores

were measured using the XK_Lex test (Masrai, 2009) and a

Vocabulary Size Test (Nation & Beglar, 2007b).

Subsequently, the students’ IELTS band scores and end of

programme grade point averages were correlated with the

vocabulary scores. The results of the study showed

vocabulary knowledge measures to be a more reliable

predictor of academic performance than IELTS. Following on

from this, the pedagogical implications were explored and

suggestions for in-session support were made.

DECLARATION FORM

DECLARATIONThis work has not previously been accepted in substance forany degree and is not being concurrently submitted incandidature for any degree.

Signed: Date: 14th October2014

STATEMENT 1This dissertation is the result of my own independentwork/investigation, except where otherwise stated. Othersources are acknowledged by giving explicit references. Abibliography is appended.

Signed: Date: 14th October2014

STATEMENT 2I understand the college policy on plagiarism as set out inthe “College of Arts and Humanities Handbook for MAStudents”, and accept that this dissertation may be copied,stored and used for purposes of plagiarism detection. Itherefore certify that I have submitted an electronic copyof this dissertation via the Turnitin system on Blackboard.

Signed: Date: 14th October2014

CONTENTS

List of Tables and Figures

Acknowledgements

1. Introduction

2. Literature review

2.1. What is a word?

2.2. What does it mean to know a word?

2.3. How many words does a learner need to know in a

second language?

2.4. Vocabulary knowledge as a predictor of language

performance

2.5. IELTS scores as a predictor of academic

performance

2.6. What makes a good vocabulary test?

2.7. Test types

3. Methodology

3.1. Aims and objectives

3.2. Research Questions

3.3. Participants

3.4. Data collection and testing procedure

4. Results

5. Discussion

5.1. How well do the two vocabulary size measures

correlate with academic performance and IELTS?

5.2. Do any of these tests have a good enough

correlation to enable it to be used as a predictive

test of subsequent academic performance?

5.3. What scores are associated with failure or poor

academic performance to guide us identifying students

in need of subsequent in-session support?

5.4. Can the scores identify areas of knowledge to be

covered by in-session support classes?

6. Conclusion

Appendices

Bibliography

LIST OF TABLES AND FIGURES

TABLES

2.1 What is involved in knowing a word?

2.2 Vocabulary size and coverage

2.3 Comparison of the most frequent vocabulary from three

sources

4.1 Raw IELTS, XK_Lex, VST and GPA scores

4.2 Mean scores on vocabulary size tests and GPAs

4.3 Correlations of vocabulary tests and IELTS with GPAs

4.4 Component matrix

4.5 Group statistics

4.6 Independent samples test

5.1 The four strands and their application with a focus on

vocabulary

FIGURES

2.1 Example of a vocabulary levels test

2.2 Example of a Yes/No test

2.3 Example of X_Lex test

2.4 Example of X_Lex results

4.1 Factor analysis scree plot

5.1 Sample frequency output

5.2 Sample Tex Lex Compare output

ACKNOWLEDGEMENTS

This thesis has been particularly challenging as I havecompleted it whilst working full time and trying to devotetime and care to my loving family. So, my first thanks mustgo to them, husband Mike, daughter Millie and son Charlie,for putting up with all of the long evenings and weekendsthat I have shut myself away in the study and not takenpart in all of the family fun and activity.

Next I would like to thank my supervisor, Professor JamesMilton, for not only enthusing me with his expert knowledgein the field, but also for having such calmness of presencethat I always left his office feeling de-stressed and ableto continue with the next chapter. Thank you Jim.

Finally, I must extend my thanks to my two line managers,Sarah Huws-Davies and Kevin Child, who have supported mewith study leave at crucial times, particularly during thelatter part of this project, when I have found the goingespecially tough.

Thank you all.

1

INTRODUCTION

Vocabulary is considered to be a core component in second

language learning; the foundation blocks essential in any

learner’s development in second language proficiency;

“Without grammar very little can be conveyed, without

vocabulary nothing can be conveyed” (Wilkins, 1972:111). It

is no wonder then, that interest in vocabulary testing has

experienced considerable growth amongst second language

acquisition researchers since the early ‘90s, a trend which

seems to be continuing. Even relatively early studies suggest

that a learner’s vocabulary size has a significant influence

on second language development; and having a large vocabulary

is an influential factor in the development of other

linguistic competencies in the target language (Meara, 1996),

such as reading, writing and listening (Staehr 2008).

There is no shortage of vocabulary tests available to

linguistics students for academic study, and it is not the

purpose of this study to develop a new vocabulary size test.

Many influential researchers in the field such as Read

6

(1993), Meara and Milton (2003), Nation (1990) and Nation and

Beglar (2007) have developed valid vocabulary size tests.

Therefore, the aim here is to establish which test will be

most valuable in order to provide reliable results to the

research enquiry.

The interest in this study was sparked in relation to the

specific issue of large numbers of non-native speakers of

English entering degree programmes at Swansea University and

requiring varying levels of in-sessional language support, if

indeed they attempted to seek the support at all, as it is

not compulsory. A comprehensive language programme is already

in existence, but with the need to improve the knowledge of

students with vastly differing levels of proficiency and

representing a wide range of academic disciplines, there is a

requirement to identify an effective way of distinguishing

potentially problematic students and their particular

knowledge shortfalls. The desired outcome would be that if a

valid method is established, it would allow for tailored and

timely support packages to be put in place, with the likely

7

end result that non-native students have the increased

capacity to achieve more successful degree outcomes.

Consequently, this study aims to follow-up on the generally

accepted view that vocabulary levels tests can be used for

diagnostic purposes in higher education to predict an L2

student’s ability to be linguistically capable of undertaking

academic study in the second language. More specifically,

consideration will be given as to the value of diagnostic

vocabulary testing as a predictor of a student’s subsequent

academic performance at degree level study in the UK. The

vocabulary scores will be compared to the students’

International English Language Testing System (IELTS) scores on entry

to the university to enquire whether one or the other might

appear a more reliable method of predicting subsequent

academic performance.

IELTS is one of the most popular English language tests for

higher education entry, principally in the UK, Australia and

New Zealand, but also growing in the US and Canada, being

accepted by around 8000 institutions in over 135 countries

8

(IELTS.org). This would imply that tertiary organisations are

very reliant on the test’s reliability in determining to what

extent a learner is linguistically capable of performing on

an academic programme of study.

In general, more objective test methods, such as multiple

choice and vocabulary size tests, have good records of

reliability and can be easily measured for equivalence with

retests of multiple versions being straightforward to

administer. Less direct testing methods, requiring extensive

writing, comprehension and oral examinations such as IELTS,

require an additional set of skills from the learner and as

such are open to subjectivity from the marker (Milton, 2009).

Therefore, this study aims to examine whether a learner’s

vocabulary score on entry to university may reliably forecast

suitability to study in the L2 and potentially be a good

predictor of academic progress in comparison to widely

accepted standardised entry tests.

9

2

LITERATURE REVIEW

The intention of this chapter is to explore word knowledge

and the various ways to measure it. The chapter will attempt

to define what is meant by a word and word knowledge, paying

particular attention to vocabulary breadth, rather than

depth, as tests of word recognition and not productive

knowledge are used in this study. The notion of learning

goals and associated vocabulary requirements will be examined

to determine how many words a non-native speaker is likely to

need to communicate appropriately in the target language.

The chapter will move on to discuss the literature around how

effectively vocabulary measures can potentially predict

language proficiency and subsequent performance on an

academic degree programme. This question will then be applied

to research around the predictive capabilities of the

standardised IELTS examination.

10

Finally, the chapter will explore vocabulary test

construction in terms of what is required to develop a valid

and reliable test which also has high face validity for both

students and the wider non-linguist academic community alike.

2.1 What is a word?

There is a wide range of vocabulary tests available to

measure vocabulary knowledge, but controversially, they do

not all give equal results, even when used with the same

learners. For example, the Seashore and Eckerson (1940) and

Diller (1978) tests estimated that the average vocabulary

size of native speakers of English was as much as 200,000

words. This contrasts markedly with Milton’s (2007) estimate

of around 10,000 words for undergraduate native English

speakers. Such variations highlight the fact that it is not

easy to measure how much language an individual knows at all.

Space, distance, time and volume can all be reliably measured

with standard units of measurement. But a unit of measurement

for vocabulary knowledge can vary depending on how we are

defining the words.

11

Tests such as Seashore and Eckerson (1940) were developed

using dictionaries and by counting all forms of a word. As

such, the words think, thought and thinking would all be

individually counted. Later studies, such as Goulden et al.

(1990), attempt to rationalise word counts by focusing on

frequency information for enhanced accuracy. In this way,

derived forms and common inflections become a word family and

for the purpose of measurement, counted as a single word.

There will be no surprise that this method leads to a far

smaller word count of around 17,000 for an educated native

speaker. Therefore, consideration should be given as to the

many different ways to count words before a valid and

reliable method of testing can be assumed.

Despite there being numerous ways to count words, the result

will always be referred to as a word count regardless of

method. However, words can be more specifically defined by

their specialist terms of tokens, types, word families or lemmas.

The number of tokens is calculated by counting every word in a

text regardless of whether the word is repeated. So the

12

sentence ‘the cat sat on the mat’ counts as six tokens even though

the word ‘the’ occurs twice. Tokens are sometimes referred to

as running words. This unit of counting can be useful if we

want to know how many words are in a journal article or an

academic essay, or to assess reading speed for example.

Counting words as types lets us know how many different words

a text contains, so if a word is encountered more than once

it is not counted the second and each subsequent time.

Therefore, in a types count, the sentence ‘the cat sat on the mat’

would only have a word count of five, as the word ‘the’ is

repeated and only counted once. This could be useful if we

want to know how many words a dictionary contains. However,

using types as the unit of measurement may not be so useful

for testing a person’s vocabulary size. Seashore and Eckerson

(1940) used a types count and estimated that an educated

native speaker would know around 200,000 words, which would

appear to be an impenetrable learning burden for the L2

learner. Further, there is a degree of regularity in the ways

in which inflections are developed, such as by adding ‘s’ to

form a plural in English. Therefore the learner can apply the

13

rules using their knowledge of inflections without having

encountered the word previously.

A number of tests use word families. “A word family consists of a

headword, its inflected forms, and its closely derived

forms.” (Nation, 2001: 8). By this method, no derivations or

inflections are counted even when there is a change to the

part of speech. As such, the words rebel (v) rebel (n), rebellion (n),

rebellious (adj) and rebelliously (adv) would be counted as one word.

Avril Coxhead’s (2000) academic word list is an example of a

word families list and is a useful tool for L2 learners of

English in academic contexts. However, when considering

whether to use word families for developing vocabulary size

tests, the researcher may encounter problems in deciding what

to include or not to include in a word family. The learners’

level of proficiency is likely to determine their level of

knowledge of affixes and thus, have an influence on a word

families vocabulary size test.

More acceptable results may be gained from tests developed

using lemmas (Gardener, 2007), which group words according to

14

a stem word and its regular inflections along with any common

derivations which do not affect the part of speech. Francis

and Kuĉera (1982) produced a lemmatised list using the Brown

Corpus, where comparatives and superlatives are not part of

the lemma, nor are the same word forms used as different

parts of speech. Thus, the words rebel (v) and rebel (n) would be

counted as two words. In forming lemmatised lists, one

consideration is what to do with irregular verbs and plural

nouns. This relates to the notion of learning burden, or the

degree of effort required in learning a word (Swenson and

West, 1934). Naturally, the learning burden of words which

follow the rules of regular inflections is lower than those

which are irregular. Consequently, although not without its

issues, using lemmas as the basis of vocabulary levels tests

would appear to give the most accurate results over those

using types, tokens or word families which could lead to

overestimation (Nation, 2001).

2.2 What does it mean to know a word?

Once the unit of measurement has been decided, researchers

should also consider the degree of understanding that needs

15

to be tested. However, there can be variations in the idea of

what it means to know a word. Nation (2001) suggests that

words are not isolated items, but parts of a more complex

interconnected system that functions on a number of levels,

while Laufer and Paribakht propose that “no clear and

unequivocal consensus exists as to the nature of word

knowledge” (1998: 366). Word knowledge, therefore, must also

exist on a number of different levels. Richards (1976)

asserts that features such as semantic character, syntax,

morphological features and appropriate register are all

involved in the extent to which one knows a word.

A useful distinction is provided by Anderson and Freebody

(1981), which examines the difference between breadth and depth

of vocabulary knowledge. Put simply, breadth refers to the

number of words known, while depth refers to a learner’s level

of understanding of vocabulary items, what is known about

meaning, connotation, register, collocation and other

morphological aspects.

16

Nation (2001) cites the work of Palmer (1921), West (1938)

and Crow (1986) who made the distinction between receptive and

productive vocabulary. Receptive knowledge is that which is

more passive and required for reading and listening in order

to comprehend meaning; whereas productive, or active

knowledge, is employed in speaking and writing where meaning

must be conveyed to others. Nation (2001) further develops

this distinction by separating word knowledge into three

aspects. Firstly, form or what the word looks like, sounds

like, its phonology and affixes. Secondly, meaning, this

links form and definition, its contextual references and

associations. Finally, use, which refers to the word’s

grammatical function, collocations and how to use it. Each of

these aspects can be further subdivided and neatly organised

into receptive and productive elements as shown in the table

below.

17

Table 2.1 What is involved in knowing a word?

Form

Spoken R What does the word sound like?P How is the word pronounced?

Written R What does the word look like?P How is the word written and spelled?

Word partsR What parts are recognisable in this

word?

P What word parts are needed to express meaning?

Meaning

Form and meaning

R What meaning does this word form signal?

P What word form can be used to express this meaning?

Concepts and referents

R What is included in the concept?P What items can the concept refer to?

AssociationsR What other words does this word make

us think of?

P What other words could we use instead of this one?

Use

Grammatical functions

R In what patterns does the word occur?

P In what patterns must we use this word?

CollocationsR What words or type of word occur with

this one

P What words or types of words must we use with this one?

Constraints on use

R Where, when and how often would we meet this word?

P Where, when and how often can we use this word?

R = Receptive, P = ProductiveSource: Nation (2001: 27)

18

2.3 How many words does a learner need to know in a second

language?

The significance of vocabulary acquisition in second language

learning should not be underestimated. In the early years a

child will begin to communicate with individual words in the

first language, and combined with facial expression, body

language and contextual referents, a degree of meaning can be

conveyed. Similar can be said of early L2 learning. But how

many words a learner needs to know to communicate effectively

is of considerable interest, as this will depend on the goal

of learning and the L2 context. Consideration also needs to

be given to what these words should be.

High frequency words, or function words, along with some

content vocabulary are considered essential for the majority

of L2 learners, regardless of learning goals, due to the high

rate of occurrence in all types of written and spoken texts.

West’s General Service List (1953) has long been the

benchmark high frequency vocabulary list; and more recently,

Nation’s Range List (2004), drawn from the British National

19

Corpus (BNC). Such lists include around 2000 word families,

which typically give an average of 80% coverage, considered

to be sufficient for general understanding.

The importance of the 2000 most frequent words in a foreign

language is demonstrated in Staehr’s study of low level

learners (2008). Staehr aimed to assess the connection

between vocabulary size and a learner’s performance in

reading, writing and listening skills. The results

demonstrated above average performance in the tests by

learners with a sound knowledge of the first 2000 most

frequent words. Reading and writing scores were below average

for those with low vocabulary scores at the 2000 word level.

Notably, listening scores were markedly better with 65% of

the same group performing above average. Presumably, other

factors such as contextual referencing played a part in the

skill of listening, thus aiding understanding (Staehr 2008).

However, whilst knowledge of the first 2000 most frequent

words may be essential for general understanding, this does

not take into account any specific learning goals.

20

In some contexts, 2000 words and 80% coverage is only

adequate for gist understanding. Laufer (1989) estimated that

university students need 95% coverage of the running words of

academic tests in order to have “reasonable” comprehension.

Similarly, Hu and Nation (2000) suggested that 98% coverage

would be required in contexts such as written discourse and

reading for pleasure. Conversely, there is some debate as to

how many words such coverage actually equates to. Laufer

(1989) calculated this at almost 5000 words for 95% coverage,

while Carroll et al (1971, cited in Milton 2009) placed this

figure at just over 12000, as seen in table 2.2 below.

Table 2.2 Vocabulary size and coverage

Number ofwords

% textcoverage

86,741 10043,831 9912,448 955,000 89.44,000 87.63,000 85.22,000 81.31,000 74.1100 4910 23.7

Source: Carroll et al. (1971, cited in Nation, 2001, p.15)

21

Nation (2001) offers an explanation for this discrepancy

noting that the unit of measurement in Laufer’s study is not

clear and the evidence was taken from Dutch texts and applied

to English. Further, the type and length of text has an

effect on the results; Laufer’s corpora were predominantly

academic, whereas Carroll et al used very large and varied

corpora which may produce a greater vocabulary size (Milton

2009). Schmitt’s 2008 study seems to address this imbalance.

Referring to Nation’s 2006 research, Schmitt concludes that a

vocabulary of 8000-9000 is necessary to perform adequately in

reading. The figures are lower for speech and listening,

which may be because these skills are more likely to involve

more informal discourse, repetition and visual clues, but

still suggests that as many as 5000-7000 words may be

required for good comprehension.

Such vocabulary goals may seem rather ambitious and

unattainable to an L2 learner, but in a multiplicity of

situations, varying numbers and types of words are necessary,

which brings us back to the aims of learning where

22

consideration of not only high-frequency words, but also

academic, technical, specialist and low-frequency words may

be required. The prerequisite vocabulary would naturally be

different for a 6 year old, an adult tourist and a student

wishing to study in a non-native university. Table 1.2

compares the frequency of occurrence of lexis taken from

three sources: the BNC, car manuals and children’s EFL books.

It can be seen that lexis like oil, valve, remove and replace, which

are technical, topic relevant words, become high frequency

words in the lexis of car manuals; while look, say and dog are

more frequent in children’s EFL books; none of which feature

in the BNC’s most frequent 15 words. Therefore, the most

frequent 2000 words should be always be a preliminary goal,

but the less frequent thematic vocabulary which is essential

for the context of a text should not be excluded.

Table 2.3 Comparison of the most frequent vocabulary fromthree sources

Order BNC Car Children’s EFL1 The And is2 Be The the3 Of To A

23

4 And Of Your5 A In No6 In As Look7 To Or Where8 Have With For9 It remove But10 To A Did11 For replace From12 I For Say13 That Oil As14 You Be Very15 He Valve Dog

Source: Milton (2009)

Over the years, a number of tests have focussed on vocabulary

where the learner’s goal is to pursue a course of academic

study in the second language. To this end, Avril Coxhead

(2000) developed the Academic Word List (AWL) to assist in

the acquisition of the recommended 95-98% coverage students

are likely to need to study in English at university level

(Laufer, 1989; Hu & Nation, 2000). Coxhead’s research

examined a 3.5 million word corpus from academic texts across

a range of disciplines. The resulting AWL contains 570

academic word families arranged into sub-lists based on

frequency, providing 10% coverage of the corpus, which in

conjunction with West’s GSL (1953) gives an average of 90%

coverage. There still remains a shortfall of 5-8% for

24

acceptable understanding of the academic discourse students

are likely to encounter. This shortage is probably made up of

the specialist lexicons found in different academic

disciplines which do not necessarily occur in general lists

or the AWL, and not forgetting that this is only relevant to

learners working with academic texts.

In an attempt to bridge the gap in one discipline,

Konstantakis (2007) uses similar criteria to Coxhead to

produce a list of 498 word families resulting in the Business

Word List (BWL). However, this additional coverage of

specialist items still only amounts to around 2.5% when items

such as abbreviations and proper nouns are omitted. More

recently, Minshall (2013) investigated the viability of a

Computer Science technical word list. This study resulted in

the CSWL which consists of 433 headwords, and the additional

Computer Science Multi Word List (CSMWL) which contains 23

computer science compound phrases, providing the desired 95%

coverage for discipline specific learners when combined with

the GSL and the AWL.

25

Overall it can be seen that learners require a substantial

vocabulary in order to perform reasonably well in a variety

of tasks in the second language. The importance of the 2000

most frequent words has been demonstrated, giving about 80%

coverage, without which, very little real meaning can be

obtained in either spoken or written discourse. Further, it

is widely believed that a higher figure of at least 95%

coverage is more realistic in order to achieve full

comprehension. Therefore, any test of vocabulary size should

be aimed at the first 2000 most frequent words for general

competence and at least 5000 for more academic assessments

(Milton 2009:70).

2.4 Vocabulary knowledge as a predictor of language

proficiency

It has long been recognised that measures of vocabulary

knowledge are generally assumed to be good predictors of

proficiency in the four skills and recent studies have served

to strengthen this hypothesis. Staehr (2008), for example,

demonstrates a correlation between vocabulary size and the

26

skills of reading, writing and listening amongst EFL learners

from a lower secondary school in Denmark. In particular,

there was a marked association between receptive vocabulary

size and reading and writing. His findings suggest that

vocabulary size accounts for 72% variance in the ability of

the learners to gain an average (or above) score in the

reading test administered. Similarly, his analysis shows a

52% variance in writing and 39% in the listening scores.

Overall, Staehr’s results indicate that vocabulary size is a

significant factor in the learners’ ability to perform in the

tested skills. Other research similarly demonstrates such

correlations, in particular within the realms of vocabulary

size and reading comprehension (Laufer, 1992; Henriksen,

Albrechtsen & Haastrup, 2008).

However, Milton et al (2010) highlight the fact that research

into the relationship between vocabulary size and language

performance has rarely taken account of the learner’s

speaking ability. Milton et al further recognised that the

results of Staehr (2008) and others demonstrate weaker

27

correlations with aural listening skills than with writing

and reading which are written skills. This could be

attributed to the testing method used, that is orthographic

measures, or possibly that vocabulary knowledge is retained

in different formats namely, aural and/or written.

Therefore, Milton et al investigated whether vocabulary

knowledge could in fact be held separately in aural and

written formats, and whether measuring written format and

aural format would provide a more complete explanation of

performance in the four skills. Vocabulary was measured using

X_Lex (Meara and Milton, 2003) for written form and A_Lex

(Milton and Hopkins, 2006) as an equivalent test for aural

form. IELTS sub-scores were used to compare performance in

the four skills. The data was collected from intermediate to

advanced students of mixed nationalities studying on a UK

pre-sessional programme. As with Staehr, the results suggest

strong correlations between vocabulary size and language

performance.

28

The findings also show that written and aural vocabulary

knowledge need not be held in combination, but that words can

be held in one form. Interestingly, Milton et al found that the

lower level learners tended to retain predominantly aural

vocabulary knowledge, while written word knowledge was more

evident amongst the advanced learners. Significantly then,

this study concurs with others that vocabulary size is a

reliable measure of language proficiency, but in this

instance, a relationship has been found to be significant in

the skill of speaking. Milton et al thus maintain that where

vocabulary size is appropriately measured, statistically

significant performance correlations can be shown for the

four skills (Milton et al, 2010).

2.5 IELTS scores as a predictor of academic performance

A number of studies have examined the predictive validity of

the IELTS test to determine whether it adequately measures

the target goals of the test taker and the HE institution

they intend to enter.

29

Of these predictive validity investigations, some have found

there to be little or no statistically significant

correlation between IELTS and academic performance. For

example, Cotton and Conrow (1998) studied 33 international

university students and found no noteworthy positive

correlations between IELTS scores and the performance of

students in their academic coursework. Despite weak

connections between IELTS reading scores and subsequent

academic performance, low IELTS entry levels of 5.5 did not

necessarily suggest poor academic results. Likewise, high

entry levels of 7.0 and above did not always result in

consequent academic success (Cotton & Conrow, 1998). Dooey’s

study corroborates this as no evidence was found to indicate

that students who did not meet the university’s specified

entry requirement of IELTS 6.0 were more likely to perform

inadequately (Dooey, 1999).

On the other hand, some studies have found significant,

although sometimes weak and inconsistent, positive

correlations between IELTS entry levels and projected

30

academic performance. Bellingham (1993), for example, found

that low IELTS scores and academic failure have a significant

relationship, while Hill, Stoch and Lynch (1999) demonstrated

a fairly strong correlation between IELTS proficiency scores

and academic performance in semester one. However, linear

regression estimates also concluded that the overall

relationship was not particularly strong. Positive, but weak,

connections were also found in Kerstjen’s and Nery’s (2000)

study of 113 international higher education students.

Meanwhile, Feast (2002) measured the relationship between

IELTS proficiency and university students’ grade point

averages, with results showing a significantly positive

association.

More recently, Woodrow’s (2006) study correlated IELTS

subtest scores against the semester one GPAs of international

postgraduate students and found that while connections with

overall bands and GPAs were weak, more valid significance was

found amongst the subsets of writing, listening and speaking.

Analysis indicated that the relationship was stronger at the

31

lower IELTS level. Thus, at 6.5 and below, IELTS scoring may

be a good predictor of achievement, whereas at 7.0 and above,

no noticeable influence was perceived. However, it was noted

that there were a number of additional significant

influencers on academic performance; previous experience with

the English language and differing professional backgrounds

were also found to be viable.

The research demonstrates, therefore, that the connection

between IELTS test results and subsequent performance on an

academic programme in the second language remains

hypothetical and further research needs to be done.

2.6 What makes a good vocabulary test?

In most circumstances, native speakers will develop

vocabulary knowledge naturally throughout their lives, giving

them the ability to manage the situations they face

appropriately according to their age and experience. Second

language learners however, can experience vocabulary barriers

which may result in them being unable to communicate

effectively. As such, there is good reason to test vocabulary

32

knowledge and monitor further development. The results of

this testing may assist in drawing conclusions as to whether

a learner’s vocabulary knowledge meets their L2 needs and

importantly, what further support and instruction may be

required. Consequently, the reasons for testing vocabulary

knowledge are clear, but consideration must also be given as

to what to test and how.

When devising vocabulary size tests for L2 learners, it is

the norm to first consider what native speakers know. This

provides a paradigm for assessing second language competency.

However, as previously discussed, there have been

considerable variations in vocabulary size test results

historically, ranging from about 3000 to over 200,000 for an

adult native speaker. Goulden et al (1990) provided a more

acceptable benchmark of 17,000, upon which subsequent studies

have been modelled and developed. In Goulden’s study, words

were classified into base, derived, proper, compound and others,

drawing on the work of Nagy and Anderson (1984) who

differentiated base and derived words by taking into account

33

their relationship of meaning and the amount of additional

learning required. Comparisons were made to a Thorndike and

Lorge (1944) list, producing a final sample of 571 words.

Once the researchers were satisfied that the sample of words

was representative, a fair and valid test had to be devised.

Looking at previous tests particularly that of Diack (1975),

which was a self-assessment test based on a series of words

arranged into frequency lists, Goulden et al also produced

tests based on frequency levels arranged into manageable 5x50

word sections, with each item representing 500 words.

Therefore, the raw results would be multiplied by 500 to

produce an estimated vocabulary size. The results show that

adult native speakers have an average vocabulary size of

17,000 words and a learning rate of 2-3 words per day.

With this benchmark set, it ensured that subsequent test

developers gave consideration as to the unit of measurement,

whether multi-word items are to be counted as lexical or

grammatical, whether to measure breadth or depth and what

34

constitutes knowing a word. Most importantly, the reliability and

validity of the test being used must be established.

Reliability is concerned with consistency and accuracy; the test

must be stable over time and produce equal results if the

same student is tested more than once on the same day.

Equivalence might also be judged, where different versions of

a test have comparable results when the same student takes

two forms of the test on the same day (Milton, 2009).

A test has validity when it can be confirmed that it measures

what it is designed to test and not something different.

There are various elements to validity, specifically content,

construct, concurrent and face validity.

Content validity ensures that the test has appropriate content to

measure what is required. For example, tests of vocabulary

breadth, such as Nation’s Vocabulary Levels Test (Nation,

1990), most often use frequency bands and lemmatised

wordlists, which assumes valid content to demonstrate a good

range of vocabulary knowledge and recall.

35

Construct validity on the other hand, is a measure of the desired

skill or construct to be measured. Constructs are often

measured by way of written or oral language production where

measurement must be inferred, thus posing issues of

subjectivity and a reliance on the learner to produce the

words they know within the given task. It is therefore

difficult to develop a productive vocabulary test which

demonstrates good construct validity. Measurement of

receptive vocabulary knowledge can be less problematic as the

test creator can decide which words are to be investigated so

enhancing construct validity.

At times, testers want to measure equivalence, so two

different test types of equal quality are given to the same

set of learners. If the results of the two tests compare

well, then the testing procedure is said to show good

concurrent validity.

If the above validity measures have been considered when

creating and administering a vocabulary test, then

researchers can be fairly confident that accurate and

36

appropriate results will be delivered. However, there is one

further, and no less important, area of validity to be

considered; that of face validity. This is concerned with the

views of the users. Learners can feel challenged by tests,

especially if they do not see the results they expected, or

they may not have taken a test of the same type before, so

not recognise it as valid. Additionally, vocabulary tests can

appear to be relatively simple, leaving the face validity

questionable to researchers outside of the field (Milton:

2009). This issue has been addressed with the X_Lex (Meara &

Milton, 2003) and Y_Lex (Meara & Miralpeix, 2006) vocabulary

size tests. The tests are computerised, giving them high face

validity. Learners tend to like working with the programmes

and get instant results. Guessing is minimised as the

frequency range is random and test administration is

straightforward.

2.7 Test Types

37

There are many different types of vocabulary test. This

section will give an overview of some of them and then focus

more specifically on the types used in this study.

Generally speaking, a vocabulary test should have a minimum

of around 30 items to be reliable (Nation, 2001) and should

use a test item type which will test the kind of knowledge

the researcher or teacher is looking for. But as test item

types can vary considerably, choosing a test may be easier

said than done. Some will focus on form, others meaning; or

the requirement may be to test knowledge of collocations.

Examples of test types include asking students to choose the

appropriate word(s) such as Read’s vocabulary depth test (1995),

definition completion (Read 1995) or multiple choice matching

like A sensitive multiple-choice test (Joe, 1994) to test whether the

learner understands word meaning. Nation’s Vocabulary Levels Test

(Nation, 1990, revised by Schmitt et al., 2001) is an example

of a recognition test where learners are presented with words

in the foreign language along with a number of definitions or

38

explanations. The learner must choose the correct explanation

to match with the test word.

Instructions:In each question, choose the correct meaning to go with the word in CAPITAL letters. Circle the letter with the best meaning for the word.

1. PATIENCE: He has no patience.a. Will not wait happilyb. Has no free timec. Has no faithd. Does not know what is fair

Figure 2.1 An example of a vocabulary levels test item

Recognition tests allow for both the knowledge of words and

their meanings to be estimated. The format of having

multiple-choice items is easy to mark and allows learners to

draw on partial knowledge (Nation, 2001). This type of test

may also have good validity as it is used in a number of

standardised tests such as TOEFL, and learners may be

familiar and comfortable with its format. There are of course

limitations in that the test can be difficult to construct

and reliable results depend on students not only being

familiar with the target word, but also the vocabulary used

in the explanations. Further, such tests are open to

39

guesswork and calculation, for which there appears to be no

obvious way of compensating.

Another test type which has increased in popularity over the

last few decades and appears in many forms is the Yes/No, or

checklist test. Such tests measure passive word recognition

to establish a learner’s vocabulary breadth or size.

Instructions:Please look at these words. Some of these words are real Englishwords and some are not but are made to look like real words.Please tick the words that you know or can use.Example:

New commer

ce

organ

ise

accus

e

Victo

ryGumme

r

tindle wooke

y

candi

sh

Skave

Figure 2.2 Example of a Yes/No test. From XK_Lex Vocabulary Test

1 – Version A (Masrai, 2009)

Although Yes/No tests have been popular since the 1970’s,

momentum really began after 1983 when Anderson and Freebody

began working with them and included some nonsense words to

check for accuracy in learner response (Anderson & Freebody,

40

Catdita

1983). Since the late 1980’s Paul Meara and his colleagues

have continued to refine checklist tests resulting in the

computerised Eurocentre’s Vocabulary Size Test (Meara and

Jones, 1990) and more recently the X_Lex (Meara & Milton,

2003).

The Yes/No format requires learners to simply tick the words

they know. Words to be tested are usually taken from trusted

word frequency lists, with an equal sample taken from each

frequency band to be tested. Paul Meara and James Milton have

been leading the way in the development of checklist tests

for some time. The X_Lex (Meara & Milton, 2003) for example

tests up to the first 5000 most frequent words in English and

is also available in other languages, while the Y_Lex (Meara

& Miralpeix, 2006) works with the 6000 – 10,000 range. The

advantages are that this format is easy to devise, mark and

administer. Learners will be able to get through the test

quickly, thus reducing the chance of boredom. The tests are

also computerised and show instant results once completed,

giving them high face validity. The main disadvantage is that

learners may be quite tempted to guess which could lead to

41

overestimation. However, the use of trusted frequency lists

and addition of false words helps to address this. The false

words are constructed to look and read like real words in the

target language, but do not actually exist. Therefore, the

learner cannot have encountered them before and should not

tick them. If false words are ticked, then there is a

calculation which determines the degree of overestimation and

the scores are adjusted accordingly (Milton, 2009).

Figure 2.3 Example of X_Lex test

The test shows a randomised series of 120 words from frequency lists up

to 5000 words. Learners click the smiley or sad face depending whether

they know the word or not.

42

Figure 2.4 Example of X_Lex results

Instant results once the test has been completed.

There are considerable benefits of having such standardised

and reliable methods of measuring vocabulary breadth, both in

terms of placement and instruction. Milton (2009) also

suggests that it is now possible to compare results not only

between different groups of learners within particular

institutions, but also between countries. Further, he asserts

that comparisons may also be possible between different

languages, although recognising that this raises issues of

the differences in how languages may inflect and combine

words. This study will now attempt to ascertain whether a

measure of vocabulary breadth compares well with IELTS scores

43

in determining the predictive performance of L2 learners

studying through the medium of English in a UK university.

44

3

METHODOLOGY

This study investigates the notion that vocabulary breadth

measures could be a reliable predictor of academic

performance amongst second language students in a UK higher

education setting compared to IELTS scores. The vocabulary

test scores will be evaluated against the participants IELTS

results on entry to university and their exit grade point

averages (GPA). The research is of interest as vocabulary

breadth measures are widely believed to be relevant in

assessing language proficiency levels and more recent

research suggest that academic performance and vocabulary

knowledge inter-relate (Milton, 2009, 2010; Daller & Phelan,

2013). The study further examines how such data could be used

by Higher Education institutions in decisions to provide

appropriate levels of in-session language support.

3.1 Aims and Objectives

The overarching objective of this study is to explore the

relevance of vocabulary levels testing as a predictor of

45

academic performance for L2 learners of English entering UK

universities in comparison to the predictive capabilities of

IELTS testing and to further ascertain the potential

pedagogical implications for effective in-session language

support. Therefore, the principle aims are:

To compare the results of learners’ vocabulary test

scores and IELTS scores.

Examine whether one testing method or the other might be

a better predictor of future academic performance.

To assess the usefulness of vocabulary scores in

devising appropriate in-session support for L2 speakers

in a formal academic setting.

3.2 Research questions:

How well do the two vocabulary size measures correlate

with academic performance and IELTS?

Do any of these tests have a good enough correlation to

enable it to be used as a predictive test of subsequent

academic performance?

46

What scores are associated with failure or poor academic

performance to guide us identifying students in need of

subsequent in-session support?

Can the scores identify areas of knowledge to be covered

by in-session support classes?

3.3 Participants

The participants were 20 Chinese students from two different

universities in China, who came to Swansea University on a

Visiting + 1 exchange programme within the College of Law.

There were 32 students on the programme in total, but only 20

elected to participate in this study, which means therefore,

that the sample size is relatively low.

The students were just starting the final year of a BA in Law

at Swansea University, having completed the first two years

in China, with the expectation that they would continue to

study a further year on a master’s degree, subject to a

satisfactory bachelor degree outcome. All of the students

entered the university with IELTS scores ranging from 6.0 to

47

7.5, with 6.0 being the minimum requirement for the

programme.

Importantly, although the sample size was low, it did provide

a fairly homogenous group to work with, having similar

educational backgrounds prior to entry and undergoing the

same academic input and in-session language support during

their degree programmes in the UK.

3.4 Data collection and testing procedure

As the requirement of this study is to explore the notion

that learners’ vocabulary measures are closely linked to

their subsequent academic performance, the participants

completed two proven vocabulary levels tests: Masrai’s XK_Lex

(2009) and Nation and Beglar’s Vocabulary Size Test (2007b).

Instruments

The XK_Lex (appendix 2) was developed as part of a master’s

thesis (Masrai, 2009) with the support of Professor James

Milton at Swansea University. The test is an expansion of the

widely used and validated X_Lex (Meara & Milton, 2003),

48

providing a checklist test to the 10k frequency level. The

XK_Lex contains 100 real words, with every ten words

representing each thousand of the 10k frequency bands. In

addition, 20 pseudo words are included to minimise guessing

and overestimation. The real words are arranged into

frequency band columns with the first five columns being

represented by vocabulary from Nation’s frequency list

(1984). The other five columns are extracted from

Kilgarriff’s frequency list (2006), bringing the test up to

the 10k range. The pseudo words are mixed up with the real

words in equal proportion of two per frequency column. This

is a Yes/No type of test which aims to test orthographic

receptive recognition, providing a vocabulary size measure in

a format which is quick and easy to administer.

The Vocabulary Size Test (appendix 3) contains 100 target words,

ten at each of the 10k levels. The words are drawn from the

fourteen 1000 BNC word family list. Nation and Beglar suggest that

using a word family list is more appropriate because once

learners are beyond a basic level of proficiency they “have

some control of word building devices and are able to see

49

that there is both a formal and a meaning relationship

between regularly affixed members of a word family” (Nation &

Beglar, 2007a, p. 10). Although the complete test has 140

items to represent the 14k frequency list, this study

administered up to the first 10k only for comparison with the

10k XK_Lex test above. The VST is a multiple choice test

designed to measure orthographic receptive vocabulary size.

The multiple choice format aims to demonstrate the learners’

knowledge of the words.

Additional data

In addition to the measures of vocabulary size gained from

the tests above, the students’ overall IELTS scores and mean

final grade point averages were also required. Authorisation

was gained form the students at the time of taking the tests

(appendix 1). The IELTS scores were taken form student

records, but only the overall scores were available, so no

correlations could be made with the sub-skills. The GPA was

also made available from student records. The students took

five modules, each carrying an equal weighting of 20 credits,

50

so the mean GPA was derived by dividing the total scores of

the modules by five.

Method

The participants were presented with a letter explaining the

process (appendix 1) and the research purpose was outlined

orally. At this point students were allowed to leave the room

should they not wish to participate. Test instructions were

both verbalised and attached to the tests for clarity. Both

tests were given out at the same time and the students were

given around an hour to complete them. However, a further

five or 10 minutes was allowed for a small number of students

who had not finished within the given time. It was considered

that completion was more important than working to time. The

tests were collected and the scores calculated.

The XK_Lex score was calculated by giving 100 marks for each

real word ticked and deducting 500 marks for each non-word

ticked. The Vocabulary Size Test is calculated by multiplying

each correct answer by 100 to give an overall vocabulary

51

size. The calculations for each test give an overall score

for the words known out of 10,000.

Statistical analysis

The data was analysed with reference to the research

questions using the SPSS software package. The following

results will be presented:

Overall results to present the mean scores and standard

deviations of the vocabulary scores and GPAs.

Correlations to compare the vocabulary tests and IELTS

with GPAs.

Factor analysis to assess what factor the vocabulary

tests, IELTS and GPAs are testing.

Extraction method for component analysis of the XK_Lex,

VST, IELTS and GPA.

Group statistics to determine the mean scores and

standard deviation of the low GPA students compared to

the higher GPAs.

Levene’s test for equality of variances.

T-test for equality of means.

52

The statistical analysis may be challenged by the fact that

not all of the data presented is interval data. The

vocabulary tests are countable and so could be said to be

interval data but the IELTS scores and GPAs are grades and

testing for a wide range of language aspects and subject

knowledge.

53

4

RESULTS

The participants’ IELTS results, GPA grades and the raw

scores of the two vocabulary tests are represented in table

4.1. It is worth noting that students E, J and K did not

complete the VST, but we have no way of knowing whether this

was due to lack of time, lack of knowledge or motivation.

However, this does bring the marking on this test in line

with the academic exams and IELTS, because if students do not

finish a task in these assessments, they will not get marks

for it and no allowances are made. Therefore, it was decided

that the results should be presented as is, even though they

may appear as anomalies.

Table 4.1 Raw scoresStudent

IELTS score

XKLex Score

VST Score % GPA

A 6.0 4900 5100 61B 6.5 4900 4000 62C 6.0 7000 5100 66D 7.0 7500 6600 66E 6.0 4600 3100 55F 6.5 5900 5400 58G 6.5 7600 6500 67H 6.5 5400 2700 61

54

I 6.0 4500 5700 60J 6.0 3000 4600 52K 6.5 6700 3400 58L 6.5 6300 4800 63M 7.5 7100 6600 59N 6.0 3500 4200 62O 6.5 4600 5000 59P 6.5 4600 4500 67Q 6.5 6800 6300 65R 6.5 5200 4100 61S 6.5 7500 6400 65T 6.0 6200 4100 62

It would be reasonable to expect that students with higher

IELTS results would demonstrate larger vocabulary scores and

ultimately perform better academically. However, this only

appears to be the case with student D. If we compare the

IELTS results with GPAs, there is no clear pattern.

Conversely, there appears to be a relationship between higher

XK_Lex measures and higher GPAs, the only exception being

student M, who not only had the highest IELTS result of 7.5,

but also achieved 7100 and 6600 in the XK_Lex and VST

respectively, yet an academic grade of only 59% overall.

There are a number of possible reasons for this such as ill

health, personal issues or poor study strategies for example,

but we are not able to examine these as part of this study.

55

The raw scores in table 4.1 are examined in more detail in

the following tables to ascertain statistical significance.

A summary of the vocabulary size data for the whole group is

presented in table 4.2. The assumption is that the VST and

XK_Lex would compare well, but with the potential to

overestimate in the VST as the scoring makes no allowance for

guessing. However, the VST mean score is lower than the

XK_Lex. The differences in the mean scores was shown to be

significant in the t test [t=2.812; sig=.011]. The

significance of these differences will be discussed in more

detail in the next chapter.

Table 4.2 Mean scores on vocabulary size tests and GPA

Mean Scores Standard deviation

Max possible

VST 4910 1185.39 10,000XK_Lex 5690 1365 10,000GPA 61.45 3.98

Table 4.3 below reports the correlation coefficients between

the vocabulary tests and IELTS with the GPAs. As expected,

the XK_Lex vocabulary test has a good correlation of 0.567

56

against the GPA and is statistically significant to the 0.01

level. Surprisingly, the VST has a lower correlation of

0.421, the possible reasons for this will be discussed in the

next chapter. Notably, the IELTS/GPA correlation of 0.203 is

weak, which is alarming when IELTS is one of the most popular

English language tests for university entry in 135 countries

across the world (www.ielts.org).

Table 4.3 Correlations of vocabulary tests and IELTS with GPAs

IELTS XKlex vst GPA

IELTS

Pearson Correlation 1 .540* .436 .203

Sig. (2-tailed) .014 .055 .390

N 20 20 20 20

XKlex

Pearson Correlation .540* 1 .535* .567**

Sig. (2-tailed) .014 .015 .009

N 20 20 20 20

VST

Pearson Correlation .436 .535* 1 .421

Sig. (2-tailed) .055 .015 .065

N 20 20 20 20

GPA

Pearson Correlation .203 .567** .421 1

Sig. (2-tailed) .390 .009 .065

N 20 20 20 20

57

*. Correlation is significant at the 0.05 level (2-tailed).**. Correlation is significant at the 0.01 level (2-tailed).

There are differences in the type of data being compared;

that is measurable vocabulary scores in comparison with IELTS

proficiency levels and the GPA percentage grades. It would be

expected that the two vocabulary scores would test the same

factor, although they have produced different scores, but

that IELTS and the GPA scores are drawing different knowledge

and skills. IELTS is intended as a test of performance in the

sub-skills reading, writing, listening and speaking. GPA

should, it is expected, tap into subject knowledge rather

than language knowledge. As such, a factor analysis was

prepared to check what is being tested. However, the scree

plot in figure 4.1shows that in fact they all appear to be

testing the same factor, with only one factor gaining a score

above 1. The component matrix (table 4.4) shows the highest

scores for the four possible factors these tests might

produce, are all in factor 1.

58

Figure 4.1 Factor analysis

Table 4.4 Component Matrixa

Component

1 2 3 4

XKlex .815 -.041 -.519 -.254

GPA .745 -.619 .003 .249

IELTS .727 .641 -.063 .235

VST .772 .036 .605 -.194

Extraction Method: Principal Component Analysis.

a. 4 components extracted.

59

Table 4.5 Group Statistics

GPA N MeanStd.

Deviation

Std. Error

Mean

XKlex

>=

58.0018 5900.00001241.44129 292.61052

< 58.00 2 3800.00001131.37085 800.00000

The group statistics presented in table 4.5 above are

statistically relevant as they show a mean XK_Lex vocabulary

score of 5900 for students gaining an academic grade greater

than 58%, while for those with grades below 58% the mean

vocabulary score is 3800. A further t-test was run for

equality of means. The results can be seen in table 4.6

below, showing that the t score is statistically significant

at the 0.05 level.

Table 4.6 Independent Samples Test

Levene's Test for Equality of Variances

t-test for Equality of Means

60

F Sig. t df Sig. (2-tailed)

Mean Difference

Std. Error Difference

95% Confidence Interval of the Difference

Lower Upper

XKlex

Equal variances assumed

.403 .534 2.28018 .035 2100.00000 920.94959 165.15671 4034.84329

Equal variances not assumed

2.4651.284.199 2100.00000 851.83385 -4424.37864 8624.37864

61

5

DISCUSSION

This chapter will examine the results of the study in greater

detail paying particular attention to the research questions

and the pedagogical implications of the enquiry. In addition,

the limitations of the research will be highlighted along

with suggestions for further investigation.

5.1 How well do the two vocabulary size measures correlate

with academic performance and IELTS?

Not surprisingly, in line with previous studies demonstrating

correlations between vocabulary size and language

proficiency, the results of this study emphasise the

association between vocabulary size and academic performance.

In particular, the correlation between XK_Lex scores and

grade point averages was reasonably good at 0.57, which is

statistically significant at the 0.05 level. A likely

explanation for this could be that academic study on the BA

Law, as with many academic degrees, is heavily text focussed

62

and the relationship shown may be predominantly between

vocabulary size and reading. It is fair to assume that if the

student has a large vocabulary size, then he/she will have a

better chance of knowing a high enough percentage of the

lexical items within the texts being studied for reasonable

comprehension (Staehr, 2008); that is, around 95-98% for

academic purposes (Laufer, 1989; Hu & Nation, 2000). Of

course, the students in this study also needed to write

extended essays and sit written examinations and thus the

correlations with XK_Lex seem to show a significant

correlation between academic reading and writing. An

additional test of productive vocabulary measure such as

Laufer and Nation (1999) might have served to strengthen this

correlation.

It was expected that the Vocabulary Size Test would associate

well with GPA, but in this study, the VST demonstrated a

weaker relationship than the XK_Lex generating a correlation

of 0.42 and the correlation is not statistically significant.

There may be a number of reasons for this. Firstly, three of

63

the participants did not complete the VST which may have

scored them lower than their actual vocabulary size, but as

we have no way of knowing the reason for non-completion,

these results were still included as true. Additionally, it

places the marking in line with IELTS testing and academic

assessments which make no allowances for incomplete answers.

Secondly, the VST was created from a language teaching

perspective and as such, partial knowledge is marked and

there is no scope in calculation for guessing. This can lead

to either overestimation or underestimation. The multiple-

choice format means the participants must also have a

moderately developed knowledge of the meaning of the word and

of the lexicon presented in the definitions. This makes the

test more difficult than the XK_Lex as “the correct answer

and distractors usually share the same elements of meaning”

(Nation & Beglar, 2007a).

As regards the IELTS correlations with GPA, these were found

to be weak in comparison to either of the vocabulary tests at

just 0.2. This is presumably because IELTS is a subjective

64

examination and despite attempts to introduce objective

measures to the test over the years (www.IELTS.org), it must

still be open to broad interpretation by examiners.

Similarly, apart from assessing language proficiency, it also

requires the test takers to interpret the questions and

respond appropriately in terms of content, style and

register.

5.2 Do any of these tests have a good enough correlation to

enable it to be used as a predictive test of subsequent

academic performance?

The IELTS examination results showed the weakest correlation

with GPA (0.2) and therefore, in line with previous research,

do not appear to provide a significant indicator of academic

performance. Meanwhile, the Vocabulary Size Test only

demonstrated a moderate association with the GPA, producing a

correlation of 0.41, signalling that this study has not shown

the correlation to be sufficiently significant for the VST to

be used as a reliable predictor of academic achievement. This

may be because the test was principally designed to track

65

learners’ vocabulary development “both longitudinally and

across a group of learners” (Nation & Beglar, 2007a: 9), so

is appropriate for charting growth and not for diagnostic

purposes per se. Additionally, Nation (2013) suggests that

the VST is appropriate for measuring the vocabulary knowledge

required for reading, but not listening, speaking or writing,

and perhaps therefore, not representative of all of the sub-

skills required for academic study.

The test in this study which showed the most significant

correlation of 0.57 was the XK_Lex. The purpose of the XK_Lex

is to provide a reliable and consistent measure of learners’

vocabulary size to the ten thousand level. It was designed by

Professor James Milton at Swansea University and validated by

Masrai (2009) as part of a master’s thesis. Results of two

forms of the test (A and B) were compared and found to

correlate well. The XK_Lex scores in Masrai’s study

correlated significantly with the tried and tested EVST

(Meara & Jones, 1990), demonstrating concurrent validity.

Therefore it is safe to assume that the results of this test

66

show reliable measures of the participants’ vocabulary size.

With this in mind, along with the significant correlation

with GPA in our study, it can be concluded that the XK_Lex

may be sufficiently effective to enable it to be used as a

predictive test of subsequent academic performance.

5.3 What scores are associated with failure or poor academic

performance to guide us identifying students in need of

subsequent in-session support?

None of the participants in this study failed their academic

degrees. However, some achieved a low or bare pass and as

such, could have been considered as at risk. Table 4.5

presents the group statistics and specifically, the mean

vocabulary scores for students with GPAs > = 58 and < 58. For

students with GPAs > = 58, the mean vocabulary score was 5900

with a standard deviation of 1241. The mean drops to 3800

with a standard deviation of 1131 for students with GPAs <

58. This strongly suggests that below the 4000 word level of

vocabulary knowledge, a student will be at risk of

67

unsatisfactory performance and likely to benefit from

subsequent in-sessional language support.

It is worth noting that the participants in this study were

supported in-sessionally with compulsory language and writing

skills classes amounting to 60 hours of tuition. Some

students also self-selected additional language classes

through the university’s Academic Success Programme. This

could be significant as it suggests that the students with

low vocabulary scores and GPAs might have achieved less well

without this provision in place.

5.4 Can the scores identify areas of knowledge to be covered

by in-session support classes?

This study demonstrates that in general, the students with

lower vocabulary measures also achieved lower GPAs. This

strongly suggests that in-session support focussing on growth

of learners’ vocabulary knowledge will assist them in

performing better academically. The pedagogical

considerations are a) What vocabulary should this be? b) How

should it be taught?

68

As The XK_Lex test is presented in sections at each thousand

frequency level up to the first 10,000, it is possible to see

from the test scores how a learner has performed at each

frequency band level and teaching could be developed around

these.

It is widely believed that the first 2000 most frequent words

are essential for general understanding, providing about 80%

coverage, so it goes without saying that some effort should

be concentrated in this area if test results highlight

deficiencies. However, Laufer (1989) suggests that university

students require 95% coverage for reasonable comprehension,

while Hu and Nation (2000) consider the figure to be higher

at 98%. As the focus of this study is concerned with

potential academic performance, then we can assume that the

students should aim for the 95-98% coverage or around 8000 -

9000 words for reading (Nation, 2006; Staehr, 2008). All of

the students in our study fell short of this figure and there

were no especially high achievers academically, with GPAs

ranging from 52% to 67% and vocabulary levels between 3000

69

and 7600 words. Notably, the student at the 3000 word level

achieved the 52% and the student at the 7600 word level

gained 67%. This evidence suggests that that beyond the first

2000 word frequency level, much effort also needs to be

directed into developing the learners’ overall vocabulary

size across the levels, particularly the infrequent

vocabulary which is essential for comprehension in their

academic subjects.

As good academic performance is the goal for these learners,

then it would be safe to assume that academic words would

form part of the input. Furthermore, technical or discipline

specific vocabulary should also be given consideration,

relevant to the student’s subject. Therefore, as part of a

needs analysis, frequency profiling would be useful in

defining words to be learned.

One method which might be adopted to help redress the

language knowledge gap in-sessionally is an extensive reading

programme. The reading should be based around subject

specific core texts in order to deliver academic credibility,

70

usefulness and face validity for the students and their

academic lecturers. Many of the articles and chapters that

students will be required to read as part of their coursework

will be in digital format; this lends itself to lexical

frequency profiling using Tom Cobb’s Lextutor site

(www.lextutor.ca). The teacher can run the texts through a

variety of software packages depending on the required

output. For example, the Frequency software (figure 5.1)

allows the user to input texts and extract frequency lists

which can be arranged in order of least to most frequent and

vice versa.

Tokens: 200Types: 122Ratio: 0.6100Sort: descending---------------------------------------------------------RANK FREQ COVERAGE COVERAGE WORD individual cumulative ---------------------------------------------------------1. 11 5.50% 5.50% THE2. 9 4.50% 10.00% HIS3. 6 3.00% 13.00% FOR4. 5 2.50% 15.50% OF5. 5 2.50% 18.00% TO6. 4 2.00% 20.00% DAY7. 4 2.00% 22.00% HANGER8. 4 2.00% 24.00% HE9. 4 2.00% 26.00% IN10. 4 2.00% 28.00% MP

Figure 5.1 Sample Frequency output (www.lextutor.ca)

71

Meanwhile, Tex Lex Compare (figure 5.2) is useful to establish

how often a word is repeated over two or more texts. The

tokens are shown along with a choice of types, families or

phrases, which could provide useful target vocabulary across

a range of core material relevant to a whole module for

example. The sample output below shows only the first 10 most

frequent words as an example, but for the purposes of

university students it would be more relevant to take the

less frequent vocabulary into consideration as the high

frequency words are likely to be known.

TOKEN Recycling Index: (481 repeated tokens : 658 tokens in new text)= 73.10%     

   FAMILIES Recycling Index: (150 repeated families : 305 families innew text) = 49.18%     

   (Token recycling will normally be the most interesting measure of e.g. text comprehensibility, as it is with VPs.)

Unique to first255 tokens216 families

001.  report 6002.  love 4003.  before 3004.  courtship 3005.  young 3006.  apparent 2007.  chad 2008.  do 2009.  during 2010. girlfriend 2

Shared481 tokens150 families

001.  the 38002.  and 23003.  be 23004.  a 20005.  he 15006.  sarkozy 13007.  in 11008.  marry 11009.  number 11010.  of 10

Unique to second177 tokens155 families

001.  i 5002.  last 5003.  start 3004.  add 2005.  follow 2006.  get 2007.  just 2008.  pair 2009.  press 2010.  reside 2

 Same listAlpha first001.  across 1002.  add 2003.  affair 1004.  age 1005.  ago 1006.  agostinelli 1007.  another 1008.  around 1009.  backlash 1010.  bazire 1

72

VP novel items

Figure 5.2 Sample Tex Lex Compare output (www.lextutor.ca)

Once the appropriate vocabulary input has been detected, an

applicable curriculum can be developed. There are many ways

to teach and learn vocabulary and it is not the purpose of

this study to explore them in detail, but Nation’s table

(5.1) provides an excellent basis for determining appropriate

stands and application for working with the target

vocabulary.

Table 5.1 The four strands and their application with a focus

on vocabulary

Strand General Conditions

Vocabulary requirements

Activities and techniques

Meaning-focused input

Focus on the message

Some unfamiliar items

Understanding Noticing

95% + coverage(preferably 98%)

Skill at guessing form context

Opportunity tonegotiate

Incidental defining and attention drawing

Reading gradedreaders

Listening to stories

Communication activities

Language-focused learning

Focus on language items

Skill in vocabulary learning strategies

Appropriate

Direct teaching of vocabulary

Direct learning

73

teacher focus on high frequency words, and strategies forlow frequency words

Intensive reading

Training in vocabulary strategies

Meaning-focused output

Focus on the message

Some unfamiliar items

Understanding Noticing

95% + coverage(preferably 98%)

Encouragement to use unfamiliar items

Supportive input

Communication activities andwritten input

Prepared writing

Linked skills

Fluency development

Focus on the message

Little or no unfamiliar language

Pressure to perform faster

99% + coverage Repetition

Reading easy graded readers

Repeated reading

Speed reading Listening to

easy input 4/3/2 Rehearsed

tasks 10 minute

writing Linked skills

This section suggests that as the scores produced by the

XK_Lex seem to provide valid data to link vocabulary

knowledge with academic performance, that this data can

subsequently be useful to enable areas of knowledge to be

identified in order to develop in-session support classes.

Additionally, some of the methods by which this might be

approached have been discussed. However, this study is not

74

without its limitations, which will be observed in the

conclusion.

6

CONCLUSION

This study set out to explore whether vocabulary knowledge

might be a significant predictor of academic performance and

how such predictive capabilities correlated with those of

IELTS. It was found that one of the two vocabulary tests

examined showed better correlations with the participants’

overall academic GPA than did the other; and that IELTS in

this instance presented a weak correlation. These results

were deemed sufficiently significant to suggest that the

XK_Lex vocabulary test was a reliable tool for determining

potential academic achievement. Moreover, the study

determined that learners’ vocabulary scores, which can be

examined at frequency band level, enable us to uncover the

relevant vocabulary for teaching and learning purposes on

university in-sessional support programmes and some

75

suggestions were given for implementation. It might be

hypothesized that this intervention would subsequently raise

students’ vocabulary profiles, which in turn, would enhance

their potential for success on their university degree

programmes.

The study was not without its limitations, however. Firstly,

the sample size was small with just 20 Chinese participants

of similar ages, all of whom came from analogous academic

backgrounds and were pursuing the same degree in the UK. In

its favour, this does provide a fairly homogenous group for

comparison; but on the downside, it does not tell us whether

the results would have been comparable had students of

varying age, background and nationality participated.

Secondly, the data provided with the study group could have

been richer. For instance, there was only access the global

IELTS bands and not those of the sub-skills of reading,

writing, speaking and listening. It would have been a

relevant exercise to compare correlations of the sub-skills

with vocabulary size and GPAs. In terms of the academic

76

grades, correlation was only explored against the end of

degree GPAs, while in hindsight, it would have been

interesting to drill down further into the individual module

grades and compare vocabulary and IELTS scores against any

failed or high achieving modules.

A further limitation might be that the XK_Lex and VST are

both tests of orthographic word knowledge and therefore no

account of aural vocabulary knowledge was tested, whereas

there is a speaking element in IELTS. It is possible that a

future study of this nature might include use of Milton and

Hopkins Aural Lex (2005) to include a comparable test of

phonological knowledge.

Notwithstanding these considerations, it can be concluded

that the study showed to a reasonable degree that knowledge

of a learners’ vocabulary size enables the teacher to predict

potential academic performance and take appropriate steps to

ensure that the students are adequately supported with

appropriate language development.

77

Taking both the findings and limitations of this study into

consideration, there is plenty of scope to take this research

a step further in order to strengthen the validity of these

claims. A longitudinal mixed methods approach is suggested to

track students’ progress both linguistically and academically

throughout their degree programme. As well as undertaking a

quantitative methodology similar to, but more extensive than

the one in this study both in terms of numbers and range of

data; a qualitative aspect could be introduced by way of

interviews with students, language teachers, lecturers and

admissions departments to analyse perceptions and their

potential effect on performance. Such a study, being

longitudinal, would enable the implementation of in-session

support and thus, its effectiveness could correspondingly be

determined.

78

APPENDICES

Appendix 1 Participant’s Letter

Dear student

Thank you for agreeing to assist with my research

“Assessing students’ vocabulary size as a predictor of academic

performance”

You are about to take two tests which are designed to test a

learner’s vocabulary size. The accuracy of these tests depends on

how honest you are in your answers. Your answers will be kept

confidential and will be used only for the research purpose

highlighted above. Your academic department will not have access

to your answers or scores.

In addition, your IELTS scores on entry and Grade Point Average

(GPA) at the end of your programme of study will be used for the

purpose of comparison with your overall vocabulary scores from

these tests.

I appreciate your participation in this research. Your

participation in these tests means that you consent to your IELTS

scores and GPAs being used for the purpose of this study.

79

However, you have the right not to take part. If you do not wish

to continue, please let me know and hand in the test sheet.

Thank you.

Kind regardsSandy George

Please begin the test. You may ask for help at any time during the

test if anything is unclear. However, I will not be able to help

you with the answers.

80

Appendix 2 XK_Lex vocabulary size test

English XK_Lex Vocabulary Test 1 - Version: A

Please look at these words. Some of these words are real English words and some are not but are made to look like real words. Please tick the words that you know or can use. Thank you for yourhelp.

Example:

Your student number: Vocabulary Score: New commerce Organise Accuse VictoryGummer Tindle Wookey Candish SkaveWord Dust Fountain Tend JewelNear nonsense Movement Landing ReliablePeace Fond Likely Volume HardenProduce Sweat Provide Tube SorrowYou Cap Castle Liner DialWife Worry Steam Previous EncloseDo Plenty Steady Style SneezeAdd Guide Pole Outline ApparatusKilp Broy Orrade Plaudate OverendBuild Pump Guest Keeper Roast

Prosecutor addict Gulp Idleness Carnation

Samphirate treadway Darch callisthe

mia Mordue

Referral detachment Thud Blizzard Plaintively

Illuminate unsure Assassin Rut Gurgle

Gown reinforcement Wrench Incessant Heal

Verge enlightenment Backdrop Blunder go-between

Counsellor workman Unfold springboa

rd common-law

Skipper feudal Upheaval Shrapnel Locket

81

cat

dita

Authorise quartet Animation Skip Nudge

Sour psychic Banish Bastion AngerNeminary Fallity Treggle Snape Tearle

Holly appropriation

Peninsula Maroon Contrive

Adapted from: Masrai (2009), p. 82

82

Appendix 3 Vocabulary Size Test

VOCABULARY SIZE TEST – RECOGNITION

This is a vocabulary test. It has 100 questions, ten at each of 10 thousand-levels. This test should give you an idea of the number of English words you know.

Instructions:In each question, choose the correct meaning to go with the word in CAPITAL letters. Circle the letter with the best meaning for the word.

Here is an example.

1. CAT: The cat sat on the mat.

a. Animal that chases dogs

b. Animal that carries people

c. Animal that chases a mouse

d. Animal that eats fruit

In the example, the best meaning for CAT is answer c, “animalthat chases a mouse,” so you should circle c.

Thank you for your help.

Your student number: VST score:

Source: Nation, P. & Beglar, N. (2007) VST, BNC Version, 1-14K, www.lextutor.ca/tests

83

First 10001. SEE: They saw it.

a.

cut

b.

waited for

c.

looked at

d.

started

2. TIME: They have a lot of time.a.

money

b.

food

c.

hours

d.

friends

3. PERIOD: It was a difficult period.a.

question

b.

time

c.

thing to do

d.

book

4. FIGURE: Is this the right figure?a.

answer

b.

place

c.

time

d.

number

5. POOR: We are poor.a.

have no money

b.

feel happy

c.

are very interested

d.

do not like to work hard

6. DRIVE: He drives fast.a.

swims

b.

learns

c.

throws balls

d.

uses a car

7. JUMP: She tried to jump.a.

lie on top of the water

b.

get off the ground suddenly

c stop the car at the edge of the road

.d.

move very fast

8. SHOE: Where is your shoe?a.

the person who looks after you

b.

the thing you keep your money in

c.

the thing you use for writing

d.

the thing you wear on your foot

9. STANDARD: Her standards are very high.a.

the bits at the back under her shoes

b.

the marks she gets in school

c.

the money she asks for

d.

the levels she reaches in everything

10. BASIS: I don't understand the basis.a.

reason

b.

words

c.

road signs

d.

main part

Second 10001. MAINTAIN: Can they maintain it?

a.

keep it as it is

b.

make it larger

c.

get a better one than it

d.

get it

2. STONE: He sat on a stone.a.

hard thing

b kind of chair

84

.c.

soft thing on the floor

d.

part of a tree

3. UPSET: I am upset.a.

tired

b.

famous

c.

rich

d.

unhappy

4. DRAWER: The drawer was empty.a.

sliding box

b.

place where cars are kept

c.

cupboard to keep things cold

d.

animal house

5. PATIENCE: He has no patience.a.

will not wait happily

b.

has no free time

c.

has no faith

d.

does not know what is fair

6. NIL: His mark for that question was nil.a.

very bad

b.

nothing

c.

very good

d.

in the middle

7. PUB: They went to the pub.a.

place where people drink and talk

b.

place that looks after money

c.

large building with many shops

d.

building for swimming

8. CIRCLE: Make a circle.a.

rough picture

b.

space with nothing in it

c.

round shape

d.

large hole

9. MICROPONE: Please use the microphone.

a.

machine for making food hot

b.

machine that makes sounds louder

c.

machine that makes things look bigger

d.

small telephone that can be carried around

10. PRO: He's a pro.a.

someone who is employed to find out important secrets

b.

a stupid person

c.

someone who writes for a newspaper

d.

someone who is paid for playing sport etc

Third 10001. SOLDIER: He is a soldier.

a.

person in a business

b.

student

c.

person who uses metal

d.

person in the army

2. RESTORE: It has been restored.a.

said again

b.

given to a different person

c.

given a lower price

d.

made like new again

3. JUG: He was holding a jug.a.

A container for pouring liquids

b.

an informal discussion

c.

A soft cap

d.

A weapon that explodes

85

4. SCRUB: He is scrubbing it.a.

cutting shallow lines into it

b.

repairing it

c.

rubbing it hard to clean it

d.

drawing simple pictures of it

5. DINOSAUR: The children were pretending to be dinosaurs.a.

robbers who work at sea

b.

very small creatures with human formbut with wings

c.

large creatures with wings that breathe fire

d.

animals that lived a long time ago

6. STRAP: He broke the strap.a.

promise

b.

top cover

c.

shallow dish for food

d.

strip of material for holding thingstogether

7. PAVE: It was paved.a.

prevented from going through

b.

divided

c.

given gold edges

d.

covered with a hard surface

8. DASH: They dashed over it.a.

moved quickly

b.

moved slowly

c.

fought

d.

looked quickly

9. ROVE: He couldn't stop roving.a.

getting drunk

b.

travelling around

c.

making a musical sound through closed lips

d.

working hard

10. LONESOME: He felt lonesome.a.

ungrateful

b.

very tired

c lonely

.d.

full of energy

Fourth 10001. COMPOUND: They made a new compound.

a.

agreement

b.

thing made of two or more parts

c.

group of people forming a business

d.

guess based on past experience

2. LATTER: I agree with the latter.a.

man from the church

b.

reason given

c.

last one

d.

answer

3. CANDID: Please be candid.a.

be careful

b.

show sympathy

c.

show fairness to both sides

d.

say what you really think

4. TUMMY: Look at my tummy.a.

cloth to cover the head

b.

stomach

c.

small furry animal

d.

thumb

5. QUIZ: We made a quiz.a.

thing to hold arrows

b.

serious mistake

c.

set of questions

d.

box for birds to make nests in

86

6. INPUT: We need more input.a.

information, power, etc. put into something

b.

workers

c.

artificial filling for a hole in wood

d.

money

7. CRAB: Do you like crabs?a.

sea creatures that walk sideways

b.

very thin small cakes

c.

tight, hard collars

d.

large black insects that sing at night

8. VOCABULARY: You will need more vocabulary.a.

words

b.

skill

c.

money

d.

guns

9. REMEDY: We found a good remedy.a.

way to fix a problem

b.

place to eat in public

c.

way to prepare food

d.

rule about numbers

10. ALLEGE: They alleged it.a.

claimed it without proof

b.

stole the ideas for it from someone else

c.

provided facts to prove it

d.

argued against the facts that supported it

Fifth 10001. DEFICIT: The company had a large

deficit.a.

spent a lot more money than it earned

b.

went down a lot in value

c.

had a plan for its spending that used a lot of money

d.

had a lot of money in the bank

2. WEEP: He wept.a.

finished his course

b.

cried

c.

died

d.

worried

3. NUN: We saw a nun.a.

long thin creature that lives in theearth

b.

terrible accident

c.

woman following a strict religious life

d.

unexplained bright light in the sky

4. HAUNT: The house is haunted.a.

full of ornaments

b.

rented

c.

empty

d.

full of ghosts

5. COMPOST: We need some compost.a.

strong support

b.

help to feel better

c.

hard stuff made of stones and sand stuck together

d.

rotted plant material

6. CUBE: I need one more cube.a.

sharp thing used for joining things

b.

solid square block

c.

tall cup with no saucer

d.

piece of stiff paper folded in half

87

7. MINIATURE: It is a miniature.

a.

a very small thing of its kind

b.

an instrument to look at small objects

c.

a very small living creature

d.

a small line to join letters in handwriting

8. PEEL: Shall I peel it?a.

let it sit in water for a long time

b.

take the skin off it

c.

make it white

d.

cut it into thin pieces

9. FRACTURE: They found a fracture.a.

break

b.

small piece

c.

short coat

d.

rare jewel

10. BACTERUM: They didn't find a single bacterium.a.

small living thing causing disease

b.

plant with red or orange flowers

c.

animal that carries water on its back

d.

thing that has been stolen and sold to a shop

Sixth 10001. DEVIOUS: Your plans are devious.

a.

tricky

b.

well-developed

c not well thought out

.d.

more expensive than necessary

2. PREMIER: The premier spoke for an hour.a.

person who works in a law court

b.

university teacher

c.

adventurer

d.

head of the government

3. BUTLER: They have a butler.a.

man servant

b.

machine for cutting up trees

c.

private teacher

d.

cool dark room under the house

4. ACCESSORY: They gave us some accessories.a.

papers allowing us to enter a country

b.

official orders

c.

ideas to choose between

d.

extra pieces

5. THRESHOLD: They raised the threshold.a.

flag

b.

point or line where something changes

c.

roof inside a building

d.

cost of borrowing money

6. THESIS: She has completed her thesis.a.

long written report of study carried out for a university degree

b.

talk given by a judge at the end of atrial

c.

first year of employment after becoming a teacher

d.

extended course of hospital treatment

7. STRANGLE: He strangled her.a.

killed her by pressing her throat

b.

gave her all the things she wanted

c.

took her away by force

d.

admired her greatly

88

8. CAVALIER: He treated her in a cavalier manner.a.

without care

b.

politely

c.

awkwardly

d.

as a brother would

9. MALIGN: His malign influence is still felt.a.

evil

b.

good

c.

very important

d.

secret

10. VEER: The car veered.

a.

went suddenly in another direction

b.

moved shakily

c.

made a very loud noise

d.

slid sideways without the wheels turning

Seventh 10001. OLIVE: We bought olives.

a.

oily fruit

b.

scented pink or red flowers

c.

men's clothes for swimming

d.

tools for digging up weeds

2. QUILT: They made a quilt.a.

statement about who should get theirproperty when they die

b firm agreement

.c.

thick warm cover for a bed

d.

feather pen

3. STEALTH: They did it by stealth.a.

spending a large amount of money

b.

hurting someone so much that they agreed to their demands

c.

moving secretly with extreme care and quietness

d.

taking no notice of problems they met

4. SHUDDER: The boy shuddered.a.

spoke with a low voice

b.

almost fell

c.

shook

d.

called out loudly

5. BRISTLE: The bristles are too hard.a.

questions

b.

short stiff hairs

c.

folding beds

d.

bottoms of the shoes

6. BLOC: They have joined this bloc.

a.

musical group

b.

band of thieves

c.

small group of soldiers who are sent ahead of others

d.

group of countries sharing a purpose

7. DEMOGRAPHY: This book is about demography.a.

the study of patterns of land use

b.

the study of the use of pictures to show facts about numbers

c.

the study of the movement of water

d.

the study of population

8. GIMMICK: That's a good gimmick.

a.

thing for standing on to work high above the ground

b.

small thing with pockets to hold money

c.

attention-getting action or thing

89

d.

clever plan or trick

9. AZALEA: This azalea is very pretty.

a.

small tree with many flowers growing in groups

b.

light material made from natural threads

c.

long piece of material worn by women in India

d.

sea shell shaped like a fan

10. YOGHURT: This yoghurt is disgusting.

a.

grey mud found at the bottom of rivers

b.

unhealthy, open sore

c.

thick, soured milk, often with sugar and flavouring

d.

large purple fruit with soft flesh

Eighth 10001. ERRATIC: He was erratic.

a.

without fault

b.

very bad

c.

very polite

d.

unsteady

2. PALETTE: He lost his palette.a.

basket for carrying fish

b.

wish to eat food

c.

young female companion

d.

artist's board for mixing paints

3. NULL: His influence was null.a.

had good results

b.

was unhelpful

c.

had no effect

d.

was long-lasting

4. KINDERGARTEN: This is a good kindergarten.a.

activity that allows you to forget your worries

b.

place of learning for children too young for school

c.

strong, deep bag carried on the back

d.

place where you may borrow books

5. ECLIPSE: There was an eclipse.

a.

a strong wind

b.

a loud noise of something hitting the water

c.

The killing of a large number of people

d.

The sun hidden by a planet

6. MARROW: This is the marrow.a.

symbol that brings good luck to a team

b.

Soft centre of a bone

c.

control for guiding a plane

d.

increase in salary

7. LOCUST: There were hundreds of locusts.a.

insects with wings

b.

unpaid helpers

c.

people who do not eat meat

d.

brightly coloured wild flowers

8. AUTHENTIC: It is authentic.a.

real

b.

very noisy

c.

Old

d.

Like a desert

9. CABARET: We saw the cabaret.a.

painting covering a whole wall

b.

song and dance performance

c.

small crawling insect

d.

person who is half fish, half woman

10. MUMBLE: He started to mumble.a.

think deeply

b shake uncontrollably

90

.c.

stay further behind the others

d.

speak in an unclear way

Ninth 10001. HALLMARK: Does it have a hallmark?

a.

stamp to show when to use it by

b.

stamp to show the quality

c.

mark to show it is approved by the royal family

d.

Mark or stain to prevent copying

2. PURITAN: He is a puritan.a.

person who likes attention

b.

person with strict morals

c.

person with a moving home

d.

person who hates spending money

3. MONOLOGUE: Now he has a monologue.a.

single piece of glass to hold over his eye to help him to see better

b.

long turn at talking without being interrupted

c.

position with all the power

d.

picture made by joining letters together in interesting ways

4. WEIR: We looked at the weir.a.

person who behaves strangely

b.

wet, muddy place with water plants

c.

old metal musical instrument played by blowing

d.

thing built across a river to control the water

5. WHIM: He had lots of whims.a.

old gold coins

b.

female horses

c.

strange ideas with no motive

d.

sore red lumps

6. PERTURB: I was perturbed.a.

made to agree

b.

Worried

c.

very puzzled

d.

very wet

7. REGENT: They chose a regent.

a.

an irresponsible person

b.

a person to run a meeting for a time

c.

a ruler acting in place of the king

d.

a person to represent them

8. OCTOPUS: They saw an octopus.a.

a large bird that hunts at night

b.

a ship that can go under water

c.

a machine that flies by means of turning blades

d.

a sea creature with eight legs

9. FEN: The story is set in the fens.

a.

low land partly covered by water

b.

a piece of high land with few trees

c.

a block of poor-quality houses in a city

d.

a time long ago

10. LINTEL: He painted the lintel.

a.

Beam over the top of a door or window

b.

small boat used for getting to land from a big boat

c.

beautiful tree with spreading branchesand green fruit

d.

board showing the scene in a theatre

Test from: Nation, P. & Beglar, N. (2007b) VST, BNC Version, 1-14K, www.lextutor.ca/tests

91

Tenth 10001. AWE: They looked at the mountain with awe.

a.

worry

b.

interest

c.

wonder

d.

respect

2. PEASANTRY: He did a lot for the peasantry.a.

local people

b.

place of worship

c.

businessmen's club

d.

poor farmers

3. EGALITARIAN: This organization is egalitarian.a.

does not provide much information about itself tothe public

b.

dislikes change

c.

frequently asks a court of law for a judgement

d.

treats everyone who works for it as if they are equal

4. MYSTIQUE: He has lost his mystique.a.

his healthy body

b.

the secret way he makes other people think he has special power or skill

c.

the woman who has been his lover while he ismarried to someone else

d.

the hair on his top lip

5. UPBEAT: I'm feeling really upbeat about it.a.

upset

b.

good

c.

hurt

d.

confused

6. CRANNY: We found it in the cranny!a.

sale of unwanted objects

b.

narrow opening

c.

space for storing things under the roof of a house

d large wooden box

.

7. PIGTAIL: Does she have a pigtail?a.

a rope of hair made by twisting bits together

b.

a lot of cloth hanging behind a dress

c.

a plant with pale pink flowers that hang down in short bunches

d.

a lover

8. CROWBAR: He used a crowbar.a.

heavy iron pole with a curved end

b.

false name

c.

sharp tool for making holes in leather

d.

light metal walking stick

9. RUCUS: He got hurt in the rucus.a.

hollow between the stomach and the top of the leg

b.

pushing and shoving

c.

group of players gathered round the ball in some ball games

d.

race across a field of snow

92

10. LECTERN: He stood at the lectern.a.

desk to hold a book at a height for reading

b.

table or block used for church sacrifices

c.

place where you buy drinks

d.

very edge

93

BIBLIOGRAPHY

Albrechtsen, D. Haastrup, K. and Henriksen, B. (2008) Vocabulary and Writing in a First and Second Language: Processes and Development. Houndsmills: Palgrave Macmillan.

Anderson, R.C. and Freebody, P. (1981) Vocabulary knowledge. In: J.T. Gutherie (ed.) Comprehension and Teaching: Research Reviews (pp. 77-117). Newark, DE: International Reading Association.

Anderson, R.C. and Freebody, P. (1983) Reading Comprehension and the Assessment and Acquisition of Word Knowledge. In: Hutson, R. (ed.) Advances in Reading / Language Research. A research Annual, CT: JAI Press.

Bauer, L. and Nation, P. (1993) Word Families. International Journal of Lexicography, 6, 253-279.

Bellingham, L. (1993) The relationship of language proficiency to academic success for international students, New Zealand Journal of Educational Studies 30, 2: 229-232.

Cotton, F. and Conrow, F. (1998) An investigation into thepredictive validity of IELTS amongst a group of international students studying at the University of Tasmania. In: IELTS Research Reports 1, IELTS Australia PtyLtd: Canberra, 72-115.

Coxhead, A. (2000) A new academic word list. TESOL Quarterly, 34 (2), 213-238.

Daller, H., Milton, J. & Treffers-Daller, J. (eds.) (2007) Modelling and Assessing Vocabulary Knowledge. Cambridge: Cambridge University Press.

Diack, H. (1975) Test Your Own Wordpower, St Albans: Paladin.

Diller, K.C. (1978) The Language Teaching Controversy. Rowley, Massachusettes: Newbury House.

94

Dooey, P. and Oliver, R. (2002) An investigation into the predictive validity of the IELTS test. Prospect, 17, 36 -54.

Feast, V. (2002) The impact of IELTS scores on performance at university. International Education Journal, 3, 70-85.

Francis, W.N. and Kucera, H. (1982) Frequency Analysis of English Usage. Boston: Houghton Mifflin.

Gardener, D. (2007) Validating the construct of words in applied corpus-based vocabulary research: a critical survey. Applied Linguistics, 28, 241-265.

Goulden, R., Nation, I.S.P. and Read, J. (1990) How large can a receptive vocabulary be? Applied Linguistics. 11, 341-363.

Hill, K. Storch, N. and Lynch, B. (1999) A comparison of IELTS and TOEFL as predictors of academic success, English Language Testing System Research Reports, 2, 52-63.

Hu, M. and Nation, I.S.P. (2000) Unknown vocabulary densityand reading comprehension. Reading in a Foreign Language 13(1), 403-430.

Kerstjens, M. & Nery, C. (2000) Predictive validity in the IELTS test: A study of the relationship between IELTS scores and students' subsequent academic performance. IELTS Research Reports, 3, 85-108.

Kilgarriff, A. (2006) BNC database and word frequency lists. http://www.kilgarriff.co.uk/bnc-readme.html#lemmatised. (last accessed 03/10/2014).

Konstantakis, N. (2007) Creating a Business Word List for Teaching Business English. Estudios de Linguistica Inglesa Aplicada (ELIA) 7, 103-126.

Laufer, B. (1989) What percentage of text is essential for comprehension? In C. Lauren and M. Nordman (eds)

95

Special Language; from Humans to Thinking Machines (pp.316-323). Clevedon: Multilingual Matters.

Laufer, B. (1992) How much lexis is necessary for reading comprehension? In: P.J.L. Arnaud and H. Bejoint (eds.)Vocabulary and Applied Linguistics (pp. 126-132). London: Macmillan.

Laufer, B. and Paribakht, T.S. (1998) The relationship between passive and active vocabularies: effects of language learning context. Language Learning, 48, 365-391.

Masrai, A.M. (2009) Measuring the English Vocabulary Size of Saudi University Students: Validating a New 10,000 Word Vocabulary Size Test.Unpublished MA thesis: Swansea University.

Meara, P. (1992) Yes-No, Test 1, Levels 1-5, Swansea University. http://www.lextutor.ca/tests/yes_no_eng/test_1/ (last accessed 20/03/2014).

Meara, P. (1996) The dimensions of lexical competence. In: G. Brown, K. Malmkjaer & J. Williams (eds.) Performance and competence in second language acquisition. Cambridge: Cambridge University Press, 35-53.

Meara, P. and Jones, G. (1990) Eurocentre’s Vocabulary Size Test. User’s Guide. Zurich: Eurocentres.

Meara, P. & Milton, J. (2003) X_Lex, The Swansea Levels Test. Newbury: Express.

Meara, P. M. and Miralpeix, I. (2006) Y_Lex: the Swansea Advanced Vocabulary Levels Test. v2.05. Swansea: Lognostics

Milton, J. (2007) Lexical profiles, learning styles and theconstruct validity of lexical size tests. In: H. Daller, J. Milton & J. Treffers-Daller (eds.) Modelling and Assessing Vocabulary Knowledge. (pp. 47-58) Cambridge: Cambridge University Press.

96

Milton, J. (2009) Measuring Second Language Vocabulary Acquisition. Bristol: Multilingual Matters.

Milton, J. and Hopkins, N. (2005) Aural Lex. Swansea: SwanseaUniversity.

Milton, J. and Hopkins, N. (2006) Comparing phonological and orthographic vocabulary size: Do vocabulary tests underestimate the knowledge of some learners. The Canadian Modern Language Review 63 (1), 127-147.

Milton, J., Wade, J. and Hopkins, N. (2010) Aural word recognition and oral competence in a foreign language.In: R. Chacon-Beltran, C. Abello-Contesse & M. Torreblanca-Lopez (eds.) Further Insights into Non-native Vocabulary Teaching and Learning (pp. 83-98). Bristol: Multilingual Matters.

Minshal, D. M. (2013) A Computer Science Wordlist. Unpublished MA thesis: Swansea University.

Morris, L. & Cobb, T. (2004) Vocabulary profiles as predictors of TESL student performance. System 32(1), 75-87.

Nagy, W. E. and Anderson, R. C. (1984) How many words are there in printed school English? Reading Research Quarterly19, 304-330.

Nation, I. S. P. (ed.) (1984) Vocabulary Lists, Words, Affixes and Stems. English University of Wellington, New Zealand: English Language Institute.

Nation, I. S. P. (1990) Teaching and Learning Vocabulary. New York: Heinle & Heinle Publishers.

Nation, I .S. P. (2001) Learning Vocabulary in Another Language. Cambridge: CUP.

Nation, I. S. P. (2004) A study of the most frequent word families in the British National Corpus. In: P.

97

Bogaards & P. Laufer (eds.) Vocabulary in a Second Language: Selection, Acquisition and Testing (pp. 3-13) Amsterdam: John Benjamins.

Nation, I .S. P. (2006) How large a vocabulary is needed for reading and listening. The Canadian Modern Language Review, 63,1 (September), 59-82.

Nation, I. S. P. and Beglar, D. (2007a) A vocabulary size test. The Language Teacher,

31(7), 9-13.

Nation, P. and Beglar, D. (2007b) Vocabulary size test, BNCVersion (1-14k), VUW: New Zealand. http://www.lextutor.ca/tests/levels/recognition/1-14k/(last accessed (20/03/2014).

Nation, I. S. P. (2012) http://www.victoria.ac.nz/lals/about/staff/publications/paul-nation/Vocabulary-Size-Test-information-and-specifications.pdf (last accessed 03/10/2014).

Read, J. (1993) The development of a new measure ofL2 vocabulary knowledge. Language Testing 10 (3), 355-371.

Read, J. (1995) Refining the word associates format as a measure of depth of vocabulary knowledge. New Zealand Studies in Applied Linguistics 1, 1-17.

Read, J. (2000) Assessing Vocabulary. Cambridge: CUP.

Read, J. (2007) Second Language Vocabulary Assessment: current practice and new directions. International Journal of English Studies 7, 2, 105-125.

Richards, J. (1976) The role of vocabulary teaching. TESOL Quarterly, 77–89.

Schmitt, N. (2008) Instructed second language vocabulary learning. Language Teaching Research 12, 3, 329-363.

98

Schmitt, N., Schmitt, D. and Clapham, C. (2001) Developing and exploring the behaviour of two new versions of theVocabulary Levels Test. Language Testing, 18, 55-88.

Seashore, R. & Eckerson, L. (1940) The measurement of individual differences in general English vocabulary. Journal of Educational Psychology, 31, 14-38.

Staehr, L.S. (2008) Vocabulary size and the skills of listening, reading and writing. Language Learning Journal,36(2), 139-152.

Swenson, E. and West, P. (1934) On the counting of new words in textbooks for teaching foreign languages. Bulletin of the Department of Educational Research, University of Toronto, 1.

Thorndike, E. and Lorge, I. (1944) The Teacher's Word Book of 30,000 Words, Columbia University: Teachers College.

Webb, S. (2005) Receptive and productive vocabulary learning: The effects of reading and writing on word knowledge. Studies in Second Language Acquisition, 27,33-52.

West, M. (1953) A General Service List of English Words, London, Longman.

Wilkins, D. (1972) Linguistics in Language Teaching. London: Arnold.

Woodrow, L. (2006) Academic success of international postgraduate education students and the role of English proficiency. University of Sydney Papers in TESOL, 1, 51-70.

http://www.IELTS.org (last accessed 26/09/2014).

http://www.lextutor.ca (last accessed 13/10/2014)

http://www.lognostics.co.uk/tools (last accessed 20/03/2014)

99

100