Post on 24-Apr-2023
A Comparative Study of Measures of Vocabulary
Knowledge and IELTS Proficiency Scores as
Potential Predictors of Academic Performance
Sandra George (589880)
College of Arts and Humanities
Swansea University
Submitted to Swansea University in fulfilment of the
requirements for the degree of Master of Arts (MATEFL)
14th October 2014
ABSTRACT
This study was motivated by work related observations at
Swansea University, which noticed the need for improved
predictive assessments of international students’ potential
academic performance, in order to better support them in-
sessionally. Additionally, positive results from a range of
studies linking vocabulary knowledge measures with
proficiency levels and their capacity for predicting future
performance in the four skills (Laufer, 1992; Staehr, 2008,
Milton et al, 2010) directed an investigation into how
effectively vocabulary scores could predict academic
performance for L2 learners on a degree programme, and how
these results would compare with the predictive
capabilities of the International English Language Testing
System (IELTS) proficiency bands.
Data was collected from 20 students on a Swansea University
Law exchange programme. The participants’ vocabulary scores
were measured using the XK_Lex test (Masrai, 2009) and a
Vocabulary Size Test (Nation & Beglar, 2007b).
Subsequently, the students’ IELTS band scores and end of
programme grade point averages were correlated with the
vocabulary scores. The results of the study showed
vocabulary knowledge measures to be a more reliable
predictor of academic performance than IELTS. Following on
from this, the pedagogical implications were explored and
suggestions for in-session support were made.
DECLARATION FORM
DECLARATIONThis work has not previously been accepted in substance forany degree and is not being concurrently submitted incandidature for any degree.
Signed: Date: 14th October2014
STATEMENT 1This dissertation is the result of my own independentwork/investigation, except where otherwise stated. Othersources are acknowledged by giving explicit references. Abibliography is appended.
Signed: Date: 14th October2014
STATEMENT 2I understand the college policy on plagiarism as set out inthe “College of Arts and Humanities Handbook for MAStudents”, and accept that this dissertation may be copied,stored and used for purposes of plagiarism detection. Itherefore certify that I have submitted an electronic copyof this dissertation via the Turnitin system on Blackboard.
Signed: Date: 14th October2014
CONTENTS
List of Tables and Figures
Acknowledgements
1. Introduction
2. Literature review
2.1. What is a word?
2.2. What does it mean to know a word?
2.3. How many words does a learner need to know in a
second language?
2.4. Vocabulary knowledge as a predictor of language
performance
2.5. IELTS scores as a predictor of academic
performance
2.6. What makes a good vocabulary test?
2.7. Test types
3. Methodology
3.1. Aims and objectives
3.2. Research Questions
3.3. Participants
3.4. Data collection and testing procedure
4. Results
5. Discussion
5.1. How well do the two vocabulary size measures
correlate with academic performance and IELTS?
5.2. Do any of these tests have a good enough
correlation to enable it to be used as a predictive
test of subsequent academic performance?
5.3. What scores are associated with failure or poor
academic performance to guide us identifying students
in need of subsequent in-session support?
5.4. Can the scores identify areas of knowledge to be
covered by in-session support classes?
6. Conclusion
Appendices
Bibliography
LIST OF TABLES AND FIGURES
TABLES
2.1 What is involved in knowing a word?
2.2 Vocabulary size and coverage
2.3 Comparison of the most frequent vocabulary from three
sources
4.1 Raw IELTS, XK_Lex, VST and GPA scores
4.2 Mean scores on vocabulary size tests and GPAs
4.3 Correlations of vocabulary tests and IELTS with GPAs
4.4 Component matrix
4.5 Group statistics
4.6 Independent samples test
5.1 The four strands and their application with a focus on
vocabulary
FIGURES
2.1 Example of a vocabulary levels test
2.2 Example of a Yes/No test
2.3 Example of X_Lex test
2.4 Example of X_Lex results
4.1 Factor analysis scree plot
5.1 Sample frequency output
5.2 Sample Tex Lex Compare output
ACKNOWLEDGEMENTS
This thesis has been particularly challenging as I havecompleted it whilst working full time and trying to devotetime and care to my loving family. So, my first thanks mustgo to them, husband Mike, daughter Millie and son Charlie,for putting up with all of the long evenings and weekendsthat I have shut myself away in the study and not takenpart in all of the family fun and activity.
Next I would like to thank my supervisor, Professor JamesMilton, for not only enthusing me with his expert knowledgein the field, but also for having such calmness of presencethat I always left his office feeling de-stressed and ableto continue with the next chapter. Thank you Jim.
Finally, I must extend my thanks to my two line managers,Sarah Huws-Davies and Kevin Child, who have supported mewith study leave at crucial times, particularly during thelatter part of this project, when I have found the goingespecially tough.
Thank you all.
1
INTRODUCTION
Vocabulary is considered to be a core component in second
language learning; the foundation blocks essential in any
learner’s development in second language proficiency;
“Without grammar very little can be conveyed, without
vocabulary nothing can be conveyed” (Wilkins, 1972:111). It
is no wonder then, that interest in vocabulary testing has
experienced considerable growth amongst second language
acquisition researchers since the early ‘90s, a trend which
seems to be continuing. Even relatively early studies suggest
that a learner’s vocabulary size has a significant influence
on second language development; and having a large vocabulary
is an influential factor in the development of other
linguistic competencies in the target language (Meara, 1996),
such as reading, writing and listening (Staehr 2008).
There is no shortage of vocabulary tests available to
linguistics students for academic study, and it is not the
purpose of this study to develop a new vocabulary size test.
Many influential researchers in the field such as Read
6
(1993), Meara and Milton (2003), Nation (1990) and Nation and
Beglar (2007) have developed valid vocabulary size tests.
Therefore, the aim here is to establish which test will be
most valuable in order to provide reliable results to the
research enquiry.
The interest in this study was sparked in relation to the
specific issue of large numbers of non-native speakers of
English entering degree programmes at Swansea University and
requiring varying levels of in-sessional language support, if
indeed they attempted to seek the support at all, as it is
not compulsory. A comprehensive language programme is already
in existence, but with the need to improve the knowledge of
students with vastly differing levels of proficiency and
representing a wide range of academic disciplines, there is a
requirement to identify an effective way of distinguishing
potentially problematic students and their particular
knowledge shortfalls. The desired outcome would be that if a
valid method is established, it would allow for tailored and
timely support packages to be put in place, with the likely
7
end result that non-native students have the increased
capacity to achieve more successful degree outcomes.
Consequently, this study aims to follow-up on the generally
accepted view that vocabulary levels tests can be used for
diagnostic purposes in higher education to predict an L2
student’s ability to be linguistically capable of undertaking
academic study in the second language. More specifically,
consideration will be given as to the value of diagnostic
vocabulary testing as a predictor of a student’s subsequent
academic performance at degree level study in the UK. The
vocabulary scores will be compared to the students’
International English Language Testing System (IELTS) scores on entry
to the university to enquire whether one or the other might
appear a more reliable method of predicting subsequent
academic performance.
IELTS is one of the most popular English language tests for
higher education entry, principally in the UK, Australia and
New Zealand, but also growing in the US and Canada, being
accepted by around 8000 institutions in over 135 countries
8
(IELTS.org). This would imply that tertiary organisations are
very reliant on the test’s reliability in determining to what
extent a learner is linguistically capable of performing on
an academic programme of study.
In general, more objective test methods, such as multiple
choice and vocabulary size tests, have good records of
reliability and can be easily measured for equivalence with
retests of multiple versions being straightforward to
administer. Less direct testing methods, requiring extensive
writing, comprehension and oral examinations such as IELTS,
require an additional set of skills from the learner and as
such are open to subjectivity from the marker (Milton, 2009).
Therefore, this study aims to examine whether a learner’s
vocabulary score on entry to university may reliably forecast
suitability to study in the L2 and potentially be a good
predictor of academic progress in comparison to widely
accepted standardised entry tests.
9
2
LITERATURE REVIEW
The intention of this chapter is to explore word knowledge
and the various ways to measure it. The chapter will attempt
to define what is meant by a word and word knowledge, paying
particular attention to vocabulary breadth, rather than
depth, as tests of word recognition and not productive
knowledge are used in this study. The notion of learning
goals and associated vocabulary requirements will be examined
to determine how many words a non-native speaker is likely to
need to communicate appropriately in the target language.
The chapter will move on to discuss the literature around how
effectively vocabulary measures can potentially predict
language proficiency and subsequent performance on an
academic degree programme. This question will then be applied
to research around the predictive capabilities of the
standardised IELTS examination.
10
Finally, the chapter will explore vocabulary test
construction in terms of what is required to develop a valid
and reliable test which also has high face validity for both
students and the wider non-linguist academic community alike.
2.1 What is a word?
There is a wide range of vocabulary tests available to
measure vocabulary knowledge, but controversially, they do
not all give equal results, even when used with the same
learners. For example, the Seashore and Eckerson (1940) and
Diller (1978) tests estimated that the average vocabulary
size of native speakers of English was as much as 200,000
words. This contrasts markedly with Milton’s (2007) estimate
of around 10,000 words for undergraduate native English
speakers. Such variations highlight the fact that it is not
easy to measure how much language an individual knows at all.
Space, distance, time and volume can all be reliably measured
with standard units of measurement. But a unit of measurement
for vocabulary knowledge can vary depending on how we are
defining the words.
11
Tests such as Seashore and Eckerson (1940) were developed
using dictionaries and by counting all forms of a word. As
such, the words think, thought and thinking would all be
individually counted. Later studies, such as Goulden et al.
(1990), attempt to rationalise word counts by focusing on
frequency information for enhanced accuracy. In this way,
derived forms and common inflections become a word family and
for the purpose of measurement, counted as a single word.
There will be no surprise that this method leads to a far
smaller word count of around 17,000 for an educated native
speaker. Therefore, consideration should be given as to the
many different ways to count words before a valid and
reliable method of testing can be assumed.
Despite there being numerous ways to count words, the result
will always be referred to as a word count regardless of
method. However, words can be more specifically defined by
their specialist terms of tokens, types, word families or lemmas.
The number of tokens is calculated by counting every word in a
text regardless of whether the word is repeated. So the
12
sentence ‘the cat sat on the mat’ counts as six tokens even though
the word ‘the’ occurs twice. Tokens are sometimes referred to
as running words. This unit of counting can be useful if we
want to know how many words are in a journal article or an
academic essay, or to assess reading speed for example.
Counting words as types lets us know how many different words
a text contains, so if a word is encountered more than once
it is not counted the second and each subsequent time.
Therefore, in a types count, the sentence ‘the cat sat on the mat’
would only have a word count of five, as the word ‘the’ is
repeated and only counted once. This could be useful if we
want to know how many words a dictionary contains. However,
using types as the unit of measurement may not be so useful
for testing a person’s vocabulary size. Seashore and Eckerson
(1940) used a types count and estimated that an educated
native speaker would know around 200,000 words, which would
appear to be an impenetrable learning burden for the L2
learner. Further, there is a degree of regularity in the ways
in which inflections are developed, such as by adding ‘s’ to
form a plural in English. Therefore the learner can apply the
13
rules using their knowledge of inflections without having
encountered the word previously.
A number of tests use word families. “A word family consists of a
headword, its inflected forms, and its closely derived
forms.” (Nation, 2001: 8). By this method, no derivations or
inflections are counted even when there is a change to the
part of speech. As such, the words rebel (v) rebel (n), rebellion (n),
rebellious (adj) and rebelliously (adv) would be counted as one word.
Avril Coxhead’s (2000) academic word list is an example of a
word families list and is a useful tool for L2 learners of
English in academic contexts. However, when considering
whether to use word families for developing vocabulary size
tests, the researcher may encounter problems in deciding what
to include or not to include in a word family. The learners’
level of proficiency is likely to determine their level of
knowledge of affixes and thus, have an influence on a word
families vocabulary size test.
More acceptable results may be gained from tests developed
using lemmas (Gardener, 2007), which group words according to
14
a stem word and its regular inflections along with any common
derivations which do not affect the part of speech. Francis
and Kuĉera (1982) produced a lemmatised list using the Brown
Corpus, where comparatives and superlatives are not part of
the lemma, nor are the same word forms used as different
parts of speech. Thus, the words rebel (v) and rebel (n) would be
counted as two words. In forming lemmatised lists, one
consideration is what to do with irregular verbs and plural
nouns. This relates to the notion of learning burden, or the
degree of effort required in learning a word (Swenson and
West, 1934). Naturally, the learning burden of words which
follow the rules of regular inflections is lower than those
which are irregular. Consequently, although not without its
issues, using lemmas as the basis of vocabulary levels tests
would appear to give the most accurate results over those
using types, tokens or word families which could lead to
overestimation (Nation, 2001).
2.2 What does it mean to know a word?
Once the unit of measurement has been decided, researchers
should also consider the degree of understanding that needs
15
to be tested. However, there can be variations in the idea of
what it means to know a word. Nation (2001) suggests that
words are not isolated items, but parts of a more complex
interconnected system that functions on a number of levels,
while Laufer and Paribakht propose that “no clear and
unequivocal consensus exists as to the nature of word
knowledge” (1998: 366). Word knowledge, therefore, must also
exist on a number of different levels. Richards (1976)
asserts that features such as semantic character, syntax,
morphological features and appropriate register are all
involved in the extent to which one knows a word.
A useful distinction is provided by Anderson and Freebody
(1981), which examines the difference between breadth and depth
of vocabulary knowledge. Put simply, breadth refers to the
number of words known, while depth refers to a learner’s level
of understanding of vocabulary items, what is known about
meaning, connotation, register, collocation and other
morphological aspects.
16
Nation (2001) cites the work of Palmer (1921), West (1938)
and Crow (1986) who made the distinction between receptive and
productive vocabulary. Receptive knowledge is that which is
more passive and required for reading and listening in order
to comprehend meaning; whereas productive, or active
knowledge, is employed in speaking and writing where meaning
must be conveyed to others. Nation (2001) further develops
this distinction by separating word knowledge into three
aspects. Firstly, form or what the word looks like, sounds
like, its phonology and affixes. Secondly, meaning, this
links form and definition, its contextual references and
associations. Finally, use, which refers to the word’s
grammatical function, collocations and how to use it. Each of
these aspects can be further subdivided and neatly organised
into receptive and productive elements as shown in the table
below.
17
Table 2.1 What is involved in knowing a word?
Form
Spoken R What does the word sound like?P How is the word pronounced?
Written R What does the word look like?P How is the word written and spelled?
Word partsR What parts are recognisable in this
word?
P What word parts are needed to express meaning?
Meaning
Form and meaning
R What meaning does this word form signal?
P What word form can be used to express this meaning?
Concepts and referents
R What is included in the concept?P What items can the concept refer to?
AssociationsR What other words does this word make
us think of?
P What other words could we use instead of this one?
Use
Grammatical functions
R In what patterns does the word occur?
P In what patterns must we use this word?
CollocationsR What words or type of word occur with
this one
P What words or types of words must we use with this one?
Constraints on use
R Where, when and how often would we meet this word?
P Where, when and how often can we use this word?
R = Receptive, P = ProductiveSource: Nation (2001: 27)
18
2.3 How many words does a learner need to know in a second
language?
The significance of vocabulary acquisition in second language
learning should not be underestimated. In the early years a
child will begin to communicate with individual words in the
first language, and combined with facial expression, body
language and contextual referents, a degree of meaning can be
conveyed. Similar can be said of early L2 learning. But how
many words a learner needs to know to communicate effectively
is of considerable interest, as this will depend on the goal
of learning and the L2 context. Consideration also needs to
be given to what these words should be.
High frequency words, or function words, along with some
content vocabulary are considered essential for the majority
of L2 learners, regardless of learning goals, due to the high
rate of occurrence in all types of written and spoken texts.
West’s General Service List (1953) has long been the
benchmark high frequency vocabulary list; and more recently,
Nation’s Range List (2004), drawn from the British National
19
Corpus (BNC). Such lists include around 2000 word families,
which typically give an average of 80% coverage, considered
to be sufficient for general understanding.
The importance of the 2000 most frequent words in a foreign
language is demonstrated in Staehr’s study of low level
learners (2008). Staehr aimed to assess the connection
between vocabulary size and a learner’s performance in
reading, writing and listening skills. The results
demonstrated above average performance in the tests by
learners with a sound knowledge of the first 2000 most
frequent words. Reading and writing scores were below average
for those with low vocabulary scores at the 2000 word level.
Notably, listening scores were markedly better with 65% of
the same group performing above average. Presumably, other
factors such as contextual referencing played a part in the
skill of listening, thus aiding understanding (Staehr 2008).
However, whilst knowledge of the first 2000 most frequent
words may be essential for general understanding, this does
not take into account any specific learning goals.
20
In some contexts, 2000 words and 80% coverage is only
adequate for gist understanding. Laufer (1989) estimated that
university students need 95% coverage of the running words of
academic tests in order to have “reasonable” comprehension.
Similarly, Hu and Nation (2000) suggested that 98% coverage
would be required in contexts such as written discourse and
reading for pleasure. Conversely, there is some debate as to
how many words such coverage actually equates to. Laufer
(1989) calculated this at almost 5000 words for 95% coverage,
while Carroll et al (1971, cited in Milton 2009) placed this
figure at just over 12000, as seen in table 2.2 below.
Table 2.2 Vocabulary size and coverage
Number ofwords
% textcoverage
86,741 10043,831 9912,448 955,000 89.44,000 87.63,000 85.22,000 81.31,000 74.1100 4910 23.7
Source: Carroll et al. (1971, cited in Nation, 2001, p.15)
21
Nation (2001) offers an explanation for this discrepancy
noting that the unit of measurement in Laufer’s study is not
clear and the evidence was taken from Dutch texts and applied
to English. Further, the type and length of text has an
effect on the results; Laufer’s corpora were predominantly
academic, whereas Carroll et al used very large and varied
corpora which may produce a greater vocabulary size (Milton
2009). Schmitt’s 2008 study seems to address this imbalance.
Referring to Nation’s 2006 research, Schmitt concludes that a
vocabulary of 8000-9000 is necessary to perform adequately in
reading. The figures are lower for speech and listening,
which may be because these skills are more likely to involve
more informal discourse, repetition and visual clues, but
still suggests that as many as 5000-7000 words may be
required for good comprehension.
Such vocabulary goals may seem rather ambitious and
unattainable to an L2 learner, but in a multiplicity of
situations, varying numbers and types of words are necessary,
which brings us back to the aims of learning where
22
consideration of not only high-frequency words, but also
academic, technical, specialist and low-frequency words may
be required. The prerequisite vocabulary would naturally be
different for a 6 year old, an adult tourist and a student
wishing to study in a non-native university. Table 1.2
compares the frequency of occurrence of lexis taken from
three sources: the BNC, car manuals and children’s EFL books.
It can be seen that lexis like oil, valve, remove and replace, which
are technical, topic relevant words, become high frequency
words in the lexis of car manuals; while look, say and dog are
more frequent in children’s EFL books; none of which feature
in the BNC’s most frequent 15 words. Therefore, the most
frequent 2000 words should be always be a preliminary goal,
but the less frequent thematic vocabulary which is essential
for the context of a text should not be excluded.
Table 2.3 Comparison of the most frequent vocabulary fromthree sources
Order BNC Car Children’s EFL1 The And is2 Be The the3 Of To A
23
4 And Of Your5 A In No6 In As Look7 To Or Where8 Have With For9 It remove But10 To A Did11 For replace From12 I For Say13 That Oil As14 You Be Very15 He Valve Dog
Source: Milton (2009)
Over the years, a number of tests have focussed on vocabulary
where the learner’s goal is to pursue a course of academic
study in the second language. To this end, Avril Coxhead
(2000) developed the Academic Word List (AWL) to assist in
the acquisition of the recommended 95-98% coverage students
are likely to need to study in English at university level
(Laufer, 1989; Hu & Nation, 2000). Coxhead’s research
examined a 3.5 million word corpus from academic texts across
a range of disciplines. The resulting AWL contains 570
academic word families arranged into sub-lists based on
frequency, providing 10% coverage of the corpus, which in
conjunction with West’s GSL (1953) gives an average of 90%
coverage. There still remains a shortfall of 5-8% for
24
acceptable understanding of the academic discourse students
are likely to encounter. This shortage is probably made up of
the specialist lexicons found in different academic
disciplines which do not necessarily occur in general lists
or the AWL, and not forgetting that this is only relevant to
learners working with academic texts.
In an attempt to bridge the gap in one discipline,
Konstantakis (2007) uses similar criteria to Coxhead to
produce a list of 498 word families resulting in the Business
Word List (BWL). However, this additional coverage of
specialist items still only amounts to around 2.5% when items
such as abbreviations and proper nouns are omitted. More
recently, Minshall (2013) investigated the viability of a
Computer Science technical word list. This study resulted in
the CSWL which consists of 433 headwords, and the additional
Computer Science Multi Word List (CSMWL) which contains 23
computer science compound phrases, providing the desired 95%
coverage for discipline specific learners when combined with
the GSL and the AWL.
25
Overall it can be seen that learners require a substantial
vocabulary in order to perform reasonably well in a variety
of tasks in the second language. The importance of the 2000
most frequent words has been demonstrated, giving about 80%
coverage, without which, very little real meaning can be
obtained in either spoken or written discourse. Further, it
is widely believed that a higher figure of at least 95%
coverage is more realistic in order to achieve full
comprehension. Therefore, any test of vocabulary size should
be aimed at the first 2000 most frequent words for general
competence and at least 5000 for more academic assessments
(Milton 2009:70).
2.4 Vocabulary knowledge as a predictor of language
proficiency
It has long been recognised that measures of vocabulary
knowledge are generally assumed to be good predictors of
proficiency in the four skills and recent studies have served
to strengthen this hypothesis. Staehr (2008), for example,
demonstrates a correlation between vocabulary size and the
26
skills of reading, writing and listening amongst EFL learners
from a lower secondary school in Denmark. In particular,
there was a marked association between receptive vocabulary
size and reading and writing. His findings suggest that
vocabulary size accounts for 72% variance in the ability of
the learners to gain an average (or above) score in the
reading test administered. Similarly, his analysis shows a
52% variance in writing and 39% in the listening scores.
Overall, Staehr’s results indicate that vocabulary size is a
significant factor in the learners’ ability to perform in the
tested skills. Other research similarly demonstrates such
correlations, in particular within the realms of vocabulary
size and reading comprehension (Laufer, 1992; Henriksen,
Albrechtsen & Haastrup, 2008).
However, Milton et al (2010) highlight the fact that research
into the relationship between vocabulary size and language
performance has rarely taken account of the learner’s
speaking ability. Milton et al further recognised that the
results of Staehr (2008) and others demonstrate weaker
27
correlations with aural listening skills than with writing
and reading which are written skills. This could be
attributed to the testing method used, that is orthographic
measures, or possibly that vocabulary knowledge is retained
in different formats namely, aural and/or written.
Therefore, Milton et al investigated whether vocabulary
knowledge could in fact be held separately in aural and
written formats, and whether measuring written format and
aural format would provide a more complete explanation of
performance in the four skills. Vocabulary was measured using
X_Lex (Meara and Milton, 2003) for written form and A_Lex
(Milton and Hopkins, 2006) as an equivalent test for aural
form. IELTS sub-scores were used to compare performance in
the four skills. The data was collected from intermediate to
advanced students of mixed nationalities studying on a UK
pre-sessional programme. As with Staehr, the results suggest
strong correlations between vocabulary size and language
performance.
28
The findings also show that written and aural vocabulary
knowledge need not be held in combination, but that words can
be held in one form. Interestingly, Milton et al found that the
lower level learners tended to retain predominantly aural
vocabulary knowledge, while written word knowledge was more
evident amongst the advanced learners. Significantly then,
this study concurs with others that vocabulary size is a
reliable measure of language proficiency, but in this
instance, a relationship has been found to be significant in
the skill of speaking. Milton et al thus maintain that where
vocabulary size is appropriately measured, statistically
significant performance correlations can be shown for the
four skills (Milton et al, 2010).
2.5 IELTS scores as a predictor of academic performance
A number of studies have examined the predictive validity of
the IELTS test to determine whether it adequately measures
the target goals of the test taker and the HE institution
they intend to enter.
29
Of these predictive validity investigations, some have found
there to be little or no statistically significant
correlation between IELTS and academic performance. For
example, Cotton and Conrow (1998) studied 33 international
university students and found no noteworthy positive
correlations between IELTS scores and the performance of
students in their academic coursework. Despite weak
connections between IELTS reading scores and subsequent
academic performance, low IELTS entry levels of 5.5 did not
necessarily suggest poor academic results. Likewise, high
entry levels of 7.0 and above did not always result in
consequent academic success (Cotton & Conrow, 1998). Dooey’s
study corroborates this as no evidence was found to indicate
that students who did not meet the university’s specified
entry requirement of IELTS 6.0 were more likely to perform
inadequately (Dooey, 1999).
On the other hand, some studies have found significant,
although sometimes weak and inconsistent, positive
correlations between IELTS entry levels and projected
30
academic performance. Bellingham (1993), for example, found
that low IELTS scores and academic failure have a significant
relationship, while Hill, Stoch and Lynch (1999) demonstrated
a fairly strong correlation between IELTS proficiency scores
and academic performance in semester one. However, linear
regression estimates also concluded that the overall
relationship was not particularly strong. Positive, but weak,
connections were also found in Kerstjen’s and Nery’s (2000)
study of 113 international higher education students.
Meanwhile, Feast (2002) measured the relationship between
IELTS proficiency and university students’ grade point
averages, with results showing a significantly positive
association.
More recently, Woodrow’s (2006) study correlated IELTS
subtest scores against the semester one GPAs of international
postgraduate students and found that while connections with
overall bands and GPAs were weak, more valid significance was
found amongst the subsets of writing, listening and speaking.
Analysis indicated that the relationship was stronger at the
31
lower IELTS level. Thus, at 6.5 and below, IELTS scoring may
be a good predictor of achievement, whereas at 7.0 and above,
no noticeable influence was perceived. However, it was noted
that there were a number of additional significant
influencers on academic performance; previous experience with
the English language and differing professional backgrounds
were also found to be viable.
The research demonstrates, therefore, that the connection
between IELTS test results and subsequent performance on an
academic programme in the second language remains
hypothetical and further research needs to be done.
2.6 What makes a good vocabulary test?
In most circumstances, native speakers will develop
vocabulary knowledge naturally throughout their lives, giving
them the ability to manage the situations they face
appropriately according to their age and experience. Second
language learners however, can experience vocabulary barriers
which may result in them being unable to communicate
effectively. As such, there is good reason to test vocabulary
32
knowledge and monitor further development. The results of
this testing may assist in drawing conclusions as to whether
a learner’s vocabulary knowledge meets their L2 needs and
importantly, what further support and instruction may be
required. Consequently, the reasons for testing vocabulary
knowledge are clear, but consideration must also be given as
to what to test and how.
When devising vocabulary size tests for L2 learners, it is
the norm to first consider what native speakers know. This
provides a paradigm for assessing second language competency.
However, as previously discussed, there have been
considerable variations in vocabulary size test results
historically, ranging from about 3000 to over 200,000 for an
adult native speaker. Goulden et al (1990) provided a more
acceptable benchmark of 17,000, upon which subsequent studies
have been modelled and developed. In Goulden’s study, words
were classified into base, derived, proper, compound and others,
drawing on the work of Nagy and Anderson (1984) who
differentiated base and derived words by taking into account
33
their relationship of meaning and the amount of additional
learning required. Comparisons were made to a Thorndike and
Lorge (1944) list, producing a final sample of 571 words.
Once the researchers were satisfied that the sample of words
was representative, a fair and valid test had to be devised.
Looking at previous tests particularly that of Diack (1975),
which was a self-assessment test based on a series of words
arranged into frequency lists, Goulden et al also produced
tests based on frequency levels arranged into manageable 5x50
word sections, with each item representing 500 words.
Therefore, the raw results would be multiplied by 500 to
produce an estimated vocabulary size. The results show that
adult native speakers have an average vocabulary size of
17,000 words and a learning rate of 2-3 words per day.
With this benchmark set, it ensured that subsequent test
developers gave consideration as to the unit of measurement,
whether multi-word items are to be counted as lexical or
grammatical, whether to measure breadth or depth and what
34
constitutes knowing a word. Most importantly, the reliability and
validity of the test being used must be established.
Reliability is concerned with consistency and accuracy; the test
must be stable over time and produce equal results if the
same student is tested more than once on the same day.
Equivalence might also be judged, where different versions of
a test have comparable results when the same student takes
two forms of the test on the same day (Milton, 2009).
A test has validity when it can be confirmed that it measures
what it is designed to test and not something different.
There are various elements to validity, specifically content,
construct, concurrent and face validity.
Content validity ensures that the test has appropriate content to
measure what is required. For example, tests of vocabulary
breadth, such as Nation’s Vocabulary Levels Test (Nation,
1990), most often use frequency bands and lemmatised
wordlists, which assumes valid content to demonstrate a good
range of vocabulary knowledge and recall.
35
Construct validity on the other hand, is a measure of the desired
skill or construct to be measured. Constructs are often
measured by way of written or oral language production where
measurement must be inferred, thus posing issues of
subjectivity and a reliance on the learner to produce the
words they know within the given task. It is therefore
difficult to develop a productive vocabulary test which
demonstrates good construct validity. Measurement of
receptive vocabulary knowledge can be less problematic as the
test creator can decide which words are to be investigated so
enhancing construct validity.
At times, testers want to measure equivalence, so two
different test types of equal quality are given to the same
set of learners. If the results of the two tests compare
well, then the testing procedure is said to show good
concurrent validity.
If the above validity measures have been considered when
creating and administering a vocabulary test, then
researchers can be fairly confident that accurate and
36
appropriate results will be delivered. However, there is one
further, and no less important, area of validity to be
considered; that of face validity. This is concerned with the
views of the users. Learners can feel challenged by tests,
especially if they do not see the results they expected, or
they may not have taken a test of the same type before, so
not recognise it as valid. Additionally, vocabulary tests can
appear to be relatively simple, leaving the face validity
questionable to researchers outside of the field (Milton:
2009). This issue has been addressed with the X_Lex (Meara &
Milton, 2003) and Y_Lex (Meara & Miralpeix, 2006) vocabulary
size tests. The tests are computerised, giving them high face
validity. Learners tend to like working with the programmes
and get instant results. Guessing is minimised as the
frequency range is random and test administration is
straightforward.
2.7 Test Types
37
There are many different types of vocabulary test. This
section will give an overview of some of them and then focus
more specifically on the types used in this study.
Generally speaking, a vocabulary test should have a minimum
of around 30 items to be reliable (Nation, 2001) and should
use a test item type which will test the kind of knowledge
the researcher or teacher is looking for. But as test item
types can vary considerably, choosing a test may be easier
said than done. Some will focus on form, others meaning; or
the requirement may be to test knowledge of collocations.
Examples of test types include asking students to choose the
appropriate word(s) such as Read’s vocabulary depth test (1995),
definition completion (Read 1995) or multiple choice matching
like A sensitive multiple-choice test (Joe, 1994) to test whether the
learner understands word meaning. Nation’s Vocabulary Levels Test
(Nation, 1990, revised by Schmitt et al., 2001) is an example
of a recognition test where learners are presented with words
in the foreign language along with a number of definitions or
38
explanations. The learner must choose the correct explanation
to match with the test word.
Instructions:In each question, choose the correct meaning to go with the word in CAPITAL letters. Circle the letter with the best meaning for the word.
1. PATIENCE: He has no patience.a. Will not wait happilyb. Has no free timec. Has no faithd. Does not know what is fair
Figure 2.1 An example of a vocabulary levels test item
Recognition tests allow for both the knowledge of words and
their meanings to be estimated. The format of having
multiple-choice items is easy to mark and allows learners to
draw on partial knowledge (Nation, 2001). This type of test
may also have good validity as it is used in a number of
standardised tests such as TOEFL, and learners may be
familiar and comfortable with its format. There are of course
limitations in that the test can be difficult to construct
and reliable results depend on students not only being
familiar with the target word, but also the vocabulary used
in the explanations. Further, such tests are open to
39
guesswork and calculation, for which there appears to be no
obvious way of compensating.
Another test type which has increased in popularity over the
last few decades and appears in many forms is the Yes/No, or
checklist test. Such tests measure passive word recognition
to establish a learner’s vocabulary breadth or size.
Instructions:Please look at these words. Some of these words are real Englishwords and some are not but are made to look like real words.Please tick the words that you know or can use.Example:
New commer
ce
organ
ise
accus
e
Victo
ryGumme
r
tindle wooke
y
candi
sh
Skave
Figure 2.2 Example of a Yes/No test. From XK_Lex Vocabulary Test
1 – Version A (Masrai, 2009)
Although Yes/No tests have been popular since the 1970’s,
momentum really began after 1983 when Anderson and Freebody
began working with them and included some nonsense words to
check for accuracy in learner response (Anderson & Freebody,
40
Catdita
1983). Since the late 1980’s Paul Meara and his colleagues
have continued to refine checklist tests resulting in the
computerised Eurocentre’s Vocabulary Size Test (Meara and
Jones, 1990) and more recently the X_Lex (Meara & Milton,
2003).
The Yes/No format requires learners to simply tick the words
they know. Words to be tested are usually taken from trusted
word frequency lists, with an equal sample taken from each
frequency band to be tested. Paul Meara and James Milton have
been leading the way in the development of checklist tests
for some time. The X_Lex (Meara & Milton, 2003) for example
tests up to the first 5000 most frequent words in English and
is also available in other languages, while the Y_Lex (Meara
& Miralpeix, 2006) works with the 6000 – 10,000 range. The
advantages are that this format is easy to devise, mark and
administer. Learners will be able to get through the test
quickly, thus reducing the chance of boredom. The tests are
also computerised and show instant results once completed,
giving them high face validity. The main disadvantage is that
learners may be quite tempted to guess which could lead to
41
overestimation. However, the use of trusted frequency lists
and addition of false words helps to address this. The false
words are constructed to look and read like real words in the
target language, but do not actually exist. Therefore, the
learner cannot have encountered them before and should not
tick them. If false words are ticked, then there is a
calculation which determines the degree of overestimation and
the scores are adjusted accordingly (Milton, 2009).
Figure 2.3 Example of X_Lex test
The test shows a randomised series of 120 words from frequency lists up
to 5000 words. Learners click the smiley or sad face depending whether
they know the word or not.
42
Figure 2.4 Example of X_Lex results
Instant results once the test has been completed.
There are considerable benefits of having such standardised
and reliable methods of measuring vocabulary breadth, both in
terms of placement and instruction. Milton (2009) also
suggests that it is now possible to compare results not only
between different groups of learners within particular
institutions, but also between countries. Further, he asserts
that comparisons may also be possible between different
languages, although recognising that this raises issues of
the differences in how languages may inflect and combine
words. This study will now attempt to ascertain whether a
measure of vocabulary breadth compares well with IELTS scores
43
in determining the predictive performance of L2 learners
studying through the medium of English in a UK university.
44
3
METHODOLOGY
This study investigates the notion that vocabulary breadth
measures could be a reliable predictor of academic
performance amongst second language students in a UK higher
education setting compared to IELTS scores. The vocabulary
test scores will be evaluated against the participants IELTS
results on entry to university and their exit grade point
averages (GPA). The research is of interest as vocabulary
breadth measures are widely believed to be relevant in
assessing language proficiency levels and more recent
research suggest that academic performance and vocabulary
knowledge inter-relate (Milton, 2009, 2010; Daller & Phelan,
2013). The study further examines how such data could be used
by Higher Education institutions in decisions to provide
appropriate levels of in-session language support.
3.1 Aims and Objectives
The overarching objective of this study is to explore the
relevance of vocabulary levels testing as a predictor of
45
academic performance for L2 learners of English entering UK
universities in comparison to the predictive capabilities of
IELTS testing and to further ascertain the potential
pedagogical implications for effective in-session language
support. Therefore, the principle aims are:
To compare the results of learners’ vocabulary test
scores and IELTS scores.
Examine whether one testing method or the other might be
a better predictor of future academic performance.
To assess the usefulness of vocabulary scores in
devising appropriate in-session support for L2 speakers
in a formal academic setting.
3.2 Research questions:
How well do the two vocabulary size measures correlate
with academic performance and IELTS?
Do any of these tests have a good enough correlation to
enable it to be used as a predictive test of subsequent
academic performance?
46
What scores are associated with failure or poor academic
performance to guide us identifying students in need of
subsequent in-session support?
Can the scores identify areas of knowledge to be covered
by in-session support classes?
3.3 Participants
The participants were 20 Chinese students from two different
universities in China, who came to Swansea University on a
Visiting + 1 exchange programme within the College of Law.
There were 32 students on the programme in total, but only 20
elected to participate in this study, which means therefore,
that the sample size is relatively low.
The students were just starting the final year of a BA in Law
at Swansea University, having completed the first two years
in China, with the expectation that they would continue to
study a further year on a master’s degree, subject to a
satisfactory bachelor degree outcome. All of the students
entered the university with IELTS scores ranging from 6.0 to
47
7.5, with 6.0 being the minimum requirement for the
programme.
Importantly, although the sample size was low, it did provide
a fairly homogenous group to work with, having similar
educational backgrounds prior to entry and undergoing the
same academic input and in-session language support during
their degree programmes in the UK.
3.4 Data collection and testing procedure
As the requirement of this study is to explore the notion
that learners’ vocabulary measures are closely linked to
their subsequent academic performance, the participants
completed two proven vocabulary levels tests: Masrai’s XK_Lex
(2009) and Nation and Beglar’s Vocabulary Size Test (2007b).
Instruments
The XK_Lex (appendix 2) was developed as part of a master’s
thesis (Masrai, 2009) with the support of Professor James
Milton at Swansea University. The test is an expansion of the
widely used and validated X_Lex (Meara & Milton, 2003),
48
providing a checklist test to the 10k frequency level. The
XK_Lex contains 100 real words, with every ten words
representing each thousand of the 10k frequency bands. In
addition, 20 pseudo words are included to minimise guessing
and overestimation. The real words are arranged into
frequency band columns with the first five columns being
represented by vocabulary from Nation’s frequency list
(1984). The other five columns are extracted from
Kilgarriff’s frequency list (2006), bringing the test up to
the 10k range. The pseudo words are mixed up with the real
words in equal proportion of two per frequency column. This
is a Yes/No type of test which aims to test orthographic
receptive recognition, providing a vocabulary size measure in
a format which is quick and easy to administer.
The Vocabulary Size Test (appendix 3) contains 100 target words,
ten at each of the 10k levels. The words are drawn from the
fourteen 1000 BNC word family list. Nation and Beglar suggest that
using a word family list is more appropriate because once
learners are beyond a basic level of proficiency they “have
some control of word building devices and are able to see
49
that there is both a formal and a meaning relationship
between regularly affixed members of a word family” (Nation &
Beglar, 2007a, p. 10). Although the complete test has 140
items to represent the 14k frequency list, this study
administered up to the first 10k only for comparison with the
10k XK_Lex test above. The VST is a multiple choice test
designed to measure orthographic receptive vocabulary size.
The multiple choice format aims to demonstrate the learners’
knowledge of the words.
Additional data
In addition to the measures of vocabulary size gained from
the tests above, the students’ overall IELTS scores and mean
final grade point averages were also required. Authorisation
was gained form the students at the time of taking the tests
(appendix 1). The IELTS scores were taken form student
records, but only the overall scores were available, so no
correlations could be made with the sub-skills. The GPA was
also made available from student records. The students took
five modules, each carrying an equal weighting of 20 credits,
50
so the mean GPA was derived by dividing the total scores of
the modules by five.
Method
The participants were presented with a letter explaining the
process (appendix 1) and the research purpose was outlined
orally. At this point students were allowed to leave the room
should they not wish to participate. Test instructions were
both verbalised and attached to the tests for clarity. Both
tests were given out at the same time and the students were
given around an hour to complete them. However, a further
five or 10 minutes was allowed for a small number of students
who had not finished within the given time. It was considered
that completion was more important than working to time. The
tests were collected and the scores calculated.
The XK_Lex score was calculated by giving 100 marks for each
real word ticked and deducting 500 marks for each non-word
ticked. The Vocabulary Size Test is calculated by multiplying
each correct answer by 100 to give an overall vocabulary
51
size. The calculations for each test give an overall score
for the words known out of 10,000.
Statistical analysis
The data was analysed with reference to the research
questions using the SPSS software package. The following
results will be presented:
Overall results to present the mean scores and standard
deviations of the vocabulary scores and GPAs.
Correlations to compare the vocabulary tests and IELTS
with GPAs.
Factor analysis to assess what factor the vocabulary
tests, IELTS and GPAs are testing.
Extraction method for component analysis of the XK_Lex,
VST, IELTS and GPA.
Group statistics to determine the mean scores and
standard deviation of the low GPA students compared to
the higher GPAs.
Levene’s test for equality of variances.
T-test for equality of means.
52
The statistical analysis may be challenged by the fact that
not all of the data presented is interval data. The
vocabulary tests are countable and so could be said to be
interval data but the IELTS scores and GPAs are grades and
testing for a wide range of language aspects and subject
knowledge.
53
4
RESULTS
The participants’ IELTS results, GPA grades and the raw
scores of the two vocabulary tests are represented in table
4.1. It is worth noting that students E, J and K did not
complete the VST, but we have no way of knowing whether this
was due to lack of time, lack of knowledge or motivation.
However, this does bring the marking on this test in line
with the academic exams and IELTS, because if students do not
finish a task in these assessments, they will not get marks
for it and no allowances are made. Therefore, it was decided
that the results should be presented as is, even though they
may appear as anomalies.
Table 4.1 Raw scoresStudent
IELTS score
XKLex Score
VST Score % GPA
A 6.0 4900 5100 61B 6.5 4900 4000 62C 6.0 7000 5100 66D 7.0 7500 6600 66E 6.0 4600 3100 55F 6.5 5900 5400 58G 6.5 7600 6500 67H 6.5 5400 2700 61
54
I 6.0 4500 5700 60J 6.0 3000 4600 52K 6.5 6700 3400 58L 6.5 6300 4800 63M 7.5 7100 6600 59N 6.0 3500 4200 62O 6.5 4600 5000 59P 6.5 4600 4500 67Q 6.5 6800 6300 65R 6.5 5200 4100 61S 6.5 7500 6400 65T 6.0 6200 4100 62
It would be reasonable to expect that students with higher
IELTS results would demonstrate larger vocabulary scores and
ultimately perform better academically. However, this only
appears to be the case with student D. If we compare the
IELTS results with GPAs, there is no clear pattern.
Conversely, there appears to be a relationship between higher
XK_Lex measures and higher GPAs, the only exception being
student M, who not only had the highest IELTS result of 7.5,
but also achieved 7100 and 6600 in the XK_Lex and VST
respectively, yet an academic grade of only 59% overall.
There are a number of possible reasons for this such as ill
health, personal issues or poor study strategies for example,
but we are not able to examine these as part of this study.
55
The raw scores in table 4.1 are examined in more detail in
the following tables to ascertain statistical significance.
A summary of the vocabulary size data for the whole group is
presented in table 4.2. The assumption is that the VST and
XK_Lex would compare well, but with the potential to
overestimate in the VST as the scoring makes no allowance for
guessing. However, the VST mean score is lower than the
XK_Lex. The differences in the mean scores was shown to be
significant in the t test [t=2.812; sig=.011]. The
significance of these differences will be discussed in more
detail in the next chapter.
Table 4.2 Mean scores on vocabulary size tests and GPA
Mean Scores Standard deviation
Max possible
VST 4910 1185.39 10,000XK_Lex 5690 1365 10,000GPA 61.45 3.98
Table 4.3 below reports the correlation coefficients between
the vocabulary tests and IELTS with the GPAs. As expected,
the XK_Lex vocabulary test has a good correlation of 0.567
56
against the GPA and is statistically significant to the 0.01
level. Surprisingly, the VST has a lower correlation of
0.421, the possible reasons for this will be discussed in the
next chapter. Notably, the IELTS/GPA correlation of 0.203 is
weak, which is alarming when IELTS is one of the most popular
English language tests for university entry in 135 countries
across the world (www.ielts.org).
Table 4.3 Correlations of vocabulary tests and IELTS with GPAs
IELTS XKlex vst GPA
IELTS
Pearson Correlation 1 .540* .436 .203
Sig. (2-tailed) .014 .055 .390
N 20 20 20 20
XKlex
Pearson Correlation .540* 1 .535* .567**
Sig. (2-tailed) .014 .015 .009
N 20 20 20 20
VST
Pearson Correlation .436 .535* 1 .421
Sig. (2-tailed) .055 .015 .065
N 20 20 20 20
GPA
Pearson Correlation .203 .567** .421 1
Sig. (2-tailed) .390 .009 .065
N 20 20 20 20
57
*. Correlation is significant at the 0.05 level (2-tailed).**. Correlation is significant at the 0.01 level (2-tailed).
There are differences in the type of data being compared;
that is measurable vocabulary scores in comparison with IELTS
proficiency levels and the GPA percentage grades. It would be
expected that the two vocabulary scores would test the same
factor, although they have produced different scores, but
that IELTS and the GPA scores are drawing different knowledge
and skills. IELTS is intended as a test of performance in the
sub-skills reading, writing, listening and speaking. GPA
should, it is expected, tap into subject knowledge rather
than language knowledge. As such, a factor analysis was
prepared to check what is being tested. However, the scree
plot in figure 4.1shows that in fact they all appear to be
testing the same factor, with only one factor gaining a score
above 1. The component matrix (table 4.4) shows the highest
scores for the four possible factors these tests might
produce, are all in factor 1.
58
Figure 4.1 Factor analysis
Table 4.4 Component Matrixa
Component
1 2 3 4
XKlex .815 -.041 -.519 -.254
GPA .745 -.619 .003 .249
IELTS .727 .641 -.063 .235
VST .772 .036 .605 -.194
Extraction Method: Principal Component Analysis.
a. 4 components extracted.
59
Table 4.5 Group Statistics
GPA N MeanStd.
Deviation
Std. Error
Mean
XKlex
>=
58.0018 5900.00001241.44129 292.61052
< 58.00 2 3800.00001131.37085 800.00000
The group statistics presented in table 4.5 above are
statistically relevant as they show a mean XK_Lex vocabulary
score of 5900 for students gaining an academic grade greater
than 58%, while for those with grades below 58% the mean
vocabulary score is 3800. A further t-test was run for
equality of means. The results can be seen in table 4.6
below, showing that the t score is statistically significant
at the 0.05 level.
Table 4.6 Independent Samples Test
Levene's Test for Equality of Variances
t-test for Equality of Means
60
F Sig. t df Sig. (2-tailed)
Mean Difference
Std. Error Difference
95% Confidence Interval of the Difference
Lower Upper
XKlex
Equal variances assumed
.403 .534 2.28018 .035 2100.00000 920.94959 165.15671 4034.84329
Equal variances not assumed
2.4651.284.199 2100.00000 851.83385 -4424.37864 8624.37864
61
5
DISCUSSION
This chapter will examine the results of the study in greater
detail paying particular attention to the research questions
and the pedagogical implications of the enquiry. In addition,
the limitations of the research will be highlighted along
with suggestions for further investigation.
5.1 How well do the two vocabulary size measures correlate
with academic performance and IELTS?
Not surprisingly, in line with previous studies demonstrating
correlations between vocabulary size and language
proficiency, the results of this study emphasise the
association between vocabulary size and academic performance.
In particular, the correlation between XK_Lex scores and
grade point averages was reasonably good at 0.57, which is
statistically significant at the 0.05 level. A likely
explanation for this could be that academic study on the BA
Law, as with many academic degrees, is heavily text focussed
62
and the relationship shown may be predominantly between
vocabulary size and reading. It is fair to assume that if the
student has a large vocabulary size, then he/she will have a
better chance of knowing a high enough percentage of the
lexical items within the texts being studied for reasonable
comprehension (Staehr, 2008); that is, around 95-98% for
academic purposes (Laufer, 1989; Hu & Nation, 2000). Of
course, the students in this study also needed to write
extended essays and sit written examinations and thus the
correlations with XK_Lex seem to show a significant
correlation between academic reading and writing. An
additional test of productive vocabulary measure such as
Laufer and Nation (1999) might have served to strengthen this
correlation.
It was expected that the Vocabulary Size Test would associate
well with GPA, but in this study, the VST demonstrated a
weaker relationship than the XK_Lex generating a correlation
of 0.42 and the correlation is not statistically significant.
There may be a number of reasons for this. Firstly, three of
63
the participants did not complete the VST which may have
scored them lower than their actual vocabulary size, but as
we have no way of knowing the reason for non-completion,
these results were still included as true. Additionally, it
places the marking in line with IELTS testing and academic
assessments which make no allowances for incomplete answers.
Secondly, the VST was created from a language teaching
perspective and as such, partial knowledge is marked and
there is no scope in calculation for guessing. This can lead
to either overestimation or underestimation. The multiple-
choice format means the participants must also have a
moderately developed knowledge of the meaning of the word and
of the lexicon presented in the definitions. This makes the
test more difficult than the XK_Lex as “the correct answer
and distractors usually share the same elements of meaning”
(Nation & Beglar, 2007a).
As regards the IELTS correlations with GPA, these were found
to be weak in comparison to either of the vocabulary tests at
just 0.2. This is presumably because IELTS is a subjective
64
examination and despite attempts to introduce objective
measures to the test over the years (www.IELTS.org), it must
still be open to broad interpretation by examiners.
Similarly, apart from assessing language proficiency, it also
requires the test takers to interpret the questions and
respond appropriately in terms of content, style and
register.
5.2 Do any of these tests have a good enough correlation to
enable it to be used as a predictive test of subsequent
academic performance?
The IELTS examination results showed the weakest correlation
with GPA (0.2) and therefore, in line with previous research,
do not appear to provide a significant indicator of academic
performance. Meanwhile, the Vocabulary Size Test only
demonstrated a moderate association with the GPA, producing a
correlation of 0.41, signalling that this study has not shown
the correlation to be sufficiently significant for the VST to
be used as a reliable predictor of academic achievement. This
may be because the test was principally designed to track
65
learners’ vocabulary development “both longitudinally and
across a group of learners” (Nation & Beglar, 2007a: 9), so
is appropriate for charting growth and not for diagnostic
purposes per se. Additionally, Nation (2013) suggests that
the VST is appropriate for measuring the vocabulary knowledge
required for reading, but not listening, speaking or writing,
and perhaps therefore, not representative of all of the sub-
skills required for academic study.
The test in this study which showed the most significant
correlation of 0.57 was the XK_Lex. The purpose of the XK_Lex
is to provide a reliable and consistent measure of learners’
vocabulary size to the ten thousand level. It was designed by
Professor James Milton at Swansea University and validated by
Masrai (2009) as part of a master’s thesis. Results of two
forms of the test (A and B) were compared and found to
correlate well. The XK_Lex scores in Masrai’s study
correlated significantly with the tried and tested EVST
(Meara & Jones, 1990), demonstrating concurrent validity.
Therefore it is safe to assume that the results of this test
66
show reliable measures of the participants’ vocabulary size.
With this in mind, along with the significant correlation
with GPA in our study, it can be concluded that the XK_Lex
may be sufficiently effective to enable it to be used as a
predictive test of subsequent academic performance.
5.3 What scores are associated with failure or poor academic
performance to guide us identifying students in need of
subsequent in-session support?
None of the participants in this study failed their academic
degrees. However, some achieved a low or bare pass and as
such, could have been considered as at risk. Table 4.5
presents the group statistics and specifically, the mean
vocabulary scores for students with GPAs > = 58 and < 58. For
students with GPAs > = 58, the mean vocabulary score was 5900
with a standard deviation of 1241. The mean drops to 3800
with a standard deviation of 1131 for students with GPAs <
58. This strongly suggests that below the 4000 word level of
vocabulary knowledge, a student will be at risk of
67
unsatisfactory performance and likely to benefit from
subsequent in-sessional language support.
It is worth noting that the participants in this study were
supported in-sessionally with compulsory language and writing
skills classes amounting to 60 hours of tuition. Some
students also self-selected additional language classes
through the university’s Academic Success Programme. This
could be significant as it suggests that the students with
low vocabulary scores and GPAs might have achieved less well
without this provision in place.
5.4 Can the scores identify areas of knowledge to be covered
by in-session support classes?
This study demonstrates that in general, the students with
lower vocabulary measures also achieved lower GPAs. This
strongly suggests that in-session support focussing on growth
of learners’ vocabulary knowledge will assist them in
performing better academically. The pedagogical
considerations are a) What vocabulary should this be? b) How
should it be taught?
68
As The XK_Lex test is presented in sections at each thousand
frequency level up to the first 10,000, it is possible to see
from the test scores how a learner has performed at each
frequency band level and teaching could be developed around
these.
It is widely believed that the first 2000 most frequent words
are essential for general understanding, providing about 80%
coverage, so it goes without saying that some effort should
be concentrated in this area if test results highlight
deficiencies. However, Laufer (1989) suggests that university
students require 95% coverage for reasonable comprehension,
while Hu and Nation (2000) consider the figure to be higher
at 98%. As the focus of this study is concerned with
potential academic performance, then we can assume that the
students should aim for the 95-98% coverage or around 8000 -
9000 words for reading (Nation, 2006; Staehr, 2008). All of
the students in our study fell short of this figure and there
were no especially high achievers academically, with GPAs
ranging from 52% to 67% and vocabulary levels between 3000
69
and 7600 words. Notably, the student at the 3000 word level
achieved the 52% and the student at the 7600 word level
gained 67%. This evidence suggests that that beyond the first
2000 word frequency level, much effort also needs to be
directed into developing the learners’ overall vocabulary
size across the levels, particularly the infrequent
vocabulary which is essential for comprehension in their
academic subjects.
As good academic performance is the goal for these learners,
then it would be safe to assume that academic words would
form part of the input. Furthermore, technical or discipline
specific vocabulary should also be given consideration,
relevant to the student’s subject. Therefore, as part of a
needs analysis, frequency profiling would be useful in
defining words to be learned.
One method which might be adopted to help redress the
language knowledge gap in-sessionally is an extensive reading
programme. The reading should be based around subject
specific core texts in order to deliver academic credibility,
70
usefulness and face validity for the students and their
academic lecturers. Many of the articles and chapters that
students will be required to read as part of their coursework
will be in digital format; this lends itself to lexical
frequency profiling using Tom Cobb’s Lextutor site
(www.lextutor.ca). The teacher can run the texts through a
variety of software packages depending on the required
output. For example, the Frequency software (figure 5.1)
allows the user to input texts and extract frequency lists
which can be arranged in order of least to most frequent and
vice versa.
Tokens: 200Types: 122Ratio: 0.6100Sort: descending---------------------------------------------------------RANK FREQ COVERAGE COVERAGE WORD individual cumulative ---------------------------------------------------------1. 11 5.50% 5.50% THE2. 9 4.50% 10.00% HIS3. 6 3.00% 13.00% FOR4. 5 2.50% 15.50% OF5. 5 2.50% 18.00% TO6. 4 2.00% 20.00% DAY7. 4 2.00% 22.00% HANGER8. 4 2.00% 24.00% HE9. 4 2.00% 26.00% IN10. 4 2.00% 28.00% MP
Figure 5.1 Sample Frequency output (www.lextutor.ca)
71
Meanwhile, Tex Lex Compare (figure 5.2) is useful to establish
how often a word is repeated over two or more texts. The
tokens are shown along with a choice of types, families or
phrases, which could provide useful target vocabulary across
a range of core material relevant to a whole module for
example. The sample output below shows only the first 10 most
frequent words as an example, but for the purposes of
university students it would be more relevant to take the
less frequent vocabulary into consideration as the high
frequency words are likely to be known.
TOKEN Recycling Index: (481 repeated tokens : 658 tokens in new text)= 73.10%
FAMILIES Recycling Index: (150 repeated families : 305 families innew text) = 49.18%
(Token recycling will normally be the most interesting measure of e.g. text comprehensibility, as it is with VPs.)
Unique to first255 tokens216 families
001. report 6002. love 4003. before 3004. courtship 3005. young 3006. apparent 2007. chad 2008. do 2009. during 2010. girlfriend 2
Shared481 tokens150 families
001. the 38002. and 23003. be 23004. a 20005. he 15006. sarkozy 13007. in 11008. marry 11009. number 11010. of 10
Unique to second177 tokens155 families
001. i 5002. last 5003. start 3004. add 2005. follow 2006. get 2007. just 2008. pair 2009. press 2010. reside 2
Same listAlpha first001. across 1002. add 2003. affair 1004. age 1005. ago 1006. agostinelli 1007. another 1008. around 1009. backlash 1010. bazire 1
72
VP novel items
Figure 5.2 Sample Tex Lex Compare output (www.lextutor.ca)
Once the appropriate vocabulary input has been detected, an
applicable curriculum can be developed. There are many ways
to teach and learn vocabulary and it is not the purpose of
this study to explore them in detail, but Nation’s table
(5.1) provides an excellent basis for determining appropriate
stands and application for working with the target
vocabulary.
Table 5.1 The four strands and their application with a focus
on vocabulary
Strand General Conditions
Vocabulary requirements
Activities and techniques
Meaning-focused input
Focus on the message
Some unfamiliar items
Understanding Noticing
95% + coverage(preferably 98%)
Skill at guessing form context
Opportunity tonegotiate
Incidental defining and attention drawing
Reading gradedreaders
Listening to stories
Communication activities
Language-focused learning
Focus on language items
Skill in vocabulary learning strategies
Appropriate
Direct teaching of vocabulary
Direct learning
73
teacher focus on high frequency words, and strategies forlow frequency words
Intensive reading
Training in vocabulary strategies
Meaning-focused output
Focus on the message
Some unfamiliar items
Understanding Noticing
95% + coverage(preferably 98%)
Encouragement to use unfamiliar items
Supportive input
Communication activities andwritten input
Prepared writing
Linked skills
Fluency development
Focus on the message
Little or no unfamiliar language
Pressure to perform faster
99% + coverage Repetition
Reading easy graded readers
Repeated reading
Speed reading Listening to
easy input 4/3/2 Rehearsed
tasks 10 minute
writing Linked skills
This section suggests that as the scores produced by the
XK_Lex seem to provide valid data to link vocabulary
knowledge with academic performance, that this data can
subsequently be useful to enable areas of knowledge to be
identified in order to develop in-session support classes.
Additionally, some of the methods by which this might be
approached have been discussed. However, this study is not
74
without its limitations, which will be observed in the
conclusion.
6
CONCLUSION
This study set out to explore whether vocabulary knowledge
might be a significant predictor of academic performance and
how such predictive capabilities correlated with those of
IELTS. It was found that one of the two vocabulary tests
examined showed better correlations with the participants’
overall academic GPA than did the other; and that IELTS in
this instance presented a weak correlation. These results
were deemed sufficiently significant to suggest that the
XK_Lex vocabulary test was a reliable tool for determining
potential academic achievement. Moreover, the study
determined that learners’ vocabulary scores, which can be
examined at frequency band level, enable us to uncover the
relevant vocabulary for teaching and learning purposes on
university in-sessional support programmes and some
75
suggestions were given for implementation. It might be
hypothesized that this intervention would subsequently raise
students’ vocabulary profiles, which in turn, would enhance
their potential for success on their university degree
programmes.
The study was not without its limitations, however. Firstly,
the sample size was small with just 20 Chinese participants
of similar ages, all of whom came from analogous academic
backgrounds and were pursuing the same degree in the UK. In
its favour, this does provide a fairly homogenous group for
comparison; but on the downside, it does not tell us whether
the results would have been comparable had students of
varying age, background and nationality participated.
Secondly, the data provided with the study group could have
been richer. For instance, there was only access the global
IELTS bands and not those of the sub-skills of reading,
writing, speaking and listening. It would have been a
relevant exercise to compare correlations of the sub-skills
with vocabulary size and GPAs. In terms of the academic
76
grades, correlation was only explored against the end of
degree GPAs, while in hindsight, it would have been
interesting to drill down further into the individual module
grades and compare vocabulary and IELTS scores against any
failed or high achieving modules.
A further limitation might be that the XK_Lex and VST are
both tests of orthographic word knowledge and therefore no
account of aural vocabulary knowledge was tested, whereas
there is a speaking element in IELTS. It is possible that a
future study of this nature might include use of Milton and
Hopkins Aural Lex (2005) to include a comparable test of
phonological knowledge.
Notwithstanding these considerations, it can be concluded
that the study showed to a reasonable degree that knowledge
of a learners’ vocabulary size enables the teacher to predict
potential academic performance and take appropriate steps to
ensure that the students are adequately supported with
appropriate language development.
77
Taking both the findings and limitations of this study into
consideration, there is plenty of scope to take this research
a step further in order to strengthen the validity of these
claims. A longitudinal mixed methods approach is suggested to
track students’ progress both linguistically and academically
throughout their degree programme. As well as undertaking a
quantitative methodology similar to, but more extensive than
the one in this study both in terms of numbers and range of
data; a qualitative aspect could be introduced by way of
interviews with students, language teachers, lecturers and
admissions departments to analyse perceptions and their
potential effect on performance. Such a study, being
longitudinal, would enable the implementation of in-session
support and thus, its effectiveness could correspondingly be
determined.
78
APPENDICES
Appendix 1 Participant’s Letter
Dear student
Thank you for agreeing to assist with my research
“Assessing students’ vocabulary size as a predictor of academic
performance”
You are about to take two tests which are designed to test a
learner’s vocabulary size. The accuracy of these tests depends on
how honest you are in your answers. Your answers will be kept
confidential and will be used only for the research purpose
highlighted above. Your academic department will not have access
to your answers or scores.
In addition, your IELTS scores on entry and Grade Point Average
(GPA) at the end of your programme of study will be used for the
purpose of comparison with your overall vocabulary scores from
these tests.
I appreciate your participation in this research. Your
participation in these tests means that you consent to your IELTS
scores and GPAs being used for the purpose of this study.
79
However, you have the right not to take part. If you do not wish
to continue, please let me know and hand in the test sheet.
Thank you.
Kind regardsSandy George
Please begin the test. You may ask for help at any time during the
test if anything is unclear. However, I will not be able to help
you with the answers.
80
Appendix 2 XK_Lex vocabulary size test
English XK_Lex Vocabulary Test 1 - Version: A
Please look at these words. Some of these words are real English words and some are not but are made to look like real words. Please tick the words that you know or can use. Thank you for yourhelp.
Example:
Your student number: Vocabulary Score: New commerce Organise Accuse VictoryGummer Tindle Wookey Candish SkaveWord Dust Fountain Tend JewelNear nonsense Movement Landing ReliablePeace Fond Likely Volume HardenProduce Sweat Provide Tube SorrowYou Cap Castle Liner DialWife Worry Steam Previous EncloseDo Plenty Steady Style SneezeAdd Guide Pole Outline ApparatusKilp Broy Orrade Plaudate OverendBuild Pump Guest Keeper Roast
Prosecutor addict Gulp Idleness Carnation
Samphirate treadway Darch callisthe
mia Mordue
Referral detachment Thud Blizzard Plaintively
Illuminate unsure Assassin Rut Gurgle
Gown reinforcement Wrench Incessant Heal
Verge enlightenment Backdrop Blunder go-between
Counsellor workman Unfold springboa
rd common-law
Skipper feudal Upheaval Shrapnel Locket
81
cat
dita
Authorise quartet Animation Skip Nudge
Sour psychic Banish Bastion AngerNeminary Fallity Treggle Snape Tearle
Holly appropriation
Peninsula Maroon Contrive
Adapted from: Masrai (2009), p. 82
82
Appendix 3 Vocabulary Size Test
VOCABULARY SIZE TEST – RECOGNITION
This is a vocabulary test. It has 100 questions, ten at each of 10 thousand-levels. This test should give you an idea of the number of English words you know.
Instructions:In each question, choose the correct meaning to go with the word in CAPITAL letters. Circle the letter with the best meaning for the word.
Here is an example.
1. CAT: The cat sat on the mat.
a. Animal that chases dogs
b. Animal that carries people
c. Animal that chases a mouse
d. Animal that eats fruit
In the example, the best meaning for CAT is answer c, “animalthat chases a mouse,” so you should circle c.
Thank you for your help.
Your student number: VST score:
Source: Nation, P. & Beglar, N. (2007) VST, BNC Version, 1-14K, www.lextutor.ca/tests
83
First 10001. SEE: They saw it.
a.
cut
b.
waited for
c.
looked at
d.
started
2. TIME: They have a lot of time.a.
money
b.
food
c.
hours
d.
friends
3. PERIOD: It was a difficult period.a.
question
b.
time
c.
thing to do
d.
book
4. FIGURE: Is this the right figure?a.
answer
b.
place
c.
time
d.
number
5. POOR: We are poor.a.
have no money
b.
feel happy
c.
are very interested
d.
do not like to work hard
6. DRIVE: He drives fast.a.
swims
b.
learns
c.
throws balls
d.
uses a car
7. JUMP: She tried to jump.a.
lie on top of the water
b.
get off the ground suddenly
c stop the car at the edge of the road
.d.
move very fast
8. SHOE: Where is your shoe?a.
the person who looks after you
b.
the thing you keep your money in
c.
the thing you use for writing
d.
the thing you wear on your foot
9. STANDARD: Her standards are very high.a.
the bits at the back under her shoes
b.
the marks she gets in school
c.
the money she asks for
d.
the levels she reaches in everything
10. BASIS: I don't understand the basis.a.
reason
b.
words
c.
road signs
d.
main part
Second 10001. MAINTAIN: Can they maintain it?
a.
keep it as it is
b.
make it larger
c.
get a better one than it
d.
get it
2. STONE: He sat on a stone.a.
hard thing
b kind of chair
84
.c.
soft thing on the floor
d.
part of a tree
3. UPSET: I am upset.a.
tired
b.
famous
c.
rich
d.
unhappy
4. DRAWER: The drawer was empty.a.
sliding box
b.
place where cars are kept
c.
cupboard to keep things cold
d.
animal house
5. PATIENCE: He has no patience.a.
will not wait happily
b.
has no free time
c.
has no faith
d.
does not know what is fair
6. NIL: His mark for that question was nil.a.
very bad
b.
nothing
c.
very good
d.
in the middle
7. PUB: They went to the pub.a.
place where people drink and talk
b.
place that looks after money
c.
large building with many shops
d.
building for swimming
8. CIRCLE: Make a circle.a.
rough picture
b.
space with nothing in it
c.
round shape
d.
large hole
9. MICROPONE: Please use the microphone.
a.
machine for making food hot
b.
machine that makes sounds louder
c.
machine that makes things look bigger
d.
small telephone that can be carried around
10. PRO: He's a pro.a.
someone who is employed to find out important secrets
b.
a stupid person
c.
someone who writes for a newspaper
d.
someone who is paid for playing sport etc
Third 10001. SOLDIER: He is a soldier.
a.
person in a business
b.
student
c.
person who uses metal
d.
person in the army
2. RESTORE: It has been restored.a.
said again
b.
given to a different person
c.
given a lower price
d.
made like new again
3. JUG: He was holding a jug.a.
A container for pouring liquids
b.
an informal discussion
c.
A soft cap
d.
A weapon that explodes
85
4. SCRUB: He is scrubbing it.a.
cutting shallow lines into it
b.
repairing it
c.
rubbing it hard to clean it
d.
drawing simple pictures of it
5. DINOSAUR: The children were pretending to be dinosaurs.a.
robbers who work at sea
b.
very small creatures with human formbut with wings
c.
large creatures with wings that breathe fire
d.
animals that lived a long time ago
6. STRAP: He broke the strap.a.
promise
b.
top cover
c.
shallow dish for food
d.
strip of material for holding thingstogether
7. PAVE: It was paved.a.
prevented from going through
b.
divided
c.
given gold edges
d.
covered with a hard surface
8. DASH: They dashed over it.a.
moved quickly
b.
moved slowly
c.
fought
d.
looked quickly
9. ROVE: He couldn't stop roving.a.
getting drunk
b.
travelling around
c.
making a musical sound through closed lips
d.
working hard
10. LONESOME: He felt lonesome.a.
ungrateful
b.
very tired
c lonely
.d.
full of energy
Fourth 10001. COMPOUND: They made a new compound.
a.
agreement
b.
thing made of two or more parts
c.
group of people forming a business
d.
guess based on past experience
2. LATTER: I agree with the latter.a.
man from the church
b.
reason given
c.
last one
d.
answer
3. CANDID: Please be candid.a.
be careful
b.
show sympathy
c.
show fairness to both sides
d.
say what you really think
4. TUMMY: Look at my tummy.a.
cloth to cover the head
b.
stomach
c.
small furry animal
d.
thumb
5. QUIZ: We made a quiz.a.
thing to hold arrows
b.
serious mistake
c.
set of questions
d.
box for birds to make nests in
86
6. INPUT: We need more input.a.
information, power, etc. put into something
b.
workers
c.
artificial filling for a hole in wood
d.
money
7. CRAB: Do you like crabs?a.
sea creatures that walk sideways
b.
very thin small cakes
c.
tight, hard collars
d.
large black insects that sing at night
8. VOCABULARY: You will need more vocabulary.a.
words
b.
skill
c.
money
d.
guns
9. REMEDY: We found a good remedy.a.
way to fix a problem
b.
place to eat in public
c.
way to prepare food
d.
rule about numbers
10. ALLEGE: They alleged it.a.
claimed it without proof
b.
stole the ideas for it from someone else
c.
provided facts to prove it
d.
argued against the facts that supported it
Fifth 10001. DEFICIT: The company had a large
deficit.a.
spent a lot more money than it earned
b.
went down a lot in value
c.
had a plan for its spending that used a lot of money
d.
had a lot of money in the bank
2. WEEP: He wept.a.
finished his course
b.
cried
c.
died
d.
worried
3. NUN: We saw a nun.a.
long thin creature that lives in theearth
b.
terrible accident
c.
woman following a strict religious life
d.
unexplained bright light in the sky
4. HAUNT: The house is haunted.a.
full of ornaments
b.
rented
c.
empty
d.
full of ghosts
5. COMPOST: We need some compost.a.
strong support
b.
help to feel better
c.
hard stuff made of stones and sand stuck together
d.
rotted plant material
6. CUBE: I need one more cube.a.
sharp thing used for joining things
b.
solid square block
c.
tall cup with no saucer
d.
piece of stiff paper folded in half
87
7. MINIATURE: It is a miniature.
a.
a very small thing of its kind
b.
an instrument to look at small objects
c.
a very small living creature
d.
a small line to join letters in handwriting
8. PEEL: Shall I peel it?a.
let it sit in water for a long time
b.
take the skin off it
c.
make it white
d.
cut it into thin pieces
9. FRACTURE: They found a fracture.a.
break
b.
small piece
c.
short coat
d.
rare jewel
10. BACTERUM: They didn't find a single bacterium.a.
small living thing causing disease
b.
plant with red or orange flowers
c.
animal that carries water on its back
d.
thing that has been stolen and sold to a shop
Sixth 10001. DEVIOUS: Your plans are devious.
a.
tricky
b.
well-developed
c not well thought out
.d.
more expensive than necessary
2. PREMIER: The premier spoke for an hour.a.
person who works in a law court
b.
university teacher
c.
adventurer
d.
head of the government
3. BUTLER: They have a butler.a.
man servant
b.
machine for cutting up trees
c.
private teacher
d.
cool dark room under the house
4. ACCESSORY: They gave us some accessories.a.
papers allowing us to enter a country
b.
official orders
c.
ideas to choose between
d.
extra pieces
5. THRESHOLD: They raised the threshold.a.
flag
b.
point or line where something changes
c.
roof inside a building
d.
cost of borrowing money
6. THESIS: She has completed her thesis.a.
long written report of study carried out for a university degree
b.
talk given by a judge at the end of atrial
c.
first year of employment after becoming a teacher
d.
extended course of hospital treatment
7. STRANGLE: He strangled her.a.
killed her by pressing her throat
b.
gave her all the things she wanted
c.
took her away by force
d.
admired her greatly
88
8. CAVALIER: He treated her in a cavalier manner.a.
without care
b.
politely
c.
awkwardly
d.
as a brother would
9. MALIGN: His malign influence is still felt.a.
evil
b.
good
c.
very important
d.
secret
10. VEER: The car veered.
a.
went suddenly in another direction
b.
moved shakily
c.
made a very loud noise
d.
slid sideways without the wheels turning
Seventh 10001. OLIVE: We bought olives.
a.
oily fruit
b.
scented pink or red flowers
c.
men's clothes for swimming
d.
tools for digging up weeds
2. QUILT: They made a quilt.a.
statement about who should get theirproperty when they die
b firm agreement
.c.
thick warm cover for a bed
d.
feather pen
3. STEALTH: They did it by stealth.a.
spending a large amount of money
b.
hurting someone so much that they agreed to their demands
c.
moving secretly with extreme care and quietness
d.
taking no notice of problems they met
4. SHUDDER: The boy shuddered.a.
spoke with a low voice
b.
almost fell
c.
shook
d.
called out loudly
5. BRISTLE: The bristles are too hard.a.
questions
b.
short stiff hairs
c.
folding beds
d.
bottoms of the shoes
6. BLOC: They have joined this bloc.
a.
musical group
b.
band of thieves
c.
small group of soldiers who are sent ahead of others
d.
group of countries sharing a purpose
7. DEMOGRAPHY: This book is about demography.a.
the study of patterns of land use
b.
the study of the use of pictures to show facts about numbers
c.
the study of the movement of water
d.
the study of population
8. GIMMICK: That's a good gimmick.
a.
thing for standing on to work high above the ground
b.
small thing with pockets to hold money
c.
attention-getting action or thing
89
d.
clever plan or trick
9. AZALEA: This azalea is very pretty.
a.
small tree with many flowers growing in groups
b.
light material made from natural threads
c.
long piece of material worn by women in India
d.
sea shell shaped like a fan
10. YOGHURT: This yoghurt is disgusting.
a.
grey mud found at the bottom of rivers
b.
unhealthy, open sore
c.
thick, soured milk, often with sugar and flavouring
d.
large purple fruit with soft flesh
Eighth 10001. ERRATIC: He was erratic.
a.
without fault
b.
very bad
c.
very polite
d.
unsteady
2. PALETTE: He lost his palette.a.
basket for carrying fish
b.
wish to eat food
c.
young female companion
d.
artist's board for mixing paints
3. NULL: His influence was null.a.
had good results
b.
was unhelpful
c.
had no effect
d.
was long-lasting
4. KINDERGARTEN: This is a good kindergarten.a.
activity that allows you to forget your worries
b.
place of learning for children too young for school
c.
strong, deep bag carried on the back
d.
place where you may borrow books
5. ECLIPSE: There was an eclipse.
a.
a strong wind
b.
a loud noise of something hitting the water
c.
The killing of a large number of people
d.
The sun hidden by a planet
6. MARROW: This is the marrow.a.
symbol that brings good luck to a team
b.
Soft centre of a bone
c.
control for guiding a plane
d.
increase in salary
7. LOCUST: There were hundreds of locusts.a.
insects with wings
b.
unpaid helpers
c.
people who do not eat meat
d.
brightly coloured wild flowers
8. AUTHENTIC: It is authentic.a.
real
b.
very noisy
c.
Old
d.
Like a desert
9. CABARET: We saw the cabaret.a.
painting covering a whole wall
b.
song and dance performance
c.
small crawling insect
d.
person who is half fish, half woman
10. MUMBLE: He started to mumble.a.
think deeply
b shake uncontrollably
90
.c.
stay further behind the others
d.
speak in an unclear way
Ninth 10001. HALLMARK: Does it have a hallmark?
a.
stamp to show when to use it by
b.
stamp to show the quality
c.
mark to show it is approved by the royal family
d.
Mark or stain to prevent copying
2. PURITAN: He is a puritan.a.
person who likes attention
b.
person with strict morals
c.
person with a moving home
d.
person who hates spending money
3. MONOLOGUE: Now he has a monologue.a.
single piece of glass to hold over his eye to help him to see better
b.
long turn at talking without being interrupted
c.
position with all the power
d.
picture made by joining letters together in interesting ways
4. WEIR: We looked at the weir.a.
person who behaves strangely
b.
wet, muddy place with water plants
c.
old metal musical instrument played by blowing
d.
thing built across a river to control the water
5. WHIM: He had lots of whims.a.
old gold coins
b.
female horses
c.
strange ideas with no motive
d.
sore red lumps
6. PERTURB: I was perturbed.a.
made to agree
b.
Worried
c.
very puzzled
d.
very wet
7. REGENT: They chose a regent.
a.
an irresponsible person
b.
a person to run a meeting for a time
c.
a ruler acting in place of the king
d.
a person to represent them
8. OCTOPUS: They saw an octopus.a.
a large bird that hunts at night
b.
a ship that can go under water
c.
a machine that flies by means of turning blades
d.
a sea creature with eight legs
9. FEN: The story is set in the fens.
a.
low land partly covered by water
b.
a piece of high land with few trees
c.
a block of poor-quality houses in a city
d.
a time long ago
10. LINTEL: He painted the lintel.
a.
Beam over the top of a door or window
b.
small boat used for getting to land from a big boat
c.
beautiful tree with spreading branchesand green fruit
d.
board showing the scene in a theatre
Test from: Nation, P. & Beglar, N. (2007b) VST, BNC Version, 1-14K, www.lextutor.ca/tests
91
Tenth 10001. AWE: They looked at the mountain with awe.
a.
worry
b.
interest
c.
wonder
d.
respect
2. PEASANTRY: He did a lot for the peasantry.a.
local people
b.
place of worship
c.
businessmen's club
d.
poor farmers
3. EGALITARIAN: This organization is egalitarian.a.
does not provide much information about itself tothe public
b.
dislikes change
c.
frequently asks a court of law for a judgement
d.
treats everyone who works for it as if they are equal
4. MYSTIQUE: He has lost his mystique.a.
his healthy body
b.
the secret way he makes other people think he has special power or skill
c.
the woman who has been his lover while he ismarried to someone else
d.
the hair on his top lip
5. UPBEAT: I'm feeling really upbeat about it.a.
upset
b.
good
c.
hurt
d.
confused
6. CRANNY: We found it in the cranny!a.
sale of unwanted objects
b.
narrow opening
c.
space for storing things under the roof of a house
d large wooden box
.
7. PIGTAIL: Does she have a pigtail?a.
a rope of hair made by twisting bits together
b.
a lot of cloth hanging behind a dress
c.
a plant with pale pink flowers that hang down in short bunches
d.
a lover
8. CROWBAR: He used a crowbar.a.
heavy iron pole with a curved end
b.
false name
c.
sharp tool for making holes in leather
d.
light metal walking stick
9. RUCUS: He got hurt in the rucus.a.
hollow between the stomach and the top of the leg
b.
pushing and shoving
c.
group of players gathered round the ball in some ball games
d.
race across a field of snow
92
10. LECTERN: He stood at the lectern.a.
desk to hold a book at a height for reading
b.
table or block used for church sacrifices
c.
place where you buy drinks
d.
very edge
93
BIBLIOGRAPHY
Albrechtsen, D. Haastrup, K. and Henriksen, B. (2008) Vocabulary and Writing in a First and Second Language: Processes and Development. Houndsmills: Palgrave Macmillan.
Anderson, R.C. and Freebody, P. (1981) Vocabulary knowledge. In: J.T. Gutherie (ed.) Comprehension and Teaching: Research Reviews (pp. 77-117). Newark, DE: International Reading Association.
Anderson, R.C. and Freebody, P. (1983) Reading Comprehension and the Assessment and Acquisition of Word Knowledge. In: Hutson, R. (ed.) Advances in Reading / Language Research. A research Annual, CT: JAI Press.
Bauer, L. and Nation, P. (1993) Word Families. International Journal of Lexicography, 6, 253-279.
Bellingham, L. (1993) The relationship of language proficiency to academic success for international students, New Zealand Journal of Educational Studies 30, 2: 229-232.
Cotton, F. and Conrow, F. (1998) An investigation into thepredictive validity of IELTS amongst a group of international students studying at the University of Tasmania. In: IELTS Research Reports 1, IELTS Australia PtyLtd: Canberra, 72-115.
Coxhead, A. (2000) A new academic word list. TESOL Quarterly, 34 (2), 213-238.
Daller, H., Milton, J. & Treffers-Daller, J. (eds.) (2007) Modelling and Assessing Vocabulary Knowledge. Cambridge: Cambridge University Press.
Diack, H. (1975) Test Your Own Wordpower, St Albans: Paladin.
Diller, K.C. (1978) The Language Teaching Controversy. Rowley, Massachusettes: Newbury House.
94
Dooey, P. and Oliver, R. (2002) An investigation into the predictive validity of the IELTS test. Prospect, 17, 36 -54.
Feast, V. (2002) The impact of IELTS scores on performance at university. International Education Journal, 3, 70-85.
Francis, W.N. and Kucera, H. (1982) Frequency Analysis of English Usage. Boston: Houghton Mifflin.
Gardener, D. (2007) Validating the construct of words in applied corpus-based vocabulary research: a critical survey. Applied Linguistics, 28, 241-265.
Goulden, R., Nation, I.S.P. and Read, J. (1990) How large can a receptive vocabulary be? Applied Linguistics. 11, 341-363.
Hill, K. Storch, N. and Lynch, B. (1999) A comparison of IELTS and TOEFL as predictors of academic success, English Language Testing System Research Reports, 2, 52-63.
Hu, M. and Nation, I.S.P. (2000) Unknown vocabulary densityand reading comprehension. Reading in a Foreign Language 13(1), 403-430.
Kerstjens, M. & Nery, C. (2000) Predictive validity in the IELTS test: A study of the relationship between IELTS scores and students' subsequent academic performance. IELTS Research Reports, 3, 85-108.
Kilgarriff, A. (2006) BNC database and word frequency lists. http://www.kilgarriff.co.uk/bnc-readme.html#lemmatised. (last accessed 03/10/2014).
Konstantakis, N. (2007) Creating a Business Word List for Teaching Business English. Estudios de Linguistica Inglesa Aplicada (ELIA) 7, 103-126.
Laufer, B. (1989) What percentage of text is essential for comprehension? In C. Lauren and M. Nordman (eds)
95
Special Language; from Humans to Thinking Machines (pp.316-323). Clevedon: Multilingual Matters.
Laufer, B. (1992) How much lexis is necessary for reading comprehension? In: P.J.L. Arnaud and H. Bejoint (eds.)Vocabulary and Applied Linguistics (pp. 126-132). London: Macmillan.
Laufer, B. and Paribakht, T.S. (1998) The relationship between passive and active vocabularies: effects of language learning context. Language Learning, 48, 365-391.
Masrai, A.M. (2009) Measuring the English Vocabulary Size of Saudi University Students: Validating a New 10,000 Word Vocabulary Size Test.Unpublished MA thesis: Swansea University.
Meara, P. (1992) Yes-No, Test 1, Levels 1-5, Swansea University. http://www.lextutor.ca/tests/yes_no_eng/test_1/ (last accessed 20/03/2014).
Meara, P. (1996) The dimensions of lexical competence. In: G. Brown, K. Malmkjaer & J. Williams (eds.) Performance and competence in second language acquisition. Cambridge: Cambridge University Press, 35-53.
Meara, P. and Jones, G. (1990) Eurocentre’s Vocabulary Size Test. User’s Guide. Zurich: Eurocentres.
Meara, P. & Milton, J. (2003) X_Lex, The Swansea Levels Test. Newbury: Express.
Meara, P. M. and Miralpeix, I. (2006) Y_Lex: the Swansea Advanced Vocabulary Levels Test. v2.05. Swansea: Lognostics
Milton, J. (2007) Lexical profiles, learning styles and theconstruct validity of lexical size tests. In: H. Daller, J. Milton & J. Treffers-Daller (eds.) Modelling and Assessing Vocabulary Knowledge. (pp. 47-58) Cambridge: Cambridge University Press.
96
Milton, J. (2009) Measuring Second Language Vocabulary Acquisition. Bristol: Multilingual Matters.
Milton, J. and Hopkins, N. (2005) Aural Lex. Swansea: SwanseaUniversity.
Milton, J. and Hopkins, N. (2006) Comparing phonological and orthographic vocabulary size: Do vocabulary tests underestimate the knowledge of some learners. The Canadian Modern Language Review 63 (1), 127-147.
Milton, J., Wade, J. and Hopkins, N. (2010) Aural word recognition and oral competence in a foreign language.In: R. Chacon-Beltran, C. Abello-Contesse & M. Torreblanca-Lopez (eds.) Further Insights into Non-native Vocabulary Teaching and Learning (pp. 83-98). Bristol: Multilingual Matters.
Minshal, D. M. (2013) A Computer Science Wordlist. Unpublished MA thesis: Swansea University.
Morris, L. & Cobb, T. (2004) Vocabulary profiles as predictors of TESL student performance. System 32(1), 75-87.
Nagy, W. E. and Anderson, R. C. (1984) How many words are there in printed school English? Reading Research Quarterly19, 304-330.
Nation, I. S. P. (ed.) (1984) Vocabulary Lists, Words, Affixes and Stems. English University of Wellington, New Zealand: English Language Institute.
Nation, I. S. P. (1990) Teaching and Learning Vocabulary. New York: Heinle & Heinle Publishers.
Nation, I .S. P. (2001) Learning Vocabulary in Another Language. Cambridge: CUP.
Nation, I. S. P. (2004) A study of the most frequent word families in the British National Corpus. In: P.
97
Bogaards & P. Laufer (eds.) Vocabulary in a Second Language: Selection, Acquisition and Testing (pp. 3-13) Amsterdam: John Benjamins.
Nation, I .S. P. (2006) How large a vocabulary is needed for reading and listening. The Canadian Modern Language Review, 63,1 (September), 59-82.
Nation, I. S. P. and Beglar, D. (2007a) A vocabulary size test. The Language Teacher,
31(7), 9-13.
Nation, P. and Beglar, D. (2007b) Vocabulary size test, BNCVersion (1-14k), VUW: New Zealand. http://www.lextutor.ca/tests/levels/recognition/1-14k/(last accessed (20/03/2014).
Nation, I. S. P. (2012) http://www.victoria.ac.nz/lals/about/staff/publications/paul-nation/Vocabulary-Size-Test-information-and-specifications.pdf (last accessed 03/10/2014).
Read, J. (1993) The development of a new measure ofL2 vocabulary knowledge. Language Testing 10 (3), 355-371.
Read, J. (1995) Refining the word associates format as a measure of depth of vocabulary knowledge. New Zealand Studies in Applied Linguistics 1, 1-17.
Read, J. (2000) Assessing Vocabulary. Cambridge: CUP.
Read, J. (2007) Second Language Vocabulary Assessment: current practice and new directions. International Journal of English Studies 7, 2, 105-125.
Richards, J. (1976) The role of vocabulary teaching. TESOL Quarterly, 77–89.
Schmitt, N. (2008) Instructed second language vocabulary learning. Language Teaching Research 12, 3, 329-363.
98
Schmitt, N., Schmitt, D. and Clapham, C. (2001) Developing and exploring the behaviour of two new versions of theVocabulary Levels Test. Language Testing, 18, 55-88.
Seashore, R. & Eckerson, L. (1940) The measurement of individual differences in general English vocabulary. Journal of Educational Psychology, 31, 14-38.
Staehr, L.S. (2008) Vocabulary size and the skills of listening, reading and writing. Language Learning Journal,36(2), 139-152.
Swenson, E. and West, P. (1934) On the counting of new words in textbooks for teaching foreign languages. Bulletin of the Department of Educational Research, University of Toronto, 1.
Thorndike, E. and Lorge, I. (1944) The Teacher's Word Book of 30,000 Words, Columbia University: Teachers College.
Webb, S. (2005) Receptive and productive vocabulary learning: The effects of reading and writing on word knowledge. Studies in Second Language Acquisition, 27,33-52.
West, M. (1953) A General Service List of English Words, London, Longman.
Wilkins, D. (1972) Linguistics in Language Teaching. London: Arnold.
Woodrow, L. (2006) Academic success of international postgraduate education students and the role of English proficiency. University of Sydney Papers in TESOL, 1, 51-70.
http://www.IELTS.org (last accessed 26/09/2014).
http://www.lextutor.ca (last accessed 13/10/2014)
http://www.lognostics.co.uk/tools (last accessed 20/03/2014)
99