Post on 02-Feb-2023
Lisbeth M Brevik
Mapping Tests:
Comparing Upper Secondary Students’
Reading Skills in L1 and L2
ECER 2014
Main interest
• How students read in English as a second
language (L2)
• Unlike first language (L1) reading,
second language (L2) reading involves
two languages. For this reason, L2 reading is
crosslinguistic and, thus, more complex than
L1 reading (Koda 2005, 2007)
A few words about Norway (context)
• Upper secondary school
• 15-16 years old (year 1)
• National reading tests
– L1: reading literacy in Norwegian (paper)
– L2: reading literacy in English (digital)
• Same population as PISA 2012
– PISA in April-May (4.686 students)
– Reading tests in September (60.000 students)
– My population: 10.331 students
ECER, 2014
Lisbeth M Brevik
Research Questions
1. What is the relationship between reading in
Norwegian (L1) and English (L2)?
2. What insights about groups of students can
be derived from the results, and how do the
groups differ?
ECER, 2014
Lisbeth M Brevik
Method
• Quantitative analysis (SPSS)
– L1 literacy (test score)
– L2 literacy (test score)
– Gender, study programme, mother tongue
• Merging the two datasets
– Analysis: frequency, reliability, factor, regression
ECER, 2014
Lisbeth M Brevik
Student categories
ECER, 2014
Lisbeth M Brevik
L1 low L1 higher
L2 low
Struggling readers
Low L1 and L2 performance.
The 20% lowest performing students
in both languages.
Outliers:
Weak L2 – good L1 readers
L2 (English) : Poor readers (20%)
L1: Good readers (60–100%)
L2
higher
Outliers:
Weak L1- good L2 readers
L1: Poor readers (20%)
L2 (English): Good readers (60–100%)
Mainstream readers
Equal (or almost equal) in L1 and L2.
Students in the 40-100% groups,
plus those bordering on these
groups.
Findings
1. Relationship: Strong relationship between
reading comprehension in L1 and L2
2. Mainstream: the majority of students read
equally across L1 and L2
3. Outliers: two groups of outliers read
significantly different in the two languages
ECER, 2014
Lisbeth M Brevik
Students read equally in L1 and L2
… but general study students read
better than vocational students
ECER, 2014
Lisbeth M Brevik
Relationship: 30% overlap between L1
and L2 reading comprehension
• 30% overlap in correlation analysis
– Regression: R Square change .297
– Correlation: .545 (significant at the 0.01 level)
ECER, 2014
Lisbeth M Brevik
Mainstream: the majority of students
read equally across L1 and L2
ECER, 2014
Lisbeth M Brevik
Outliers:
4,5% (N=463)
Outliers:
4,9% (N=505) Mainstream:
74% (N=7,601)
Weak L1 – good L2: language
Mother tongue
– 91,6% Norwegian
– 8,4% other languages
Researcher: Are you more motivated to read when it is in English?
Student: Yes. […] I just find English easier for me to read.
I actually know many more words in English than in
Norwegian… even though I’m from Norway.
ECER, 2014
Lisbeth M Brevik
Discussion: the outliers
• Why do the weak L1 – good L2 outliers read
significantly better in L2 than in L1?
• Why do boys both outnumber and outperform
the girls in this group?
• Could it be linked to digital assessment, the
English language, shorter texts, and topic of
interest?
ECER, 2014
Lisbeth M Brevik
Conclusion
• The shift – from testing what the students know about language to
assessing what students can do with language
• The challenge – Mainstream: the majority read equally well - or poorly – in L1
and L2
– Outliers: a minority read significantly different in L1 and L2
– How can test scores be used to make inferences about the
needs of student growth - to improve learning and instruction?
– Is training in transfer of reading strategies one solution?
ECER, 2014
Lisbeth M Brevik
Selected references
• Bernhardt, E. B. (2011). Understanding advanced second-language reading.
NY: Routledge
• Brevik, L. M. (2013). Research ethics: An investigation into why school leaders
agree or refuse to participate in educational research. Problems of Education in
the 21st Century, 52, 7–20. Retrieved from http://urn.nb.no/URN:NBN:no-34383
• Brevik, L. M. (2014). Making implicit practice explicit: How do upper secondary
teachers describe their reading comprehension strategies instruction?
International Journal of Educational Research. Doi: 10.1016/j.ijer.2014.05.002
• Frones, T. S., Narvhus, E. K., & Aasebo, M. C. (2013). Nordic results from the
PISA digital reading assessment. Nordic Journal of Digital Literacy, 8(01-02),
pp. 13–31
• OECD (2009). PISA 2009 Assessment Framework. Key competencies in
reading, mathematics and science. Paris: OECD Publications.
• OECD (2013). PISA 2012 Results: Ready to Learn – Students’ Engagement,
Drive and Self-beliefs (Vol. III). Paris: OECD Publishing.
doi:10.1787/9789264201170-en
ECER, 2014
Lisbeth M Brevik
Thank you
Lisbeth M Brevik,
University of Oslo, Norway
E-mail: l.m.brevik@ils.uio.no
ECER, 2014
Lisbeth M Brevik