Learners’ Self-assessment of their Oral
Performance in EFL Classrooms:
The Case of Wollega University
Rufael Disasa Worabu
Addis Ababa University
Addis Ababa
June 2013
Learners’ Self-assessment of their Oral Performance in EFL
Classrooms: The Case of Wollega University
Rufael Disasa Worabu
A Doctoral Thesis Submitted to
The Department of Foreign Languages and Literatures
Presented in Fulfilment of the Requirements for the Degree of Doctor of
Philosophy in Teaching English as Foreign Language (TEFL)
Addis Ababa University
Addis Ababa
June 2013
ADDIS ABABA UNIVERSITY
SCHOOL OF GRADUATE STUDIES
This is to certify that the thesis prepared by Rufael Disasa entitled: ‘Learners’
Self-assessment of Oral Performance in University EFL Classrooms: the case of Wollega
University’ submitted in fulfilment of the requirements for the Degree of Doctor
of Philosophy in Teaching English as a Foreign Language complies with the
regulations of the University and meets the accepted standards with respect to originality
and quality.
Signed by the Examiners Committee:
Examiner _________________________ Signature ___________ Date ___________
Examiner _________________________ Signature ___________ Date ___________
Advisor __________________________ Signature ___________ Date ___________
_____________________________________________________________
Chair Person, Department Foreign Languages and Literature
iii
Abstract
This thesis reports a study of conception and validity of learners’ self-assessment of English
oral performance and the effect of training learners in self-assessment on the validity and the
conception in university EFL classrooms.
In the study, the concurrent mixed methods design was opted for to address different questions
and to validate the finding through data triangulation. In the qualitative aspect, group and
individual interviewing was employed to study the conceptions, and in the quantitative part of
the study, single-group experiment design that involved training learners in self-assessment of
English oral performance was used to study the validity of learners’ self-assessment.
Questionnaire survey was used to supplement the data from interview. The subjects of the
study were 46 2nd
year EFL learners and 19 EFL instructors in the English program of
Wollega University. Data were collected through self-developed interview schedule, oral
performance assessment tools and questionnaires. Training materials for the students and the
instructor were also developed and used in the experiment. Individual interviews were
conducted with five purposively selected instructors; and group interviews were conducted
with eight students before and after an intervention. English oral performance assessments in
which the students self-assessed their own performance in classroom were administered to the
students prior to and after the training intervention. The training was explicit and integrated
into Spoken English II (Enla 342). It was conducted for six weeks (two-hours/week).
The data from the interviews were analysed and interpreted qualitatively following the
qualitative content analysis approach on the bases of the themes that constituted the
conception of learner self-assessment. The data from the performance assessment and the
questionnaires were analysed quantitatively using descriptive and inferential statistics. The
extent of validity of learners’ self-assessment was determined by comparing the mean values
of learners’ self-scoring with the instructors’ scoring of the oral performances using the
paired-samples t-test. Similarly, the effect of the training on students’ conception was
evaluated by comparing the findings from the qualitative analysis of the data on conception
before and after the training.
The analysis of data on the subjects’ conception of learners’ self-assessment indicated that
both groups of subjects held less positive conception of learners’ self-assessment. They had
the belief and understanding that classroom assessment is carried out primarily to check off
and to describe learning outcomes; they described learners’ self-assessment in its non-
technical and simplest level; and they could identify only few benefits of involving learners in
assessment scheme. They also held the belief that doing classroom assessment is the
responsibility of the instructor only and students should not participate in the complete process
of assessment because learners’ self-assessment cannot be trustworthy. Analysis of the scores
of the English oral performance prior to the intervention indicated that the learners
considerably overrated their own performance as compared to their instructor’s scoring. The
iv
analysis of the post intervention data indicated that the students’ conception was improved,
and the difference between the learners’ scores of their own performance and that of the
instructor was not statistically significant; indicating that the training positively influenced the
learners’ conception and validity of their self-assessment. The result of study suggested that to
encourage the use of learners’ self-assessment in classrooms, both the instructors and the
students need to be provided with opportunity of exposure to explicit teaching and practices of
learners’ self-assessment.
v
Acknowledgements
I am indebted to a number of persons among whom I mention a few just because of the space,
and I hope the rest would understand me that I am grateful to them for their invaluable
contribution to my success in this study.
First of all, my heart-felt gratitude goes to my supervisor Dr. Nuru Mohammed-Tahir who
helped me unreservedly just from the scratch to the final stage of my work. Without his
guidance, it would have been difficult for me to successfully accomplish my work. I am also
grateful to Dr. Taye Regassa, Dr. Mendida barkessa and Ato Tibebe Alemayehu for their
invaluable comments and suggestions during the examination of the proposal and the pilot
work.
Next, my deepest appreciation is to my wife Wubayehu Kitessa and my children Naol,
Umeran and Lemifol for their patience and persistent moral support throughout the duration of
my study. I owe special thank to my wife for carrying all the burden of managing the family
and making all arrangements for my study leave, leaving her personal needs aside.
I would like to extend my thanks to the students and the instructors in the English program of
Wollega University for their cooperation during the data collection. I am also grateful to Dr.
Fikiru Tafesse (MD), Tesfaye Dechasa, Adeko Wolde and Ebissa Dhaba for reading and
commenting on the instruments and materials developed for the study, and for coming in with
financial, moral and material support whenever I need them. Finally, my thanks go to
Maddalena Taras, University of Sunderland, UK and Mark Winterbottom, University of
Cambridge, for their support in providing me with information and materials.
vi
Dedication
To the memory of my parents Adde Zawditu Yadessa and Obbo Disasa Worabu who were
longing for my success as a student and did all their best to make a man of me and inspired
me with determination and tolerance. Mom, Dad, I believe that you are looking on me down
from the heaven, and dedicated this work to your honour.
vii
Table of Content
page
Abstract ...................................................................................................................................... iii
Acknowledgements ...................................................................................................................... v
List of tables and figure ............................................................................................................ xv
Acronyms and Abbreviations .................................................................................................. xvii
Chapter One
Background of the Study
1.0 Introduction .................................................................................................................. 1
1.1 Organisation of the Thesis ........................................................................................... 2
1.2 Educational Context of the Study ................................................................................ 3
1.3 Statement of the Problem ............................................................................................. 7
1.4 Rationale of the Study .................................................................................................. 9
1.5 Objectives of the Study .............................................................................................. 11
1.6 Significance of the Study ........................................................................................... 12
1.7 Chapter Summary ....................................................................................................... 13
Chapter Two
Review of Related Literature
2.0 Introduction ................................................................................................................ 15
viii
2.1 Major Developments in Assessment Tradition .......................................................... 15
2.2 Learners’ Self-Assessment ......................................................................................... 22
2.2.1 Description and Typology of Learners’ Self-assessment .......................................23
2.2.2 LSA in University EFL Classrooms .......................................................................29
2.2.3 LSA and Learning...................................................................................................32
2.2.4 LSA and Learning Gains ........................................................................................38
2.3 Issues Related to LSA ................................................................................................ 41
2.3.1 Validity and Reliability ..........................................................................................42
2.3.1.1 Reliability of LSA ........................................................................................... 42
2.3.1.2 Validity of LSA ............................................................................................... 42
2.3.1.3 Determinants of Validity of LSA .................................................................... 47
2.3.2 Conception of LSA .................................................................................................49
2.3.2.1 The Concept of Conception ............................................................................ 49
2.3.2.2 Conception and Practice .................................................................................. 50
2.3.2.3 Conceptions of LSA ........................................................................................ 52
2.3.3 Training Learners to Self-assess..............................................................................55
2.4 Assessment of Oral Performance ............................................................................... 59
2.4.1 Performance Assessment.........................................................................................59
2.4.2 Assessing Oral Performance ...................................................................................60
2.4.2.1 Defining Constructs of oral Performance ....................................................... 61
2.4.2.2 Measuring the Constructs of Oral Performance .............................................. 64
2.4.2.3 Rating Scales for Oral Performance ................................................................ 67
2.4.2.4 Tasks for Assessing Oral Performance ........................................................... 68
ix
2.5 Chapter Summary ....................................................................................................... 70
Chapter Three
Methodology
3.0 Introduction ................................................................................................................ 71
3.1 Approach to the Study ................................................................................................ 71
3.2 Design of the Study .................................................................................................... 72
3.2.1 Interviewing.............................................................................................................72
3.2.2 Single-group quasi-Experiment...............................................................................74
3.2.3 Questionnaire Survey .............................................................................................75
3.3 Population and Subjects of the Study......................................................................... 75
3.4 Instruments ................................................................................................................. 78
3.4.1 The Interview Schedule...........................................................................................80
3.4.2 The Questionnaire ...................................................................................................81
3.4.3 The Training Materials............................................................................................84
3.4.4 The Oral Performance Assessment Tools...............................................................87
3.4.4.1 The Tasks ........................................................................................................ 87
3.4.4.2 The Marking Scheme ...................................................................................... 88
3.5 Data Gathering Procedure .......................................................................................... 88
3.5.1 The Pre-intervention Stage......................................................................................89
3.5.2 The Intervention (Training).....................................................................................92
3.5.3 The post-intervention Stage....................................................................................94
3.6 Data Analyses Procedure ........................................................................................... 95
x
3.6.1 The Qualitative Data...............................................................................................95
3.6.2 The Quantitative Data.............................................................................................97
3.7 The Pilot Study ........................................................................................................... 98
3.7.1 Summary of the Pilot Study.....................................................................................99
3.7.2 Lesson Learnt from the Pilot Study.........................................................................99
3.8 Chapter Summary ..................................................................................................... 103
Chapter Four
Data Presentation and Analysis
4.0 Introduction .............................................................................................................. 104
4.1 Conceptions of LSA ................................................................................................. 104
4.1.1 The Instructors’ Conception of LSA.....................................................................104
4.1.1.1 Data from the Interview ................................................................................ 104
4.1.1.1.1 Main Purpose of Classroom Assessment .................................................. 105
4.1.1.1.2 Who is Responsible for Classroom Assessment ....................................... 110
4.1.1.1.3 Description of LSA ................................................................................... 111
4.1.1.1.4 Appropriateness of Involving Learners in Assessment Process ............... 112
4.1.1.1.5 Validity of LSA ........................................................................................ 114
4.1.1.1.6 Benefits and/or Downsides of LSA .......................................................... 115
4.1.1.2 Data from the Questionnaire ......................................................................... 117
4.1.1.2.1 Main Purpose of Classroom Assessment .................................................. 118
4.1.1.2.2 Whether or not students should be involved in the assessment scheme 120
4.1.1.2.3 Description of LSA ................................................................................... 121
xi
4.1.1.2.4 Validity of LSA ........................................................................................ 122
4.1.1.2.5 Significance of LSA ................................................................................. 123
4.1.1.2.6 Relevance of LSA ..................................................................................... 126
4.1.2 Students’ Conception of LSA................................................................................127
4.1.2.1 Data from the Interviews ............................................................................... 128
4.1.2.1.1 Summary of the Finding from CoPre1 Interview ...................................... 128
4.1.2.1.2 Data from Pre-intervention Interview (CoPre2) ........................................ 130
4.1.2.1.2.1 Main Purpose of Classroom Assessment ........................................... 130
4.1.2.1.2.2 Who Should Do the Assessment ........................................................ 133
4.1.2.1.2.3 Description of LSA............................................................................. 134
4.1.2.1.2.4 Appropriateness of Involving Learners in Assessment Process ......... 135
4.1.2.1.2.5 Validity of LSA .................................................................................. 136
4.1.2.1.2.6 Benefits and/or Problems of LSA....................................................... 137
4.1.3 Effect of the Training Intervention on Students’ Conception of LSA..................138
4.1.3.1 Data from Post-intervention Interview .......................................................... 138
4.1.3.1.1 Main Purpose of Classroom Assessment .................................................. 139
4.1.3.1.2 Who Should Do the Classroom Assessment ............................................ 141
4.1.3.1.3 Description of LSA ................................................................................... 142
4.1.3.1.4 Appropriateness of Involving Learners in Assessment Process ............... 143
4.1.3.1.5 Validity of LSA ........................................................................................ 144
4.1.3.1.6 Benefits and/or Problems of LSA ............................................................. 145
4.1.3.2 Data from Students Questionnaires ............................................................... 149
4.1.3.2.1 Main Purpose of Classroom Assessment .................................................. 150
xii
4.1.3.2.2 Whether Students should be Involved in the Assessment Scheme ........ 153
4.1.3.2.3 Description of LSA ................................................................................... 155
4.1.3.2.1 Validity of LSA ........................................................................................ 156
4.1.3.2.2 Benefits and/or Problems of LSA ............................................................. 158
4.1.3.2.3 Relevance of LSA ..................................................................................... 162
4.2 Validity of LSA of English Oral Performance ......................................................... 163
4.3 Effect of the Training on Validity of LSA of English Oral performance ................ 166
4.4 Summary of the findings from the Post Intervention Data from the Non-Experiment
Group........................................................................................................................ 167
4.4.1 Summary of Result of the interview......................................................................168
4.4.2 Summary of the result of the data from oral performance assessment (non-
experiment group)..................................................................................................169
4.5 Discussions of the Findings ..................................................................................... 170
4.5.1 Conception of LSA................................................................................................170
4.5.1.1 The Instructors’ Conception of LSA ............................................................. 171
4.5.1.2 The Students’ Conception of LSA before the intervention ........................... 174
4.5.1.3 Validity of LSA of English Oral Performance .............................................. 177
4.5.2 The Effect of the Training.....................................................................................179
4.5.2.1 On the Validity of LSA ................................................................................. 179
4.5.2.2 On the Students’ Conception of LSA............................................................ 180
4.6 Chapter Summary ..................................................................................................... 182
xiii
Chapter Five
Summary and Conclusions
5.0 Introduction .............................................................................................................. 184
5.1 Summary .................................................................................................................. 184
5.2 Conclusions .............................................................................................................. 186
5.3 Implications .............................................................................................................. 189
5.3.1 Implication for Classroom.....................................................................................190
5.3.2 Implication for in-service and pre-service Teacher Training and Development _ 192
5.4 Limitations of this Study .......................................................................................... 193
5.5 Recommendation for Further Research ................................................................... 195
References........................................................................................................................197
Appendices
Appendix 1: Students’ and instructors’ interview schedule ................................................... 214
Appendix 2 Instructors’ interview Transcript ......................................................................... 215
Appendix 2A: Instructor A ................................................................................................... 215
Appendix 2B: Instructor B ..................................................................................................... 218
Appendix 2C: Instructor C ...................................................................................................... 221
Appendix 2D: Instructor D ..................................................................................................... 223
Appendix 2E: Instructor E ...................................................................................................... 225
Appendix 3 Students’ Pre-intervention interview transcript ................................................. 227
xiv
Appendix 4 Students’ post intervention interview transcript ................................................. 234
Appendix 5: Questionnaire filled out by students .................................................................. 240
Appendix 6 Questionnaire filled out by instructors ................................................................ 243
Appendix 7 Training Material (learners’ Handout) ................................................................ 246
Appendix 8 Training Material (Instructor’s Guide) ............................................................... 266
Appendix 9: Marking Scheme for students ............................................................................ 286
Appendix 10 : Marking scheme for instructor ........................................................................ 288
Appendix 11 ............................................................................................................................ 292
Appendix 12 ............................................................................................................................ 298
xv
List of tables and figure
page
Table 1 Major shifts in the assessment tradition ----------------------------------------------------- 21
Table 2 Correspondence between the behaviours studied and the instruments employed ---- 79
Table 3 Category of Purposes of Classroom Assessment (by instructors) ---------------------- 107
Table 4 Benefits of LSA (instructors) ---------------------------------------------------------------- 116
Table 5 Classroom assessment for describing achievement --------------------------------------- 118
Table 6 Classroom assessment for improving learners' achievement --------------------------- 119
Table 7 Appropriateness of learners' involvement in assessment process ---------------------- 120
Table 8 Instructors’ understanding of the concept of LSA --------------------------------------- 121
Table 9 Validity of LSA ------------------------------------------------------------------------------ 122
Table 10 Cognitive benefits of LSA ------------------------------------------------------------------ 124
Table 11 Non-cognitive benefits of LSA ------------------------------------------------------------ 125
Table 12 Relevance of learners' SA ------------------------------------------------------------------- 126
Table 13 categories of classroom assessment purposes (students’ pre-intervention) ---------- 131
Table 14 Category of purpose of classroom assessment (students’ post-intervention
interview) -------------------------------------------------------------------------------------- 140
Table 15 Benefits of LSA (learners' post) ----------------------------------------------------------- 145
Table 16 Classroom assessment for describing learners' achievement -------------------------- 151
Table 17 Classroom assessment for improving learning ------------------------------------------ 152
Table 18 Appropriateness of learners' involvement in assessment process -------------------- 154
Table 19 Description of LSA ------------------------------------------------------------------------- 156
Table 20 Validity of LSA ------------------------------------------------------------------------------ 157
xvi
Table 21 Cognitive benefits of LSA ------------------------------------------------------------------ 159
Table 22 Non-cognitive Benefits of LSA ------------------------------------------------------------ 161
Table 23 Relevance of LSA ---------------------------------------------------------------------------- 163
Table 24 Statistical summary of the first round oral performance score (Vpre1) -------------- 164
Table 25 Descriptive statistics of the oral performance scores (pre- and post
intervention) --------------------------------------------------------------------------------- 166
Table 26 Descriptive statistics of the oral performance assessment scores of the non-
experiment group --------------------------------------------------------------------------- 169
Figure: Conceptual model of how LSA contributes to improving learning
(Adapted from Ross 2006)....................................................................................37
xvii
Acronyms and Abbreviations
ACTFL: American Council on the Teaching of Foreign Languages.
AfL : Assessment for Learning
ALCM: Applied Linguistics and Communication
AoL: Assessment of Learning
Bed: Bachelor in Education
CEF: Common European Framework of Reference
CoPre1: First round pre-intervention data on learners’ conception
CoPre2: Second round pre-intervention data on learners’ conception
EGSECE: Ethiopian General Secondary Education Certificate Examination
EHEECE: Ethiopian Higher Education Entrance Certificate Examination
ELIC: English Language Improvement Centre
FDRE: Federal Democratic Republic of Ethiopia
HDP: Higher Diploma Program
LSA: Learners Self-assessment
MA: Masters of Art
MoE: Ministry of Education
TDP: Teacher Development Program
TESO: Teacher Education System Overhaul
TSE: Test of Spoken English
TTC: Teacher Training College
Vpre1: First round pre-intervention data on validity of self-assessment
Vpre2 Second round pre-intervention data on validity of self-assessment
1
Chapter One
Background of the Study
1.0 Introduction
It is self evident that the 21st
century society has been experiencing fundamental changes
in all aspects. New knowledge and new technologies are emerging continuously and
information is made available from multiple sources through multiple media
concurrently. This has accelerated changes in all aspects of human life styles, which in
turn, has posed high demand for new skills and competencies (Knapper and Cropley,
2000; Partnership for 21st Century Skills, 2008). Therefore, the important message
teachers and teacher educators are confronting is the need to build learning society that
cope with the ever changing and fiercely competitive world, and raise educational
standards and qualities and quantity of learners’ learning.
In the light of these needs, the ultimate goal of education has become empowering
learners to take responsibility for their own learning and grow independent and lifelong
learners so that they fit into the 21st century life. This in turn, called for re-engineering
education.
One prominent feature in the recent reforms in education is re-establishing the
relationship among and alignment of the teaching/learning and assessment. Assessment
has been re-defined and the assessment culture has been changed from being a means of
auditing learning outcomes as a way of controlling the students and the schools
accountability to the means of helping learners to improve their learning. In generic
terms, it has moved away from controlling to supporting. Learners are put in the
2
mainstream of the assessment scheme. This is because the more learners know about and
participate in decisions about the goals of their own learning and assessment process, the
more likely they are to make and direct their own learning efforts effectively. Therefore,
involving students in the assessment scheme through learners’ self-assessment, the crux
of this study, is at the heart of the means of achieving this goal of education (Dearing,
cited in Taras, 2002; Havnes and McDowell, 2008; Marzano et al., 1993; Sebba et al.,
2008; Segers, Dochy and Cascallar, 2003; Tan, 2007).
1.1 Organisation of the Thesis
This thesis reports a study that explored conceptions and validity of learner’s self-
assessment in university EFL classrooms in relation to oral performance. It is organised
into five chapters. The first chapter provides general background to the study. It describes
context of the educational setting in which the study was planned focusing on the role of
English language in Ethiopian education, effectiveness of its teaching, and use of
learners’ self-assessment in tertiary levels. It also states the problem and gives the
rationale, objectives, and significance of the study. It finishes by summarising the
chapter.
The second chapter presents the review of relevant literature. First, it gives theoretical
and conceptual framework that embrace learners’ self-assessment. It also summarises
writings on the significance, issues, and concerns related to learners’ self-assessment. It
also discusses issues related to conception and validity of learners’ self-assessment, oral
performances, and training learners on assessing oral performances.
The third chapter deals with the methodological framework of the study. It explains the
approach to the study, identifies the site and subjects, and describes the design of the
3
study and the strategies to be employed to gather data. It also explains the instruments
and their development and validation, and the procedure of the data collection and
analysis. In addition, it briefly describes the pilot study and lessons learnt from the
piloting.
The fourth chapter is devoted to presentation and analysis of the data. It also covers the
discussion of the findings from the data analysis in line with the research questions. The
last chapter gives summary of the whole thesis and concludes the findings of the study. It
also indicates the implications of the result for classroom teachings and teacher
development. Lastly, it points out some limitations of the study and suggests areas for
further research.
1.2 Educational Context of the Study
In EFL/ESL context where Ethiopia is the case in point, the role of English language in
quality of education is immense. The role of English language in Ethiopian socio-
economic activities has grown over several years. It was introduced into the education
system in 1908 as a school subject. After the 1928-35 Ethio-Italian war, it replaced
French as major medium of international relation and eventually took over the role of
medium of instruction in 1941 (McNab, 1989, Alem 1974).
Today, because of the rapid technological advancement and growing internationalisation
that increased the country’s contact with the globe, English is being widely used in
business activities, media and other communicative needs. Particularly, English plays
decisive role in the educational system; its contribution to the learners’ success in
secondary and tertiary education where English is the medium of instruction is immense.
4
Students are required to read and listen and comprehend academic materials and lectures.
They are also required to participate in classroom activities, take notes, produce term
papers and essays, make presentations, and defend their papers. To cope with all these
circumstances, learners need to have good command of English that enable them to carry
out the academic learning tasks effectively and efficiently in all the language skills.
However, learners’ low performance in English is prevalent in all levels and is serious in
tertiary levels. Having summarised the situation of the decline of English language in
Ethiopia a quarter of century back, Taye (2008: 182) contends that after three decades
“the situation nowadays does not seem to be any better, if not worse, than what it was
during those days”. The Ethiopian first, second and third National Learning Assessment
(NLA) (MoE, 2008) also confirms this observation. The report shows that grade eight
students’ performance in English in regional examination declined for three consecutive
years.
In addition, it seems that there is wrong perception about learners’ oral performance in
higher education. These days, students who come to universities from very few privileged
families who afford sending their children to the costly private schools in cities like
Addis Ababa, Adama and Jimma seem to be better in informal oral communication than
their fellow classmates from rural areas. Therefore, they have the perception that they are
good at English oral skills. Not few people also feel that university students have better
competence in oral English nowadays than the past. Here, it should also be noted that
according to the FDRE Central Statistics Agency (2008) report, the urban dwellers
constitute only about 17% of Ethiopian population, and among these, it is possible to
5
estimate that less than a quarter of the urban families afford the school fees of the best
private schools.
However, these students do not perform any better than other students do in academic
tasks. Though there is no recent study that measured oral proficiency of university
students, it is commonplace to hear instructors complaining that in EFL classrooms, for
those students who have high perception about their oral skills, participating in the simple
academic oral performance is really a struggle. As Cummins and Yee-Fun (2007) warn,
the conversational fluency that require the use of only high frequency words and simple
grammatical constructions concealed significant gaps in the competence in academic task
performances.
The researcher has also experienced this ‘superficial proficiency’ in EFL classes in
Wollega University while conducting Communicative English Skills courses. Although
such students can be said better in informal interactions, they have great difficulty in
using the language in academic learning tasks. For instance, they exhibit vivid difficulties
in producing grammatically correct and logically acceptable string of sentences during
oral and written performances.
Generally, English language plays crucial role in Ethiopian higher education.
Nevertheless, learners’ low performance is prevalent at all levels and seems serious in
tertiary level. In addition, students’ oral performance is being misunderstood by both the
students themselves and the school community. This indicates that teaching/learning of
English at all levels and particularly in higher education needs further attention from
different perspectives.
6
One of the major issues to be addressed is the need to harmonise the assessment practice
with the classroom teaching/learning. From this perspective, there are indicators that the
English language teaching is not aligned with the current reform in the assessment
tradition, i.e., classroom instruction has not been allied with the major shift of emphasis
from Assessment of Learning (AoL) to Assessment for Learning (AfL) in which the
learners’ involvement is the prominent feature.
Ministry of Education recommends the use of learner-centred active learning method and
formative assessment in tertiary levels (MoE, 2003, 2005) to improve learners learning
out come. However, local studies that assessed implementation of formative assessment
in Ethiopian Teacher Training Colleges (TTCs) have shown that learners’ self-assessment
that is the major component of formative assessment was very rarely used in the tertiary
levels. For example, Baye (2006) surveyed the alternative assessment modes used by
EFL classrooms in Dessie CTE and found that learners’ self-assessment is very rarely
used. In addition, Silashi (2007), Zergahun (2007), and Dagne (2009) in their studies of
the implementation of continuous assessment asked students and teacher educators to rate
the assessment techniques they used in their classes. They reported that while some
techniques of assessment are practised, learners’ self-assessment is used very rarely or
not at all. Moreover, a country-wide survey of the implementation of ETP (Education and
Training Policy) reported that learners’ self-assessment is the second least frequently
used (17%) next to oral examinations and defences among the alternative modes of
assessment used in tertiary classrooms (MoE, 2008). This indicates that learners’ self-
assessment, a core element of classroom assessment, is missing in tertiary EFL
classrooms.
7
All the above issues discussed in the wider context apply to Wollega University. It is one
of the recently established government universities and shares almost all of the features
of Ethiopian English language education, including the prevalent students’ low
performance in all skills in general and oral skills in particular. In addition, learners’ self-
assessment has not been used in the university.
Although there is no study on the implementation of learners’ self-assessment in Wollega
University, being the staff of the English program in the university, the researcher has the
experience and information that learners’ self-assessment that is the major component of
formative assessment is not used in the university at large and EFL classrooms in
particular. Hence, the learner-centred active learning method opted for is alien to the
assessment practice it is supposed to affiliate and is less likely to effect the intended
improvement of learners learning.
1.3 Statement of the Problem
Learners’ self-assessment plays a pivotal role in improving students learning by
positively affecting the cognitive emotional and social behaviours. Research has
confirmed that, though not unanimously, it fosters deep approach to learning, increases
quality and quantity of learning gains, promotes critical thinking, improve learning
behaviours, enriches meta-cognitive skills and raises willingness and ability of sharing
responsibility of one’s own learning. These all eventually make inroads to the
development of autonomous lifelong learning skills (Falchikov, 2005; Ross, 2006; Sebba,
et al., 2005; Struyven, et al., 2003; Topping, 2003).
8
The role of self-assessment skill in language learning is also promising. It enables the
learner to use the occasions of the target language use into opportunity for out-of-
classroom language learning (Little, 2005)
However, there are some contentious issues related to learners’ self-assessment. Of these
are concerns about the extent to which it can be valid; what and how the stakeholders
think about the practice; and whether or not training improves the thinking held by the
concerned party. Firstly, it seems that there is contrasting ‘popular’ views about self-
assessment; a person is unlikely to assess him/herself realistically; and the reverse of this
supposition. In addition, in the findings of the studies of validity of self-assessment there
is little consensus on the dependability of learners’ grading of their own works. The
findings vary with context of the studies like the educational cultures, methods used, the
level of the learners, and the subject areas (Falchikov, 2005; Topping, 1998; Boud and
Falchikov, 1989).
Secondly, the studies of conceptions held by the learners, the teachers and administrators
about self-assessment are few for generalization; and the results are not uniform. Thirdly,
although there is an assumption, by common sense, that training and exercise could
improve the validity and reliability and the associated less positive conceptions, the
studies carried out to cross check these concerns have not unanimously supported the
assumption (see Section 2.3.3 below).
On top of that, the studies of learners’ self-assessment are all done out of Ethiopian
cultural and educational context that, as Marshall (2004) noted, differs in many aspects
from the western cultures. Teachers’ and students’ conception of education in general and
9
the way they assume their respective role in teaching/ learning process, and especially,
the very idea of learners’ self-assessment is highly influenced by societal and educational
culture they pass through. These all make clear that there is a gap of research on learners’
self-assessment in general and Ethiopian context in particular.
1.4 Rationale of the Study
One finding of the local studies of the implementation of formative assessment in TTCs
that catches attention is the instructors’ tendency of perceiving assessment as something
secret and only their responsibility. In Baye’s (2006) and Dagne’s (2009) studies, the
students’ responses to questions about the types of the frequently used assessment
techniques and why they were chosen showed that the instructors never communicate to
them the what and the why of the assessment mode they use and they (learners) do not
question any of the process in evaluating their achievements.
The researcher’s experience as an instructor and Higher Diploma Training Program
(HDP) trainee conforms to the above findings. In fact, the idea of this research topic
emerged from the researcher’s experiences as HDP trainee in Wollega University. During
peer observation, the researcher observed that instructors plan to use learners’ self-
assessment in their classrooms, but do not allow learners to assess their own works.
During the post-observation collegial discussions, casual attempt made to elicit why they
did not do that showed that though they discuss the issues of learners’ self-assessment
during the HDP training sessions, they do not consider it appropriate and feel comfortable
with the practice. They also feel that the learners’ ability to assess themselves is
unreliable.
10
This scepticism will not be surprising in Ethiopian educational culture where the teacher
is expected to be the decisive figure, otherwise considered not doing his/her job, and
where the students are expected to submit without questioning (Nuru, 2000; Yalew,
2004; Yonas, 2008). As Pajares, (1992) explains, there appears to be a strong relationship
between previous experience and the development of conception about teaching and
learning: beliefs about teaching are formulated through many years of formal schooling
and this conception highly influences the teachers’ and students’ multiple classroom
activities (Pajares, 1992; Williams and Burden, 1997).
Therefore, to implement the learner-centred and active learning approaches that have
been opted for (MoE, 2003) in its complete sense, it is important that the conceptions
held by the teachers and the students about learners’ self-assessment are made explicit
and visible, and unfavourable behaviours need to be modified so that learners are
involved in the assessment scheme. In addition, for education to be a lifelong enterprise
that continues far after the student completes her/his schooling, it is important that the
learners’ self-assessment should be valued and count to students’ final grade, otherwise it
would not appeal to both students and teachers. Thus, it is clear that the judgements
provided by students about their own learning must be dependable. Therefore, it is crucial
that validity of learners’ self-assessment and ways of improving it be the focus of study.
Generally, to exploit the potential benefits of learners’ self-assessment to improve the
effectiveness of education in general and English education in particular, there should be
at least initial awareness of how it works in Ethiopian context; otherwise, the active
learning and learner-centred approaches fly blind of their meaning and core objectives.
Therefore, studying conceptions and validity of learners’ self-assessment in Ethiopian
11
context seems not a matter of choice but necessity. It is with these backdrops that the
present study of learners’ self-assessment has been attempted.
1.5 Objectives of the Study
The general objective of this study was to contribute an input to the very scarce local
literature that recognizes the critical role student self-assessment plays in improving
learning outcomes, and the part that students assume in assessment process. Specifically,
this study was aimed at exploring conceptions and validity of learners’ self-assessment in
university EFL classrooms from both the students’ and the instructors’ perspectives. It
tried to find out the conceptions held by the instructors and the learners and the extent of
validity of learners’ self-assessment of their English oral performance. It also aimed at
disclosing whether training improves the conception and the validity.
To achieve the above-mentioned general and specific objectives, the following research
questions were formulated and answered.
RQ1. A) What is the EFL instructors’ conception of learners’ self- assessment?
B) What is the University EFL students’ conception of learners’ self-
assessment?
RQ2. To what extent can learners provide a valid assessment of their own
English language oral performances?
RQ3. A) How does training in self-assessment affect the validity of learners’
assessment of their own English oral performances?
B) How does training in self-assessment influence the learners’
conception of learners’ self-assessment?
12
The research questions are set in such a way that they provide information on both the
visible (actual practices) and invisible (conceptions) aspects of the topic in focus from
different perspectives. While RQ1A & B are designed to study the invisible aspects, i.e.,
the conception of the participants, RQ2 is designed to look into the actual practice, i.e.,
the extent of validity of learners’ self-assessment. RQ3 A, & B are formulated to explain
the likelihood of improving the practice by training the learners.
The assumptions underlying this research is that actual practice is highly influenced by
knowledge, beliefs and values one attaches to the practice (Ashcroft, 1996; Pajares, 1992;
Williams and Burden, 1997; Woods, 1996); and that better understanding of the existing
behaviours can help to improve the situation. Related to this is the assumption that the
conceptions and practices can be improved by training and exercises. Although the
debate about whether to change conception first in order to change practices or the vice
versa is still going on in the literature (James and Pedder, 2007), it is unlikely that
learners’ self-assessment as strategies for improving learning and learner behaviour can
simply be added to the repertoire of teachers and students alongside the practices
designed for other purposes such as developing language skills or cognitive thinking; it
needs particular attention.
1.6 Significance of the Study
In essence, as Larsen-Freeman and Long (1991) noted, pursuing knowledge for its own
sake is very important because all knowledge is practical. However, this may not be
possible for study of this kind, which is expected to contribute at least a tiny drop to the
improvement of an existing practice. As a classroom research, significance of the
findings of this study is primarily be helping as opposed to merely knowing, though
13
distinction between “wanting to know” and “wanting to help” is blurred one, (Van Lier,
1989).
If the practise of and the conceptions of learners’ self-assessment held by the participants,
and how better it should be conceived and practiced is explored and made explicit, it can
prompt and help people in the profession to modify their conceptions and behaviour.
Thus, this research is primarily hoped to help the students and the instructors to create
and/or improve their awareness about learners’ self-assessment so that they could be
encouraged to use it in their classrooms and enhance their teaching/learning outcomes.
Secondly, the information to be made available about how the instructors and students
view and utilize this assessment mode would also inform the educational administrators
and curriculum designers and increase their awareness about ways of exploiting the
potential benefits of using this assessment technique in EFL classrooms. This is
particularly important in view of the persisting criticism about the low performance of
Ethiopian English language education programs. Lastly, it can attract attention and
prompt other inquiries to get wider and deeper understanding from different perspectives.
1.7 Chapter Summary
To sum up this chapter, there has been a pressing need for development of autonomous
lifelong learners who can cope with the sweeping and rapidly changing 21st century life.
Learners’ self-assessment plays pivotal role in achieving this need, and it is at high stake
in higher education.
English language education has been given attention because it plays significant role in
the quality of education in Ethiopian. However, it does not seem it is doing well;
14
therefore, it needs more attention from different angles. Empirical literature has indicated
that learners’ self-assessment improves learners’ approach to learning in general and
language learning in particular, but there are questions hanging over issues related to its
validity, conception and training. In addition, there are little, if at all, studies of learners’
self-assessment that explore how it works in Ethiopian classrooms. Therefore, this study
is intended to explore some issues related to the use of learners’ self-assessment in
university EFL classrooms; and by doing so, hoped to contribute to the national efforts
being made to improve the quality of Ethiopian English language education. The next
section turns on the review of the related literature.
15
2 Chapter Two
Review of Related Literature
2.0 Introduction
This section reviews the conceptual and empirical literatures that are related to learners’
self-assessment. First, it provides the theoretical ground in which learners’ self-
assessment is rooted. It describes the major development in the field of language
assessment, situates LSA within the Assessment for learning (AfL) tradition and relates it
to Humanism and Cognitive constructivism. Next, it clears the ambiguity in the use of
the term learners’ self-assessment by providing definitions and its typology. It also
describes features that make learners’ self-assessment in higher institution EFL classroom
different from other subject areas. Then, it provides conceptual and empirical literature
that supports the significance of learners’ self-assessment in contributing to the
improvement of learners’ learning outcomes and development of autonomous lifelong
learners. Then, it discusses some of the major issues like its validity, conception and
training that are related to the practical application of learners’ self-assessment. Next, it
discusses performance assessment and constructs of oral performance, and explains how
these constructs can be assessed. It also points out that these discussions informed the
assessment of English oral performance used in this study. Lastly, it concludes the
chapter by summarising the main points.
2.1 Major Developments in Assessment Tradition
There have been shifts in the approaches to and methods in language learning and
language teaching since the early nineteenth century. The approaches and methods in
16
language testing also followed the shifting sands of the teaching methodology both in its
purpose and in method. Five stages of development in language testing are briefly
outlined here. See Richards and Rodgers (2001) for the description of approaches to
language teaching.
During the grammar translation method when language teaching/learning was
characterised by detailed analysis of its grammar followed by translating sentences and
texts into or from the target language (Richards & Rodgers, 2001), language testing also
took the essay translation approach. Language tests usually consist of essay writing,
translating to or from the target language and analysis of the grammar. It did not require
any expertise or special skill in testing and the teacher’s subjective judgement was
considered superior. This is labelled as the pre-scientific era (Heaton 1990).
The development of Behaviourism brought a slight change in language teaching.
Language was seen as a structure of discrete linguistic items, and language learning is
considered as the mastery of sets of these linguistic items by mechanical habits
(behaviours) formation through the process of conditioning subjects (learners) to stimuli
through reward and punishment. From this perspective, assessment is seen as an
inventory of the items learners are supposed to master by using discrete items that are
short, atomized, and de-contextualized and that require the learner to recall fragmented
bits of information (James et al., 2007; Richards and Rodgers, 2001; Williams and
Burden, 1997). This approach to assessment is termed as ‘discrete-point approach’
(Brown, 2003) or ‘psychometric-structural approach’ (Heaton, 1990).
17
This conception and practice of language teaching and assessment was changed with the
emergence of cognitive theory of language learning that see language learning as mental
process in which the learners employ mental strategies to receive, analyse and transform
information. Language competence is seen as a unified set of interacting abilities that
cannot be separated and requiring simultaneous control over different aspects of the
language system as well as the different language skills (Brown, 2003; Williams and
Burden, 1997; James et. al., 2007). From this perspective assessment is seen as the
procedure of collecting data on the learning process using holistic and integrative tests
such as cloze and dictations (Brown, 2003; Falchikov, 2005; Heaton, 1990; Weir, 1990).
Weir terms this approach to assessment practice as psycholinguistic-sociolinguistic
approach and Falchikov terms it as ‘assessment as procedure’ approach.
The 1960s development of communicative approaches to language teaching, which is
based on the view that language is a vehicle for interaction between individuals for
performance of social transactions, brought the communicative language testing
(Bachman, 1990 Weir, 1990). Communicative language testing claims to assess the
effectiveness of the learner in specific real-like communication context by using the
components of communicative language ability: language competence, strategic
competence, and psycho physiological mechanisms (Bachman, 1990; Weir, 1990).
The common features of all the above-mentioned approaches to assessment are that they
primarily have the purpose of measuring learning outcomes; are external to the learners
even when developed and used by teachers; and are perceived as a separate entity of the
educational program and appended to a program of instruction. They are also mainly in
the paper-pencil format (Falchikov, 2005; Lewkowicz and Moon, 1985; Weir, 1990).
18
Another major development of the late 1960s is the emergence of humanism concurrently
with cognitive constructivism (Davis, 1971; Stevick, 1990; Williams and Burden, 1997).
These two developments brought major changes on the conceptions and practices of
assessment tradition, and resulted in the idea of assessment for learning (AfL) of which
learners’ involvement in the assessment process is the main feature (Butterfield, 1995;
Clarke, 2005; Marzano, et al., 1993; Segers, et al., 2003).
Humanism prioritizes the human aspect in language teaching/learning process. The basic
principle of this theory is the notion that the emotional and social aspects of teaching
matter as much as, if not more than, the cognitive aspects. The underlying assumptions
here are first, a person is perceived as largely responsible for his/her own destiny.
Second, education should aim at preparing the learner to cope with the questions “put by
life” (Tolstoy in Davis 1971:76). Third, to achieve this aim, the individual’s inner world
(thoughts, feelings and emotions) are considered to play vital role in human development.
From this perspective, learning is highly affected by the way the individual learner views
him/herself within the context and by the extent to which the learner assumes control
over the learning process. Therefore, humanists emphasise the ‘engagement of the whole
person’-- emotional, social and intellectual aspects -- in the learning process to facilitate
the development of independent lifelong learning skills (Stevick, 1990; Davis, 1971).
While the intellect promotes the development of cognitive thinking, the emotional and
social aspects encourage cooperation and friendship, feeling of pleasure and appreciation,
sense of self-concept and true inner quality, confidence and responsibility. Therefore,
teaching should focus on the development of the learner’s (as an individual and member
19
of the social group) initiative, sense of worthiness, personal identity and responsibility for
his/her own learning (Davis, 1971; Nunan, 1988; Reagan, 1999; Stevick, 1990).
Cognitive constructivism is an extension of Piaget’s cognitive learning theory. Drawing
insight from Jean Piaget, cognitive constructivism asserts that learning is a constructive
process in which the learner is building an internal representation of knowledge and
a personal interpretation of experiences (Bednar et al. cited in Kwon, 2004). The
basic assumption about learning is that it comes about not by adding up discrete
knowledge bit by bit but constructing the holistic map of the interconnections of the facts,
concepts and experiences because the whole is greater than the sum of its parts. Persons
also do not simply receive this holistic knowledge but construct it by actively engaging in
interaction with their environment (Williams and Burden, 1997).
The common tenets of these two theories are their focus on development of autonomous
lifelong learner and their emphasis on the need for the engagement of the whole person in
the learning process. Autonomous lifelong learning is needed for it is impossible to teach
everything a learner needs within the relatively little class time available. Therefore,
education should aim at preparing learner for not only the here and now but helping
him/her proceed further on his/her own so that s/he can fit into the ever and rapidly
changing fiercely competitive world (Dickinson & Carver, 1980; Nunan, 1988). This can
be achieved when the whole person (cognitive, emotional and social aspect) is involved
and actively interact in the learning process (Stevick, 1990; Williams and Burden 1997).
These tenets of the humanism and the constructivism are realised through the learner-
centred approach to language teaching (Nunan, 1988). In learner-centred approach,
20
learners are encouraged to take fuller and more active participatory role in the whole
process of their language learning. Classroom decision- making is a negotiated and
shared process between the learner and the teacher because this encourages the learners
to assume a sense of responsibility, ownership and control over their own affairs (Breen
and Littlejohn, 2000; Tudor, 1996; Nunan, 1988; Wenden, 1991).
These complementary developments resulted in rethinking about the congruence in the
conceptions and practices between learning, instruction and assessment. The ‘traditional’
assessment was inconsistent with the idea of holistic learning and learner empowerment
as ultimate goal of language education. As a result, the purpose and modes of assessing
learners’ learning has changed. Assessment has become an integral part of the instruction
to provide opportunity for the learners’ full participation in the process and holistic
information about their learning. Thus, it concords with the basic tenets of humanism and
constructivism (Brown with Bull and Pendlebury, 1997; Gardner, 1999; Havenes and
McDowell, 2008; Nunan, 1988; Marzano et al., 1993; Sebba et al., 2008; Segers et al.,
2003; Zariski, 1996).
This development is termed as Assessment for Learning (AfL) or Formative assessment.
Formative assessment differs from its predecessors in that first, its primary purpose and
objective is not describing achievement for certification but improving learning.
Therefore, it is considered not just an integral part but the major component of the
classroom instruction. Second, it comes not only in the paper-pencil format but include
variety modes like portfolio, journal writing, project works, and task performances. The
third and prominent feature is the involvement of the learners in the whole process of
21
assessing their own learning outcomes through peer- and self-assessment. The major
shifts in the current assessment tradition can roughly be summarised in Table 1 below.
Table 1 Major shifts in the assessment tradition
(Based on Brown with Bull & Pendlebury, 1997; Havnes & McDowell, 2008)
A points needs to be mentioned here. As can be observed from the table above, it does
not mean that the AoL and the AfL are exclusive of each other. Rather, the shifts can be
seen on a continuum with items in the lists on both side of the table are on the two
extremes of the continuum. For instance, an assessment can serve multiple purposes like
improving and at the same time for grading, with different emphasis on the right or the
left end. Nevertheless, it should be stressed that for assessment to serve its purpose, the
focus of attention has been shifted towards the right column of the table.
from assessment of learning (AoL) towards
towards
Assessment for learning (AfL)
Separated
integrated
periodical continuous
for judgement, selection, certification etc. for improving learning
de-contextualized/atomistic contextualized
product assessment process & product assessment
paper-pencil format variety of techniques
teacher-controlled learner-involved/negotiated
stored responsibility shared responsibility
opaque transparent
competitive collaborative
22
One of the marked shifts in the assessment purposes and methods particularly in language
teaching is the shift of locus of control i.e., the shift from the teacher-controlled
assessment to learner-involved/negotiated through self- and peer-assessment (Brown et
al., 1997; Falchikov & Goldfinch, 2000; Falchikov, 2005; Havnes & McDowell, 2008).
These modes of assessment have got the most significant place in the thinking in the
direction of learner autonomy. The next subsequent subsections describe learners’ self-
assessment and conceptualises how it contributes to the improvement of learners’
learning outcomes.
2.2 Learners’ Self-Assessment
Self-assessment is not alien concept to human behaviour. Human constantly engage,
consciously or subconsciously, in an on-going process of evaluating the self and the
others. In non-technical language, every day one part of the self assesses the actions and
successes of another part of the self and the others. In most of religious rites, it is part of
meditation that involves examination of conscience and private confession. In
psychology, the ego assesses and controls the id and the super ego, and in pedagogy
learners always question their own performance and achievement (how well have I
done?) with the need to know and fill the gap between the actual and expected standard.
At the work place persons usually engage in evaluating colleagues, workmates and others
with the intention of comparing oneself with others.
It is in the mid 20th
century that this common sense emerged as part of the meta-
cognitive process (thinking about one’s own thinking and learning about ones’ own
learning), and its significance as part of human thinking and learning process gained
popular acceptance in cognitive psychology. Since then, it has been provided with
23
theoretical explanations and research findings in human learning (Brown with Bull and
Pendlebury, 1997; Coombe, 2002; Lewkowicz and Moon, 1985).
2.2.1 Description and Typology of Learners’ Self-assessment
The concept of learners’ self-assessment (hereinafter LSA) is open to be defined loosely.
It goes with variety of terms like, self-evaluation, self-testing, self-grading and self-
marking which refer to different practices that can be labelled self-assessment, but do not
actually conform to the very idea of self-assessment. As Boud & Brew (1995:2) argue,
though these terms are similar in certain ways to LSA they “are sufficiently different to
warrant separate consideration and the use of alternative descriptors”.
The terms self-testing, self-grading and self-marking refer to the practices in which the
learners check their performance, for example, on objective type test items where the
correct answers for the items are provided by the teacher after the exam session or at the
back of the book/ material as is common in text books and distance learning materials. In
such cases the items and the answers are designed and provided by the teacher or material
writer and the testees have no way of engaging in setting or questioning the criteria of the
evaluation (Boud and Brew, 1995; Brown & Knight, 1994).
Although the terms ‘self-assessment’ and ‘self-evaluation’ are sometimes used
interchangeably (for example Black, 1999; Sebba et al., 2008; Ross, 2006), they are still
different in the basic concept they stand for. As Oscarson in (Geeslin, 2003) argued,
while self-assessment refers to the process of setting criteria and assigning value (mark or
grade) to describe achievement in performance, product or process, self-evaluation is
judging and interpreting the meaning and quality of that achievement by reflecting on the
24
result of the assessment. To illustrate, a self-assessor marks his/her writing performance
seven out of ten, then this mark can be judged (evaluated) as ‘good’ or ‘poor’ depending
on contexts. Thus, self-evaluation does not include the two components of self-
assessment.
Some definitions of LSA in the literature also fail to differentiate it from the other terms
that flock with it. For instance, Topping (2003, p. 58) defines LSA as “an arrangement
for learners and/or workers to consider and specify the level, value or quality of their own
products or performances.” This definition is not comprehensive because it does not
mention the role of the learners in the process of the assessment. An illustrative definition
of LSA is given by Boud & Brew (1995) and Boud & Falchikov (1989). They explained
that the two main defining characteristics of any assessment process are making decisions
about the explicit criteria and standards of performance expected and making judgments
about the quality of the performance in relation to these standards. Therefore, they
argued that as the term suggests, LSA should ideally involve students in both of these
aspects. Thus, LSA can be defined as a form of assessment in which students are
involved in, at least, identifying standards and/or criteria for evaluating their work and
making judgements based on the criteria about the quality of their own performance and
the extent to which they have met the standards.
Because of the less precise use of the term LSA and its associates, LSA is usually
misunderstood and/or misused in the literature. Especially in the empirical literature on
LSA, only few studies provide clear or implied meaning of the term as it is used there.
This made difficult to compare findings of different studies with and among each other
(Boud & Falchikov, 1989; Boud & Brew, 1995). In response to this problem, Boud &
25
Brew (1995) developed a typology of LSA. They put in to three levels based on the
knowledge and cognitive level required and the extent to which the learners are allowed
to take part in the assessment process.
Technical interest level is the first and the lowest level of LSA which requires students
to check knowledge, performance or skill level against a model answer or criteria
provided by the teacher. The use of answer keys or teacher-provided criteria or
questionnaire-like competency statements can be examples of assessment involving
students at this level. These tasks do not require high level of cognitive ability and
knowledge of the subject matter itself; it serves only technical knowledge level where the
interest is assessing the understanding of simple and objective facts and ideas
independent of their context using objective statements, and does not require subjective
and contextual interpretation of phenomena. The concepts of the terms mentioned above
that are synonymously but mistakenly used with LSA are categorised under this level.
Communicative interest level: At this second level, involvement is greater and students
are allowed or encouraged to discuss and negotiate criteria or consider what constitutes a
good answer before applying these standards to their own work. This level of LSA
involves elements of communication and interpretation and discussion of relationships
between the elements of the assessment. In this process the students derive practical
knowledge and develop their own meaning. For example, students in a class test situation
that involve subjective answer, first each student outline an answers to each question.
Then, one student describes her/his answer and others add ideas to form a collective view
of what are good answers. Students then score their own answers in relation to the good
answers.
26
Emancipatory level: The third level has an emancipatory component in that at this level
of involvement students not only build on their knowledge but also make meta-level
analysis where they reflect on the process of building the knowledge. They are required
to give critical attention to the criteria, question and comment on the competencies
themselves. At this domain, the discussion goes well beyond the question whether or not
the students met the set criteria and standard and focuses on justification of the standard
and the given level of competency. As an example, students perform tasks, set up their
own competency standards and assess themselves. They then justify these to their teacher
and may change their assessment or the criteria in the light of these discussions.
From the point of view of constructivist and humanist philosophy of education, these
second and third types of LSA are directly related to the improvement of learning
outcomes. In these types, learners first participate directly in deciding on the assessment
criteria; then they do a task-based performance of the target language aspect (in authentic
or simulated situation) and finally, score their own performance based on the criteria
developed for that specific task. After all, the second and third types of LSA fit into the
why of LSA.
Beyond clearing the confusion in the terminology, Boud & Brew’s typology suggests two
points that need consideration. The first and may be more important is the question that
which type of LSA should be used at which level with what kind of learners. A clear-cut
answer to this question obviously should come from research findings. However, it can
be logically argued that the second and the third types of LSA would be difficult for
students in the primary and first cycle secondary education in Ethiopian. This is because
it is self evident that the tasks of assessment in these levels requires relatively developed
27
cognitive ability and more complex schema that could be attained during the late
adolescence Piaget’s cognitive development stage. In similar logic, it is possible to argue
that the second type of LSA can be used with upper secondary (preparatory) and tertiary
level students.
The second and related point is the kind of skill and knowledge to be fostered. This again
depends on the grade level and cognitive maturity. Here again it can be asserted that for
LSA to serve its purpose, i.e., facilitate the development of autonomous lifelong learning
skills, it is necessary to involve learners in at least the second type of LSA which
encourage construction of personal meaning and knowledge in the process of doing the
assessment.
Based on the format by which students self-assess, LSA of language skills can take
global (indirect) or direct (task-based) performance format (Brown, 2003; Tudor, 1996).
Global (indirect) self-assessment is a format in which learner is provided with partial or
extended description of a language skill or descriptions of situation of the language use
and asked to rate him/herself on a five-or ten-point scale questionnaire by indicating the
description that s/he thinks best corresponds to her/his assumed level of competence and
skills. Alternatively, learner can self-assess by completing the questionnaires about what
s/he thinks s/he can do with the language or one of its aspect in certain situation on a
yes/no or a Likert-like scale. This format corresponds to the first type of self-assessment
in Boud & Brew’s typology.
Although this format is easier to construct and administer and can be used to help
learners make rough and indirect estimate of their perceived overall proficiency, it does
28
not specify the detailed constructs of each language skills to be assessed. In addition, it
fails to help learners detect what s/he can or cannot actually perform (Lewkowicz and
Moon, 1985; Tudor, 1996).
Direct (task-based) self-assessment requires the learner (self-assessor) to accomplish
certain communicative task in an authentic or simulated situation in the language skill to
be assessed. Then, s/he evaluates how well the performance was using the set criteria.
The task-based self-assessment format, though not easy to develop and administer,
enables the learner to detect the specific constructs that cannot be elicited by the indirect
test formats. It involves the actual performance of the task and provides the testees with
opportunity to demonstrate their ability to organise the language materials using their
own words and ideas. This enables the self-assessor to see the direct application of the
micro and macro skills (Brown, 2003; Tudor, 1996).
In this study, based on the observations and perceived relevance drawn from the typology
and description of self-assessment, the Communicative interest level of LSA was used in
the Direct (task-based) assessment format. Considering the students’ age and level of
education, using the first type seemed simplistic and too mechanical. In addition, it does
not fit the main goal of higher education for which LSA should be pursued (Taras, 2003;
Tan 2007). Again using the emancipator type of LSA seemed to be complex for the
students. Though it can be argued that the third level is appropriate for undergraduate
students, their lack of any experience in any kind of LSA and their perceived low quality
of educational ground does not seem to allow realistic use of the highest level. Hence, in
this study the second type of LSA was employed. That means the students were involved
in discussing and developing the criteria of assessment and used the criteria to evaluate
29
their own performance. Therefore, subsequent discussions related to LSA are made in
light of this conception.
2.2.2 LSA in University EFL Classrooms
LSA can be employed in a wide range of subject areas and levels. However, there are
reasons to focus on EFL classes in tertiary levels. First, because the language learning
process by itself is complex and the nature of the constructs of what is to be learned and
assessed is elusive (Bachman, 1990; Coombe, 2002; Dickinson & Carver, 1980), the idea
that learners can assess themselves appears to be more doubtful in language classrooms
than the content area classrooms for various reasons. In the first place, unlike assessment
of abilities in other areas of knowledge where language is used in the process of
measuring something else, in foreign language classroom language is both what is to be
learned, and at the same time, the means by which it is learned and assessed.
Consequently, there is high probability of occurrence of a number of unnoticeable errors
of certain sort (Bachman, 1990; Tsui, 1996; Allwright and Bailey, 1991). For example,
while attempting to express her/his thought in the language being learned but not yet
mastered, a learner may get it right in terms of content, but commit very elusive errors in
any of the linguistic aspects. This makes learners’ assessment of themselves more
difficult when compared with the content area classrooms where the focus of the
evaluation is the degree of acceptability of the content information.
Next, foreign language classrooms are characterized by high prevalence of anxiety.
Following the introduction of communicative approaches, foreign language classrooms
put greater emphasis on oral as well as written interaction rather than knowledge
transmission. As Tsui (1996) noted, this has the risk of subjecting the learners’
30
performance to public scrutiny and unfavourable criticism and evaluation. This causes
excess anxiety from fear of making mistakes in front of the teacher and peers in settings
where the culture of self-assessment and constructive peer evaluation is not practised;
Ethiopian EFL classrooms are the case in point.
In addition, the different constructs like proficiency, communicative competence,
linguistic competence and strategic skills are not, at least for the learner, easy to identify
and pin point. For the learners, it is not easy as such to demarcate these skills and see
whether they are separate and comparable for assessment. Besides, different skills require
different focus of attention. For example, oral performances demand more fluency and
online processing than the written ones; and communication needs different focus than
linguistic analysis. These situations i.e., the inevitability of errors and the high possibility
of failure to catch these errors in assessing one’s and/or peer’s foreign language
performances than the other school subjects make foreign language teachers sceptical
about the dependability of LSA.
While the need of incorporating LSA in EFL classrooms, regardless of the perceived
threat, is nicely relevant to EFL classrooms at all levels, there are more reasons to
underscore the need for the learners’ fuller participation in their own assessment in higher
education. The first reason for promoting student self-assessment in higher education is
connected to the fact that it equips students to become lifelong learners who can evaluate
their own performance after they have finished formal study’ (Brown and Glasner, 1999
in Tan, 2007).
31
It is widely recognised that the main goal of higher education is to help students to
develop into reflective practitioners who are able to reflect critically upon their own
professional practice (Falchikov & Boud, 1989). University students are at the verge of
joining the world of work. These graduates could be ready to take responsibility of their
own action and judgement when they get substantial amount of control over assessing
their own learning outcomes in an informed manner (Brown and Knight, 1994; Dearing,
1997 in Taras, 2002). For instance, the fact that many of the graduates of TEFL from
Ethiopian universities could join the EFL teaching staff in secondary schools makes LSA
immediate necessity for the to-be- teachers. Besides developing the skills of assessing
themselves, they have to have the awareness of the need for and the skill of practising
LSA with their learners (Miller and Ng, n.d).
The second reason is related to students’ perception of higher education. In the recent
trend of perceiving education as business in higher learning institutions, students, as
paying customers, have invested in higher education and perceive the reward of their
investment “to materialize in the form of assessment grades” (Taras, 2001:606). When
the grade fails to turn out to their expectation, they become cynical. As Smith (2000)
noted, allowing learners to participate in assessment is the major step to correct this
negative perception and decrease the learners’ cynicism because it enhances learners’
awareness of the standard required of them. This, in turn, minimises their distrust about
grades they earn and leads them to understand and share responsibility of their learning,
and consequently, improves learner-lecturer relationship which is conducive situation to
learner autonomy.
32
Generally, learners’ participation in assessment of their foreign language learning is at
the heart of learner empowerment at all levels and especially in tertiary education.
However, the concerns of complexity of features that are peculiar to language learning
make it more imperative in foreign language than other school subjects. It is with this
understanding that this study was planned to be carried out in university EFL classrooms.
2.2.3 LSA and Learning
As briefly discussed in section 2.1, LSA derives its theoretical justification from
humanism and constructivism that are realised in learner-centred approach to language
teaching. The basic premise of involving learners in assessment through LSA is that the
direct involvement of the learners in the whole process enhances the quality and standard
of learning by affecting learners’ cognitive, emotional and social learning skills thereby
enhancing the academic and non academic learning outcomes.
The role of LSA does not end with the class end. Learners who master the art of self-
assessment can extend their learning successfully beyond the classroom and curriculum
structure, and the presence of a teacher or tutor and examination (Brown, 2003; Brown
with Bull and Pendlebury, 1997; Davis, 1971; Dickinson, 1987; Geeslin, 2003; Little,
2005; Sebba et al., 2008). Underscoring the centrality of LSA in the development of
autonomous lifelong learner, Davis (1971) puts:
The learning process ─ also known as education ─ cannot end at the end of the
professor’s grading pencil. As soon as that happens [,] the professor takes the
responsibility for what is learned, not the student. It is only meaningless
education which needs the professor’s evaluation, for what is meaningful is
continually evaluated by the learner, who further does not need the goad of
grades to motivate him [her] to learn. What [s/] he does need, of course, is the
experience of evaluating his[/her] own learning and his[/her] own behaviour.
33
This experience [s/]he cannot get if the professor takes all the responsibility for
the quality of [her/]his work, leaving [her/]him with only the responsibility to
satisfy the professor’s demands. (p. 76)
Little (2005) particularly stresses the critical role of LSA skill in language learning for
the-beyond-classroom learning. He writes “... a capacity for accurate self-assessment is
an essential part of the toolkit that allows learners to turn occasion of target language use
into opportunity for further explicit language learning.” (p. 322). Generally, in Brown and
colleagues words “[learners’] self-assessment is sine qua non of course design and
delivery” (p. 178).
So, what does self-assessment require from students in terms of cognitive, meta-cognitive
and social-affective demands? Through what processes might these benefit students?
How does the premise work?
In the first place, assessment is a cognitively complex undertaking that requires
understanding of the goals of the task(s), the criteria for success and the ability to make
judgements about the relationship of the product or performance to the goals and the
criteria. Thus, LSA is reflective activity that involves critical thinking, comparing,
contrasting and reasoning. It also involves the assessor in reviewing, summarizing,
clarifying, giving feedback, diagnosing misconceived knowledge, identifying missing
knowledge, and considering deviations from the ideal.
When the criteria for assessment is discussed, negotiated and clarified by all participants
and used in practice, learners are most likely to be clear concerning what constitutes
high-quality work and direct their attention to crucial elements. In addition, the process of
self-assessment guides the participants to monitor their progress. It helps them to identify
34
their strengths and weaknesses, recognize sources of difficulty, identifying ways to
improve and making decision like whether to go on to the next item or the need for
remedial work and how much time and effort to allocate by focusing on specific aspects
of their performance (Dickinson, 1987; Geeslin, 2003; Smith, 2000; Topping, 1998,
2003).
Another, LSA makes available quicker feedback in greater quantity. In the event of
misconception, it might prevent consolidation of confusion and the fossilization of error.
Even where assessed products show no major error, feedback could prompt higher order
or better quality thinking. Furthermore, LSA also encourages increased on-task
behaviour, thinking, comparing, contrasting and communicating (Topping, 1998).
Communicating the assessment result to justify to a peer or the tutor also heightens the
learners' sense of personal accountability and responsibility as well as confidence,
motivation and self-efficacy. All of these features are cognitive, emotional and social
activities that lead the learners to deep approach to learning (Geeslin, 2003; Topping,
1998, 2003).
Students who adopt a deep approach to learning are intrinsically interested in the task.
They aim for understanding the meaning of what is being learnt by relating the different
aspects of the information and ideas together and to their own experience, and looking for
patterns, principles and meaning in the texts. This approach leads to higher-quality
learning outcomes for the student (Bloxham & Boyd, 2007; Tang, 1994).
In addition to the above mentioned core academic and non-academic benefits, engaging
learners in LSA bears additional systemic positive side effects. It improves transparency
35
and develops sense of trust between the teacher/tutor and learner. It can also give students
greater insight into institutional assessment processes. Students might thus develop more
confidence in these processes and greater tolerance of the inevitable difficulties of
discrimination at the margin. Even, if institutional assessment procedures are inadequate,
students could contribute to improvement of the inadequacies. These conditions surely
improve the learner-teacher relationship and minimize students’ disruptive behaviours,
which is a common ‘headache’ in tertiary levels (Dickinson, 1987; Race, Brown &
Smith, 2005; Smith, 2000; Topping, 1998)
LSA can also reduce assessment burden on the teacher. It can save the teacher’s/tutor’s
time. However, it should be noted that there might be no saving of time in the short to
medium term, since establishing the validity and reliability of the practice requires time
for organization, training, and monitoring (Topping, 1998; Race et al., 2005).
In a nut shell, LSA enhances learners’ academic and non-academic learning outcomes by
improving their awareness and ability of setting goals, directing their attention to the
more important aspect, motivating to commit the necessary amount of effort and more
personal resources, directing and monitoring the effort and resources towards the
achievement of the goal, evaluating the achievement of the goal to see the knowledge gap
and set the next goal.
The following figure was adapted from Ross (2006) to conceptualise and illustrate how
LSA plays its role in enhancing learning outcomes. In Ross’s model, the process and
outcome of self-assessment is shown in rectangular fashion using arrows. The arrows
indicate the direction and steps along the process. In this model, self-assessment involves
36
three steps, self-observation, self-judgment and self-reaction that students undergo; and
results in self-efficacy. Next, self-efficacy leads to two different directions, goals setting
and making efforts. Lastly, the goals and the efforts lead to achievement from the
different directions and the cycle of learning is completed.
Though the model can help to conceptualise how self-assessment contributes to the
improvement of learners’ learning, it has two basic shortcomings. Firstly, it missed
learners’ participation in the setting of standard and criteria. As discussed in section
2.2.1, one of the basic features of LSA is the learners’ participation in discussion of
standard and criteria of the performance expected of them; and LSA that is intended to
improve learning must include this element. Secondly, it assumes development of self-
efficacy belief as the only product of the process. However, as it is discussed in this
section, self-efficacy is just one among the many cognitive and non-cognitive benefits
identified by empirical and meta-analysis studies. The figure also does not visually
represent the incremental cycle of learning. Therefore, the figure was modified in the way
it includes all the fundamental features of LSA.
The modified figure conceptualises the incremental recycling of the learning process that
involved LSA and the subsequent causes and effects. The self-assessment procedure
embodies four mental processes that students undergo. First, going through the discussion
of standard and criteria creates for the students a better awareness of their learning goals
that initiates the launching of organised and focused effort to achieve better results in the
next performance. This will be followed by students’ self-observations (scoring of their
own performance), deliberately focusing on specific aspects of their performance related
to their subjective standards of success. Next, students judge (determine) how well they
37
have achieved their general and specific goals (self-evaluation); and lastly, they reflect on
and, interpret the degree of their satisfaction with the result (self-observation). Students
who perceive themselves to have been successful on the current task (i.e., who recognize
it as a mastery experience) are more likely to boost their self-efficacy belief and
confidence. They will also be motivated and tend to believe that they will be successful in
the future.
Figure 1 Conceptual model of how LSA contributes to improving learning
(Adapted from Ross 2006)
Critical thinking
Reasoning
Confidence
Motivation
Self-efficacy
Accountability
Monitoring
.
.
.
Goal 1
Effort 1
Achievement
LSA
Goal 2 Effort 2
Improved
learning
outcome
Self-reflection
Self judgment
Self observation
Cognitive, emotional &
social behaviours
Goal 3 Effort 3
Discussion of
standards & criteria
38
The process also helps the learners to see the gap between their performance and the goal
set, and the strategy to be employed to close the gap, the kind of effort to be made and
monitored. This leads to the development of positive emotional social feelings that in turn
encourages the setting of higher goals and exerting and monitoring the necessary amount
of effort to achieve the newly set higher goal.
The different broken-lined arrows indicate the upward development of the learning cycle.
When students evaluate their performance positive i.e., they perceive that they are
successful and the gap between the performance and the goal seems little, they would be
encouraged to set higher and more difficult goals (goal2) and commit more personal
effort and resources (effort2) which leads to achievement, and the cycle goes on.
But, as Ross (2006) warns, the self-assessment processes can result in negative outcomes
if learners assess themselves negatively often. A stream of negative LSA can lead
students to select personal goals that are unrealistic, adopt learning strategies which are
ineffective, exert low effort and make excuses for low performance.
2.2.4 LSA and Learning Gains
What have been conceptualized in the above diagram has also been demonstrated in the
empirical literature. The empirical and meta analysis studies that are concerned with
examining the effect of LSA on learning, despite the inconsistency in the use of the terms
and variety in the purpose and design of the studies, have generally, though not
unanimously, reported positive effects of the practice on cognitive and non-cognitive
learning outcomes. Few writers have however, expressed their doubt about these
potential benefits of LSA.
39
Falchikov (2005) and Topping (2003) summarised the results of the studies of benefits of
involving learners in assessment scheme in generic terms. The benefits they reported
include improving higher order cognitive skills, fostering deep approaches to learning,
developing reflective skills, sharpening critical abilities, increased autonomy, increased
on-task behaviour, increased commitment to subsequent performance and increased
participation. LSA as a continuous longitudinal process also facilitates activation and
integration of the learners' prior knowledge and reveals developmental pathways in
learning. In the longer term, it impacts self-management of learning - facilitating
continuous adaptation, modification and tuning of learning by the learner, rather than
waiting for others to intervene.
In other studies, the results were reported in specific terms. Ross (2006) reviewed
research evidences on the impact of LSA on students’ performance and reported positive
results of studies in different grade levels and subjects for academic and non-academic
outcomes. The review indicates that students trained in LSA outperformed the control
samples in narrative writing (grade 4-6), mathematics problem solving (grade 5-6), and
Geography (grade 11) with the Effect sizes ranging from small to medium size; i.e.,
ES=.58 (for weaker writers), .40, and .50 respectively. In this article, positive effects on
non-academic outcomes such as increase in positive interactions and a decline in off-task
behaviours, higher student self-direction, and reduced disruptive behaviours have also
been reported.
Sebba et al. (2008) have also reviewed studies of the impact of LSA on students’
academic achievement and non-cognitive outcomes in secondary schools. They reported
that their review showed three types of improvements. These are increased Pupil
40
attainment across a range of subject areas (9 out of 15 studies showed a positive effect),
improvement in Pupil self-esteem (7 out of 9 studies showed a positive effect) and
improved learning to learn, especially goal setting, clarifying objectives, taking
responsibility for learning and/or increased confidence were reported by 17 out of 20
studies.
McDonald and Boud (2003) in their study of the introduction of LSA across a range of
subjects on a large scale found out that training learners in self-assessment has significant
positive effect on their performance. They trained students from Business Studies,
Humanities, Sciences and Technical Studies in self-assessment and compared their
performance on external examination with those who were not involved in the training.
They reported “On average, students with self-assessment training outperformed their
peers who had been exposed to teaching without such training in all curriculum areas
with effect size ranging from 0.13 to 0.26.” (p. 9)
Results of studies of the benefits of LSA on development of language skills are also
consistent with the above reports. For example, Atai & Meratzadeh (n.d.) compared the
effect of self, peer and teacher evaluation on Iranian female EFL students’ writing
composition. They found out that self-evaluation has had a significant effect (F-Observed
= 32.92 > F-Critical = 2.42) on the improvement of the writing components: content,
organization, vocabulary, grammar and mechanics. Ross, Rolheiser and Hogaboam
(1998) have also studied the effect of self-evaluation training on students’ narrative
writing skills and reported that the treatment group outperformed the control group on
narrative writing, but the overall effect was small (ES = .18).
41
However, it does not mean that the studies unanimously supported LSA; negative effects
of have also been reported in few studies. Ross (2006) has reported two studies in which
LSA resulted in negative outcomes. In both studies the experimental group that used LSA
underperformed when compared to a control group. However, Ross indicates that the
credibility of the results was questionable because of the quality of the design of the
studies. In one of the studies the subjects in the control and experimental classes are not
of comparable ability group and in the other the number of the treatment differ for the
control and experimental group.
Generally, the empirical and review studies reported above, affirm that use of LSA has
considerable beneficial contribution to improvement cognitive, emotional and social
learning behaviours. As Boud & Falchikov (1989) and Smith (2000) concluded, even
studies that report poor or negative evidences on the validity and reliability of using LSA
as measurement tools witness that its pedagogical advantages outweigh the
disadvantages. However, the implementation of LSA is not straightforward. There are a
number of issues to be considered to exploit the potential benefits. The next subsection
turn on the discussion of problems related to the use of LSA.
2.3 Issues Related to LSA
LSA, being a relatively recent development in the area of educational assessment and an
activity related to the mental structure of individual student, needs consideration of a
number of issues including its description discussed in 2.2.1 above. In this section, some
of the rest major issues are discussed.
42
2.3.1 Validity and Reliability
Validity and reliability are at the heart of any assessment in education. They are the
measures of the quality of the assessment instrument (a test, an exam, a task performance
etcetera). They indicate from different dimensions how well an assessment serves the
purpose which it is intended for. While validity is related to the quality and
appropriateness of the assessment instrument itself, reliability is related to the
employment of the instrument (Alderson, Clapham & Wall, 1995; Bachman, 1990;
Brown, 2003; Hughes, 2003). While reliability affects all modes of assessment in similar
way, validity becomes the major problem when it comes to LSA.
2.3.1.1 Reliability of LSA
Reliability is the extent to which the score obtained from an assessment result is
consistent and dependable. An assessment is said to be reliable if it yields the same or
nearly similar result when administered to the same or matched group of testees twice or
more, or when marked by two or more raters (Bachman, 1990; Brown, 2003). From this
point of view, reliability of LSA is measured by the extent of consistency between two or
more self-scoring of the same or similar performance over time. Reliability of LSA can
be improved through practice.
2.3.1.2 Validity of LSA
Validity in language assessment is the extent to which the instrument actually measures
the language abilities and skills it is purported to measure so that the evidences and
inferences derived from the result is adequate, appropriate, meaningful and useful to
make appropriate decisions. The authors in the field of assessment generally identified
five ways or perspectives from which validity can be tested though they used different
43
terms. These are content-related (internal or rational) validity, criterion-related (external
or empirical) validity, construct-related validity, consequential validity and face validity
(for details on validity and reliability see Alderson, et al., 1995; Brown, 2003; Heaton,
1990; Hughes, 2003).
Validity is particularly of paramount importance in LSA. For LSA to serve its main
purpose, i.e., to help learners improve their learning behaviours, it must be valid. First of
all, as Taras (2002) underscores, involving learners in the central aspect of the
learning and a very important means of developing their autonomy and independence
should include their involvement in the “assessment that is graded and contributes to
their academic results” ( P. 504). Secondly, learners should be able to see clearly the gap
between the quality and quantity of performance expected of them and what they actually
performed. If they fail to identify their weakness from strength or tend to be unrealistic or
overlook their weakness because of dishonesty, it is unlikely that they strive to exert the
necessary effort and appropriately monitor it to close the gap. In addition, as Todd
(2002) argued, to encourage increased motivation, greater independence and awareness
and positive learners’ attitude as well as the students’ genuine effort in assessing their
own works the self-assessment should form a proportion of the students’ final grade. This
also necessitates the relative validity of LSA.
Although all aspects of validity are important, criterion-related validity is of more
importance in validating LSA because it is the most susceptible to subjectivity. There are
also a number of related issues. In the first place, it does not seem that the terms validity
and reliability are being used consistently in the studies of LSA. In some studies, e.g.,
Cheng & Warren (2005) and the studies reported in Topping (1998), the term reliability
44
is used whereas the studies intended to validate learners’ scoring against that of the
teacher/tutors which means checking criterion related validity. As Falchikov & Goldfinch
(2000) and Topping, (2003) argue, if we are looking for the extent of agreement
between/among two or more learners’ self-scoring of the same test over time or different
peer ratings it could be said we are examining reliability. If, however, our primary
concern is comparing the scores from LSA with assessments made by professionals,
rather than with that of the same self over time, then we are validating the assessment
result.
In this sense, the criterion-related validity of LSA can be tested in two ways. One is
looking for the extent of agreement of the LSA results with that of an external
standardised examination like placement tests or proficiency test. The other is validating
the LSA results against that of expert markings (most likely the teacher/tutor) of the
subject assessed. In this study, validity of LSA was tested using this latter technique.
The other problem is related to relatively specificity of self-assessment of language
performance. LSA of foreign language skills performance is highly vulnerable, relative
to the other modes of assessment, to subjectivity and bias to influence the scoring of the
assessment. First, scoring task performance in language skills is by its nature a subjective
activity even when clear criteria are used. In addition, because it is human nature,
learners are likely to be biased when scoring their own and/or their classmate’s language
performance. Especially when it is of high stake, learners cannot help being dishonest
and exaggerate their mark (Boud and Falchikov, 1989; Dickinson, 1987; Peirce, Swain
and Hart, 1993; Wen-ming and Xiao-zhen, 2008).
45
Moreover, as mentioned in Section 2.2.2 above, the language skills themselves and the
constructs to be assessed are complex. Therefore, it would not be easier for learners to
demarcate clearly between and among the different constructs; and they may fail to
assess the same constructs using the same standards as their teacher or an expert assessor,
as observed in Cheng and Warren (2005), Falchikov, and Boud (1989) studies, for
instance.
It is because of these observed problems that most of the empirical and meta-analysis
studies on LSA have concentrated primarily on studying the extent to which learners can
assess themselves in realistic manner, and the determinants that are likely to affect the
practice. However, the studies are diversified in their designs: the metrics used to report
their findings, the forms of assessment tasks used and the criteria against which to check
the validity; and reported inconsistent and inconclusive results. For example, while some
reported their findings in percentage, others used correlations and still others used the
measures of central tendencies like mean and standard deviations. In their finding, while
some studies report considerable agreement between teacher/faculty marking and learner
marking, others report students overrating or underrating themselves when compared to
the other criteria. The studies also vary in the type of self-assessment existed there. Some
used the first type (Boud & Brew, 1995) or the global format (Brown, 2003; Tudor, 1996)
of LSA whereas others used the second type or the task-based format of LSA (Boud and
Falchikov, 1989; Chen, 2008; Falchikov, 2005; Topping, 1998; Falchikov and Boud,
1989; Falchikov and Goldfinch, 2000; Ross, 2006, Topping, 2003).
Boud and Falchikov (1989) conducted meta-analysis of 48 earlier studies in wide variety
of subject areas that compared LSA marks with the teacher’s/tutor’s marks. They found
46
out that 17 studies reported overrating and 12 underrating. Another review of similar
studies by the same authors, Falchikov and Boud (1989) that included 57 studies also
showed that on average students graded themselves higher than did 68% of tutors
marking their works. Ross’s (2006) review of research evidences on LSA has also shown
that studies of self-teacher agreement yielded mixed results and, in most cases, students’
self-markings are higher than teacher marking is.
The inconsistency of findings of the studies of the extent of student-teacher/tutor
agreement in the wider subject areas has also been observed in the studies that
particularly focused on foreign language skills; while some report positive results, others
express their doubt. For example, Chen, (2005) in a study that compared self, peer and
teacher evaluation of oral performance in a university EFL classroom observed a
significant match between teacher, self and peer assessment and concluded that “...
significantly students could assess themselves in a manner comparable to the teacher
when they had more practice in assessment...” (p. 8). But, Peirce, Swain and Hart’s
(1993) compared LSA of their own French proficiency to the objective measures of
language proficiency and reported that the LSA correlated only weakly with the objective
measures.
In addition, Ross (1998) in a meta-analysis of 60 correlation studies of validity of
learners’ self-assessment in second language learning also found mixed results for self-
teacher agreement. He found wide range of correlation coefficients ranging from 0.090 to
0.800 among the studies with an average of r = 0.63 across the four skills. The analysis of
differential validity (Ross, 1998) also indicated that the extent of validity of self-
47
assessment varies among the language skills; correlations being the strongest in reading
skill and weakest in speaking skill.
Generally, what the studies of validity in all subjects in general and in EFL classrooms in
particular indicate is that the issues of dependability of LSA has not been resolved yet
which implies further investigation is in order.
2.3.1.3 Determinants of Validity of LSA
The other focus of attention in the issues related to LSA is the elements that may
influence its validity. The determinants of the extent of closeness of correspondence
between LSA and other measures like teacher/tutor markings or other objective or
standardised tests were found to include, course level, subject area, format of the task and
level of language proficiency, task. Like the studies of validity, findings in these aspects
are also not conclusive.
Falchikov and Boud (1989); Falchikov (2005) and Ross (2006) in their meta-analysis
studies reported that course level and subject area are found to influence validity of LSA.
Studies within the broad area of science were associated with closer correspondence
between students’ LSA and teacher/expert assessment than those from other disciplines
like social sciences and languages; and students in advanced course levels appeared to be
more "accurate" in assessing themselves than students in introductory courses.
Some studies have also indicated that the format of the assessment is another factor that
influences validity of LSA. For example, Butler and Lee (2006) compared the validity of
off-task (global) and on-task (task-based) LSA of their English oral performance. The
result indicated that the students could self-assess their oral performance more accurately
48
in an on-task format than they do in an off-task format. In Peirce, Swain and Hart’s
(1993) study also, it was reported that self-assessment measures on specific task
performance are more correlated with tested proficiency scores than global self-
assessment measures.
Although studies of LSA were not found, researches on peer assessment in the area of
foreign language skills have indicated that level of learners’ proficiency in the language
is related with the extent of agreement between students’ marks and other criteria though
the direction of the relationship is not uniform. In Miller and Ng’s (n.d.) study of how
realistically students can assess each other’s speaking skills proficiency, it is reported that
students of high proficiency level could assess their peers’ oral skills realistically than
their low proficient counter parts. However, Mika (2006) compared three proficiency
group (Lower intermediate, Upper intermediate and Advanced) of students’ peer
assessment of their oral presentation in EFL classroom to that of the instructors’
assessment. The results indicated that the rating of the Advanced proficiency level
students was the least closely correlated with the instructor’s ratings, the Upper
intermediate students’ correlated most closely and the Lower intermediate the next.
Further, both the Lower Intermediate and the Advanced gave higher scores than the
instructor. As there is close relationship between LSA and learners peer assessment it is
very likely that language proficiency can affect validity of LSA.
Generally, validity of LSA is likely to be influenced by a number of variables like task
format, course level, and language proficiency. However, as the studies are few and the
results are not consistent, it is difficult to predict the kind of relationship between the
variables and validity of LSA.
49
2.3.2 Conception of LSA
Another major area of concern related to LSA is the conceptions of LSA held by the
stakeholders (teachers, learners, educational administrators and parents). A few studies
available on this area show that there exist different conceptions among students and
teachers. However, one of the problems in this area is the inconsistency of the concept
with which the term conception itself is used in different literature. Therefore, it is
necessary to make clear the intended meaning used in this research.
2.3.2.1 The Concept of Conception
It is difficult to precisely demarcate the distinction between/among the plethora of terms
like belief, knowledge, attitudes, assumptions, values judgments, axioms, opinions,
perceptions, conceptions, conceptual systems, preconceptions, dispositions, implicit
theories, and personal theories that are used to refer to psychological constructs that
define and describe the structure and content of mental states that drive a person’s actions
(Pajares, 1992; Woods, 1996). The interest, however, here is the term conception and its
affiliates like knowledge, perception and belief. In some literature (e.g., Abiy, 2005)
these terms are used interchangeably and in some others (e.g., Struyven, et al., 2005;
Kreber, 2003; Awol, 1999) the researchers bypass defining the terms. However, although
these terms have some commonality, they have differences in some aspects; therefore,
they should be used accordingly.
Perception refers to a person’s understanding of her/his surrounding world by organising
brief experiences, sensory information and feelings. Knowledge is a set of relatively
universal facts and information a person possesses about his/her world. Belief is a valued
Knowledge that refers to relatively more developed idea or opinion acquired through
50
evaluation and judgement of the knowledge, perception and reflection or experience
which felt to be true, deeply personal and is more stable than knowledge and perception
(Bunts-Anderson, 2004; Heal, 2003; Pajares, 1992).
Conception, as invoked by Thompson (1992), represents general mental representations
of phenomena encompassing knowledge, perceptions, beliefs, attitudes, meanings,
preferences, and other mental images that explain complex and difficult categories of
experience. It represents comprehensive, organized, and unified body of knowledge
(Brown, 2004; Freeman & Richards, 1993). As the context of teaching learning and
assessment is characterised by such complex phenomena and the attempt of this study is
to gain insight of it, this meaning of the term conception is maintained in this study and
used accordingly.
2.3.2.2 Conception and Practice
There is consensus on the recognition that the conceptions individuals hold about
phenomena are the best indicators of the decisions they make in the course of everyday
life (Williams and Burden, 1997; Woods, 1996). Accordingly, conceptions teachers and
students hold of teaching, learning and assessment have strong effect on their classroom
pedagogical acts than their knowledge and recommendations of the pedagogical experts.
Conceptions act as filters through which educators view and interpret their own teaching
environment (Brown, 2004; Marton, 1981). For example, a teacher can have a good
knowledge of learner-centred pedagogy, but insist on knowledge-transmission because of
his conception of learners as unable to discover by themselves.
51
Similarly, studies have revealed that the teachers’ and the students’ classroom decision
and actions regarding the assessment practice is closely linked to their conception of
assessment (Brown, 2004; Struyven, et al., 2003). It is critical that, as Brown (2004) and
Vandeyar and Killen (2007) stressed, such conceptions are made explicit and visible,
challenged and changed or modified if it is considered necessary that the practices be
improved.
Conceptions develop through experiences. They are rooted in and highly influenced by
the person’s societal and educational cultural ground (Pajares, 1992; Williams and
Burden, 1997; Woods, 1996). For example, in Western cultures independence and
individuality is favoured, and children who are outgoing, expressive, eager to question
and explore are encouraged. Students from such cultures have positive conceptions about
themselves.
In contrast, in most of Southern European, Latin American, and African cultures,
harmonious interdependence is emphasised; and reticence and submissiveness is
considered the characteristics of a well-behaving child by parents and teachers. Students
from such cultures are less expressive and have less positive conception about themselves
(Marshall, 2004). In such cultures the expectation is that the teacher is the only person
responsible for teaching, learning and assessment of students.
Ethiopia is one of the countries with such cultures (Derebssa, 2007; Yalew, 2004). In
Ethiopia, “obedience and politeness are overriding goals in bringing up children”.
Moreover, in the educational culture ‘silence, unless demanded’ is an established
classroom norm, and trying to participate in the decision about their learning is
52
considered impolite and being rude to the teacher (Derebssa, 2007: 131). As Harris,
(1997) observes, the very idea of LSA goes against such educational culture. However,
Marshall (2004) notes that it does not mean that all persons or families from certain
ethnic or racial group necessarily share common conceptions; there are differences within
a culture and even within a family. This is true for Ethiopian context where students,
especially in higher education, come from a wider society in which a number of ethnic
and cultural groups live together and socially interact. This indicates that it is not
possible to rightly predict the kind of the learners’ and the teachers’ conception of LSA in
Ethiopian context.
2.3.2.3 Conceptions of LSA
Conceptions of LSA emanate from the general conception of assessment that, in turn,
emerges out of understandings, socially and culturally shared working traditions,
experiences and educational practices one underwent. In the conceptual and empirical
literature, it is possible to discern that the teachers’, learners’ and school administrators’
conception of LSA is closely linked to their thinking about purpose of classroom
assessment and who is responsible to do it; their understanding and interpretation of the
concept of LSA itself and its significance in the teaching learning process; and their view
of its validity.
In his study of teachers’ conception of assessment, Brown (2004) summarised three
purposes for which assessment may be carried out: improvement of teaching and
learning, school accountability and student accountability. These can be put as
assessment for improving learning and for auditing learners’ achievement. The view of
assessment as a means of auditing may come from the dominant traditional thinking of
53
assessment as a means of summative purpose. The assessment as a means of improving
learning may come from awareness of the redefined purpose of assessment (see Havnes
& McDowell, 2008; Boud & Falchikov, 2007).
Regarding where the authority and responsibility of assessment resides, it is possible to
hypothesise that assessment is exclusively the job and responsibility of the teacher or a
joint venture of the teacher and the students or should come from external body. The
conception of assessment as the teachers’ role emanates from educational cultures where
the teacher is an authority. In school systems where the teacher is seen as the authority in
decision-making, judgements about learning outcomes should be made by the teacher
only. Shifting this locus of control is seen as ‘breaching the law’. The major premise of
the view of assessment as a collaborative activity of the teacher and the students can be
the thinking from perspective of constructivism and humanism, and learner-centeredness.
From this point of view, assessment is an integral part of the teaching learning process in
which learners are the active partners (Havnes & McDowell, 2008).
As discussed in section 2.2.1 above, the concept of LSA can be interpreted at different
levels and there are different meanings that are attached to it. In addition, it is reasonable
to assume that the significance one attaches to LSA is related to the meaning s/he has in
mind. The source of the different interpretations and meanings, and the understanding
and value given to LSA in the teaching learning process can be one’s experience as
student and professional knowledge and experience as teacher.
54
The premise of the belief related to validity of LSA is one’s belief about whether learners
are able to understand the assessment constructs and, even if they could, whether they
would be able to control their ego and assess themselves realistically.
One more point related to conception of LSA and can be expected is that LSA may be
seen as an irrelevant act. This conception may arise from the thinking of the conventional
assessment tradition. Assessment seen as solely an instrument for judgement, LSA can be
considered as irrelevant and just waste of time because the result of the assessment
provided by learners on their own work cannot be taken for granted (Brown and Knight,
1994; Brown et al., 1997; Falchikov, 2005; Sebba, et al., 2005; Smith, 2000)
There are few studies attempted to see learners’ conception about LSA qualitatively, and
these did not include such a comprehensive patterns of conception as outlined above. In
some studies, the researchers just asked the participant to express how they feel about
assessing self or what is meant by self-assessment. Some studies used questionnaire-like
instrument in which students rate their feeling of being involved in assessment scheme
where as others used interviews. As a result, the findings of studies on this area are not
consistent and conclusive. For example, Chen (2005) asked learners’ feelings about doing
assessment of their public speaking skill and reported that learners perceived it positively.
Conversely, Struyven, et al. (2003) reported Mires, Ben-David, Preece and Smiths’
(2001) study of the feasibility and reliability of LSA in which the learners expressed
negative feelings. The learners found the process stressful, tedious and time consuming.
They felt that the teachers were offloading their responsibility and were also uncertain
about their own marking.
55
A qualitative study of students’ conception of learning and self-assessment by Bourke
(2000) which asked the students to describe self-assessment indicated that learners’
conceptualisation of self-assessment ranges from the less sophisticated: the simplest type
of LSA, to the more sophisticated one: using own set criteria and measuring one’s own
performance. One more qualitative study was Tan’s (2007) study of academics
conception of learner self-assessment. In this study the academics identified three types
of self-assessment based on the purpose for which it serves: teacher-drive, program-
driven and future-driven. These studies besides being inconsistent in their method and
finding, the number of studies reported so far is few that the findings do not suffice for
generalisation.
2.3.3 Training Learners to Self-assess
One more issue related to LSA is concerning learners’ training. The skills of self-
assessment, as a higher cognitive skill do not occur as peripheral consequences of
language classroom instruction that aimed at developing the language skills and
competences. On the other hand, while conceptually, there is common general
assumption that training could improve knowledge and practice; this may or may not
work for LSA. Some authors (e.g., Brown and Knight, 1994; Tudor, 1996) have also
suggest that training is needed to help learners to improve their skills of assessing
themselves and overcome the influence of the traditional culture of assessment.
Nevertheless, this general assumption has not been unanimously supported by empirical
literature. For instance, Ross, Rolheiser and Hogaboam (1998) trained grade 4-6 students
in self-evaluation of their writing skills and reported that the treatment group became
more accurate in their self-evaluation and outperformed the control group in their
56
narrative writing, but with small effect size (ES = .18). Chen’s (2008) study in which
learners were trained in assessing their own English oral performance has also shown that
the training improved the students’ skill of self-assessment. However, Jafarpur and
Yamini (1995) in their study that aimed to determine if training learners in self- and peer
assessment improves language learners’ skills of assessing their own and peers’ language
ability concluded that the training had no any significant effect on the learners’ assessing
skills. In addition, in Ross & Starling (2005) study of effect of self-evaluation in grade 11
computer-supported geography classes indicated that while self-evaluation improved the
students achievement, it had negative effect on the students self-efficacy belief.
Generally, whereas there is a general assumption that behaviours can be modified or
changed through training, this is not uniformly exhibited in the studies of training
learners in self-assessment. Moreover, the number of studies cannot be considered
enough for conclusion that suggests the need of further study.
Like the empirical literature on effect of training students in self-assessment, there is
scarcity of both conceptual and empirical literature on teaching students to self-assess.
However, it is reasonable to draw on literature on learner training in general and second
language-learning strategy training in particular to deal with the major and general
concerns in learner training. The main concerns in language learning strategy training are
whether the training should be implicit or explicit, separate or integrated, how long it
should take, which model to follow and which material to use (O’Malley & Chamot,
1987; Oxford, 1990).
57
Regarding the issue of implicit/explicit and separate/integrated, these days there is more
agreement than disagreement among the scholars in the area that explicit training is more
effective than the implicit one in fostering the learners’ ability to use the strategy in
focus. However, there is less agreement on whether strategy instruction should be
integrated into or provided separate from the language courses (Chamot, 2004, 2005).
Chamot (2004), referring to the others’ work, explain the point of disagreement that
while many argue that integrated instruction provides students with opportunities to
practice learning strategies with authentic language learning tasks, others voice their
concerns that strategies learned in integration within a language class are less likely to
transfer to other tasks. Taking position, Chamot (2004) advises that “Given the current
state of knowledge about explicit and integrated learning strategy instruction, teachers
should certainly opt for explicit instruction and should probably integrate the instruction
into their regular course work, rather than providing a separate learning strategies course”
(P. 19). Harris (1997:18) also considers the problems of feasibility of implementing LSA
in terms of time and equipments in the busy schedules of language classrooms and
suggests, “The answer to this is the integration of self-assessment with everyday
classroom activities....”
Concerning the training model to be followed, the variants of CALLA (Cognitive
Academic Language Learning Approach) model that begins by awareness-raising
activities and working through to evaluation is favoured (Chamot 2004; Oxford, 1990).
Particular to LSA training, Ross and his colleagues, drawing on CALLA, developed a
four-staged model. The stages begin with awareness raising introductory activities and
work through i) involving students in defining assessment criteria (e.g., developing
58
rubrics that describe performance expectations in language meaningful to students); ii)
teaching students how to apply the criteria (e.g., model application of the rubric by
assessing examples of performance); iii) giving students feedback on their self-
assessments (e.g., engage students in evidence-based discussions of the differences
between their self-assessments and assessments by peers or the teacher); and (iv) helping
students use assessment data to develop action plans (e.g., find trends in performance and
identify short and long term strategies for overcoming weaknesses).
Ross and colleagues, for example, Ross, Rolheiser and Hogaboam-Gray (1999) and Ross
and Starling (2005) used the model and found it effective. Then, Rolheiser and Ross
(2007) proposed this model for training students in self-assessment.
Regarding duration of learning strategy training in general and LSA in particular, there is
no clear-cut idea. The scholars’ advice is that too short a period may mean unreliable
results and too long a time may result in a high dropout rate among learners; what appeals
to sense is it should be optimum (Chamot, 2005). Still, how long is optimum? Obviously,
answer to this question should come from empirical literature. Nevertheless, it was
necessary to draw on researchers’ experience for the purpose of this study. In Ross,
Rolheiser and Hogaboam-Gray (1999) and Ross and Starling (2005), the trainings lasted
for eight to twelve hours and, as reported here, the studies yielded positive results.
Therefore, the eight to twelve hours duration is taken as optimum.
To sum up, the common perception is that training and practice can help to improve the
students’ skills of self-assessment, but the research findings do not verify the expressed
perception. In addition, the issues related to how the training should be carried out are not
59
direct and clear as such. In fact, the literature does not provide any standardised model of
LSA training. Nevertheless, drawing on literature from language learning strategy
training, LSA training should be given explicitly and integrated into the normal language
learning program. The model and duration of training suggested and used by Ross and
colleagues can be a springboard for organising and conducting training in LSA.
Therefore, in this study explicit and integrated training was used following Rolheiser and
Ross (2007) model.
2.4 Assessment of Oral Performance
2.4.1 Performance Assessment
Performance assessment is one of the major features of alternative assessment tradition. It
was developed as a response to the dissatisfaction with the paper-pencil selective item
test format that does not allow the demonstration of the actual ability of accomplishing
certain task in a particular situation. Performance assessment is a form of assessment that
requires students to perform a task rather than select an answer from a ready-made list. It
involves students demonstrating what they can actually do rather than simply speculating
what they could or would do by completing test items that contains objective type
fragmented ideas. It requires students to apply what they know and can do in real-life
situations and demonstrate in integrative manner. This helps to make systematic
observation for gathering valid data required to make sound decisions (Brown, 2004;
Havnes and McDowell 2008; Pierson and Beck, 1998; Smits, Sluijsmans, and Jochems,
in Havnes and McDowell, 2008).
60
In the context of foreign language teaching where the language is detached from
authentic situations, performance assessment is not easy and efficient in planning,
designing, and administering. Nevertheless, the extra efforts pay off in the form of
validity, backwash effect, and transferability. Because students are assessed as they are
performing real world or simulated tasks that provide opportunity to demonstrate the
target constructs, performance assessments can be a more content and construct valid
indicator of students' knowledge and abilities. Performance assessment can also provide
impetus for improving instruction, and increase students' understanding of what they
need to know and be able to do. In addition, this can be transferred to the actual work
places (Brown, 2004; Sweet, 1993).
2.4.2 Assessing Oral Performance
Although the assessment of language skills, in principle, should be based on students’
performance, performance assessment is especially important in assessing oral skills.
This is because knowledge of the discrete element of the language counts for little unless
the user is able to combine them in a new and appropriate way to meet the demand of the
situation in which s/he wishes to use the language. Therefore, it is difficult to give fair
decision about someone’s oral ability based on paper-pencil test without observing
her/his performance. For example, there is a big difference between answering multiple-
choice questions on describing a place and describing the place orally.
Assessing oral performance is a difficult undertaking because the constructs in oral
proficiency by their nature are complex and multi-componential. Oral performance
entails the use of both linguistic and non-linguistic competences at a time online.
Therefore, assessing it requires careful planning of the aspect of the performance to be
61
assessed, the kind of task to be designed, the type of criteria to be used, and the way the
performance should be scored. The planning should also be based on the type and
purpose of the assessment and/or course/curriculum objectives.
2.4.2.1 Defining Constructs of oral Performance
There can be a number of reasons for assessing language in general and oral performance
in particular. It can be as general as when it is to describe the overall proficiency level for
making decision about placement or promotion; or it can be as specific as when the
intention is to promote the achievement of a particular curricular objective/s as a part of
the instructional process. In all cases, it is necessary to have a framework of reference to
explain what it means by the constructs assessed and how the scores are related to the
constructs (Luoma, 2004).
One way of establishing the relationship between the assessment result and the constructs
is relating the assessment to a model or several models of language competence that
is/are compatible with our purpose of assessing. Alternatively, usually when it comes to
learning-related assessment, the sample of language assessed should be related to the
general goal or specific objectives of the course or curriculum which themselves are most
likely to be based, explicitly or implicitly, on a theoretical view or an eclectic
combination of views about language competence. It is also possible to use the
combination of the two approaches; viewing the assessment constructs from the
perspective of model/s of language ability as well as the teaching goals (Luoma, 2004).
A number of theoretical models of language skills have been developed based on their
respective orientation about language and language learning. Luoma, (2004) presents the
62
summary of the models. Specific to speaking, a summary of oral skills that are useful in
classroom assessment is given by Bygate, (1987). The model provides interrelated multi-
dimensions and sub dimensions of knowledge and skills that are processed and executed
simultaneously, especially, during extensive oral performance. The model has got two
major dimensions: knowledge and skills. The knowledge dimension encompasses the
planning, selection and production sub dimensions that interplay with the skills
dimension that includes message planning, management skill, negotiation of meaning,
production skills and accuracy skills which themselves are realised through further
practical elements.
In the assessment of oral performance, these principal dimensions of the oral skills can be
captured under the notions of complexity, accuracy and fluency (CAF), which are
themselves multi-componential that encompass subcomponents. However, in the
literature, there do not seem unanimous agreement on the definition and interpretation of
these constructs. Even among language teachers and researchers, who are supposed to
share common concept of oral language skills, various definitions coexist (Chambers,
1997; Housen & Kuiken, 2009). Therefore, explicit definition is required to make clear
what is meant by the terms in relation to a specific context. In this study the most
comprehensive and consistent concepts of the constructs as perceived from the
perspective of the objectives of the course Spoken English II (EnLa 342) are given and
used.
Accuracy (correctness) is probably the most consistently defined construct. It refers to the
degree of congruence with the established linguistic norm. Deviance from the particular
norm is considered as error. Then, accuracy in oral performance is described as the extent
63
to which the production is error-free of linguistic elements like grammar vocabulary,
pronunciation etc. possibly reflecting higher levels of control in the linguistic skill as well
as a conservative orientation, that is, avoidance of challenging structures that might
provoke error. This seemingly straightforward description however, poses difficulty
when it comes to the decision of what is to be taken as correct and what is not, and why;
which is relatively subjective. For example, some usages that are acceptable in some
social context or community may not be in others (Ellis, 2009; Housen & Kuiken, 2009).
Fluency, in non-technical meaning, refers to the overall oral proficiency, as it is common
to say/hear that ‘Ms/Mrs/Mr. X is fluent in English’ to mean that the person has a native-
like performance. It refers to the extent of ease, smoothness, and eloquence with which
the speaker produces a string of extended discourse spontaneously. In the technical sense,
it is a multi-dimensional construct that include sub-dimensions like speed fluency (speech
rate), breakdown fluency (pause) and repair fluency (gap fillers, false starts, hesitations).
It is the speakers’ capacity to articulate speech without unintentional interruption and
delay and less attention to the form (Chambers, 1997; Housen & Kuiken, 2009; Luoma,
2004).
Complexity may be the most ambiguous and complex aspect of language performance.
As Housen & Kuiken (2009) summarised the use of the term in the SLA literature, it can
be used to refer both to properties of language task (task complexity) and to properties of
language performance (L2 complexity). L2 complexity in turn can be interpreted in at
least two different ways: as cognitive complexity and linguistic complexity. Again,
whereas cognitive complexity is defined from the perspective of the L2 learner/user,
linguistic complexity is defined from the perspective of the L2 features (items,
64
patterns, structures, rules) or subsystems (phonological, morphological, syntactic,
lexical).
In this research, the interest is the definition related to linguistic complexity that is
commonly interpreted as the size, elaborateness, richness, and diversity of the learner’s
L2 linguistic system. It refers to the capacity to use advanced language, i.e., use of
complex and variety of syntactic patterns and lexical elements (Ellis, 2009; Housen &
Kuiken, 2009).
In addition to these three major constructs that are indicators of the ability and skills of
manipulating mainly the linguistic elements, oral performance also requires the
performer’s use of different strategy of executing the performance. This includes
techniques of compensating for language gap, gaining and maintaining audiences’
attention throughout, overcoming psychological and physiological barriers, etc. that are
likely to affect the performance.
Thus, any assessment of extensive oral performance that involves the use of an amalgam
of the array of the linguistic and non-linguistic knowledge and skills necessitates the
inclusion of these four major constructs but with different degree of focus depending on
the purpose and nature of the assessment.
2.4.2.2 Measuring the Constructs of Oral Performance
The way that the construct of oral performance are measured is related to the main
purpose of measuring them. In research where the purpose of measuring the constructs
has to do something with trying to understand and explain issues related to oral skills,
there are different units of measurements to be used to quantify the constructs. For
65
example, fluency, as speech rate can be measured by counting the number of syllables or
words uttered per time unit; or as break down fluency, it can be measured by counting the
number and length of pauses in time unit.
Accuracy is usually measured by counting error-free T-units and dividing by the total
number of T-units. (Error-free T-units are main clauses and subordinate clauses attached
to or embedded in them that contained no grammatical, syntactic, lexical, or spelling
errors).
Complexity is measured in terms of the ratio or frequency of T-units, (clauses, verb
phrases and sentences), amount of embedding subordination and coordination,
range of structural types, and structural sophistication. That is the number of t-units
(lexical or syntactic) in a text divided by total words multiplied by 100 (Chambers, 1997;
Housen & Kuiken, 2009; mehrang, 2010).
However, these kinds of measurements are too technical to be used in learning-related
assessment where the assessment purpose is mainly fostering learning and checking the
achievement of the particular course objectives. Instead, the scholars in the area of
language testing, (e.g., Bachman & Palmer, 1996; Brown, 2003; Heaton, 1990; Hughes,
2003) propose three techniques of scoring language performances primarily developed
for assessing responsive and extensive writing performance, and later adapted for scoring
assessing oral performance. These techniques are error counting, impressionistic and
analytic scoring. Brown adds one more scoring technique, primary trait scoring.
In the error counting method, the marker counts the number of errors in the texts and
deduct from the total marks assigned to the work. However, it ignores the different
66
constructs and sub-skills and fails to take in to account of the kind of error to be or not to
be emphasised. The impression (holistic) method involves awarding a single whole mark
to students’ work based on the total impression of the production (written or oral) as a
whole. Though this method is good for efficiency for marking large number of
composition, it is vulnerable to the markers’ bias. It also ignores the aspects of error to be
focused on and it is difficult for students to get feedback.
In the primary traits scoring method, the scorer assign marks based on the effectiveness
of the text in achieving a narrowly defined purpose or function of language for example,
persuading, describing, or commenting. The scoring can be done on a four or five point
scale. The drawback of this method is it totally ignores other aspects of competences like
fluency, organization accuracy etc (Brown, 2003).
The analytic method involves the use of marking scheme that helps the marker to identify
the language aspects and skills the testee is required to demonstrated. It also guides on the
amount of mark to be given for each element. Therefore, it is relatively reliable than the
others and helps to give feedback to students on their performance. This technique is
usually preferred to the others because of its perceived relative reliability and validity.
The two practical advantages of the analytic scoring are:
First, it allows us to provide a ‘profile’ of the areas of language ability that are
rated. ... A second advantage is that analytic scales tend to reflect what raters
actually do when rating samples of language use.” Bachman & Palmer
(1996:211).
In this research, because of this perceived relative advantage, this latter method was used
to score the learners’ English oral performance.
67
2.4.2.3 Rating Scales for Oral Performance
In classroom assessment where one of the techniques of scoring discussed above is to be
used, it necessitates the use of a certain sort of rating scale. The scale can take a form of
plain numbers representing set of descriptors or verbal expression categories like
‘excellent’, ‘fair’ etc. in ascending series of levels. There are varieties of rating scales for
assessing speaking skills developed by different examination bodies for different
purposes. Some of the scales are holistic and others are analytic, (e.g., ACTFL & TSE)
some are for testing general proficiency and others are for task performance in specific
context (e.g., CEF), etc. (Luoma, 2004).
It may seem attractive to use one of the existing scales that are used by a formal
examination body. However, scales must always be related to the purpose of the test and
the definition of the construct to be assessed. This makes it difficult to find one that fits
exactly to the purpose of a specific classroom assessment. Besides, it is difficult to trust
that many existing scales have been developed through valid and reliable research
procedures. Therefore, Luoma, (2004:82) advises that “even if existing scales were used
as a basis, it would therefore be a good idea to modify them...” in such a way that they
serve the purpose for which they are to be used.
For the purpose of this research, The Associated Examining Board Test in English for
Educational Purpose: Assessment Criteria for the Oral Test (Weir, 1990) was adapted
based on the course objectives and specific assessment purpose. This was chosen because
the constructs and the descriptors of the constructs are more or less similar to the
elements of oral performance to be assessed in Spoken English II (Enla 243), the course
that was being taught and used for this study (see Section 3.4.4.2)
68
2.4.2.4 Tasks for Assessing Oral Performance
In the assessment context, tasks are simulated or real life activities that provide learners
with context and opportunity to exhibit their skills of using the language for certain
purpose in certain situation. To be able to capture the aspect of the language
competence/s in focus, an appropriate task whose performance necessarily trigger the
production and manipulation of the target constructs should be designed.
Designing tasks for assessing oral production skills needs considering a number of issues
in relation to the purpose of the assessment. These include the type of task (what the
assessee is asked to do), the format of the performance (individual, pair or group), and the
purpose of the language (academic or real-life simulation). It also needs considering
whether it is integrated or discrete, and issue of difficulty in terms of content (background
knowledge) and language level, length of time, etc. Readers can refer Luoma, (2004) for
comprehensive description of the types, formats, and purposes of speaking tasks.
There are various types of tasks among which choice can be made and adapted to the
level and purpose of the assessment. These task types range from imitative through
controlled interview to oral presentations. Brown (2003:141-2) gives a comprehensive
list of types of tasks into five inclusive categories.
Imitative (parroting back words, phrase or short sentences)
Intensive (direct response, reading aloud, sentence and dialogue
completion, simple picture cued tasks)
Responsive (greetings; short conversation, requests and comments; and
the like)
69
Interactive (extended transactional or/and interpersonal exchanges)
Extensive (speeches, oral presentations, storytelling, descriptions etc)
The categories are listed in their ascending order in terms of their length and complexity.
While the imitative tasks are used to assess the purely phonetic level of production some
lexical and grammatical elements can be incorporated in the assessment criteria. The
intensive type of speaking involves the production of short chunks to demonstrate
competences in lexical, phrasal, and grammatical relationship. The responsive tasks are
used to test competence in very short conversation guided by prompt stimulus. The
difference between the interactive tasks and the responsive ones is length and complexity.
The interactive tasks include multiple exchanges and/or participants. The extensive or
monologue is the task type where there is limited or no overt oral interaction from
listeners and requires demonstrating relatively higher level of competence.
Then the choice of the kind of task to be used in assessing oral performance depends on
the level of the assessee, the objective of the particular assessment and the construct
intended to be demonstrated. In this study, the last two types of tasks (interactive and
extensive oral performances) were used because the objectives of the course used require
the students to involve in extended oral interactions and presentations.
In this study, LSA of their English oral performance is informed by the forgoing
discussions about performance assessment in general and oral performance assessment.
The selection and defining of the construct, design of the tasks, and the measuring
instrument (marking scheme) are all based on these discussions.
70
2.5 Chapter Summary
To wind up this chapter attempt is made to position LSA in its theoretical and empirical
grounds. Humanism and constructivism are the educational theories that support LSA. It
has been shown that LSA enhances learning by affecting the cognitive, social and
emotional aspects of the learners’ learning. The empirical evidences have also supported
the theoretical propositions about beneficial effects LSA on students’ learning. LSA is
described and categorised in terms of the extent it permits students’’ involvement in the
assessment scheme. The issues of its validity, conception held by learners and others, and
ways of training learners have been reviewed. The literature indicates that issues
surrounding its application need further consideration. It has also been pointed out that
oral performance assessment is more appropriate than the paper-pencil discrete items.
Designing oral performance assessment needs to consider the construct to focus on and
the way it is captured. Finally, the preference of performance-based LSA format to the
global one and analytic marking to the other techniques is indicated. The type and format
of task used is also justified. The next chapter deals with the methodology of the study.
71
3 Chapter Three
Methodology
3.0 Introduction
This chapter deals with the methodology of the study. First, it describes the approach
used in this study. It aliens this study with the mixed-methods approach, provides
justification for the choice of the approach, and explains how the quantitative and the
qualitative methods are mixed. Next, it portrays the design of the study, the instruments
and their development, the data gathering procedures, and data analysis techniques
employed. Lastly it overviews the lessons learned from piloting the instruments and the
methods. It finishes by providing the summary of the chapter
3.1 Approach to the Study
An approach to research is informed by the researchers’ worldview-- assumptions and
belief about knowledge and reality. Drawing on pragmatic research paradigm, the
concurrent mixed-methods design was adapted in this research. This design was opted for
to expand understanding by looking into different aspects of the phenomenon and
corroborating the data through triangulation. This study tried to understand LSA from
three dimensions. It explored conceptions of LSA held by the instructors and the learners.
It also assessed the validity of learners’ assessment of their own English oral performance
and the effect of training on the existing conceptions and practice (cause and effect
relationship).
Accessing conceptions involves capturing subjective meanings, opinions, and feelings
individuals attach to phenomena. Although it was possible to use quantitative approach
72
through questionnaire, the fact that the question items need to be simple and
straightforward limits the depth of the data. Getting to such behaviour required
qualitative methods such as interviews and discussions that enable the researcher to probe
deep into such invisible behaviours (Cohen, Manion and Morrison, 2000; D rnyei, 2007).
The rest two constructs, i.e., the validity of learners’ LSA and effect of training on the
validity and conceptions, required data from the students’ self-marking and the
instructors’ marking; and test-retest procedure of the quantitative method and, obviously,
generated quantitative data that lend themselves to quantitative analysis. In addition,
mixing the methods also had the purpose of corroborating the qualitative data by
quantitative data. Therefore, the compelling reason for adapting mixed methods approach
is the need to address the different components of the research with different nature of the
constructs to be studied and the perceived need of validating the qualitative data through
triangulation.
3.2 Design of the Study
As mentioned above, this study involved both qualitative and quantitative methods. For
the qualitative aspect, interviewing strategy was used and the quantitative aspect followed
the single-group quasi-experimental design. In addition, a survey questionnaire was
employed as a means of method triangulation. These three strategies are described
below.
3.2.1 Interviewing
Interviewing is the common communication strategy that enables participants
(interviewers and interviewees) to discuss their interpretations of the world in which they
73
live, and to express how they regard situations from their own point of view. It is
particularly important means of accessing the perceptions, beliefs, and meanings the
participants attach to phenomena. Therefore, it is suitable and most commonly used
strategy to study the conceptions held by participants about their teaching and learning
practices (Cohen, et al., 2000; D rnyei, 2007; Berg, 2001). However, as it involves small
number of subjects and is susceptible to subjectivity because of its qualitative nature,
there is threat to validity. Nevertheless, this weakness was complemented by
corroborating the interview by questionnaire. In addition, care was taken by including as
many subjects as possible and avoiding personal judgement during data analysis.
In this study group and one-to-one format interview was used as the main research
strategy to collect data on the students’ and the instructors’ conception of LSA. The
group interview format was used with the student subjects and the individual format was
used with the instructors to respect the preference of both groups of subjects: the students
preferred the group interview and the instructors preferred the individual interview
format. Both individual and group forms of interviewing share the same features,
advantages, and disadvantages, but group interview, as it involves small group (usually
6/7-10/12), is more advantageous than the individual interview in that it is often
timesaving and involves little interruption. One more advantage of this interview format
is that it might also be less intimidating for the participants than individual interviews
when the group members are of similar characteristics like educational level and age
group and have acquaintance.
74
3.2.2 Single-group quasi-Experiment
The single-group variant of quasi-experimental design was adapted to study whether or
not training learners in self-assessment affects learners’ skills of assessing their own oral
performance and their conception of LSA. This design was opted for because of its
perceived advantages over the true experimental design. As it employs only one group
(the experiment group), it enables to overcome the practical constraints to do true
experiment in educational setting in general and classroom research in particular where
there are a number of serious limitations on the researcher to manipulate the situations.
First, it enables to bypass the inevitable language program administrators’, teachers’, and
students resistance “to disturb their ongoing programs and allow reorganization of classes
in order to randomize the assignment of subjects to different experimental groups”
(Seliger and Shohamy, 1989:148). Secondly, it enables to eliminate the between-subjects
extraneous variables that would be one of the potential threats to internal validity even in
randomly assigned groups experiment. Thirdly, it avoids the resentment and
demoralisation by one of the groups that could happen when it is perceived that the
treatment of one group brings an advantage or disadvantage. In such cases, the subjects
try hard to change their behaviour to compensate the perceived advantage or
disadvantage (Marczyk, DeMatteo and Festinger, 2005), which is another potential threat
to internal validity in experimental design. Lastly, it also helps to overcome the threat to
validity because of diffusion of information by communication between/among groups or
subjects.
One criticism about the single-group experiment is doubt about its internal validity, i.e.,
how confidently the change in the dependent variable can be attributed to the
75
intervention. However, while the true experiments rely on randomization to establish
control of extraneous variables, the single-group experiments eliminate between-subjects
variables by using only one group. In addition, relevant environmental factors are
controlled by establishing a stable baseline of the dependent variable by taking repeated
measures of the dependent variable (Marczyk, DeMatteo & Festinger 2005). In this
study, though the literature does not provide any information about how many times it
should be repeated, it was planned to take the measures of the dependent variables at least
thrice because it is hoped that this enables to see whether the baseline is stable.
3.2.3 Questionnaire Survey
Questionnaire survey can be used to gather data on biographic and demographic
characteristics, behavioural data like what the respondents do or have done; and
psychological constructs like beliefs, opinions, perceptions, understandings, interests, and
values. Questionnaire can serve as main instrument by itself or come in as a
supplementary one (D rnyei 2007). In this study, questionnaire survey was used collect
data on the subjects’ conception of LSA. It had the function of validating data through
triangulation; it came in to corroborate the qualitative interview because, first, it lends
itself for objective analysis, and second, it involves larger number of respondents. This
helps to maintain the validity and reliability of the research findings.
3.3 Population and Subjects of the Study
This study was carried out in English program of Wollega University. It could have been
done in any of Ethiopian universities EFL classrooms. However, for the reason that has to
76
do with issue of convenience and the intention of expanding research tradition to the
recently established government universities, Wollega University was chosen.
Wollega University is one of the recently established government universities located in
Nekemte, East Wollega zone. The English program of the university is one of the
departments set up in 2006 G.C when the university became functional. During the data
collection of this study, the English program has got 31 staff members of which 12 were
on study leave.
The main reason for the choice of Wollega University, among many Ethiopian
universities, has to do with convenience. The researcher is a staff of the English
department as of June 2007. It was hoped that this would ease administrative procedures
and would help to win the goodwill and cooperation of the English language program
administrators and instructors who would have been generally, as Seliger and Shohamy
(1989) observe, usually very reluctant to allow an intruder into their programme; though,
as Cohen et al. (2000) warn, such cooperation cannot be taken for granted.
The researcher’s connection to the English department was important for the qualitative
aspect of the study that needs the researcher to get as close as possible to the participants
being studied to understand better the subjects’ perceptions (Creswell, 2007). It also gave
the researcher, as Ashcroft (1996) remarks, an insider perspective that helped to
effectively explore the real classroom situation, understand better, and come up with
more valid result. The secondary reason was that as it is a recently established university
and most of the academic staffs were new graduates of MA and BA Degree, carrying out
77
a research activity in the department helps to cultivate research tradition in the university
in general and English program in particular.
The population for this study were EFL students taking the Spoken English II (Enla 342)
and EFL instructors in the English program of Wollega University. These students were
chosen because of the relevance of the course they were taking by then for gathering
appropriate data on learners’ English oral performance. The English program of Wollega
university offers two speaking skills courses: Spoken English I (EnLa 341) and Spoken
English II (EnLa 342), for two consecutive semesters to second year students. The first
course mainly focuses on providing the students with theoretical knowledge about
speaking and spoken language and linguistic elements of English. It describes the nature
and functions of spoken language. It also gives an overview of English sound system and
pronunciation of some irregular forms. Then it involves the students in practicing
everyday spoken English like self-introduction, greeting and parting, giving instruction,
etc. The second course is an extension of the first one. It focuses on developing the
students’ oral communicative skills in formal and academic contexts. It involves the
students in practicing to produce extended speech in different occasions like public
speech, debate, etc. The assessment of students’ achievement also involves a little
proportion of written formats and more of oral task performances continuously in the
forms of individual, pair and group works both in classrooms and out of class
assignments.
The second year classes of students taking Spoken English II (EnLa 342), during the time
of data collection were preferred to the first one. This is because the assessments on the
course required the students to engage in extended academic oral performance rather than
78
the production of short chunks of everyday English that require only the common high
frequent vocabulary and simple grammatical structures. This fits the rationale of this
study and provides appropriate context to generate relevant data on the learners’
relatively extensive English oral performance.
For the quantitative part of the study, all the available number of the students and the
instructors was used because the size was manageable; there were a class of forty-six
students and nineteen instructors in the English program during the time of data
collection.
For the qualitative aspect that involved interview, the student subjects were purposively
sampled. In the purposive sampling, it was attempted to include students from the three
ability groups, from both sex and from different parts of the country because these traits,
i.e., achievement, gender and cultural grounds are likely to affect individuals’ conception
of self-assessment. Following Cohen, et al.’s (2000) and D rnyei’s (2007) suggestions,
nine students (two for safety margin), who volunteered to participate in the interview
were selected. The group included three female and six male students, two each from
Oromia, Amhara and Addis Ababa, and one each from Tigray, Debub and Benishangul
regions; there were no students from the rest regions. The five instructors for the
individual interview were recruited on the availability basis because only six out of the 19
instructors were willing to be interviewed.
3.4 Instruments
As discussed in Chapter one (Section 1.5), the objective of this study was exploring the
conceptions held by students and instructors, assessing the extent of validity of LSA of
79
English oral performance and effect of training on the conceptions and the validity of the
self-assessment in university EFL classrooms. To achieve these objectives, following the
research strategies discussed above, unstructured interviews, English oral performance
assessments tools, and questionnaires were, used to gather three sets of data.
The interviews were used to obtain the set of data pertaining to conceptions and English
oral performance assessments tools were employed to generate data on the validity of
LSA of English oral performance. The questionnaires were used to collect supplementary
data on conceptions. In addition, training materials were also used to train students on
LSA. The correspondence between the behaviours studied and their corresponding
instruments for data collection is summarised in Table 2 below. The subsequent sections
describe development and validation of these instruments.
Table 2 Correspondence between the behaviours studied and the instruments
employed
Note: The double tick (√√) means that the instrument was used as the main one, and the
single tick (√) shows it was supplementary to the major one.
The constructs (behaviours)
Res
earch
Qu
esti
on
s
instruments
interview Questionnaire English Oral
performance
assessment
Instructors’ conception of
LSA
RQ1A √√ √
Learners’ conception of LSA RQ1B √√ √
Extent of validity of LSA
of learners’ SA
RQ2 √√
Effect of intervention RQ3A √√
RQ3B
√√ √
80
3.4.1 The Interview Schedule
In this study, it was opted to use the unstructured interview because it allows the
interviewer to be able to restructure and reformulate the questions in the course of the
interview as required to gather as large amount of information as possible by probing
deeply and flexibly. Therefore, unstructured guiding questions were designed and used
for both groups of respondents.
The guiding questions for this study were drawn from both the conceptual and empirical
literature on assessment in general and LSA in particular (e.g., Brown 2004; Brown &
Knight, 1994; Brown with Bull and Pendlebury, 1997; Lewkowicz & Moon, 1985; Black,
1999). In the procedure of developing the guiding questions, first broader categories of
perceptions, knowledge, beliefs, and opinions that constitute conceptions about
assessment in general and LSA in particular were pooled from the literature. The
categories were built around six themes: the main purpose of classroom assessment, the
responsible body for classroom assessment, the concept of LSA, appropriateness of LSA,
its significance, and its validity.
The initial idea of the themes was taken from Brown’s (2004) COA-III (Conception of
Assessment) questionnaire originally developed to study teachers’ conception of
assessment and later adapted for studies of students’ conception of assessment (Brown &
Hirschfeld, 2008). Reason for depending on this source is that as assessment and LSA,
and teachers and students, respectively share many behaviours in educational setting.
Then based on these initial themes, twelve open-ended questions were set as a first draft.
Next, the questions were given to two experienced teacher educators and PhD candidates
81
(one in TEFL and the other in ALCM) for comment. In addition, discussions were held
with the teacher educators about the comments they gave. The comments indicated that
some questions were redundant and some others need rephrasing of words and phrases
for clarity. Then considering the comments of the educators, the questions were revised
and reduced to seven in the final draft.
Next, as a form of pre piloting, the questions were used with a group of three students
and two instructors to explore how well the questions were clear, relevant and helpful to
generate sufficient data on the themes, and to see how long it takes. Moreover, it was also
intended to derive questions for the questionnaire items. After the interviews, discussions
were held with the students and the instructors to check whether they really understood
the language of the question regardless of the concept. These preliminary trial interviews
and the informal discussions helped to identify the questions that needed more
clarification and details to trigger the appropriate responses. Then, the clarifications were
done and the interview schedule was made ready to be piloted.
3.4.2 The Questionnaire
The questionnaire had the function of validating data through triangulation (D rnyei,
2007); it came in to supplement the unstructured interviews. It was used to collect
quantitative data on the students’ and the instructors’ conception of LSA. Two close-
ended multi-item Likert type 5-point scales questionnaires, for the instructors and the
students, were used. The 5 points scale range from 1: Strongly Agree (SA), 2: Agree (A),
3: Undecided (UD), 4: Disagree (DA) and 5: Strongly Disagree (SDA).
82
In the literature, there is lack of a validated questionnaire that helps to elicit conception
about LSA as comprehensively as done in this one. This necessitated the use of self-
developed questionnaire. Therefore, as with the interview guiding questions, the
questionnaires were developed by the researcher. Two questionnaires, one for the
students and one for the instructors containing 29 and 32 items respectively were
developed.
The questionnaires contain six thematic categories of which the three categories were
subcategorised in to two each. The themes for categorisation were the same with that of
the interview questions, and as done with interview, the items and the thematic categories
were drawn from the conceptual and empirical literature. Here also COA-III (conception
of Assessment) was used as starting point, but fundamentally improved. The themes for
both respondent groups are the same, but the wording of the question items differ so that
they fit to the level of each group of respondents.
The multi-item scale format (thematically clustering of items) is preferred to minimise
the effect of wording of a single item on the respondents’ response. As D rnyei (2007)
indicates, a minor difference in wording of an item can cause considerable difference in
respondents’ response. The use of two or more similar question items helps to minimise
this effect; the assumption is that when the item scores for the similar questions are
summed, “any idiosyncratic interpretation of any item will be averaged” (P.92). It also
helps to test the validity of the instrument by measuring the internal consistence of the
items.
83
To minimise the problem of response set in which respondents tend to respond in a sort
of pattern regardless of the content of the item, or tend to simply repeat previous
response without careful reading (Marczyk, DeMatteo and Festinger, 2005; D rnyei,
2007), the items are mixed up and some items are negatively worded. For analysis, the
item clusters were retained and the negatively worded items were reversed.
In the procedure of developing the questionnaire, first 39 items were pooled from
thorough review of the literature and the pre piloting interview (see Section 3.4.1 above).
Next, the drafted questions were given to two teacher educators; one PhD holder and the
other MA in teacher education, who are experienced in giving research courses to
undergraduate students. They commented and discussions were held with them. Then
considering their comments, the items were reduced to thirty-five and some items were
rephrased to make the language simple. It was also felt necessary that some of the items
should be stated differently for the instructors and the students, and three items should be
removed from the version for students. Consequently, two versions of the questionnaire,
one for the instructors (35 items) and one for the students (32 items) were produced as
final drafts. Then the items were checked again against the guideline for writing
questionnaire items in the literature, for example, D rnyei (2007) and Dawson (2002)
Then, the questionnaires were pre-piloted to test the reliability of the questionnaires, see
whether the language and the direction were clear enough for the respondents and how
long it takes to fill it out. The students’ questionnaire was distributed to fifteen first year
English majoring students to fill it out in a classroom. The students were informed that
they could ask for any clarification if they could not understand the instruction and the
items. While they were filling out, the items on which many students need clarification
84
and the problems were noted. Later, these items were revised and simplified. The
students took about twenty-one to twenty-five minutes to complete the questionnaire.
This implied that the questionnaire is not too long to be boring.
The reliability of the questionnaire was tested by checking internal consistency of the
items for each major and sub cluster of the items using Cronbach Alpha coefficient. The
Cronbach Alpha coefficient was computed using IBM SPSS statistics 20 and found to be
>=0.62 for each cluster which indicates reliable consistency for 3-4 number of item
scales (D rnyei, 2007). The analysis of the internal consistency also indicated three of the
items in both versions tend to affect the internal consistency of the items, i.e., their value
in the column labelled sig 2-tailed is greater than the Alpha coefficient of the scale;
which means they tend to measure different construct from the other items in the cluster.
Therefore, these three items were dropped from both versions, so the students’ version
contains twenty-nine items and that of instructors’ contain thirty-two items. Then the
questionnaires qualified for the piloting.
3.4.3 The Training Materials
The single-group quasi-experiment involved the students scoring of their own oral
performance before and after intervention, and an intervention by training the students to
assess their English oral performance.
Neither the materials used for teaching the Spoken English II (Enla 342) nor any other
course in the department contained a content related to teaching the learners to self-
assess. There was also no any material readily available in the literature to pick up and
use or adapt for training students in self-assessment. Consequently, the materials for
85
training the students on LSA were developed by the researcher. To develop these
materials the researcher drew mainly on the literature on learner training in general, for
example, Oxford (1990, 1994), Chamot (2005), O’Malley & Chamot, (1987); and Ross
(2006) for LSA in particular. The researcher also used his experience of writing materials
for teaching English language skills.
The materials used for training the students to self-assess included two booklets:
students’ material and instructors’ guide. The students’ material was organised in such a
way that it provides input on the content ideas of SA and tasks to engage students in
individual, pair, and group activities. The instructor’s material provides suggestions on
how to help the students through their material (appendices D and E). The basic outline
of the contents were derived from Rolheiser and Ross’s (2007) model (see Section 2.3.3)
and enriched with ideas from the literature. Both the students’ material and the
instructors’ guide were organized in to an introductory part and five sections in the body.
The introductory part gives a general overview of the need for training on LSA and the
general objectives and learning outcomes of the training. In the body, each section states
its specific objectives, and divided in to Activities and Tasks that engage students in
discussions, reading, responding and reflecting on tasks that led to the achievement of the
objectives stated. The sections also provide content information on some topics where it
seems necessary to help the learners and the instructor.
The first section introduces the training and clarifies how to go about it. It also deals with
issues related to purpose of classroom assessment and who should do it. Then raises issue
related to involving learners in assessment, description of LSA, its advantage,
86
disadvantage, etc. The second section focuses on developing students’ knowledge and
skills of linguistic and non-linguistic elements of English oral performance. The third and
fourth sections focus on developing students’ assessment skills, i.e., developing criteria,
scoring performances using the criteria, and giving and using feedback. The last section
provides activities for reflection and action. It guides the students how to benefit from
their self-assessment by practicing setting goals and developing action plans to improve
one’s own learning.
In the procedure of developing and validating the materials, first, the materials were
drafted and developed by the researcher. After both materials were fully developed, they
were given to two experienced material writers who have experience of writing distance
materials and course modules to get comments and suggestions. Then series of
discussions were held with the persons about their comment to get more complete and
detailed suggestions. Then their comments and suggestions were considered to improve
the materials.
Next, to see the extent the materials fit the target population, two copies of the students’
material were given to second year EFL majoring students, and one copy was given to an
instructor to get comment on the content, structure, and language of the material. Based
on the students’ and the instructor’s reflections, the content, language, and organization
of the materials were revised and then the pilot training was carried out.
At the end of the pilot training, the students and the instructor were asked to evaluate the
material in terms of the content (relevance, interesting, coverage of tasks and activities)
87
language (clarity), organisation (logical sequence, flow) and time allotted. Finally, the
materials were revised based on the comments of the teacher and the students.
3.4.4 The Oral Performance Assessment Tools
The English oral performance assessment was used to gather data on the extent of
validity of LSA of English oral performance. The tools used to generate this data were
tasks for oral performance in classroom and marking schemes.
3.4.4.1 The Tasks
Three oral performance tasks, two for pre-intervention and one for post-intervention
stage, were prepared by the researcher in collaboration with the course instructor based
on the content and objectives of the course Spoken English II (EnLa 342). The two tasks
for the pre-intervention were description (individual) and debating (in pair) and the one
for the post-intervention phases was public speech. These were selected because they
were the tasks that the instructor had planned to carry out continuous classroom
assessment by then.
Both the pre- and post training assessments were given as assignment so that the students
get time to prepare. They also had similar objectives, but with different weight given to
the different components of the assessment. They were aimed to assess the students’
ability to organise the contents logically and present it coherently, ability of using
appropriate vocabulary and expressions, their fluency and accuracy, and their stage
management techniques. Seven topics on which the students debate in pair for about 4-5
minutes and other seven topics on which they talk for 3-5 minutes were chosen by the
researcher discussing with the instructor. The topics were on the learners’ common
experiences like my dorm, my home village/town, the good person I know, how can we
88
save our forest?, What should farmers do to improve their production?, How can we
reduce air pollution?, Advantages and problems of high population. This is because it
minimises the problem of task difficulty caused not by language competence but lack of
background or content information.
3.4.4.2 The Marking Scheme
To score the performances, analytic marking scheme was adapted from The Associated
Examining Board Test in English for Educational Purpose: Assessment Criteria for the
Oral Test (Weir, 1990, 2005). The analytic method is preferred for its relative reliability
for such subjective scoring as compared to the other methods (see Section 2.4.2.3).
The assessment components, the respective weighting assigned to each component, and
the descriptors were discussed and decided on with the course instructor and the students
based on the objectives of the particular assessment, for each of the performances before
and after the training. The marking scheme included the constructs (components of the
assessment), the weighting assigned to each component, scales for each component and
descriptors for each scale and the mark given for each descriptor. The components
include content, focusing on relevance to the topic and adequacy; organisation, focusing
on the logical sequence of contents and coherence of ideas; use of language, grammar
and vocabulary, fluency and intelligibility; and strategy of delivery (see the detail in
Appendix 9).
3.5 Data Gathering Procedure
As the study involved experimental design, data were gathered at the pre-intervention and
post-intervention stages. As discussed in Section 3.2.2, it was planned to take the
89
measures of the variables thrice prior to the intervention to establish stable baseline
thereby ensure internal validity of the study. However, because of the time constraint
during the data collection, i.e., the students and their instructor complained that the
project was taking too much of their time in which they need to work on other courses,
these data were collected twice. The analysis of these data showed little variation
between the results of first round and the second round, indicating the stability of the
variables before the intervention. Therefore, there was no compelling reason for taking
additional measure of the variables. Besides, the missing of the third round data was
compensated by, collecting and analysing data on these variables after about 10 months
from similar non-treatment group of students.
3.5.1 The Pre-intervention Stage
Prior to the intervention data were collected on the two variables under study: validity of
LSA and the subjects’ conception of LSA to establish the baseline. First, the first round
quantitative and qualitative data on the students’ conception (CD1) was gathered using
the questionnaires and the interview consecutively. Next, data on validity of LSA was
collected through the oral performance assessments twice (VD1 & VD2) in an interval of
a week. Then the second round data on conception (CD2) was collected. The reason for
such sequence was to see if the practice of self-assessment by itself and/or maturation
because of time laps changes the conception of the students.
The students’ questionnaires, in both rounds, were administered in a classroom. This is
preferred to minimise the missing of the questionnaire paper and be able to clear if there
were any difficulties, considering the students below-level English language ability, in
understanding the instruction and/or the items. In classroom, they were briefed on the
90
purpose of the questionnaire and were informed the importance of their genuine
responses. They were also informed that they should do it on their own. Then they were
allowed as much time as they need to fill out it.
The group interview with students was conducted with a group of eight students in two
rounds. Though the language of the interview was supposed to be English, the
experiences from the pilot study showed that some of the students had serious difficulty
of using the language. Hence, the students were informed that they could use their mother
tongue or any language with which they are comfortable. With the consent of the
interviewees, the interview was videoed and audio recorded so that it could be transcribed
and analysed carefully later.
The two oral performance assessments to determine the baseline of the extent of validity
of students’ self-assessment were administered after the first round questionnaire and
interview. The first oral performance was in a form of debate in pair and the second one
was public speech on a topic of choice (see Section 3.4.4.1).
In the procedure of administering the first oral performance assessments in the classroom
first, the students were briefed about the task and the topic on which they speak were put
on the board so that everybody chooses one. Then they were informed to get ready for the
next period to talk for 3-5 minutes.
In the procedure of the second oral performance, the students were put into random pairs
and the task was explained. Then the topics on which they are required to debate were
written on the board and the pairs were allowed to choose one of the topics. Then they
were told to get ready for the next period to debate for 4-5 minutes
91
During the performances of the assessment tasks, all the performances were videoed with
the consent of the learners so that the instructors as well as the students could score the
performances by playing the video. It can be assumed that these days the students both
from town and rural areas are accustomed to being videoed. However, to be confident
that videoing in classroom did not affect the students’ performance, the students were
videoed in three sessions prior to the sessions of data gathering while they were carrying
out different activities so that they are used to it and would not be aware of it during the
sessions of data collecting.
After each oral performance, the marking criteria were discussed in the classroom and the
marking schemes were developed by the researcher in collaboration with the instructor
based on Assessment Criteria for the Oral Test adapted from Weir (1990). Then the
students were provided with copies of the marking scheme and they scored their own
performance by watching the videotape. These same performances were also scored by
instructor so that the baseline of validity of the LSA could be determined by comparing
scores from the learner’s and the instructor’s scoring (see Section 2.3.1.2 for ways of
checking validity of LSA).
To ensure the reliability of the instructor’s scoring, inter-scorers’ reliability was checked
by computing Alpha coefficient. First, two experienced instructors, the course instructor
and another one, scored ten performances independently. The Alpha coefficient was
computed using the IBM SPSS statistics 20 and found to be 0.893 indicating reliable
consistency between the two scorers. In addition the instructors discussed on some points
where their scoring varies considerably and came to agreement. Then the course
instructor marked all the performances.
92
To tap quantitative data on the instructors’ conception of LSA, the instructors’
questionnaire was distributed to the nineteen instructors face to face. All of them but one
instructor filled out and returned the papers. Then individual interview was held with five
instructors individually to gather the qualitative data. Here, it should be noted that the
instructors were interviewed individually because during the trial of the questions and the
piloting it was observed that the instructors expressed their discomfort with being
interviewed in a group, and the individual interview was successful during the pilot
study. As done during the students’ interview, these were also videoed and audio
recorded with the consent of each instructor.
3.5.2 The Intervention (Training)
After the data for demarcating the baselines of the students’ conception and the extent of
validity of LSA were taped, the next step was training the students to self-assess English
oral performance to see how intervention affects the existing behaviours. A 12hrs
integrated and explicit four-stage model training on LSA of English oral performance
was conducted for six weeks (2hrs/week). The training was carried out by the course
instructor; the researcher comes in for assistance when it was necessary, for example to
clarify concept.
The training was based on the materials developed for the training (see Section 3.4.3).
Prior to the training, the materials were introduced and explained to the instructor. The
instructor also got enough time to read by himself and anything unclear was clarified
through discussion. Then it was discussed and agreed on to devote 2 hours every week
for the training under a topic assessing your speaking skills. The student material was
distributed to every student; at least a copy for two.
93
In the training, the instructor followed the active learning approach. During the training
sessions, the students were mainly engaged in doing the tasks and activities in their
material individually, in pair, in small groups or the whole class discussion depending on
the guidance given in each activity. The role of the instructor was mainly organising,
facilitating, guiding, and helping the students.
The sessions usually begin with a brief warm up activity like, jokes or general questions
related to the training. This is followed by reflection on the previous session unfinished
tasks or assignments. Next, the instructor briefs and guides through the objectives of the
session and asks students if they want to add or omit any of the objectives. Then the
instructor directs the students to carry out the tasks and activities related to the topics in
their material according to the respective instructions. The instructor also may help the
students on how to put themselves in to groups or pairs where it is necessary. Then he
facilitates the accomplishment of the tasks and activities by, for example, giving initial
input, supervising and helping group or pair discussions etc. The sessions usually finish
by whole class discussion in which individuals, pairs or groups report what they have
done, comment on others work and get feedback from the class. The instructor
summarises the whole thing.
Forty-six students attended the training throughout and two students’ attendance was less
than 50% of the training sessions. At the end of the training, a test was prepared and
administered to the students to see the learning outcome of the training and to assess the
extent to which the objectives of the training were attained. The result of the students
ranges from 42% to 86% implying the effectiveness of the training. However, four
94
students’ result, including the two absentees, was below 35%. Therefore, the post-
training oral performances of these students were discarded during the analysis.
3.5.3 The post-intervention Stage
After the training, to determine whether the training had any effect on the two constructs,
validity and conceptions about LSA, the steps followed at the pre-intervention phase
were repeated. To collect data on the validity of LSA after the intervention, the third
assessment task (see Section 3.4.4.1) was administered in the same procedure as the
former ones. The students were given the topics to choose before hand and had time to
get ready at home. Then the learners and the instructors scored the performance. To
gather data on the learners’ conception of LSA after the training, the group interview and
the questionnaires used at the pre-intervention phase were repeated in the same way done
before.
As mentioned in Section 3.5.1, one more set of data collected after the post intervention
was the data gathered from students through interview and oral performance assessment
to check the internal validity of the finding from the single-group experiment. These data
were collected after the completion of the post-intervention data analysis to avoid any
doubt and confidently attribute the changes observed in the dependent variables to the
training, otherwise question the effect of the training. The interview was conducted with
a group of five volunteer students and the oral performance assessment was administered
to a class of 15 students in the same way as it was done during the pre-intervention stage.
95
3.6 Data Analyses Procedure
As mentioned earlier, the study employed instruments that generated both qualitative and
quantitative data. These data were analysed accordingly. This section describes the
procedures and techniques used in organising and analysing the data.
3.6.1 The Qualitative Data
To analyse the qualitative data from the interviews, qualitative content analysis technique was
used. Content analysis is the process of categorizing qualitative textual data into clusters of
similar entities, or conceptual categories, to identify consistent patterns and relationships between
variables or themes. This analytic method is a way of reducing data and making sense of them
and deriving meaning. It is a commonly used method of analyzing a wide range of textual
data, including interview transcripts, recorded observations. In qualitative content analysis the
general procedure is a generalized sequence of transcribing, coding for themes, looking for
patterns, and making interpretations (D rnyei, 2007; Given, 2008). This procedure was followed
in this study.
As the qualitative data were generated through interviews, the first step to the data
analysis was transcribing the audio and video-recorded interviews. Regarding ways of
transcription, D rnyei (2007) and Schilling (2006) suggests if the interest is analysing
the content meaning rather than linguistic elements, it is important to edit out the supra
segmental and other surface linguistic elements that provide little additional value. D rnyei
also advises the use of standard orthography to evoke the naturalness and readability of the text.
As the focus of this study was not analysing linguistic elements but content meaning,
considering Schilling’s and D rnyei’s suggestion, the interviews were transcribed using the
standard orthography to keep its readability. The paralinguistic features like hesitations,
96
pauses, repetitions, habitual expressions, etc. were edited out because observation from
repeated watching of the video indicated that these features do not affect the
interpretation of the data. The transcription included all the questions and responses and
the interviewer’s extended illustrations were transcribed only where it seemed important
for clarity. The learners’ wrong grammars were also ignored.
The interviews were transcribed by the researcher, but the reliability of the transcription
was checked first. To do this, a part of the students’ interview was transcribed
independently by the researcher and a colleague, a PhD student of ALCM at AAU. The
transcriptions were compared and found to have very little differences. In addition, the
audio and video records made it possible to crosscheck wherever there were problem of
audibility.
For validity of the transcript, printed copies of the completed transcript were given to the
students and the instructors so that they were able to comment if the responses they had
given were distorted or modified in the course of transcription.
The transcriptions then were coded by the researcher thematically using open code5,
computer software for coding textual data. As done with the transcription, the reliability
of the coding was also maintained; a part of the transcript was coded by the researcher
and the same colleague to see if there was any difference. There were few differences in
wording and those were discussed and improved. In the coding procedure, first as a pre-
coding step, the transcript was fed to the computer and then the text was reviewed
thoroughly and repeatedly to be familiar with any mention of ideas related in any ways to
the preconceived themes of the conceptions about LSA (see Section 2.3.2.3) (D rnyei,
97
2007; Yin, 2011). Next 1st level codes were assigned to instances or significant
statements, segments that carry units of meaning according to the idea they were intended
to mention. Then, the 1st level codes were put into categories, and where found necessary,
subcategories were formed. To determine the category to which an utterance is related,
the following indicators were considered:
the frequency with which it is mentioned in the text [ the number of times in
which a thought occurs in the text can be taken as an indicator of its significance
(May2001 in Blaxter, Hughes and Tight 2006)]
its position in the discourse: whether it is mentioned towards the
beginning or the end (this can also be related to the significance speakers
attach to their meaning of utterance and
the emphasis with which it is uttered
Then, the thematically coded categories were interpreted considering both the manifest
level and latent meanings (Berg, 2001; D rnyei, 2007) to catch the meanings of the
responses in relation to the categories of themes raised. Lastly, the meanings were
interpreted in relation to the categories.
3.6.2 The Quantitative Data
To analyse the two sets of quantitative data – the English oral performance scores and the
questionnaires- IBM SPSS statistics 20 was used. The data from the questionnaire and
from the oral performance assessment were first entered into the software window. To
avoid the missing and mistyping of data because of being busy going to and from paper,
key board and the screen, as suggested by D rnyei, a friend helped in taking turn in
98
dictating and keying data into the computer. Then, the data were cleaned and screened by
looking for outliers and missing data.
After the data were made ready, the data from the questionnaire were analysed and
interpreted using the descriptive statistics (the mean values and percentages). The
interpretations were done mainly from two angles: comparing the percentage of
agreement and disagreement for each statement and looking at the mean values of
responses of each statement and the corresponding scale used.
The data from the performance assessment was analysed by comparing the instructors’
scoring with that of the learners’. To do this, first descriptive statistics (mean and
standard deviation) were computed, and then to see if the mean differences were
significant, t-test of the inferential statistics were computed and the figures were
interpreted.
3.7 The Pilot Study
A pilot study is generally considered a small-scale trial of the proposed methods procedures
instruments and materials. The point of carrying out a pilot study is to try out to and revise the
methods and instruments. It helps to uncover any problems, and to address them before the main
study is carried out. It is also an important means of assessing the feasibility and usefulness of the
data collection methods and making any necessary revisions before they are used with the
research participants. This is particularly very important for novice researcher and for the self-
developed instruments as it is the case in this study (Mackey & Gass, 2005).
99
3.7.1 Summary of the Pilot Study
The pilot study was carried out after the instruments were fully developed. The primary
objective of the pilot study was to try out the instruments: questionnaires, interview
schedule, marking schemes, and training materials, developed to be used in the main
study and to check the feasibility of methods designed and to make necessary
amendments if necessary. It was also intended to help the researcher to gain experience to
tackle problems that are usually likely to encounter in educational research.
It was carried out in the same university where the main study was conducted and
involved similar group and less number of subjects. It involved a class of thirty-two
students and 10 instructors. The qualitative aspect involved a group of four students and
three instructors and the quantitative aspect participated all of the students and
instructors.
Data was collected by administering the questionnaires and carrying out interviews with
the group of students prior to and after intervention, and with the instructors individually.
The training was carried out for eight weeks, 1hr/week.
The qualitative data were analysed by transcribing coding and categorising the codes by
themes. The quantitative data were analysed using descriptive and inferential statistics
using SPSS v16.
3.7.2 Lesson Learnt from the Pilot Study
As indicated in the above section, the point in carrying out this pilot study was to check
the effectiveness of the methods, instruments, and materials before taking the course of
the full-scale research. Hence, important insights were gained from the piloting; the
100
course and the result of the piloting helped to detect the weaknesses and foresee some
unseen problems and make necessary amendments. The following lessons focus on the
aspects that needed modification and their implication for the main study.
One of the important observation concerns the language of the interview with the
students. The assumption before the piloting was that the second year EFL class would be
able to express their idea in English relatively clearly and comfortably. Nonetheless, the
piloting sessions of the interview indicated that the students had serious difficulty in
expressing themselves in English. Moreover, even when they were informed that they
could use any language they are comfortable with, many of the students in the interview
group insisted on using English even those who had considerable difficulty and were
poor to the extent they failed to produce any meaningful string of two or three sentences.
It was speculated that this problem could arise from anxiety caused by wrong perception
about the interviewer (the researcher); they could have thought that the interviewer
expect them to be perfect in English. To mitigate this problem, it was necessary that, in
addition to encouraging them to use any language they need, the interviewer should take
some time to develop friendly relationship with the target student population. This helped
to improve the situation; in the main study, the students tended to switch between English
and their native language to express their idea more clearly.
The second concern was the modification of the interview format with the instructors.
Initially the interview with both groups of respondents was intended to take the group
interview format. In practice, however, the instructors were very reluctant to be
interviewed in a group. During the piloting of the interview schedule, when asked for
101
cooperation to participate in the group interview, the instructors expressed their
willingness but preferred if it would be an individual interview. Then it was felt that if
individuals are pressed on to join the group, there could be a danger of over-reporting the
desirable behaviours and under-report the undesirable ones with an intention of being
politically correct. Therefore, to elicit the genuine behaviour and secure the validity of
the findings, it seems necessary to cope with the feeling and preference of the
respondents in the main study, as it was done in the piloting.
Another observation was regarding length of the interview with the students. It was
observed that students seemed to feel restless after about an hour-long attention.
Therefore, it was decided to limit the length of the group interview with the students to
about 45 to 50 minutes.
Two more observations from the pilot study were related to the training. The first one is
the timing and schedule. From the students’ and the instructor’s general comment on the
training (in addition to their comment on the material) and as it was observed during the
training, first, the 8 hours were not enough to cover the material, and second the 1hr/week
schedule was not convenient for the nature of the tasks and activities in the material. A
week-long gap between two training sessions seemed to cause forgetting of the previous
session point of discussions thus needed longer time devoted each period for extended
revisions. In addition, the 1 hour-long session was not suitable for nature of the training,
particularly for the practice phase. To solve these problems, 2-hours/week schedule was
used for six weeks for the main study.
102
The second thing related to the training was concerned with the part of the researcher in
the training. In the piloting, the researcher helped the instructor to get clear understanding
of the material and discussed on some issues related to presentation in the classroom, but
left the training totally to the instructor except for observations. However, the observation
of the training session and a few students’ comment show that it would be better if the
training was conducted jointly with the researcher, for example as a sort of team teaching
or the researcher being overt participant. In addition, it can be seen from the students,
results of the test on the training cannot be appreciated (the highest score was 79%). This
indicated that the training should be improved in any way. Therefore, during the main
study the researcher increased the frequency of observation and sometimes helped the
instructor in clarifying concepts and summarising whole class discussions, but refrained
from fully participating in the training to minimize researcher’s bias.
Lastly, the pilot study had suggested the sequence with which the qualitative and the
quantitative data on the students’ conceptions should be gathered. It was learned from the
experience of the piloting that if the interview is conducted prior to the administration of
the questionnaire, it can affect the awareness of the subjects i.e., the subjects could derive
certain understanding and modify their behaviour when they fill out the questionnaire and
consequently, the data from the questionnaire would be distorted. Therefore, the
questionnaire was administered first, and the interview followed during the main study.
To sum up, the observations from the piloting has helped to detect and deal with some
procedural and ethical aspects that could have been potential threats to the validity of the
findings of the study. Going through the process also booted the researchers’ experiences
in doing research and dealing with much inconveniency during the main study.
103
3.8 Chapter Summary
This chapter dealt with methodological issues of the study. It indicated that mixed
methods approach was adapted because of the nature of the constructs studied. Next, it
sketched the design of the study. It showed that interviewing, single-group quasi-
experiment, and questionnaire survey were the designs used in the study; and interview
schedule, questionnaires, and performance assessment tools were the instruments used to
collect data. The subjects were a classe of 2nd
year EFL students and the instructors in the
English program of Wollega University. They were sampled wholly for the quantitative
aspects, because the population size was manageable, and selected for the qualitative
aspect based on convenience.
It also described the nature and purpose of the instruments and the reason why they are
selected; and how they were developed by the researcher. Interview guiding questions
and performance tests were the main tools for gathering qualitative and quantitative data
respectively. Questionnaires were also used as a way of data triangulation. Training
intervention using training materials was also the part of the quasi-experimental design.
Lastly, it described the procedures followed in gathering and analysing the data. The
qualitative data were analysed following the narrative content analysis method, and the
quantitative data were analysed using the descriptive and inferential statistics. It finished
by summarising the pilot study and lesson learnt from the piloting. The following chapter
presents the data analysis and discussion.
104
4 Chapter Four
Data Presentation and Analysis
4.0 Introduction
The preceding chapter dealt with the description of the approach, design, strategies, and
instruments used in this study. It also described the procedures used for gathering and
analysing the data. This chapter presents the description and the analysis of the data. It
has two major parts. The first part the data are analysed along the three constructs
studied; the second part gives discussion and interpretation of the findings from the
analysis of the data.
4.1 Conceptions of LSA
As indicated in the preceding chapter, data on the instructors’ and the students’
conception of LSA was obtained through interviews and questionnaires from instructors
and students. These data are presented accordingly.
4.1.1 The Instructors’ Conception of LSA
4.1.1.1 Data from the Interview
The interview involved five instructors in the English program of Wollega University
referred to as InA, InB, InC InD and InE whenever necessary to keep the confidentiality
of the names of the instructor, The instructors had different level of qualification and
teaching experiences in tertiary level. One of them had Bed degree in English and one
and half year experience in teaching English in university. Two of them had MA degree
in TEFL and two and three years teaching experience in university, but had more than
105
four years in secondary schools. The rest two, one MA in TEFL and the other in
Linguistics had taught for four years in university and had had long years of experience
in teaching English and other subjects in different levels including primary schools and
TTCs. In addition, three of the instructors completed the HDP training that is given in
Ethiopian higher education institutions. By then, all of them were teaching at least two
English language courses in the university.
The transcripts of the five respondents’ response were combined for coding and analysed
as a whole to identify the conception that exist across the respondents. The transcript for
each instructor is appended as Appendix 2A, 2B, 2C, 2D and 2E for the 5 instructors
respectively. The results are presented thematically based on the six main themes that
constitute the conception of LSA, and around which the interview guiding questions were
built. These themes include the main purpose of classroom assessment, where the
responsibility of the assessment should resides, their understanding of the concept of
LSA, appropriateness of learners’ participation in the assessment process, its validity and
perceived merits and/or demerits. To make the interpretations clear, direct short and long
quotations and paraphrasing are used where deemed necessary.
4.1.1.1.1 Main Purpose of Classroom Assessment
The first theme is the main purpose for which the instructors prepare and carry out
classroom assessments. This theme is related to conception of LSA in that the
individuals’ understanding of and belief about the why of classroom assessment is
directly related to the way the person perceives students’ role in the assessment.
106
The instructors responded to questions like ‘what do you think is/are the main purpose of
classroom assessment?’ and why do you assess your students in your classroom?’ The
instructors identified different purposes that are put into seven first level categories of
purposes that again are put into two major categories containing two first levels each and
one major category containing three first levels as shown in Table 3 below. In the table,
the categories are put from top to down according to the priority they were given by the
respondents and each of the categories is described and illustrated by quoting instances
from the transcript.
Category 1: Auditing
In Table 3, auditing is assigned to the instances that explicitly or implicitly indicate that
classroom assessment is used as a tool for checking off whether or not learning has
happened. The two first level codes under this category are checking whether students
understood what they were taught and which of the objectives were achieved. All of the
instructors articulated either of these forms of checking students’ learning outcomes in
their first or second instance related to purpose of classroom assessment. In their
responses to the probes related to this theme they promptly and most frequently used
expressions like ‘to check how much students have learned’ ‘to check whether or not
students understood’, ‘to check whether or not objectives are achieved’.
In this category, the respondents gave priority to checking understanding. For instance,
the first respondent mentioned three purposes of classroom assessment: to check their
understanding, to put them into categories and to help them improve their learning.
107
Table 3 Category of Purposes of Classroom Assessment (by instructors)
2nd
level 1st level
Auditing
Checking understanding
Checking attainment of the course objective
Describing
Categorising students
Giving grade & reporting
Improving
Give feedback (teacher)
Adjust teaching methods (teacher)
Share idea (students)
When asked if he can put them in their order of priority, he said “for me the first one is
the first priority; just I assess to check whether they learnt or not what I taught them. This
is my prior purpose” (InA). InD also reflected the same feeling saying, “I feel that assessment
is a means or a tool that to check for the students their current performance ...whether they
understand or not the portion you taught them. So I think it is a means or a tool for checking
students understanding.”
The other respondents indicated that they use classroom assessment to check what the
students have learned, and this is mainly done to make sure that the instructional
objectives were met. The following respondent mention the use of assessment as a tool
for checking understanding and stresses attainment of general or specific objectives either
at certain point of a course or at the end.
108
... it is or has multipurpose not only one purpose. The first purpose of I think I mean
of assessment is I mean it enables to check whether the students understand or not
especially the objectives it could be either general or specific (InC).
The above view was also supported by another instructor. He said,
“Most of the time especially in our today time the main purpose is to check whether the
objectives set before were achieved or not. There could be two types: either at the end to
check the end result or the progress. So we can see in these two ways (InB).
Category 2: Describing students’ achievement
The purpose of assessment prioritized next to checking off learning outcomes is
‘describing’. The word is used here to refer to the assessment done for reporting
learners’ achievement for judgmental purpose in the form of grades and for grouping
students according to their achievement. The respondents used expressions like strength
and weakness, high performing and low performing, top middle low. These expressions
were also mentioned frequently and even one instructor put it as the primary one. He
said:
....So in my opinion the main purpose of assessment is to identify the students
strengths and weakness and to know their academic status whether they top
middle or low and then to give remedies on the weak area (InE).
However, for the other instructors this comes secondary. After explaining the primary purpose of
classroom assessment, InA continued,
“...the second is as I think it helps to identify the standard of our students because
identifying academic standard of our students helps us to help them differently because
students do have different standards: there are high performing students, low performing
students and medium performing students by using assessment we can identify them...
The instructors also frequently mentioned giving grade as the second priority. For
instance InC, after putting the first thing first, continued “not only that the teacher also
gives grades at the end of the course.” InD also reflected this view in the same way. He
109
extended his response “You also grade the students achievement that is grades because to decide
they promote or fail.”
Category 3: Improving students learning
The third category identified by the respondents was ‘improving learning’, which was
assigned a peripheral position. Expressions relating assessment purpose to improving
learning and teaching are mentioned sparsely and doubtfully in the discourse. For the
respondents, improving learning is not the main reason for assessing; it may come as a
side effect. InA, for example, putting it in the third place, reflected the idea that
assessment helps students indirectly i.e., students correct their errors from the feedback
they get from instructors, and instructors use it to identify the group of students who need
help. Similarly, after mentioning assessment as a means of identifying weak and strong
learners, InE added, “...then to give remedies on the weak area [areas on which students
have difficulty].” InsC, expressing his belief that it might also inform the teacher to
modify his/her teaching method, added, “Another importance of assessment might be it
helps the students to share idea among themselves.”
Other instructors raise a point about improvement only when probed further. They were
also uncertain that assessment really improves learning. When asked for confirmation of
the purpose he already mentioned, InB responded, “ sure that is it but not only that it may
also help the instructor it may help the teacher himself sometimes to device the
methodology mechanism if the mechanism devised is not of that much effect.”
Generally, the instructors’ responses indicated that while they held the belief and
understanding that classroom assessment is primarily done for either inspecting or
110
describing students learning or both, the improvement may or may not come as a
peripheral consequence. In addition, none of the respondents showed preference to or
negative feeling towards any of the three categories of purposes they identified, other
than implying the priority of the purposes they perceived.
4.1.1.1.2 Who is Responsible for Classroom Assessment
Next, the instructors were asked their belief about responsible body to carry out
classroom assessment. Instructors reflected two views about who should do classroom
assessment. Two of the respondents, mentioned teacher as the main responsible body, but
had the belief that students should assess themselves. For example, InB said, “So most of
the time the teacher should assess in the classroom based on their achievement based on
their work day to day work. Students can also assess each other”. Although these
instructors had the belief that learners should assess themselves, they couldn’t say how
the learners do the assessment; and what they mentioned as ‘assess themselves’ does not
conform to the concept of involving learners in assessment (see Section 2.2). Responding
to the question, an instructor explained:
...the students should also evaluate themselves to know at what level they are
performing, they have to know themselves. Are they performing well, are they
underperforming or are they performing at medium level. Each and every student
should evaluate himself. I don’t know how we can let them evaluate themselves
since I’m not in it practically, but basically I believe both the teacher and the
students should be involved in assessment (InA).
For the other instructors it is unquestionable that the course instructor is the only
responsible person to do assessment.
I think it is clear I mean it is obvious it should be the teacher who gives the
course. The course instructor knows the general and specific objectives of the
course and he has to check continuously whether he has achieved his objectives.
The instructor also must report the students’ grade at the end of the semester or
111
course. For this reason he is responsible for assessing his students. He is also
professionally trained. Anybody cannot do it. It is difficult for others to I mean
who are not involved in teaching the course they do not know the level of the
students which content they have covered. So this is clear as I think (InC)
InE also stressed:
...the teacher it must be the subject teacher. After finishing a topic or a portion of
the course he or she has to check whether or not the students understood it
whether or not the objectives are achieved to decide to continue or to make
remedies or revisions. He or she is also expected to give grade at the end so
he has to take continuous assessment of his students. this is one part aspect of his
job professionally. It is difficult for others to I mean for example the
department only supervise and give guidance but do not directly involve in
assessment the instructor knows the level of the students what and how he taught
which content they have covered. So, no question I think it must be the instructor.
Generally, the instructors had the belief that the teacher is responsible person to do
classroom assessment because s/he is the knower and it is his/her job. Although some
respondents had the view that students should assess themselves, they could not mention
it how they would do it.
4.1.1.1.3 Description of LSA
The third theme of the interview question was how the concept of LSA and the
procedures it involves is conceived. To elicit data pertinent to this theme, questions like
those that ‘what is LSA/ how do you describe LSA?’ were posed to all of the instructors.
In response to this question, all the instructors described LSA in its simplest level. They
defined it as the students’ activities like comparing one’s achievement to that of a
classmate after getting back test results or reflecting on how well one has performed on a
performance or how many of the questions one got right after exams or how well one can
understand, for example, while reading for exams. The following extract is from
instructor’s response to a question ‘how do you describe LSA?’
112
it is one way of assessment. Teacher’s assessment is formal one and learners’ self
assessment is another one. It is the way students evaluate themselves. They always check
whether they understand a topic or what they learned. A student can say which course is
easy and which one difficult for him. For example, when they study they can evaluate
whether he or she has understood it. Again after doing an assessment task exam or a test he
or she can check how much he or she has answered and missed. After the exam also they
evaluate themselves by comparing one’s result with that of classmate’s how well they have
done the exam therefore students always make self-assessment (InC)
InB also describes the procedures that self-assessment involves as “checking their results,
checking their end results or progress and check if they have achieved their target
objectives. InD, explaining how students assess themselves, said:
A student can assess or evaluate him/herself by thinking what he can do and
cannot do. For example, when he study he can evaluate how well he understand
and which topic is difficult for him. They can also evaluate themselves after doing
assignments or taking a test. When they see the result they can evaluate how
much they have done. A student usually evaluates himself after getting the result
of the test by comparing with their classmates. So self assessment is the way
students check their ability and learning before and after they take tests and
exams.
This indicates that their understanding of the concept of LSA is limited to the simplest
form identified by Boud & Brew (1995). The instructors’ responses show that they
equated LSA with practices like self-testing, self-grading and self-marking which are
conceptually different from LSA proper. Such equating is similar with what is observed
in the literature (see Section 2.2.1).
4.1.1.1.4 Appropriateness of Involving Learners in Assessment Process
In this theme, the subjects’ reflected their thinking about whether students should
participate in the whole process of assessment. Being cognizant of their understanding of
concept of LSA in its non-technical meaning, the interviewer directed the subjects to the
concept of advanced LSA and asked their views on whether or not it is appropriate to
113
engage students in assessment in its full-fledged form. The instructors reflected three
different, even, self-contradicting views. One view supports partial involvement, i.e.,
letting the learners to participate in the setting of criteria, but not in mark their own work
using the criteria, questioning the students’ honesty.
Reflecting the above view, a respondent commented that while it is important to involve
students in setting criteria it is difficult to trust students marking their work. He said,
“I think students should be involved in setting criteria of assessment together
with the teacher ...” “... If students assess themselves you know everybody is
selfish by nature you know, so their aim is to score high mark to achieve high
grade” (InD).
The second view is total objection of learners’ involvement in the process of assessment.
The respondents had the thinking that learners cannot be involved in setting criteria and
marking because they do not have the knowledge and skill. InC had the belief that
learners “... are not aware of what assessment is and its criteria....”, and the students’
limited ability and selfishness is a problem if they are involved in assessment scheme. He
stated:
.... If students are involved in the assessment process I think they can contribute
towards the improvement of the teaching learning process but they have no
experience they have no awareness, ...; therefore, they can put exaggerated marks
because they need to score good grade. Therefore their marking cannot be
accepted (InC).
The third view seem to support letting students to participate in both activities, setting
criteria and marking, but express fear of its practicability because of students’ lack of
knowledge and experience’, and selfishness. InB commented, “if students are allowed to
assess themselves, they can set criteria and give mark, they can do though there are
difficulties in the practical aspect.” InA also held the same, but self-contradictory view.
114
In support of the view, he said, “when we say students should assess themselves they
should have some benchmark or some criteria set by the teacher or in consensus with the
students. After setting criteria I think student can evaluate himself in terms of the already
set criteria”. Nevertheless, when probed further, he mentioned ideas that seem
contradicting to each other. He said,
I don’t think students should evaluate their tests their assignments because that
could lead them to be biased for themselves. Even about setting criteria in
practical world it is not practicable, but if we really need to make them evaluate
themselves we have to do that (InA).
Although some of the instructors support partial or holistic involvement, their response
does not seem to be grounded on any knowledge or experience because as it is indicated
in Section 4.1.1.1.3 above, they did not have clear understanding of what LSA proper
really entails. Their answers are likely to come from the speculation they made about
LSA based on what they just learned from the interview questions.
4.1.1.1.5 Validity of LSA
This theme is related to the subjects’ belief about the dependability of the students’
scoring of their own work. Having less positive view of appropriateness of students’
marking their own performance (Section 4.1.1.1.4 above), the instructors were cynical
about students assessing themselves. They expressed strong disbelief about validity of
learners’ scoring their own performance and related this to students’ dishonesty and lack
of ability and experience. One instructor, responding to the question ‘do you think
students score their own work fairly correctly?’ rightly said “no here no doubt to say that
they exaggerate their mark. The students want to get high mark good grade, so they do
not put the right mark they will be biased.” (InC). InE also persistently argued that
115
students cannot be honest scoring their own work because the students’ life after the
campus is based on the grade they can show to their hirer not on what they know and can
do; and complained that this is because of the existing problem of the recruitment system
of the country. In his words:
... you want to be confident when you get out of the campus you have to get job to
live your life you have to get your livelihood so in educational level the first
criteria is you have better mark or better grade when you where in school so hirers
the job providers search you by your mark they look at only the paper so the main
aim or criteria for job provider is what is on the paper so they look at the paper if
they see good grade they select you by the grade not with your real potential
anything else so involving students in the assessment is not that much necessary
and important because students do not know that the aim of learning is a quest for
knowledge they think learning is getting mark to get job
This implies that there is a strong belief among the instructors that learners marking of
their own performance is unacceptable because they cannot and do not score correctly
and honestly.
4.1.1.1.6 Benefits and/or Downsides of LSA
The instructors were, at last, asked to speculate anything students and teachers may
benefit from LSA despite its invalidity; or additional problem it may impose. They
identified one benefit related to cognitive ability and some none cognitive elements.
In the table, the Cognitive category is used to refer to the ways in which LSA directly
contributes to the increase of learners’ content learning outcomes, and the Non-cognitive
benefits refer to the contribution of the practice of LSA to the improvement of the
emotional and social behaviours that have direct impact on the cognitive development.
116
Table 4 Benefits of LSA (instructors)
Cognitive
‘internalising mistake’ (to mean getting a closer look at one’s
own mistake) and learning from it
Non-cognitive
encouraging independence
confidence
Increase transparency & minimise complaints
responsibility
increasing commitment
participation
In response to the question related to the contribution of LSA, InA used phrases like “... it
develops transparency in grading” and “.... they feel committed to their work”. InC said
“for sure it helps. People internalize [clearly identify] their mistake as they view than
when people assess them”. An instructor also explained how it encourages self-learning
and confidence. He relates:
“when they assess themselves when a person assesses himself or another, she or
he identify mistakes and just consult the teacher or any expert. So it can
develop confidence within the person or within the student who assess
himself or herself. So it can develop confidence. It can facilitate learning also
self-learning” (InB).
The only problem mentioned other than its invalidity was that “it takes time for the
teacher to set criteria or guiding the students needs additional time” (InB). There is also a
view that LSA is irrelevant. InE expressed the view that if it is not valid, it is meaningless
117
to spend time on it. He stresses that unless self-assessment is valid it is “just waste of
time”.
The instructors’ speculative response to the question related to the potential benefit of
LSA showed that while they could guess some of the non-cognitive aspects they failed to
think the way it helps the development of the cognitive ability of the students.
Generally, the instructors seem to have the conception that assessment is mainly used as
an instrument for measuring and describing learning outcomes and the teacher is the only
authorized party to do so. Their understanding of the concept of learners LSA is also
mistaken and consequently they did not have positive view about the learners’
involvement in the assessment process and were not clearly aware of its academic
significance in improving quality and quantity of learners’ learning.
4.1.1.2 Data from the Questionnaire
The instructors’ questionnaire was Likert-type multi-item 5-point scales questionnaire
developed to tap quantitative data on their conception of LSA. The questionnaires elicited
information on the same themes as the interview questions i.e., the main purpose they
attach to a classroom assessment, who should be involved in it, their awareness of the
concept of LSA, its validity and perceived advantages and/or problems. As indicated
earlier, the instructors’ questionnaire was distributed to nineteen instructors and eighteen
instructors filled out and returned the questionnaire papers properly. The items were
clustered and analysed according to the themes to which they are related; and are
presented in six sections.
118
4.1.1.2.1 Main Purpose of Classroom Assessment
The statements in this theme of the questionnaire are subdivided in to two: describing
achievement and improving achievement. The first group of four items, item 1, 2, 7 and
10 elicited the instructors’ extent of agreement to the statements that suppose the purpose
of classroom assessment is primarily to describe learning out comes. These are presented
in Tables 5 and 6.
Table 5 Classroom assessment for describing achievement
Table 5 indicates that the majority of the instructors showed their agreement to all the
four statements. More than 66% of the respondents agree or strongly agree and small
proportion, 22.2% and less, of the respondents showed their disagreement or unable to
decide. None of them strongly disagreed with any of the four statements. In addition, the
Item
No.
Statements
The main purpose of
classroom
assessment is ...
Extent of agreement
m
ean
1 % 2 % 3 % 4 % 5 %
Tota
l
1 to check whether or
not students have
mastered what they
have learned
7
38.8
8
44.4
2
11.1
1
5.5
0
0
18
1.83
2 indentifying the
strengths and
weakness of the
learners
6
33.3
9
50
2
11.1
1
5.5
0
0
18
1.89
7 checking learners’
progress against the
course objectives
5
27.7
8
44.4
4
22.2
1
5.5
0
0
18
2.06
10 grading and
categorising students
2 11.1 10 55.5 4 22.2 2 16.6 0 0 18 2.33
119
mean value for all the four statements ranges from 1.83 to 2.33, which almost correspond
to the ‘Agree’ label in the scale. This shows that the instructors agree with the view that
describing the learning outcomes is the main purpose of classroom assessment.
The next group of three statements sought the instructors’ degree of agreement on the
view that the teacher’s purpose of carrying out classroom assessment is improving
learning. As displayed in Table 8 below, the instructors’ responses to the three statements
also indicate that most of them agree with the three statements. The percentage of the
respondents who agreed to all the three items is above 66.6%. About 22% failed to decide
to agree or disagree to items 8 & 11, and only 11.1% disagreed to item 11. The mean
values are 1.67, 2.06 and 2.44, which approximate the agreement scale. This generally
implies that most of the respondents agree with the idea that the teachers assess their
students so that it helps the students to improve their learning.
Table 6 Classroom assessment for improving learners' achievement
Item
No. Statements
The main purpose of
classroom assessment is ...
Extent of agreement
Tota
l
mea
n 1 % 2 % 3 % 4 % 5 %
3 is helping learners to
improve their learning
8 44.4 8 44.4 2 11.1 0 0 0 0 18 1.67
8 is providing feedback to
students about their own
performance
3 16.6 11 61.1 4 22.2 0 0 0 0 18 2.06
11 getting information for
the teacher on the way
s/he teaches
0 0 12 66.6 4 22.2 2 11.1 0 0 18 2.44
As can be seen from table 5 and 6, most of the respondents agreed or strongly agreed
with both views about the main purpose of the classroom assessment; they did not
120
differentiate one from the other. This implies that they may not have a clear
understanding about what the most important and the secondary purposes of classroom
assessment are.
4.1.1.2.2 Whether or not students should be involved in the assessment
scheme
The second major theme of the questionnaire focuses on the instructors’ view about the
appropriateness of learners’ participation in assessment process. This contained a group
of four statements to which the instructors responded as displayed in Table 7 below.
Table 7 Appropriateness of learners' involvement in assessment process
Item
No.
Statements
Extent of agreement
T
ota
l
m
ean
1 % 2 % 3 % 4 % 5 %
4 Students should
participate in
assessing their own
work
1 5.6 5 27.8 8 44.4 3 16.7 1 5.6 18 2.89
18 It’s the instructor’s
job to evaluate
students’ work
6 33.3 9 50 3 16.6 0 0 0 0 18 1.83
19
Assessment is one
aspect of the learning
process in which
students should take
part
3 16.6 7 38.8 7 38.8 1 5.5 0 0 18 2.33
21
I consider the real
assessment to be
assessment that is
done by the teacher
12
66.6
4
22.2
2
11.1
0
0
0
0
18
1.44
The instructors’ responses to these statements show that they seem to hold mixed view
about the appropriateness of students’ involvement in assessment process. Majority of
121
the instructors (more than 83.%) agreed to the statements that assessment is the
instructors’ job (item18) and real assessment is done by the teacher (item21). The mean
values of these items are 1.83 and 1.44 respectively. Again, about 55% agreed to item 19
that states students should take part in assessment, and 44.4% were unable to decide
whether students participate in assessing their own work (item 4). The mean value of this
item also draws near the Undecided scale (2.89). This indicates that majority of the
respondents did not have a clear idea whether or not it is necessary and appropriate to
involve the students in the assessment of their own work.
4.1.1.2.3 Description of LSA
A group of three statements elicited the instructors’ response on their understanding of
the concept of the LSA.
Table 8 Instructors’ understanding of the concept of LSA
Item
.N
o.
Statements
Extent of agreement
T
ota
l
m
ean
1 % 2 % 3 % 4 % 5 %
5 Learners’ self-assessment
means students marking their
own test or assignment using
answer keys provided by the
instructor
9
50
8
44.4
0
0
1
5.5
0
0
18
1.61
6 Learners’ self-assessment is a
technique of assessment where
students participate in setting
criteria and scoring of their
own performance
0
0
5
27.7
11
61.1
2
11.1
0
0
18
2.83
9 Self-assessment means
checking how well you
understood a topic while you
are learning or studying a
subject
2
11.1
13
72.2
3
16.6
0
0
0
0
18
2.06
122
The data in the Table 8 indicates that most of the instructors, 94.4% and 83.3%, showed
their strong and/or moderate agreement with statements in item 5 and 9 respectively that
describe LSA in its simplest form and lower level. The mean for the two items is also
1.61, and 2.06, approximating to the agreement scale. In their response to item 6 which
describe LSA in its advanced proper sense, majority of the instructors (61.1%) were
unable to decide. This indicates that the instructors do not seem to have a clear concept of
what is meant by learners’ LSA; they seem to go with statements that relate LSA to a
simple technical knowledge interest and unable to decide about the statements that relate
to a more advanced communicative and interpretive interest.
4.1.1.2.4 Validity of LSA
The next group of four statements elicited the instructors’ response with regard to the
issues of validity of LSA. Table 9 displays the result.
Table 9 Validity of LSA
Item
No.
Statements
Extent of agreement
T
ota
l
M
ean
1 % 2 % 3 % 4 % 5 %
14 Learners are more
likely to provide
inflated information
on their own
performance
14 77.8 4 22.2 0 0 0 0 0 0 18 1.22
24 Students assess their
own performance
honestly
0 0 1 5.6 3 16.7 6
33.3
8 44.4 18 4.17
27 It is difficult for
students to
understand the
assessment criteria
1 11.1 10 55.5 6 33.3 2
11.1
0 0 18 2.44
29 Students’ self-
assessment can
count to final grade
0 0 1 5.6 6 33.3 4 2.22 7
38.9
18 3.94
123
As the Table shows, All (100%) and majority (66.6%) of the instructors respectively
agreed to item 14 and 27 that state LSA cannot be valid. The mean values of these items
are also 1.22 and 2.44 that correspond to agreement scale. Similarly, Majority, 77.7% and
61% of the instructors respectively showed disagreement to items 24 and 29, which
contain the statements that suppose LSA has validity. The mean values of these items are
also 4.17 and 3.94 that are approximating to 4, the disagreement scales. This indicates
that most of the instructors held the view that LSA cannot be valid.
4.1.1.2.5 Significance of LSA
This part of the questionnaire sought instructors’ response to statements about the
benefits of learner participation in the assessment process in improving learning. The
statements are grouped in to two: a group of seven statements related to cognitive and
meta- cognitive benefits, and a group of five statements related to non-cognitive benefits
of LSA. Table 10 presents the data on the cognitive benefits and Table 11 presents the
data on the non-cognitive benefits.
Table 10 shows most of the instructors (72.2% 100%) agreed and/or strongly agreed to
all of the statements that suppose different types of cognitive and meta-cognitive benefits.
The mean value of these items also ranges from 1.61 to 2.22 which confirm the strength
of their agreement. And, majority of the respondents (66.6%) failed to agree or disagree
to statement which make the proposition that assessing their own performance help
students little to improve their learning (item 20). The mean value of this item is also 3;
corresponds to the Undecided scale. This indicates that the respondents have the view
that LSA has beneficial cognitive effect on students’ learning.
124
Table 10 Cognitive benefits of LSA
Item
No.
Statements
Extent of agreement
Tota
l
mea
n
1 % 2 % 3 % 4 % 5 %
12
Participating in
assessment scheme helps
students to improve ways
of approaching
their learning
7
38.9
11
61.1
0
0
0
0
0
0
18
1.69
13
If students assess their
own work, they would be
able to recognize their
errors easily
8
44.4
9
50
1
5.5
0
0
0
0
18
1.61
15
Involving learners in
assessing their own
learning outcomes is a
good way of providing
feedback on their own
performance
2
11.1
14
77.7
2
11.1
0
0
0
0
18
2
20
Participating in assessing
quality of their
performance has little
effect on improving
students learning
0
0
3
16.6
12
66.6
3
16.6
0
0
18
3
22
Assessing themselves
makes learners more
aware of what they need
to know in the subject
3
16.6
12
66.6
3
16.6
0
0
0
0
18
2
23
Learners’ self-assessment
enables them to
understand the quality of
performance expected of
them
6
33.3
12
66.6
0
0
0
0
0
0
18
1.67
30
Assessing themselves
makes them think what
and how much to do to
achieve the learning
objectives
4
22.2
9
50
2
11.1
3
16.6
0
0
18
2.22
In Table 11, among the items each of which supposes different types of non cognitive
benefits, majority of the instructors, 66.5% and 83.2% agreed or strongly agreed to the
125
propositions that LSA make students feel responsible (item 16 ) and it increases
participation (item 32) respectively.
Table 11 Non-cognitive benefits of LSA
Item
No.
Statements
Extent of agreement
Tota
l
Mea
n
1 % 2 % 3 % 4 % 5 %
16 Assessing their own
work makes students feel
responsible for their
learning
5 27.7 7 38.8 4 22.2 2 11.1 0 0 18 2.17
17
Getting students to
assess their own work is
a way of solving
cheating problems and
disruptiveness
0 0 2 11.1 12 66.6 4 22.2 0 0 18 3.11
25 If students are to assess
their own work, they
would get bored
0 0 3 16.6 12 66.6 3 16.6 0 0 18 3.00
28 Assessing their own
work improves the social
climate of the teaching
learning process
0 0 2 11.1 2 11.1 9 50 5 27.7 18 3.94
32 Learners’ self-
assessment is the way of
increasing students’
participation in learning
activities
7 38.8 8 44.4 2 11.1 1 5.5 0 0 18 1.83
The mean values of these items also approximate the agreement scale. About 77% of
them disagreed to the idea that LSA improves social climate of the classroom. However,
majority of the respondents (more than 77%) couldn’t decide whether or not to agree to
the statement that assume LSA solves problem of disruptiveness (item 17) and the
statement that is negatively stated: LSA add boredom (item 25). The mean values of
these items are 3.11 and 3.00.
126
Generally, the instructors identified ‘developing sense of responsibility’ and ‘increasing
participation’ as the only contribution of LSA in improving learning. This indicates most
of the respondents did not have clear understanding or were sceptical about the non-
cognitive benefits that LSA contributes to the improvement of the learning outcomes.
This scepticism can be related to their understanding of the concept of LSA because in
Section 4.1.2.1.2.3 above it was observed that respondents described LSA in its simplest
technical level that contribute little to the cognitive and non cognitive development. It
should also be noted that the view of the respondents regarding the cognitive aspects
could not be said to come out of their knowledge and experience. Therefore, it is difficult
to say that the respondents’ perception of the value of LSA is clear because the way it is
conceptualised and its contribution to learning cannot be separated.
4.1.1.2.6 Relevance of LSA
The last two items sought the instructors’ response on whether or not LSA is relevant at
all. As Table 12 below indicates, the respondents’ view about whether or not LSA is a
part of the teaching learning process is mixed.
Table 12 Relevance of learners' SA
Item
No.
Statements
Extent of agreement
Tota
l
mea
n
1 % 2 % 3 % 4 % 5 %
26 Learners’ self-assessment
is an important part of the
teaching /learning process
1 5.6 5 27.8 7 38.9 4 22.2 0 0 18 2.72
31 Learners’ self-assessment
is just putting meaningless
burden on students and
teachers
0 0 3 16.7 6 33.3 8 44.4 1 5.6 18 3.39
127
In response to the statements, (33.4) of the respondents agreed to the proposition that it is
relevant, and a considerable percentage (38.9%) of the respondents were unable to decide
while 22.2% disagreed. The mean value of this item is also 2.89, nearer to the Undecided
scale. Again, 50% of the respondents disagreed with the opposite statement i.e., the
assumption that LSA is irrelevant, but still considerable proportion, (33.3%) was unable
to say yes or no and the rest, 16.7% agreed. The mean value of this item is 3.39, nearer
to the Undecided scale. This undecidedness of the respondents’ is also the reflection the
of understanding they had about concept of LSA.
4.1.2 Students’ Conception of LSA
Data on the students’ conception were collected at two phases of the study i.e., the pre-
intervention and post-intervention phases. The purpose of gathering data on students’
conception prior to the intervention was dual: to answer RQ1 B and use it as demarcation
of a baseline for the experiment (RQ2). Like that of the instructors, data on students’
conception of LSA was gathered using interview and questionnaire.
As mentioned in sections 3.2.2 and 3.5, the pre-intervention data on the students’
conception was collected twice, labelled as CoPre1 & CoPre2, to check the stability of the
dependent behaviours at the initial stage so that valid interpretations can be drawn from
the finding of the post-intervention data analysis. That means both the interviewing and
the administration of the questionnaire were carried out twice
The two sets of data were analysed accordingly, and the results of the analysis showed
that the difference between the findings of CoPre1 and CoPre2 data was negligible.
Therefore, the result of the second round data (CoPre2) was taken as the reference for
128
comparing the results of the pre- and post intervention data. Hence, to avoid boredom of
unnecessary repetition of similar results, the findings from the first round data are briefly
summarised first and the analysis of the second round data is presented. The transcript of
the first round interview is appended as Appendix 11.
4.1.2.1 Data from the Interviews
The interview guiding questions were the same for the instructors and the students, and
were based on the same themes, i.e., the main purpose they attach to a classroom
assessment, who should be involved in it, their awareness of the concept of LSA,
appropriateness of learners’ participation in assessment scheme, its validity and perceived
advantages and/or problems.
The results of the interviews before and after the intervention are presented separately for
comparison.
4.1.2.1.1 Summary of the Finding from CoPre1 Interview
The group of students responded to the six questions related to the main themes of the
conception of LSA. In response to the first question, ‘what do you think is the main
purpose of classroom assessment?’ the respondents generally mentioned three purposes,
measuring students’ achievement to give grades and categorise students; checking the
attainment of the instructional objective; and improving learning. Among these, priority
and emphasis was given to the first one, and instances related to improving learning were
sparsely mentioned with uncertainty.
Responding to the second question, ‘whose responsibility is carrying out classroom
assessment?’ the students unanimously expressed the belief that assessment is the
129
responsibility of both the teacher and the students. However, as was observed from their
description of LSA, when asked to explain the idea of LSA, what they mentioned as the
role of the learner in assessment was not real assessment practise, but were activities like
self-reflection or self-checking as when the student compare him/her achievements to that
of classmates upon getting back test results, or contemplate on how much s/he could
memorise after reading certain text.
When they were guided to the kind of LSA which requires the involvement of the learner
in the whole process of assessment and asked whether or not it is appropriate to involve
learners in setting criteria and scoring their own work, the students disclosed two
different views. Few of them had the opinion that learners could give mark to their own
work provided that they get ready made criteria from the teacher. Majority of them,
however, completely opposed the idea of letting students to participate in setting criteria
and marking. They were probed to reason out their disagreement, and reasoned that first
of all students are selfish because they need to get good grade and second, learners cannot
understand the criteria because they have no professional skills as the teacher does.
Therefore, the students scoring cannot be accepted.
The last question was to elicit if the respondents could speculate any advantage and/or
problem that LSA has. They said that it can help to focus on errors and correct it.
Generally, the analysis show that LSA was not considered as positive practice that
contributes to their learning.
130
4.1.2.1.2 Data from Pre-intervention Interview (CoPre2)
The students’ pre-intervention interview was conducted before the training to gather data
on the existing conception of learner LSA. A group of eight students participated in the
interview. To identify one respondent from the other each respondent was assigned an
ideal number and here speakers are labelled as S1, S2, S3 . . . and S8 whenever necessary
to refer to a respondent. Similar to the analysis of the instructors’ interview, the result of
the students’ interview is presented thematically in to six sections. Note that the
ungrammatical sentences were interpreted by sensing the intended meaning of the
speakers. Where languages other than English were used, translation of the discourses is
presented here, and the original transcript is maintained in Appendix 3.
4.1.2.1.2.1 Main Purpose of Classroom Assessment
The group of students were asked their understanding and belief about the principal
purpose of classroom assessment. In their response to the question, students identified six
first level purposes that are categorised into two major or 2nd
level categories that have
two 1st level categories each; and two 1
st level category of the purposes for which the
instructors do classroom assessments. These are shown in Table 13.
The purposes perceived by the students are similar to that of the instructors, but differ in
some aspects. First, the emphasis given to ‘auditing’ and ‘describing’ categories by the
instructors is reversed here. The students gave priority to the ‘describing’ purpose
through categorising and grading. Second, the students identified one more purpose,
controlling.
131
Table 13 categories of classroom assessment purposes (students’ pre-intervention)
Category 1: Describing achievement
In the students’ responses, the describing by categorising was the most frequent and
emphasised purpose of classroom assessment. The respondents used phrases like who is
hard working and who is lazy, who scores high mark and who scores low mark, who is
dependent and who is not, to indicate their view that the instructors assess students in
classroom to put learners into categories. For example, a student explained:
... when the teacher teaching students he evaluates which or among the students
who are working hard to differentiate who is weak in his work and his study
evaluate whether they are who record the higher mark he want to know when he
is teaching who is hard working and to evaluate each from one class to another
class students... (S3).
S2 also said “that is important for the teacher because by taking assessment he can
identify students who are weak and strong ...”
2nd
level 1st level
Describing
Categorising students
grading achievement
Auditing Measuring amount of learning
Checking attainment of objectives
Controlling students
Improving learning
132
The ‘describing’ purpose through ‘grading’ was also frequently mentioned by the
respondents. After putting ‘categorizing’ as the first priority, S2 continued, “it is
important for the teacher also to give grade for the students”
Category 2: Auditing achievement
The second major category of purpose for classroom assessment perceived by the
students was ‘auditing achievement’. This includes measuring learning achievement and
checking attainment of objectives. Responding to a probe related to this theme, S4 said
“... the main purpose of assessment is nothing but evaluating or measuring performance,
the work of the students ...”, and S6 added “... as I think evaluating someone how much
the person understands the given activity or the given topic ...”
Some students viewed classroom assessment as a means of controlling students’
behaviour in and out of the classroom by making them busy. Their responses in the
following extracts indicates the students’ thinking that instructors use assessment to
control students class attendance, and give continuous assessment as a means of
surveillance of students off-class time activities. S4 explained:
... and the purpose is it helps to hold the students towards the work for example in
case of our university or Wollega university the process of teaching learning is
continuous assessment and due to this we have different tests and assignments
throughout the week due to this reason we visit library in order to answer the
assignments if no continuous assessment students don’t study always they go to
towns playing therefore in a simple way or simple manner it helps to hold the
students towards their work
S5 added
... and also the purpose of assessment is to evaluate the students depending on
their activities in the classroom and to search [check] how the student attending
133
their class during their day to day life in the classroom not only in the classroom
also in their dormitory....
The improving learning was the least frequent and the least emphasised purpose of
classroom assessment mentioned by the students. It was mentioned in only two instances
and it was mentioned as a finishing point. For example, S6 mentioned the improvement
purpose at last saying “... another, the purpose of assessment is to develop or increasing
students ability in the whole direction means speaking ability reading ability and also
writing ability in classroom ...” S5 also perceived it as a minor purpose. After explaining
the other purposes, he continued, “... in order to answer the questions which is given for
the students they share different ideas among the group to understand or to answer the
questions which their teacher gives for them”. This indicate that the respondents held the
view that classroom assessment is also carried out, though not primarily, to help learners
improve their learning
Generally, the participants’ response indicated they held the view that classroom
assessment is done for various purposes like to describe achievement, to audit amount of
learning, to control students working in and out of class, and to help learners improve
their achievement. However, they showed strong belief that describing achievement
specially to categorise students is the major reason for which teachers assess students in
their classrooms.
4.1.2.1.2.2 Who Should Do the Assessment
Regarding the question of the responsible body for classroom assessment, the students
unanimously responded that assessment should involve both the teacher and the students.
For instance, the following is response given by S2:
134
... the assessment in the classroom or during the teaching learning process is
conducted by both the students and the teacher in the classroom. For instance,
when the teacher gives a class work for example in case of writing skill the
teacher gives different assignments to students to do in the class and then after
they write whatever work given to them the students evaluate themselves through
peer editing process means they evaluate themselves in the class by sharing or
exchanging their work with each other...
S8 added “since assessment is one part of teaching learning process it must be done both
by the instructor and the students and also department.”
However, in their explanation as in the above extract, the activities they referred to, like
peer editing, are not assessment by its nature. In addition, as seen in the next two
sections, the respondents came against this view and argued that learners should not
participate in marking their own work. Therefore, this view was not based on clear
understanding of self-assessment.
4.1.2.1.2.3 Description of LSA
When students were asked what LSA means and what it involves, they could not extend
their response beyond phrases like ‘the way in which students can evaluate his or
herself’. In addition, when asked to elaborate on how that evaluation is done, they
referred to their activities like reflecting on how well they understood while reading their
note, editing one’s own paragraph, reflecting on test results, and checking one’s status by
comparing oneself with classmates. This is similar to the description given by the
instructors. For instance, when asked to give example of LSA, S8 responded “Example I
myself can write one paragraph and edit that means I evaluate myself how the
punctuation the grammar is correct that means I evaluated myself.” S7 also reflected the
same view in the following extract.
135
self-assessment means it is assessment by which we evaluate ourselves for
example if we go to library we read our exercise books or references after that we
may take some question or we may ask ourselves what I know about this question
already read it and what type of question. By this without the involvement of
others we ask ourselves and we can evaluate ourselves so this is what we call
evaluating ourselves.
This indicates that like their instructors, the students considered LSA as a simple
reflective activity that requires little involvement of students in the whole process.
4.1.2.1.2.4 Appropriateness of Involving Learners in Assessment
Process
Observing that their description of the concept of LSA is limited to the preliminary stage
(Section 4.1.2.1.2.3 above), the interviewer directed to students to the higher and more
advanced type of LSA. Then, he asked their view about whether it is appropriate to
involve learners in the whole process of assessment including setting standards and
criteria and marking their own work using the criteria. The students’ responses reflected
two different views. The first one, mentioned by only one student, was that they could
give mark to their own work if they were provided criteria by the teacher, but it is not
appropriate to let them participate in setting criteria. S4 explained it as follows.
... evaluating himself for students is good because since certain criteria is given
for her or him depending on that criteria s/he can evaluate himself but on the
designing of the criteria it is not fair to give chance for or allow students to
design criteria for evaluating
The second view was rejecting the idea at all. Other respondents argued that learners
should not participate in setting criteria as well as the scoring. The reasons they
mentioned were one, students don’t have the ability and awareness about setting criteria,
second, students are selfish or want to get the highest mark and third, students can’t
identify their errors. Expressing this second view, S5 said:
136
... for example as other students tried to raise, other than evaluating (reflecting on)
his/her achievement, it is impossible for a student to give mark to his/her own
work and to set the criteria for evaluation because it is the teacher or instructor
who should and is able to set criteria and give mark based on that criteria ....
Supporting this view, S3 added:
Now there is no experience of setting marking criteria by students and we are not
aware of it, again it is not right to do so. In addition, it can’t be fair to give mark
to ourselves. I may get a few things right and I do not focus on my error but think
as if I did all well and give the mark that I don’t deserve. Therefore it is common
and also I don’t think it is right.
Generally, the students hold the view that assessment is the part of the teachers’ job and it
is not right to involve students in it because they do not have the knowledge and
experience; they are also not expected to do as they are selfish.
4.1.2.1.2.5 Validity of LSA
The students’ view about validity of LSA was consistent with their reaction to the idea of
letting students to participate in the assessment process. As they reasoned it out above,
learners’ scoring of their own performance cannot be reliable mainly because students
compete among themselves, so everybody needs good grade. This view is reflected in the
following extension of S5’s response to the above question related to appropriateness of
learners’ involvement in assessment.
In self-assessment, as we are students and we compete among ourselves, we need
to get good grade, therefore, we all do not give low mark to ourselves; by
ignoring the marking criteria, we give high mark, we need to benefit ourselves.*
translated from A/Oromo translated from Amharic
Translated from A/Oromo
137
Other students mentioned inability to recognise their own errors for undependability of
scoring of their own work. S8 said:
... when task given to us I may do it how I know it and I consider myself this is
right I do not know my error so since I don’t know my error where my
punctuation grammar error is I don’t know so I may think this is correct this is
right so I must get due to that reason giving mark to ourselves is not fair.
These responses indicate that the students are less optimistic about the validity of their
self-scoring. this is linked to their view about teacher as a knower.
4.1.2.1.2.6 Benefits and/or Problems of LSA
The students were asked the contribution and/or problems it has other than its invalidity if
they were allowed to participate in the whole process of assessment. The students could
speculate only one benefit and one problem related to cognitive ability. They thought
that it helps them to improve their learning by helping them to focus on errors closely and
correct it. S4 explain that after self-evaluation if one finds error s/he corrects it, and this is
good because s/he never forget that area.
The disadvantage they mentioned was the
possibility of fossilising error. They thought that as students tend to overlook their weak
sides, there might not be chance of correcting error. S4 continued his explanation said if
one ignores the criteria or fails to detect error, and assumes that what s/he has done is
correct, especially in speaking and writing, it is not good.
The students’ speculations of the benefits and/or problems of self-assessment are that it
may help them to detect and correct error so that they improving their learning, but there
may also be chance of error fossilisation if students ignore the criteria and be selfish.
translated from A/Oromo
138
Nevertheless, they could not see the other many cognitive and non-cognitive advantages
of assessing themselves.
Generally, the result of the pre-training interview indicates the students had the thinking
that classroom assessment serves the purpose of describing achievement by categorising
students and grading, and this should be done by the instructor. They understood LSA in
its simplest and non-technical meaning and consequently failed to see the potential
benefit it has, and were pessimists about its validity. They were divided over the
appropriateness of being involved in the assessment scheme: some supported it and
others objected it.
4.1.3 Effect of the Training Intervention on Students’ Conception
of LSA
The training was conducted to see its effect on the students’ conception of LSA and the
validity of learners’ assessment of their English oral performance. The analysis of data on
effect of the training on the students’ conception of LSA is presented in this sections and
data on the effect of the training on validity of learners’ assessment of their English oral
performance is presented in Section 4.3
The previous section presented data from the pre-intervention interview. The next section
presents data from the post-intervention interview.
4.1.3.1 Data from Post-intervention Interview
After training in assessing their English oral performance, the same group of students in
were asked the same questions as the pre-intervention session. As done for the pre-
139
intervention interview, the analysis of these data is presented thematically. The transcript
of this interview is appended as Appendix C
4.1.3.1.1 Main Purpose of Classroom Assessment
The purposes of classroom assessment identified by the students during this post-training
interview were a little similar to those in the previous one, but there is a shift in the
priority they were given. The students’ responses showed that they prioritised the
improving achievement purpose, tend to ignore the auditing achievement purpose, and
dropped the control purpose as indicated in Table 14.
Category 1: Improving learning
The respondents frequently referred to the improvement purpose. They explained that
doing assignments or preparing for any sort of assessment makes them active and work
hard, interact and share ideas with each other and encourages them to search for different
reference materials. They perceived these things as a way of developing their skills and
knowledge. The following extracts from the students’ responses are indicators of this
view.
when we say assessment it is used for developing students activity when there is
assessment now assessment is continuous when there are assessment students
study their books and exercise book what they learned or what they got from the
lectures or from other indirectly help to developing their knowledge ... and there
is strong interaction between students then to develop experience to interpreting
and exchanging their idea the purpose of assessment in general is the main one is
to develop students’ knowledge (S1).
In S2’s response there is a statement: “The main purpose of assessment is it makes the
students active enough and also makes students search different references...”
140
In addition, they perceived that the feedback they get from assessment leads them
to work hard to improve their mark in subsequent assessments. S4 said “... when
you see your mark is less or not good as your classmate you work hard to get
more in the next time
Table 14 Category of purpose of classroom assessment (students’ post-intervention
interview)
Category 2: Describing achievement
The respondents also reflected the view that giving grade and identifying weak and
strong students as the other purpose of assessment. S3, after mentioning that the main
purpose of classroom assessment is improving learners’ ability, continued “...the other
purposes are also to give grade to see weak and strong students to give feedback these are
the main purposes of assessment.”
The students had also mention measuring learning as another purpose of classroom
assessment, but this was observed in only one instance. S3, after explaining the other
purposes, he continued, “.... and also the teacher knows whether the students get the knowledge
and understand the topic he teaches by giving different assessments. The scarcity of the instances
related to the measuring purpose shows the least priority given to it by the respondents.
2nd
level 1st level
improving
motivating
Getting feedback
Getting help
describing
Categorising
grading
auditing Measuring amount of learning
141
The students’ responses to the question about the main purpose of classroom assessment
generally indicate that the students have got the thinking that classroom assessment is
primarily carried out to improve students’ learning and the others like grading,
identifying students, are the secondary purpose of the teacher.
4.1.3.1.2 Who Should Do the Classroom Assessment
Responding to the questions directed at eliciting their view about who is responsible to do
the classroom assessment, the respondents seemed to have one clear-cut answer: both the
teacher and the students. S5 responded to the question “...as you said who should do the
assessment the responsibility to do the assessment is both teacher and students or both the
instructor and the students.” and S3 added “...your question is who should do the
assessment and my answer is the assessment should be done by both the teacher and the
students.”
The respondents were asked to elaborate their answers, to explain how students can
participate in assessment. S5 explained: “In the assessment the students can discuss with
the teacher and creates some criteria for doing that assessment and depending on those
criteria students also score their own work or their friends work ...”
This response indicates that unlike the pre-training interview, the students’ belief that
classroom assessment is a shared responsibility of both the teacher and the students is
with the awareness of the kind of assessment that fully involve learners in the process.
This is also confirmed in their description of LSA as seen in the next section.
142
4.1.3.1.3 Description of LSA
In the same way they answered the question related to the above theme, where the
responsibility of assessment resides, the interview group unanimously described LSA as
an activity of evaluating one’s performance by participating in the whole process of
assessment including setting criteria and scoring one’s work. The following S3’s
description of LSA is an example indicator of their concept about what self-assessment is
and what it entails.
self-assessment is nothing but the way you evaluate your own performance
depending on the designed criteria by both the teacher and the students first by
participating in the class discussion with the teacher to design criteria and doing
the assignment or activity in the class or at dorm and marking for yourself for
example last time when we learned self-assessment we participated by designing
criteria and giving mark to our own presentation. This is self assessment.
To add one more example of the description given by S7
self-assessment is to evaluate ourselves how we did something or to identify
capacity of doing that activity to determine our knowledge identify our error good
or bad and self-assessment should be taken by individuals from the designing of
criteria by discussing in class and giving mark for my own performance.
The students’ responses to the questions directed at eliciting students’ understanding of
the concept of LSA indicated that the respondents’ understood LSA as a mode of
assessment in which the learner participate in and contribute to the development of the
criteria, and score his/her own performance using the criteria. This indicates that they
have perceived the advanced type of LSA which involves learners in the complete cycle
of the process.
143
4.1.3.1.4 Appropriateness of Involving Learners in Assessment Process
Based on their description of LSA, the group of interviewee was asked to comment and
express their views on the appropriateness of letting students to participate in the
assessment process at all. The students’ responses reflected two different views. The first
one was that it is appropriate to involve the students in both setting criteria and scoring
their own work. S7 responded to the question:
I think to evaluate themselves the students should participate to make criteria to
which they do in the future time or the students should be involved in setting
criteria which is used to evaluate what they are going to do because it helps him to
know that criteria and work hard to do as the criteria is needed. I also support that
is good appropriate for students to give mark depending on the criteria designed
by the teacher and student because they follow that criteria and cannot cheat the
teacher by giving high mark
However, this view was accompanied by cautionary note that students must have
awareness about the purpose of assessment and ability to set criteria and marking. In the
following two extracts, while supporting the appropriateness of learners’ participation in
the process, S5 related the training experience they had had and aired his concern about
the need of creating awareness before letting students to participate in the process.
I think setting criteria and giving mark based on the set criteria is not appropriate
because students don’t know how to set a criteria or to evaluate their work
because students have no awareness about criteria or self-assessment last week we
have learned about setting criteria about self-assessment now it is good for us to
give mark for ourselves because we know the advantage and disadvantage. If they
know or have awareness about setting criteria and giving mark for themselves it is
possible and it is appropriate it is acceptable but one thing behind that if they
don’t have awareness about setting criteria and giving mark it is problem
The second view supports letting students participate in designing criteria, but rejecting
the idea of students scoring their own performance. Other respondents argued that it is
144
possible to set criteria under the guidance of the teacher, but it is unlikely that many
students score their work honestly. S3 observed:
my idea is opposite from other and my answer is yes. Students should participate
in setting criteria for evaluation here it is possible to avoid bias the teacher also
help and give direction to set the appropriate criteria but it is not fair for the
students to give mark for themselves because some students can be honest but
some students may not be honest they wish to get good grade which is not
according to their ability and knowledge
Generally, the respondents seem to have the opinion that it is appropriate and necessary
to involve students in the process of assessment, but learners should have at least a
knowhow of the issues related to LSA. Nevertheless, there are respondents who are,
irrespective of the training, less optimistic about the learners scoring their own
performance honestly. This is also reflected in the discussion point related to validity of
LSA.
4.1.3.1.5 Validity of LSA
As discussed above in relation to the issue of appropriateness of students’ involvement in
assessment, the students’ responses indicate that there are different feelings about the
validity of LSA. Some of the group members had the feeling that the students’ marking
of their own work can be valid if the students have the knowledge and practice of using
the criteria. Strengthening his belief about the appropriateness of students’ participation,
S7 expressed the view that students scoring can be valid saying “... I also support that is
good appropriate for students to give mark depending on the criteria designed by the teacher and
student because they follow that criteria and cannot cheat the teacher by giving high mark.
However, others expressed their doubt that students would not be honest in scoring their
own work. As S3 commented, they just need to get grades that they do not deserve. S5
145
also repeated similar view saying, “... other one is some students may be selfish or make
themselves get good mark not depending on that criteria.”
These responses indicate that the subjects’ belief about validity of LSA is mixed. Though
some of the respondents held the view that students can take responsibility of assessing
themselves if they get awareness and practices, some others remained sceptical about it.
4.1.3.1.6 Benefits and/or Problems of LSA
In this post-training interview, the respondents could mention more advantages in doing
self-assessment than they did in the pre-training interview session. Here they mentioned
two benefits related to cognitive ability and a number of benefits related to non-cognitive
ability as listed in the table blow.
Table 15 Benefits of LSA (learners' post)
cognitive
improve the language skills
develop skill of assessing
non-cognitive
independence
Intrinsic motivation
Planning and executing
commitment
confidence
minimise complaints
responsibility
146
The students frequently mention the role of LSA in improving learning throughout the
discussion. They saw its benefits in helping them to identify their errors weaknesses and
to improve it. For instance, S4 explained his experience of the training and concluded “...
setting criteria and giving mark for their performance is crucial to understand their
weakness and improve their knowledge. It help us to correct the wrong grammar, the
vocabulary of speaking and also how to correct writing problems.” The other benefit the
students perceived in the cognitive domain was the development of their skill of
assessing. They believed that assessing themselves can also develop their skill of
assessing their students when they become teachers. S3, after explaining other benefits,
continued:
Again, when we go the teaching profession for example we become teacher we
can teach in high schools and we must evaluate the students. If I don’t learn how
to make criteria and mark I can’t evaluate my students so it has opportunity to
learn how to assess students
The students also recognised the non-cognitive skills that are initiated and or improved in
the process of self-assessment. The following response by S2 indicates that self-
assessment leads him to planning and determination to accomplish the plan
independently and thereby become confident.
I say it have many advantage when you discuss about the criteria we learn we
know what we must do at the end the last objectives of the topic or the course so I
know what must I do which book I must read or I ask able students to fulfil that
criteria that objectives after I give mark for myself using the criteria, I can check
where is my problem my error so I try to correct it for the next time by this
process my speaking ability or writing ability develop and improve so I have self
confidence
Other contributions of self-assessment to development of non-cognitive aspects like
intrinsic motivation and trusting instructor’s mark were also mentioned as in S6’s
147
response. He clearly stated that just participating in assessment by itself creates happy
feeling (motivation) and even if one cannot self-assess (failed to use the criteria) can trust
the marking and grading of the teacher.
when you participate in self assessment it is good I became happy because I don’t
see this kind of teaching learning process another advantage is when the teacher
give me low mark I am not angry because we know the criteria and I may give
wrong mark not following the criteria but the teacher depend on the criteria so I
accept
The students also expressed the experiences they derived from the training and could see
the all rounded potential benefits of participating in self-assessment even if they could
not assess themselves accurately. S4 related his experience of participating in this study as
follows.
... For example, I don’t have seen any such activity before. After you asked to do
starting that research I saw many things, for example, we participate in some
activity and after that we give mark for ourselves I learned many things from this
how the criteria is already set depending on the criteria when I consider my
performance I considered myself where is my lack where is my strength where is
my weakness. In that case now I could get many ability and learn many things
from this through process I hope I will change it due to this reason it has many
advantages. If I don’t did this activities I don’t know this criteria and how I can
improve my weakness. Now I get self-confidence. Due to that reason the students
setting criteria and giving mark for their performance is crucial to improve their
knowledge.
Concerning the downside of LSA, the students repeated the problem of possibility of
error fossilization they mentioned during the pre training interview. For example S8
expressed her idea that if a student fails to catch his/her own error or overlook it, there
would be no chance of correcting it. In addition, the students observed that if self-
assessment is not valid, it is irrelevant. S5 commented that a mark that does not correctly
148
describe our performance does not help to improve our learning, i.e., if students do not
assess themselves genuinely they cannot learn. In his words:
If we make bias we cannot know our weak side and strong side. If we neglect
[overcome] the sense of selfishness we can evaluate ourselves depending on our
work. If I give simply high mark to get A or B for myself that is unnecessary it is
beyond my capacity I should learn from my mistakes. If I give something
equivalent to my capacity I also try to learn from the mistakes and improve my
capacity
One more problem the students observed was related to time. They felt that it is ‘time
consuming’. S4 said “The disadvantage is if students have many classes in the week they
have no time to look at the video and give mark for themselves.”
To sum up, in this post intervention interview the respondents generally seemed to have
favourable conception about LSA. They perceived classroom assessment as having
primarily the purpose of improving learning and it is a job to be shared between and
among the students and the teacher. They also described LSA in its advanced and
participatory sense. They reflected the belief that it is appropriate and necessary to put
learners in the whole process of assessment including giving marks to their own work.
Their awareness of its role in improving students’ learning was also improved when
compared to the pre-intervention interview response, but they mentioned few cognitive
benefits. However, they were divided over the validity of the marks provided by the
students. Some remained pessimistic concerning the learners’ honesty. Regarding its
downside, they perceived two problems: probability of overlooking errors that is
connected to its validity and time consuming on the part of the students.
149
4.1.3.2 Data from Students Questionnaires
As mentioned in Section 3.4.2 above, the students’ and the instructors’ questionnaire are
the same in their content and form, but some statements in the students’ questionnaire
were worded differently in an attempt to make the language easy for the students. Parallel
to the interview, the students’ questionnaire was administered before the intervention and
after the training so that the results could be compared to see the effect of the training on
their responses.
To make the comparison easier and convenient, and to reduce the number of tables used
to display the data so that readers would not be bored with referring to the unnecessarily
many tables, the results of a cluster of statements for both the pre- and post intervention
are presented together in one table. Again, to make the size of the tables manageable, the
5-point scales were reduced to three: the Strongly Agree (SA) and Agree (A) scales were
combined and assigned the value 1, the Undecided scale was assigned 2 and the Disagree
and Strongly Disagree were assigned 3.
The pre-intervention questionnaire was distributed to 46 students in a classroom and all
the papers were collected back. However, three papers were discarded because they were
not filled out properly. The post intervention questionnaire was distributed to and
collected from 44 students and all were used. Therefore, 43 papers for the pre-
intervention and 44 papers for post intervention were analysed. The statements were
analysed in theme based six clusters as presented is subsequent sections.
150
4.1.3.2.1 Main Purpose of Classroom Assessment
The first theme of this questionnaire is the purpose that students attach to classroom
assessments. As done in analysing the instructors’ questionnaire, the statements in this
theme are put into two sub categories: describing achievement and improving
achievement. The first sub-category includes four statements that relate the purpose of
classroom assessment to describing the learners learning outcomes, and the second
category includes a cluster of three statements that relate classroom assessment to
improving students’ achievement. The results are presented in Tables 16 and 17 below.
In Table 16, the students’ responses to the group of statements related to purpose of
classroom assessment in the pre-intervention questionnaire show that most of the
respondents ( more than 72.1%) agreed to all of the four statements, and few respondents
(less than 11.6 % and less) disagreed. The rest (16.3% and less) were undecided. In
addition, the mean values of all the four items range from 1.14 to 1.40 approximating the
value that corresponds to agreement scale. This indicates that the students had the belief
that instructors do classroom assessments primarily to describe learners’ learning
outcomes.
In their response to the same group of statements after the intervention, majority of the
respondents (68.2% and above) disagreed to all the four statements, 22.7% and less
agreed while l8.2% and less remained undecided. In addition, the mean values for all the
statements are 2.45 and greater, in between the undecided and disagreement scale. This
indicates that more than 68% of the respondents do think that the primary purpose of
classroom assessment is not describing achievement.
151
Comparison of the students’ pre-intervention and post intervention responses indicates
that while majority of the students had the belief that describing the students’
achievement is the main purpose of classroom assessment, this view was changed in their
response to the post intervention questionnaire.
Table 16 Classroom assessment for describing learners' achievement
Item
No.
Statements
The main
purpose of
classroom
assessment is
...
Pre-intervention
Post-intervention
To
tal
Mea
n
Extent of agreement
Tota
l
tota
l M
ean
med
ian
Extent of agreement
1 % 23 % 3 % 1 % 2 % 3 %
1
to check
whether or not
students have
mastered what
they have
learned
36 83.7 5 11.6 2 4.7 43 1.21 9 20.5 4 9.1 31 70.5 44 2.5
2
indentifying
the strengths
and weakness
of the learners
31 72.1 7 16.3 5 11.6 43 1.40 8 18.2 5 11.4 31 70.5 44 2.52
7
grading and
categorising
students
39 90.7 2 4.7 2 4.7 43 1.14 4
9.1
8
18.2
.
32 72.7 44 2.64
9
checking
whether or not
the course
objectives are
achieved
34 79.1 4 9.3 5 11.6 43 1.33 10 22.7 4 9.1 30 68.2 44 2.45
Table 17 indicates that in their response to the pre-intervention questionnaire the
percentage of the respondents agreed to all the three statements that relate purpose of
classroom assessment to improving learning outcomes ranges from 73.8% to 85.7. The
152
mean values also range from 1.17 to 1.40 nearer to the agreement scale. This indicates
that the students had the view that teachers do classroom assessment to help learners
improve their achievements.
Table 17 Classroom assessment for improving learning
Item
No
.
Statements
The main
purpose of
classroom
assessment is ...
Pre-intervention
Post-intervention
T
ota
l
M
ean Extent of agreement
T
ota
l
tota
l
Mea
n
med
ian
Extent of agreement
1 % 2 % 3 % 1 % 2 % 3 %
3 helping learners
to improve
their learning
36 85.7 5 11.9 1 2.4 42 1.17 41 93.2 3 6.8 0 0 44 1.07
5 providing
feedback to
students about
their
performance
33 76.7 6 14 4 9.3 43 1.33 43 97.7 1 2.3 0 0 44 1.02
10 providing
information for
the teacher on
the way s/he
teaches
31 73.8 5 11.9 6 14.3 42 1.40 38 88.4 4 9.3 1 2.3 43 1.14
The post intervention data also show that majority of the students agreed to all of the
same statements. Moreover, the percentage of the respondents who agreed to the
statements increased and the mean values drew closer to the agreement scale. At this
time, the percentage of the agreement ranges from 88.4 to 97.7, and the mean values are
between 1.02 and 1.14.
Comparison of the result of pre-intervention data on Table 16 and 17 indicates that most
of the respondents agreed with the two contrasting propositions, i.e., classroom
153
assessment is done primarily for describing learners’ achievement and for improving
learning. This indicates that the students had not had a clear awareness on the main
reason for which the teachers carry out classroom assessment; they did not differentiate
between the contrary suppositions about the main purpose of classroom assessment. This
failure to make distinction between the primary and secondary purposes of classroom
assessment was cleared in the data after the training. Their responses to the post-
intervention questionnaire show majority of the students’ disagreed with the propositions
supporting the describing purpose, and strengthened their degree of agreement with the
improving purpose.
Therefore, from the analysis of the data in Tables 16 and 17, it is possible to see that there
is a change in the students’ thinking about the main purpose of classroom assessment
after the intervention; it moved away from describing to improving. Thus, it can be said
that the training has had an effect on the students’ view regarding the main purpose of
classroom assessment.
4.1.3.2.2 Whether Students should be Involved in the Assessment Scheme
The next cluster of questions is related to power relationship between students and
teachers regarding classroom assessment. Before the intervention, in response to the four
statements related to the belief about who should do the assessment, the respondents
exhibited two self-contradicting views. As Table 18 shows, 76.7% and 90.5% of the
respondents agreed respectively to items 4 and 11 that suppose learners to be involved in
assessing themselves. Again, 65.1% and 67.4 of the respondents respectively agreed to
the items 14 and 20 that suppose an idea contradicting to the former. In addition the
mean values for the four statements range from 1.14 to 1.42, approximating the
154
agreement scale. This confusion can be related to their understanding about the concept
of LSA as seen in Section 4.1.3.2.3 below.
Table 18 Appropriateness of learners' involvement in assessment process
Item
No
.
Statements
Pre-intervention Post-intervention
To
tal
Mea
n Extent of agreement
To
tal
tota
l M
ean
med
ian
Extent of agreement
1 % 2 % 3 % 1 % 2 % 3 %
4 Students should
participate in
assessing their
own work
33 76.7 5 11.6 5 11.6 43 1.35 43 100 0
0 0 0 44 1
11 One of the
students’ role in
the teaching
learning process
is participating in
the whole process
of assessment
38 90.5 2 4.8 2 4.8 42 1.14 43 100 0
0 0 0 43 1
14 It’s the lecturer’s
job to evaluate
students’ work
28 65.1 12 27.9 3 7 43 1.42 0 0 4 9.3 39 90.7 43 2.91
20 I consider the real
assessment to be
assessment that is
done by the
teacher
29 67.4 10 23.3 4 9.3 43 1.42 7 15.9 12 27.3 25 56.8 44 2.41
In their response to the post intervention questionnaire, the respondents cleared this
confusion by agreeing to the proposition that students should participate in assessment
process, and disagreeing to the opposite proposition. The post-training data indicates that
all of the respondents agreed with items 4 and 11, and majority, 90.7% and 56.8%,
disagreed to items 14 and 20 respectively. But the percentages of the abstainers and those
who disagreed (43.2%) also and indicates that still there are considerable percent of
155
respondents who, regardless of the training, hold the view that real assessment is that of
the instructor.
4.1.3.2.3 Description of LSA
The other theme is related to the description of the concept of LSA itself. The students
responded to a cluster three of statements that describe LSA in its different forms. The
data from the pre-training questionnaire in Table 19 indicates that most of the
respondents, 86% and 83.7%, agreed to items 6 and 12 respectively, which give the
simplest form of LSA that is related to the technical knowledge level. The mean values of
these items are 1.23 and 1.21. In their response to item 8 which describe LSA in its
relatively advanced and related to communicative and interpretive level, majority of the
respondents neither agreed nor disagreed, 51.2% remained undecided, while 34.1%
agreed and 14.6% disagreed. The mean value for this item is 1.8 that approaches 2, the
scale for ‘UD’. This implies that they understand LSA in the simplest form that is related
to the technical knowledge level. This conforms to their response to the pre-intervention
interview.
The post-training data indicates that there are changes in the patterns of the learners’
responses to this group of statements. Most of them, 88.4% and 93%, disagreed to the
lowest and technical level description of LSA in items 6 and 12 respectively, with mean
values 2.84 and 2.88 nearing to the scale for disagreement; and 95.5% of the respondents
agreed to the statement in item 8 which gives the relatively sophisticated description
related to communicative and interpretive level. The mean value is also 1.07. This
indicates that the training helped them to advance their understanding of the concept of
LSA.
156
Table 19 Description of LSA
Item
No. Statements
Learners’ self-
assessment
means ...
Pre-intervention Post-intervention
Tota
l
Mea
n Extent of agreement
Tota
l
tota
l
Mea
n
med
ian Extent of agreement
1 % 2 % 3 % 1 % 2 % 3 %
6 students marking
their own test or
assignment using
answer keys
provided by the
instructor
37 86 2 4.7 4 9.3 43 1.23 2 4.7 3 7 38 88.4 43 2.84
8 participating in the
process of
developing criteria
and standards and
then marking
one’s own
performance
14 34.1 21 51.2 6 14.6 41 1.8 42 95.5 1 2.3 1 2.3 44 1.07
12 checking how well
you understood a
topic while you
are learning or
studying a subject
36 83.7 5 11.6 2 4.7 43 1.21 2 4.7 1 2.3 40 93 43 2.88
4.1.3.2.1 Validity of LSA
The following group of three statements sought the students’ degree of agreement to the
propositions related to the validity of their self-assessment before and after the training
on LSA of oral performance. As Table 20 below shows, in their response to the pre-
intervention questionnaire majority of the respondents rejected the view that LSA would
be valid. The data shows that 58.1% of the students disagreed to the propositions that
students would be honest in assessing themselves, item 22; and considerable percentage
of the respondents (30.2) were unable to decide. The mean value of this item is also 2.47,
157
half way between the scales for Undecided and Disagree. In addition, 79.1% disagreed to
item 26 that states LSA could count to students’ final grade.
Table 20 Validity of LSA
Moreover, 81.4% of the respondents showed their agreement to the statement in item 23
which states that students would not understand the assessment criteria. This indicates
that majority of the students had the belief that LSA cannot be valid.
The post intervention data shows minor changes in the patterns of the subjects’ response
to the three items. Majority (68.2%) of the respondents neither agreed nor disagreed to
item 22 that supposes students are honest; and less than half (45.5%) agreed to item 26
that states LSA can go into the students’ final grade while the rest were undecided or
disagreed. However, about 84% disagreed to item 23 which states that students can’t
Item
No.
Statements
Pre-intervention Post-intervention
Tota
l
Mea
n Extent of agreement
Tota
l
tota
l
Mea
n
med
ian Extent of agreement
1 % 2 % 3 % 1 % 2 % 3 %
22 Students
assess their
own
performance
honestly
5 11.6 13 30.2 25 58.1 43 2.47 5 11.4 30 68.2 9 20.5 44 2.09
23 It would be
difficult for
students to
understand the
assessment
criteria
35 81.4 4 9.3 4 9.3 43 1.28 4 9.1 3 6.8 37 84.1 44 2.75
26 Students’ self-
assessment can
count to the
final grade
1 2.3 8 18.6 34 79.1 43 2.77 20 45.5 18 40.9 6 13.6 44 1.68
158
understand the marking criteria. The mean values of these items are 2.09, 1.68 and 2.75
respectively.
This result implies that after the intervention the subjects accepted the propositions that
students can understand the assessment criteria, but were divided over the idea that the
mark would go into the final grade. This is because they did not change their beliefs that
students cannot be honest in marking their own performance; they are still not confident
to say students can be honest. The percentage of the respondents who could not decide on
and/or disagreed to the proposition that the mark from LSA can count to final grade is
greater than the respondents who agreed (54.5). Thus, the effect of the training on this
theme can be said minimal.
4.1.3.2.2 Benefits and/or Problems of LSA
There are two subcategories in this theme. The first contains a group of five statements
related to propositions about the cognitive and meta-cognitive benefits and the second
contains five statements about non-cognitive benefits. The data on these subcategories
are presented in Tables 21 and 22 respectively.
As Table 21 shows, in the pre-intervention questionnaire, most of the respondents agreed
to all the four items that mention: recognizing errors easily (item13), improving language
skills and knowledge (item16), understanding the quality of performance expected of
them (item 21), and thinking what and how much to do to achieve the learning objectives
(item 29) as the contribution of LSA to cognitive development in some way.
159
Table 21 Cognitive benefits of LSA
Item
No
.
Statements
Pre-intervention Post-intervention
To
tal
Mea
n Extent of agreement
To
tal
tota
l
Mea
n
Med
ia
n
Extent of agreement
1 % 2 % 3 % 1 % 2 % 3 %
13 If I assess my own
work, I would be
able to recognize my
errors easily
33 78.6 6 14.3 3 7.1 42 1.29 42 97.7 1 2.3 0 0 43 1.02
16 Being involved in
assessment of my
own learning
outcomes helps me
to improve my
language skills and
knowledge
35 81.4 5 11.6 3 7.0 43 1.26 39 90.7 3 7 1 2.3 43 1.12
18 Assessing my own
performance has no
importance to the
improvement of my
language skills and
knowledge
1 2.3 8 18.6 34 79.1 43 2.77 1 2.3 0 0 43 97.7 44 2.95
21 Self-assessment
enables me to
understand the
quality of
performance
expected of me
36 83.7 5 11.6 2 4.7 43 1.21 41 93.2 2 4.5 1 2.3 44 1.09
29 Assessing myself
makes me think what
and how much to do
to achieve the
learning objectives
37 86 4 9.3 2 4.7 43 1.19 39 88.6 2 4.5 3 6.8 44 1.18
The percentage of the respondents agreed to these statements ranges from 78.6 to 86 with
the mean values ranging from 1.19 to 1.29. In addition, most of the respondents (79.1%)
also disagreed to the supposition that LSA has no any contribution to learning (item 18).
The mean value of this item is also 2.95 approaching 3 which is the disagreement scale.
160
This implies that the subjects view LSA as beneficial to the development of their
cognitive and non-cognitive abilities. However, these beneficial effects they identified are
not congruent with the type of LSA they identified as revealed in their description of
LSA in the pre-intervention data in Section 4.1.3.2.3 above.
The post intervention data shows result similar to the pre-intervention one. Their
responses to these items after attending the training show the subjects strong agreement
to the positive statements and strong disagreement to the negative statement. More than
88% of the students agreed to the cognitive and meta-cognitive benefits mentioned in
items 13, 16, 21 and 29. Moreover, 97.7% of the respondents disagreed to item 18 that
does not consider any benefit of LSA, This implies that the subjects hold the view that
the practice of self-assessment contributes to the development of their cognitive and
meta-cognitive ability. This time, as the respondents improved their understanding of the
concept of LSA to the sophisticated level as indicated in the post intervention data in
Sections 4.1.3.1.3 above, it is possible to say that this view emerged out of their clear
consciousness of the advanced type of LSA they gained from the training.
In the data from the pre-intervention questionnaire displayed in Table 22, the respondents
identified two ways by which LSA contributes to their non-cognitive aspect of learning.
Majority, 90.7% agreed to the benefit of creating feeling of responsibility (item 15) and
72.1% agreed to the benefit of creating willingness of working much and harder (item
27). The mean values of these items are 1.16 and 1.35 nearing the agreement scale.
Their response to the rest statements is not uniform. In response to the statement that
LSA minimises cheating and misbehaving (item 19), majority, but less than half (47.6%)
161
remained undecided and considerable proportion (42.9%) disagreed. In response to the
statement that LSA makes students trust instructors marking (item 24), most of the
respondents (60.5%) disagreed and substantial proportion (32.6%) remained undecided.
In addition, 55.8% of the subjects neither agreed nor disagreed to the statement that LSA
causes boredom.
Table 22 Non-cognitive Benefits of LSA
Item
No.
Statements
Pre-intervention Post-intervention
Tota
l
Mea
n
Extent of agreement
Tota
l
tota
l
Mea
n
med
ian Extent of agreement
1 % 2 % 3 % 1 % 2 % 3 %
15 Assessing my own
work makes me
feel responsible for
my learning
39 90.7 1 2.3 3 7 43 1.16 41 93.2 3 6.8 0 0 44 1.07
17 If I am to mark my
own work, I would
get bored
9 20.9 24 55.8 10 23.3 43 2.02 4 9.1 8 18.2 32 72.7 44 2.64
19 Learners’
assessing their
own work helps to
minimise cheating
and misbehaving
4 9.5 20 47.6 18 42.9 42 2.33 33 76.7 9 20.9 1 2.3 43 1.26
24 Assessing my own
work makes me
trust the
instructor’s
marking
3 7 14 32.6 26 60.5 43 2.53 4 100 0 0 0 0 44 1
27 Self assessment
makes me spend
much of my time
on studying and
working hard
31 72.1 9 20.9 3 7 43 1.35 43 100 0 0 0 0
4
3
1
162
This result generally shows the respondents did not have a clear idea about the benefits
and problems related to LSA. This could also be related to their understanding of type of
LSA; their response is more likely to come from the concept they had had about it.
In the data from post-training questionnaire, the respondents responded in a clear pattern.
High proportion (ranging from 76.7% to 100%) agreed to the positively worded
statements related to benefits of LSA, i.e., creating sense of responsibility (item 15),
minimising cheating and misbehaving (item 19), creating feeling of trust in the
instructor’s marking (item 24), and encourage spending much time on working (item 27).
The mean values of these items also range from 1.00 to 1.26 and correspond to the
agreement scale. In addition, 72.7% of the subjects disagreed to the statement that LSA
causes boredom. The mean value of this item is 2.64 approaching the disagreement scale.
This view is also in line with the concept of LSA in its advanced level that the students
could conceptualise after the training. Therefore, it could be attributed to the effect of the
training.
4.1.3.2.3 Relevance of LSA
The last theme to which the students were asked to respond was whether LSA is relevant
to the teaching learning process at all. They responded to two statements of which one is
negatively stated. Table 25 below present the pre- and post intervention data.
In the pre-intervention questionnaire, as Table 23 shows, 93% of the respondents agreed
to the proposition that LSA is an important aspect of the learning process (item 25), and
most of them (65.1%) disagreed to the statement that supposes LSA as irrelevant (item
28). This indicates that they considered LSA as the integral part of the teaching learning
163
process. However, as it is indicated elsewhere in the above sections, they were referring
to the LSA in its simplest and lowest level.
Table 23 Relevance of LSA
Item
No
.
Statements
Pre-intervention Post-intervention
To
tal
Mea
n
Extent of agreement
To
tal
tota
l
Mea
n
med
ian Extent of agreement
1 % 2 % 3 % 1 % 2 % 3 %
25 Learner’ SA is an
important part of
learning teaching
process
40 93 3 7 0 0 43 1.07 42 95.5 1 2.3 1 2.3 44 1.07
28 Students’ self-
assessment is just
adding burden to
both teachers and
students
5 11.6 10 23.3 28 65.1 43 2.53 0 0 0 0 44 100 44 3.00
The data from the post-intervention questionnaire indicates that the students’ response is
similar to the former one, but with increased proportion. Almost all, 95.5% agreed to item
25 and all of them disagreed to item 28 with mean values of 1.07 and 3.00 respectively.
This shows that the respondents view LSA as the important part of their learning
activities. This time, it is possible to say they referred to the advanced type of LSA which
they derived from the training.
4.2 Validity of LSA of English Oral Performance
Data on the validity of learners’ assessment of their English oral performance was
generated using the English oral performance assessments tools: classroom oral
performance tasks and marking schemes, prior to and after the training intervention. The
164
data prior to the intervention was used to answer RQ2 and to demarcate the baseline for
the extent of validity for the experimentation.
Similar to the data on the students’ conception, the data on the validity of LSA of English
oral performance prior to the intervention was collected and analysed twice, labelled
Vpre1, and Vpre2, to get a stable baseline. The analysis showed that there was little
difference between the results of these two data. Therefore, the result of the second round
data (Vpre2) was taken as the baseline of the data on validity of the self-assessment. To
avoid unnecessary repetition of reporting the same result, summary of the statistics of the
first round data (Vpre1) is given in Table 24 and the analysis of the Vpre2 data is
presented here.
The validity of the self-assessment of oral performance was assessed by comparing
scores from instructor’s marking to that of the students’ self-marking. Therefore, the
English oral performance assessments both before and after the intervention were marked
by the students themselves and the instructor so that the validity of the marking would be
judged by comparing the two markings. The descriptive statistics (mean values) and
inferential statistics (the t-test) were computed using the IBM SPSS statistics 20 software
in analysing the data.
Table 24 Statistical summary of the first round oral performance score (Vpre1)
Students’ scoring Instructor's scoring
M SD n M SD n
7.64 .870 43 4.18 1.628 43
165
Students’ scoring (M = 7.64, SD = .870), instructor’s scoring (M = 4.18, SD =1.628),
t(42)= 11.85, P < .005 (two-tailed), mean difference = 3.46, 95% confidence interval
range 2.87 to 4. 04
The second round pre-intervention oral performance assessment (Vpre2) was administered
to all the 46 students. Among these, two students did not turn up for marking their own
performance and two other students’ marking of their own performance was discarded
because they did not follow the marking scheme correctly. Therefore, the result of 42
students was subjected to analysis. To make the comparison easier and convenient, the
result of the statistics of the pre- and post-intervention data is displayed in one table, Table
25.
As the Table shows, the descriptive statistics of the students’ scoring and the instructor’s
scoring of the pre-intervention assessment shows that the mean value of the pre
intervention students’ scoring is greater than that of the instructors’ scoring: 7.2667 >
4.9250. To see if this difference is significant, a paired-sample t-test was conducted. The
test indicated that there is statistically significant difference between the students’ scoring
(M = 7.2667, SD = .7500) and instructor’s scoring (M = 4.9250, SD = 1.6182), t (41) =
10.892, P < .005 (two-tailed). The mean difference between the scores was 2.3416 with
99% confidence interval ranging from 1.76097 to 2. 9223. The figure P < .005 (two-
tailed) indicates that the probability of attributing the difference between the mean values
to mare chance or sampling error is less than 5% and there is real difference between the
mean values. This indicates that the students overestimated their own oral performance
compared to the instructors’ scoring.
166
4.3 Effect of the Training on Validity of LSA of English Oral
performance
The post-intervention performance assessment was administered to 45 students because
one student was absent. Among these, four students’ result was discarded because their
result of the test given to see the effectiveness of the training was not satisfactory (see
Section 3.5.2). In addition, three of these students did not mark their performance.
Hence, the result of 41 students was analysed.
Table 25 Descriptive statistics of the oral performance scores (pre- and post
intervention)
The descriptive statistics of the post-intervention scores shows the mean value of the
students scoring is still greater than that of the instructors’. The paired-sample t-test
conducted to see if this difference is significant indicated that the difference between the
mean value of the students scoring (M = 6.8805, SD = .7252) and the instructors scoring
(M = 6.6073, SD = .9305), t (40) = 1.858, p > .005 (two tailed) is not significant. The
mean difference of the scores is .2731 at 95% level of confidence interval ranging from -
.12439 to .67074.
Students’ scoring Instructor's scoring
M SD n M SD n
Pre intervention 7.2667 .7500 42 4.9250 1.6182 42
Post intervention 6.8805 .7252 41 6.6073 .9305 41
167
To examine the kind and magnitude of the relationship between the students’ self-
marking and the instructor’s scoring of the performance assessment, and to see whether
or not the intervention affected the relationship, Pearson product-moment correlation was
run for both the pre- and post intervention pairs of scores using the SPSS. The statistics
showed there was weak, positive correlation between the instructor’s and the students’
self-scoring of the performance before the intervention, r = .276, n = 42, p < .05 (2-
tailed). The correlation of the two sets of scores after the intervention was found to be
medium positive: r = .334, n = 41, p < .01 (2-tailed).
This result shows that there was a real difference that can’t be attributed to chance or
sampling errors between the students’ self marking and the instructor’s marking of the
students’ performance before the training, and the correlation was also weak, implying
the less validity of the students self marking. After the intervention, though there is still
difference between the students’ and the instructor’s scoring, the gap between the means
of the scores decreased to the non-significant level, and the correlation was also improved
to medium implying that the training improved the students’ scoring of their own work.
4.4 Summary of the findings from the Post Intervention Data
from the Non-Experiment Group
As discussed in section 3.2.2, data on the students’ conception of LSA and the validity of
LSA of the oral performance were collected from a non-experiment group of students
through group interview and English oral performance assessment after the analysis of
the post-intervention data to check the internal validity of the single-group quasi-
experiment,. Here, non-experimental group refers to a class of 15 students who were
similar to the subject of the study in every aspect. Analysis of these data indicated that
168
the findings from these data are almost the same to findings from the pre-intervention
data. To avoid redundancies in describing and explaining similar results again that would
bore the reader, the findings of these data are summarised here and the transcript of the
interview is put in Appendix 12.
4.4.1 Summary of Result of the interview
This interview was conducted with a group of five volunteer students. The analysis of
this interview indicated that the students had less positive conception of LSA. First, they
had the understanding that the main purpose of classroom assessment is giving grade and
categorising students, and assessment should be carried out by both the teacher and the
students. However, they described LSA in its simplistic form like self-correction using
answer key prepared by the teacher; therefore, the role of the learner in assessment as it
was perceived by the respondents does not conform with basic concept of LSA. Next,
when the students were briefed on what LSA ‘proper’ entails and asked their opinion
about whether or not it is appropriate that students participate in developing the standard
and criteria of assessment scoring of their own performance, they objected the idea and
expressed the view that students are not in a position to set criteria and score their own
work because they do not have the skill of assessment and also cannot be honest as
everyone needs to get the highest grade if possible. Lastly, when they were asked to
speculate if LSA has any advantage and/or problem, they could see that it helps to find
and correct errors, and repeated the problem of dishonesty.
When compared to the findings from the pre- and post intervention interviews, this result
is similar to the previous finding from the two rounds of interview prior to the
intervention.
169
4.4.2 Summary of the result of the data from oral performance
assessment (non-experiment group)
The oral performance assessment was administered to all of the 15 students in the class,
but the scoring of two students was discarded because they did not use the making
scheme properly. Therefore, the result of 13 students was analysed and summarised here.
Table 26 Descriptive statistics of the oral performance assessment scores of the non-
experiment group
Students’ scoring Instructor's scoring
M SD n M SD n
7.123 .724 13 5.473 1.121 13
The shows the mean value of the students scoring is greater than that of the instructors,
7.123 > 5.473; and the paired-sample t-test indicated that the difference between the
scoring of the students (M = 7.123, SD = .724) and the instructors scoring (M = 5.473,
SD = 1.121), t (12) = 5.251, P < .005 (two-tailed) is significant. The mean difference of
the scores is 1.650 at 95% level of confidence interval ranging from .965 to 2. 335.
This result shows that the students the difference between the students’ and the
instructor’s scoring is statistically significant implying that the students scoring is less
valid that that of the instructor. This result is also conforms to the results of the data from
oral performance assessment collected in two rounds before the intervention.
170
4.5 Discussions of the Findings
The preceding sections of this chapter presented analysis of the data. This section
discusses the results. The discussion focuses on merging the findings from the qualitative
and the quantitative data, looking relationship between the data from the students and the
instructors and, where available, relationship of the findings with other research. The
discussions of the findings are presented along the constructs studied following the
research questions that framed the study. First the findings related to conceptions are
discussed, followed by findings of the validity study. Then the effect of training on both
constructs is explained.
4.5.1 Conception of LSA
Conception of LSA was studied from both the instructors’ and the students’ perspectives.
The research questions leading this construct are:
1. What is the EFL instructors’ conception of learners’ self-assessment? (RQ1. A)
2. What is the university EFL students’ conception of learners’ self-assessment?
(RQ1 B)
To be able to access conceptions held by the two groups, interviews were used as the
main instrument and questionnaires came in as a complementary instrument. Both of the
instruments are built around six major themes related to knowledge and beliefs about
LSA. These include the main purpose of classroom assessment, whose responsibility is to
do the assessment, description of LSA, its validity, its contribution to learning, and its
relevance. The instructors’ conception is discussed first and the students’ conceptions
follow.
171
4.5.1.1 The Instructors’ Conception of LSA
Results of the analysis are discussed along the themes focused on. The analysis of the
questionnaire data indicates that subjects agreed to the all items they found in the list;
they did not differentiate between the subcategories of describing achievement and
improving learning as the primary or secondary purposes of classroom assessment.
However, the analysis of the interview indicates that the instructors identified auditing
and describing learners’ achievement as the prior purpose of classroom assessment, and
mentioned improvement as peripheral consequence. This discrepancy between the
findings from the interview and questionnaire is indicator of the subjects’ less clear
understanding about purposes of classroom assessment. Nonetheless, the probing
questions during the interview helped to identify the kind of belief they had about
purpose of classroom assessment.
Taking the result of the interview for granted, such thinking of the instructors is
consistent with findings in some studies and inconsistent with some others. For example,
in Harris and Brown’s (2008) study of New Zealand primary and secondary school
teachers’ conceptions of the purpose of assessment, it is reported that the teachers
identified seven categories of description. Based on the degree of importance perceived
by the respondents, descriptions related to improvement are ranked 4th
and below, the
others like grading and reporting being at the top. However, in Shing and Fai’s (2007)
study of conception about assessment of Mainland China college lecturers in a
vocational and technical institute showed their highest agreement of all dimensions to
proposition that assessment improves quality of teaching and student learning.
172
With respect to the belief about where the responsibility of classroom assessment lies, the
analysis of both the interview and the questionnaire indicated that instructors assigned the
responsibility of assessment exclusively to the teacher. Even, few of them who think that
it should be shared by learners were not referring to the full-fledged involvement of the
learners, as it is evident from their description of LSA in the interview and the
questionnaire as well. Though there is no empirical literature on such cases, such view of
the instructors is clearly the reflection of one of the causes of teachers’ perception of LSA
aired in the conceptual literature. Instructors who came through educational culture where
teachers exercise official authority and have fixed picture of their role are very reluctant
to accept the idea that learners should be permitted to participate in their own assessment
and to devolve or share with the students any aspect of their responsibility in assessment
(Smith, 2000; Tudor, 1996).
The instructors defined LSA as some intentional or unintentional learners’ activities like
self-checking and self-reflections as when they get back their test papers and see how
well or bad they have done. This is the technical knowledge level of self-assessment that
does not require the participation of learners in the process of identifying standards and
setting and using criteria. Such teachers’ thinking about LSA can be associated with
Tan’s (2007) study in which academics identified three types of LSA of which the first
one is teacher-driven self-assessment. It is the first type of self-assessment as classified
by Boud and Brew (1995) and is characterised by teacher’s control. Less sophisticated
conception of LSA given by the instructors is also the reflection of the inconsistent and
sometimes misleading meaning of LSA used in the literature (Boud & Falchikov, 1989;
Boud & Brew, 1995).
173
Concerning the appropriateness and validity of LSA, the analysis of the interview showed
that the instructors reflected mixed and even self-contradictory views; while some seem
to support partial involvement, but not whole-heartedly, others seem to allow whole
involvement, and others opposed it at all. In addition, when it comes to the acceptability
of their self-marking, all were pessimistic and frequently mention learners’ inability and
dishonesty. One instructor commented that students cannot be honest because in the
country, the criterion for selection for job is what is on paper, grade, not the knowledge
and competencies they actually developed in educational and training institutes. This
inconsistent and self-contradictory feeling of the instructors is also corroborated by result
from the questionnaire data in which they supported the involvement of the learners, but
doubted the validity of their own marking.
As Brown and Lake (2006:3) observed, it is possible for individuals to “simultaneously
hold multiple, and possibly even contradictory, conceptions of a phenomenon without
being disturbed by such contradiction”. Here also it seems to reflect a conflict between
personal ideology, for example reluctance to lose domination in the area of assessment,
that could have developed out of experience from contextual elements and a just
intruding feeling of being labelled ‘traditional thinker’. This is because their rejection of
the involvement of the learners in setting criteria as well as marking did not come from
practical valid knowledge and professional experience, as they were not practicing it; it
comes from their educational culture. As Brown with Bull and Pendlebury (1997:173)
noted, beneath such concerns lie their expectation of the “lecturer’s role” and fear of
“giving up some of their power”
174
In relation to the last theme, the contribution of LSA to the teaching learning perceived
by the instructors was insufficient. The result of the questionnaire was also not supported
by the interview. In the questionnaire, they agreed to all the items that stated the
cognitive benefits, but could not identify the non-cognitive benefits clearly. Nevertheless,
this was not confirmed by the interview. In the interview, focusing attention on errors
was the only cognitive benefits they mentioned; and confidence, responsibility,
commitment, and transparency were the non-cognitive benefits. This indicates the
inadequacy of the instructors’ awareness of the contribution that LSA has to students’
cognitive development.
Generally, the subjects’ belief system about LSA is more or less consistent and
interrelated to each other along all the thematic categories focused on. The findings from
the interview in most cases are corroborated by the questionnaire. In addition, these
results also support the findings reported in empirical literature and the ideas and
concerns reflected in the conceptual writings.
4.5.1.2 The Students’ Conception of LSA before the intervention
The findings from the analysis of the pre-intervention interview and the questionnaire
data on the students’ conception of LSA are similar to that of the instructors’ discussed
above in all the thematic categories. Like the instructors, the students did not differentiate
between the main and peripheral purposes of classroom assessment in the questionnaire
and the analysis of the interview data show that the students attributed classroom
assessment to measuring achievement and controlling students’ behaviour that are
generally put as describing achievement. They perceived improving learning as the
175
peripheral purpose for which teachers assess students in classrooms. This is similar to the
view of the instructors discussed above.
Assuming the result of the interview data, this finding is consistent with some other
similar studies. For example, Brown & Hirschfeld (2008) reported Zeidner’s (1992) study
of Israeli junior and senior high school students’ perception of purpose of classroom
assessment. This study showed that among the four purposes of assessment (summarising
student achievement, arousing student interest and motivation, evaluating quality of
teaching, and administrative purposes) offered to them, the students showed strong
perception of assessment as summarising students’ achievement than as improving
motivation or behaviour.
Regarding their understanding of the concept of LSA, the students’ description of LSA is
the same to that of their instructors: they related the technical knowledge level of self-
assessment. In both the interview and the questionnaire data, their description includes,
for example, actions of comparing one’s test result with that of a classmate. Such
perception of LSA is similar to what Tan, Teo and Ng (2011) identified as compliant self-
assessment that involves little, if at all any, understanding or emphasis on the standards
and criteria to judge their own learning against.
Such understanding of the concept of LSA also lead them to hold self-contradictory
views about the responsible party to do assessment and whether or not students should be
involved in setting criteria and marking of their own work. Having the simplest form of
self-assessment in mind, they unanimously agreed that assessment should be a joint
venture of the teacher and the students. When guided to the advanced level of LSA,
176
however, were divided over the view students should be involved in deciding on criteria
and marking, while some supported partial involvement others supported the full-fledged
involvement. This self-contradicting view was also observed in the analysis of the pre-
intervention questionnaire data. As that of the instructors’, such view of the students can
also be related to their educational ground. As Smith (2000:61) observed, in educational
cultures where “[l]earners and teachers have been given very specific roles in the
classroom it is difficult for them to change”. However, though some of the students seem
unwilling to do the ‘teacher’s job’, the students seem to be less conservative than the
instructors were.
Regarding the issue of validity of LSA, the students unanimously disagreed to the
proposition that LSA is valid in both the interview and the questionnaire data. The
students shared the instructors’ view and frequently referred to learners’ dishonesty and
incapability of using the assessment criteria. This is also an eco of the concerns popularly
raised in the conceptual literature, (e.g., Tudor, 1996).
Regarding the benefit of LSA in the interview, the students could speculate identifying
and correcting error as the only contribution of their self-assessment, but in the
questionnaire, agreed to all the four cognitive benefits and two of the non-cognitive ones.
This discrepancy between the interview and questionnaire data could be attributed to their
unclear understanding and lack of experience with LSA. However, though they could not
clearly see its contribution, both the students and the instructors considered LSA as the
necessary component of classroom teaching learning process.
177
Generally, in the results of this part the study the instructors and the students exhibited
more or less similar conceptions about LSA. They had the views and beliefs that
classroom assessment is done primarily to audit and describe learning outcomes; had the
elementary idea about what LSA is; were reticent about involving learners in the
complete cycle of assessment and validity of students’ marking of their own work; and
could see few benefits of LSA in improving learning.
As Pajares (1992) argues, teachers’ conception of the teaching learning practices is the
product of their educational experience as student. This strongly suggests that students
and teachers are likely to develop similar conception about the phenomena in education.
Therefore, similarities between the subjects’ conception is most likely because both the
instructors and the students share similar educational culture and experiences.
4.5.1.3 Validity of LSA of English Oral Performance
The next research question guided a part of this study concerning validity of LSA is:
To what extent can learners provide a valid assessment of their own English
language oral performances? (RQ2.)
This question was answered by comparing the students’ self-score (the score they
provided to their own English oral performance) against the instructors score using
descriptive and inferential statistics and correlations. The analysis of the data from the
pre-intervention oral performance assessment shows that there was significant difference
between the mean values of the two set of scores compared, the students’ scoring being
greater than that of the instructors’ (P< 0.005). In addition, the two sets of scores
correlated positively but poorly (r = .276). Taking the assumption that the instructor’s
scoring is a reliable scoring against which the validity of the students’ scoring could be
178
tested, these results show that the students scoring of their own work is less valid as
compared to that of the instructor.
Though it is difficult to compare results of the studies of validity of LSA because of a
number of conceptual and methodological inconsistencies among the studies (Boud &
Falchikov, 1989; Topping, 2003) this finding can be aligned with the findings of
relatively older studies reported in Boud & Falchikov, (1989) and with the relatively
recent ones reported in Ross (2006) in which the students overrated their performance
compared to ratings by external part like teachers and tutors. In a more recent study, Lew
(2009) also reported that students overestimated their performance as compared to the
grade given by the tutors. LSA was as weakly correlated (ranging from .07 to .31) with
the tutors’ grade.
Chen (2005) has also reported significant difference between self-assessment of Chinese
freshman students’ English oral performance and their teacher and peer. However, the
form of relationship between the teachers’ and the student’ scoring in this study was the
reveres. In Chen’s study the Chinese underrated themselves awarded themselves lower
marks than awarded by their teacher. As Chen speculated, source of such differences may
be the societal and educational cultural expectations of the students; “modesty is a virtue
in Chinese culture” (p. 254).
Generally, these findings strengthen the popular concerns mentioned in the conceptual
literature and echoed by the subjects of this study as well.
179
4.5.2 The Effect of the Training
The other focus of this study was how or the way training may influence the students’
skill of assessing their English oral performance and the conception they had held. This is
guided by the questions:
1. What is the effect of training on the validity of learners’ assessment of
their own English oral performances? (RQ3. A) and
2. How does training in self-assessment influence the learners’ conception of
this assessment mode? (RQ3. B)
4.5.2.1 On the Validity of LSA
The finding from this aspect of the study indicated that the training improved the
students’ skill of judging the quality of their own work. Comparison between the mean
values of scores given by the students and the instructor before and after the intervention
indicated that there had been statistically significant difference before the training while
the difference after the training was reduced to be statistically non-significant. The
correlation statistics also showed that the correlation between the two sets of scores was
improved after the training: the correlation had been positive but weak before the
training, and improved to be medium after the training.
This signifies that the training helped the students to modify their view of objectives of
learning English oral performance from the point of grading to the development of oral
skills, and to better understand marking criteria they used which in turn, encourage them
to move away from egocentrism (being totally subjective) to a little objectivity.
180
This finding is consistent with some other studies like Ross, Rolheiser and Hogaboam
(1998) in which grade 4-6 students trained in self-evaluation and the treatment group
became more accurate in their self-evaluation. Chen’s (2008) also trained a group of 28
Chinese EFL learners in assessing their own English oral performance and concluded that
the training improved the validity of learners’ self-assessment.
4.5.2.2 On the Students’ Conception of LSA
The analysis of the data after the intervention indicated that the training has affected the
subjects’ pattern of conceptions of LSA positively. Firstly, the findings from the per-
intervention data analysis showed that the students prioritised measuring and describing
achievement purposes of classroom assessment, but the result of analysis after the
intervention indicated that the students prioritised improving learning as the major
purpose of classroom assessment and put the others as secondary. Secondly, the subjects
modified their understanding of the concept of LSA and they could describe the advanced
form of self-assessment and the whole process that they failed to do in the pre-
intervention data. Next, the mixed and contradictory beliefs about the appropriateness
and necessity of learners’ involvement in the process of classroom assessment wholly
which had been observed in the analysis of the data before the training was cleared in the
data after the intervention. In addition, they could articulate more number of benefits they
could derive from self-assessment as compared to what they mentioned earlier. They
even witnessed that they started experiencing the advantage of participating in the
process of self-assessment and the training.
However, the participants remained more or less sceptical about the validity of the marks
given to one’s own work. In spite of the improved validity of their self-scores, the long
181
standing beliefs that ‘the teacher is the knower’ and ‘human being is selfish’ persisted in
the subjects’ belief system. This is contrary to Chen’s (2005) finding of study of
Taiwanese self-assessment of oral performance in EFL university classroom. In Chen’s
study, learners were asked to assess themselves and then to reflect on validity of their
self-assessment. The students reflected the belief that they assessed themselves honestly
and correctly. This difference may be attributed to the contextual factors in which the
studies were carried out. In the case of this study, it may imply that such thinking has
been ingrained in to the learners’ belief system and needs more training and practice to be
improved.
Generally, improvements have been observed in the students’ understanding, beliefs, and
perceptions of LSA and validity of learners’ self-scorings in the findings of post
intervention data. This change can be attributed to the effect of the training the subjects
underwent because it is very less likely that there is any extraneous variable to affect the
students’ conception and skills of assessing their own work in general and English oral
performance in particular during the time of the study. For instance, there was no any
indication that any of the instructors of courses other than Spoken English II (Enla 243)
raised issues related to self-assessment. In addition, as discussed in section 3.5.1, to
complement the limitation of the single-group quasi-experiment design, first the
measures of the dependent variables i.e., the validity of LSA of the oral performance and
the students’ conception of LSA were taken twice and stable baselines were demarcated.
Furthermore, after the post-intervention data were collected and analysed, additional data
were collected and analysed from another class of students but with similar profile. The
findings from these data also corroborated the findings of the pre-intervention data.
182
Therefore, the changes in the dependent variables after the intervention can confidently
be attributed to the function of the training.
4.6 Chapter Summary
This chapter presented the analysis of data and discussion of the findings. The
presentation and analysis of the data is organised thematically. The qualitative data is
presented in textual form supported by tables where seemed helpful. The quantitative
data were also presented in tables and followed by descriptions and interpretations.
The analysis of the data related to conception of LSA before the intervention revealed the
respondents’ understandings of the principal purpose of classroom assessment, and
description of their concept of LSA. Their views and beliefs about who should be
involved in carrying out classroom assessment, and whether or not students are in a
position to provide a dependable judgement of their own work; the cognitive and non-
cognitive contributions of and/or problems related to learners’ SA. The analysis of the
English oral performance assessment data before the training also marked the extent of
dependability of the scores provided by the learners of their own performance. The
analysis of the data collected after the intervention indicated the direction of effect of
training on the students’ conceptions of LSA and the dependability of the learners’
marking of their own work.
The discussion of the result generally indicated that first, the two groups of subjects of
the study held nearly similar understandings and beliefs about the themes raised and the
pattern of their thinking is consistent across the thematic categories. Next, it also
indicated that the students significantly overrated their own English oral performance
183
than the instructor. The post intervention result showed the difference between the
students’ scoring of their own performance and the instructors scoring decreased to a
non-significant level, and the pattern of their thinking about LSA was also changed.
It also confirmed that some of the findings are not peculiar to this study, i.e., similar
studies carried out elsewhere out of Ethiopian context have reported similar results.
However, some findings are related to the contextual factors like educational traditions
where the studies were conducted. The next chapter turns to the conclusions and
implications of the findings and finishes the thesis.
184
5 Chapter Five
Summary and Conclusions
5.0 Introduction
The preceding chapter dealt with the presentation and analysis of the data and discussion
of the findings. This last chapter winds up the thesis. First, it summarises the thesis and
presents the conclusions drawn from the findings. Then it discusses the implications of
the findings. It finishes by suggesting direction for further research.
5.1 Summary
LSA plays significant role in improving students’ learning. The conceptual and empirical
literature has confirmed that it enhances learning by positively influencing their approach
to learning and their emotional feeling and social interaction. However, the assumptions
individuals often make about assessment based on what they have experienced in the past
rather than in terms of the new circumstances that confront them affect the practice of
assessment. The conceptual and the empirical literature on LSA indicate a number of
problems that may impede the implementation of this assessment format. The problems
of conceptions and issues related to validity are among the areas of frequent concern. In
addition, LSA has been neither used nor studied in Ethiopian context. This initiated this
research.
This study attempted to find out the patterns of conception of LSA held by students and
instructors in the English program Wollega University. It also examined the extent of
validity of learners’ scoring of their English oral performance and the effect of training
intervention on the existing validity and the students’ conception.
185
The nature of the research and the need for validation through triangulation necessitated
the use of concurrent Mixed Method design. Thus, both qualitative and quantitative
methods were used. Corresponding to the methods, interviewing along with
questionnaires was the research strategy used to study the conceptions of LSA. Oral
performance assessment and single-group quasi-experiment were used to study the
validity of the self-assessment and the effect of training on the validity and the
conception. The tools for data collection were interview schedule, pre- and post
intervention performance assessment, Likert-type 5-point scale multi-item questionnaires,
and training materials. The tools were self-developed and validated by different
techniques like expert validation and test of internal consistency. Pilot study was also
carried out to test the effectiveness of the instruments and the strategies employed.
The study involved 5 EFL instructors and 46 EFL students. In the procedure of data
collection, first, the students and the instructors filled out the questionnaires and then
interviewed to elicit data on their conception about LSA. Next, the students assessed their
own oral performance on a task based on their spoken English II (Enla 432). Then, 12hrs
training on self-assessment of English oral performance was give to the students. Lastly,
the students were interviewed and filled out the questionnaire again.
The qualitative data from the interviews were transcribed coded and interpreted
qualitatively, and the quantitative data from the oral performance assessments and the
questionnaires were analysed using descriptive and inferential statistics. The analyses of
the qualitative data were presented thematically in text form and the quantitative data
were tabulated and described.
186
The findings from the analysis of the pre-intervention interview and questionnaire data
indicate that both groups of subjects held less favourable conceptions of LSA, and the
self-assessment of oral performance data show that the students marking of their own
work is not valid; there is statistically significant difference between the students’ and the
instructors’ scoring. The post-intervention data analysis suggests that the training
improved the students’ conception and the validity of their self-assessment. The findings
were discussed in relation to both the empirical and conceptual literature and conclusions
were drawn based on the discussion. The implications of the findings to classroom and to
the teacher development have also been discussed and the need and direction for further
research is suggested.
5.2 Conclusions
This study has shed light on the issues raised in the research questions and the following
conclusions are drawn from the findings. Regarding the conceptions about LSA, firstly,
the analysis of the qualitative and the quantitative data on the participants’ conceptions
indicated that there is established belief among the subjects that classroom assessment is
primarily done for describing learners’ achievement by grading and categorising and that
the responsibility of assessing students is exclusive domain of the teacher’s job. The
subjects’ belief of the purpose of classroom assessment is in harmony with their views of
retaining teachers’ responsibility of assessment: when the solely measuring and
describing purpose of assessment is prioritised, assessment is seen as something done by
teachers on students.
Secondly, the description of LSA provided by the subjects, though can be labelled as self-
assessment, is in the lowest level where the interest is on simplest form of knowledge that
187
does not involve analysis, synthesis, interpretation and communication. Such type of self-
assessment requires little participation in the complete cycle of assessment. Thirdly, the
subjects’ were sceptical about the appropriateness of allowing students in the whole
process of assessment and the validity of the scores provided by the students on their own
works. They were also unable to recognise the contribution of LSA to cognitive and non-
cognitive aspects of learner development.
Such conception of assessment in general and of LSA in particular is not in line with the
basic tenets of constructivism and humanism and the current thinking of assessment
reform. Classroom assessment should be done primarily not as a tool for checking on
students’ with the principal aim of grading and reporting students’ achievement but as a
tool for improving learning. Even when it is intended for measuring and describing
purpose, it should be guided by the learning outcomes that encourage deep learning.
In addition, classroom assessment to serve its purpose, i.e., improve learning outcome, it
should be linked to LSA that provides opportunity for the students at least to discuss
success criteria and mark their own performance followed by reflection on their
performance; the type of self-assessment that Boud and Brew labelled it Communicative
Interest.
Thus, it is concluded that conception of the subjects i.e., their understandings, perceptions
and beliefs about LSA is less sophisticated and less positive. It is not the kind of thinking
that tertiary level instructors and students are expected to hold so that their assessment
practice would be congruent with Active Learning approach.
188
One more conclusion related to the conception is that the instructors’ and the students’
conception about LSA is interconnected and comes from the same source. As can be seen
in the analysis sections, they both held very similar understandings and beliefs about the
major themes. This less favourable conception does not seem to emanate from the
knowledge and professional practice and experience. This is because LSA, even in its
simplest form, is rarely if at all, used at all levels of Ethiopian educational institutions.
The instructors’ responses to the interview have also confirmed that they were not using
self-assessment in classrooms. Thus, the source of the existing conception is not the
existing knowledge and practice but the past educational experiences that the subjects
underwent.
In relation to the second construct studied, validity of the LSA of English oral
performance, the comparison between the scores provide by the learners on their own
English oral performance and the instructor’ score indicates that the students score is
bigger than the instructors and the difference between the mean values is statistically
significant. The correlation analysis also shows positive, but poor correlation. Then it is
possible to conclude that the students’ scoring of their own English oral performance did
not have criterion-related validity, given the instructors marking as a reliable criterion
against which the students’ marking is checked.
The third focus of this study was effect of training on the validity and the conceptions.
Regarding the validity, the analysis of the scores provided by the students and the
instructor on the post-intervention oral performance assessment indicate that though the
students’ marking is still greater than that of the instructor, the difference is statistically
not significant. The correlation analysis also shows there is positive medium correlation
189
between the two scores. This means the training helped the students to assess their oral
performance fairly and to minimise the difference between the instructor’s and the
students’ scoring. The conclusion is, thus, the training has positively affected the
students’ ability to assess their own English oral performance in the classroom.
In relation to the conception about LSA, the analysis of the post-intervention data
indicated that there is an observable change in the learners’ pattern of understanding and
beliefs about all the themes in focus, but one. They, perceived improvement as the prior
purpose of classroom assessment; got the understanding of the advanced type of LSA;
developed the belief that learners should take part in the complete cycle of assessment;
and could speak the various advantages of being involved in self-assessment and its
drawbacks. Thus, it is logical to conclude that the training brought positive effect on the
students’ conception in general. However, the training did not considerably affect the
students’ belief about honesty and ability to provide valid and reliable information about
their own performance. This may be explained by the less flexible nature of belief than
knowledge. Although it is impossible to demarcate the boundary between belief and
knowledge, belief is more stable than knowledge and needs persistent persuasion to be
modified.
5.3 Implications
Higher education is at high stake in two interrelated senses. It should provide the
prospective graduates with all the cognitive and non-cognitive (meta-cognitive, social,
and affective) skills and competencies they need at the workplaces. It is also responsible
to build and sustain a learning society, citizens who developed self-learning skills so that
190
they become lifelong learners and be able to function effectively and efficiently in the
dynamic world (Knight 2005).
Assessment, as a major component of the teaching/learning process, plays the pivotal role
in achieving these desired goals of higher education. The practices of classroom
assessment, guided by conceptions, highly influence the route the learners take to their
learning. Assessment can serve multi-purpose, when the purpose of measuring and
describing is prioritised and/or perceived by the students that way (even if it is actually
not), the students use surface approach to learning like memorisation and rote learning.
They also work ‘to beat the system’ and prefer academic dishonesty to engaging in
meaningful learning; and the reverse is true when assessment is directed at the improving
purpose.
Cognizant of this effect of assessment and the need of improving students’ learning, the
Ministry of Education recommended the use of formative assessment in higher education
(MoE, 2002, 2003). However, the practice does not seem going in the way it was meant
to be. The conclusions drawn from the findings of this study indicated some major
pedagogical implications that help to react to such problem. The implications are related
to classroom teaching and teacher training.
5.3.1 Implication for Classroom
One implication of the findings from this research relates to tertiary classrooms. In this
study, the instructors’ and the students’ responses to the interview questions implied that
LSA was not the component of the EFL classroom instruction. As mentioned in Section
5.2 above, this is also true for other tertiary levels in the country. This absence of LSA
191
shows the incongruence among the components of teaching/learning approach opted for,
which in turn implies the difficulty of achieving the goal set for English language
teaching in general and that of oral performance in particular because it is what goes on
in the classroom that determines students’ learning.
Black and Wiliam (1998) and Clarke (2005) stress that formative assessment devoid of
LSA is very less likely to serve its purpose. To enhance the contribution of classroom
assessment to the improvement of students learning outcomes, LSA should be part of the
teaching learning process. Therefore, the instructors should be encouraged to recognise
this fact and consider using it as one of the variety of forms of formative assessment
when they plan for and design assessment for their classroom. It should also be noted that
the benefit of LSA by far outweigh the problems like invalidity and consuming time.
Therefore, more focus should be on its importance than its problems.
This study has also shown that learners held less positive conception that was not
favourable to practice LSA, and they lack the knowledge, skills, and experience of
assessing themselves. The conception that was developed out of the educational system
over long period may not change overnight. In addition, LSA by itself is a cognitively
demanding task, so it cannot be simply added to the usual classroom activities. As has
been proved in this study, training can help to improve the students' skills of self-
assessment and the less positive conceptions. Therefore, the learners should be provided
with explicit training and exercise integrated to their first year first semester subject area
courses so that they benefit from self-assessment during their stay in the university or
college.
192
5.3.2 Implication for in-service and pre-service Teacher Training
and Development
This research has also indicated that the conception held by the instructors is not
consistent with the current thinking of classroom assessment in line of the constructivism
and humanism. As conception is most likely to be derived from their experiences as
students in different levels and professional training, it is logical to say that the
instructors have either not come across topics related to LSA or given as much emphasis
as it deserves. One indication of this case is that there is little, if at all, topic of LSA in the
description of the course Language Testing (TEFL 608) for post graduates students of
MA in TEFL in Addis Ababa University and other older universities, where most of the
lecturers in the universities and colleges in the country are trained. In addition, from the
experience of the researcher, the material for HDP training that is intended for teachers’
professional development in tertiary education institutions does not give due emphasis to
the topic of LSA.
It follows that the conception held by the instructors is not fertile ground for LSA to be a
component of classroom assessment. From the teachers professional development point
of view, once such conception of the teachers is made explicit, it is necessary that the
conception be modified because conception colours and shapes teachers pedagogical acts,
and it is clear that the more positive and sophisticated conception of learning and
assessment they hold, the more likely they are willing to embrace it.
Therefore, in-service instructors should be provided with training opportunities aimed at
enhancing their professional skills and conceptions related to LSA. In addition, course
193
materials for the pre-service trainees should also be revised so that they contain sufficient
contents and methods to help learners get the knowledge and skills of LSA.
This could be done through both the in-service and pre-service teacher training programs
and the TPD programs established in educational institutions. The institutions that train
English language teachers for different levels should consider including and/or
emphasising on the topics of formative assessment in general and LSA in particular.
Moreover, the TPD programs like HDP that run in universities need to give considerable
attention to the topic related to LSA in their materials.
5.4 Limitations of this Study
This study has provided some insight into Wollega University EFL classrooms regarding
the instructors’ and the learners’ thinking about and the validity of LSA in relation to oral
performance. However, it does not claim to be perfect when scrutinized from
methodological perspectives.
One of the limitations of the present study is related to the extent of generalizability of
the findings. That is, it was relatively small in scale, involving only one university
among the 31 plus government universities and other many private university colleges.
Therefore, the number of students and instructors it involved cannot be said to represent
all tertiary level EFL learning and teaching community. It would have been more
generalizable to the larger Ethiopian tertiary level EFL context if it had involved more
universities. However, this research did not aim to develop systematic generalisation of
the findings to all Ethiopian university EFL classrooms, but rather to provide detailed
insight of the issues in focus in that particular context.
194
Nonetheless, although the students and the instructors participated in this study were not
necessarily the representatives of all the Ethiopian universities EFL students and
instructors, they reflect the current situation in relation to the issues of LSA treated in this
study because Ethiopian universities are almost similar in almost all aspects like
composition of the students and the instructors, their educational backgrounds and others.
Another limitation of this study is related to the design of the experimental part. As this
research used single-group quasi-experiment design, it does not normally be internally as
valid as the true experimental designs are supposed to be. However, one way such quasi-
experiment safeguards its internal validity is by taking repeated measures of the
dependent variables to establish the baseline of the behaviour/s prior to the treatment or
In this study, the dependent variables were measured twice because of the time constraint
during the data collection. Though the literature does not say the number of times the
variable in focus should be measured to get stable baseline, the researcher believes, and it
seems logical, that it should be taken more than twice. However, it should be noted that
there was almost no variation between the first and the second measures of the
behaviours. In addition, to complement this weakness, and see whether it is possible to
confidently attribute the changes in the dependent variables to the effect of the training, at
the end of the post intervention phase data was gathered from similar group of students
(2nd
year EFL students taking the same course) after almost a year. Findings from
analysis of this data were compared to the pre- and post-intervention findings, and
indicated that this latter group’s conception about LSA and the extent of validity of LSA
is almost the same to the findings from the pre-intervention data confirming that the
195
changes in the dependent variables (conception and validity) was the function of the
training intervention.
5.5 Recommendation for Further Research
First of all, as mentioned in the preceding section, one of the limitations of this study is
the extent of the generalizability of its findings. In addition, there is no any study on this
area in Ethiopia. Therefore, to confidently generalise the result of this kind of research to
the larger population of Ethiopian universities EFL classrooms, replicating the research in
a number of universities that are, at least statistically, representative of all universities is
in order.
Secondly, another limitation of this study is related to its design that could have affected
its validity. Therefore, to come up with a study that has very good internal validity,
interested researchers could replicate this study using the true experimental designs.
Next, LSA is a crosscutting issue that operates across all curriculum areas and school
subject at all levels. Though EFL classroom shares many features with the other school
subjects, there are also features that are peculiar to each subject and grade level.
Referring to others’ works, Winterbottom et al., (2008:17) note, “Different school
subjects have different bodies of knowledge, different sets of agreed procedures, different
social norms, and exist within different epistemological frameworks”. Then, LSA as an
integral part of the classroom events is influenced directly or indirectly by these
differences. Therefore, it is suggested that practitioners and researchers would do more
research to understand LSA in each of the specific context within EFL itself and other
subject areas and different grade levels.
196
Lastly, issues related to LSA go far beyond conception and validity. It is related to
culture, language proficiency, grade level, and etcetera. Thus, to develop comprehensive
understanding of LSA, researchers could attend to all these aspects. Above all, in
Ethiopian context where LSA is less practiced and hardly studied, researchers and
practitioners would contribute much to the improvement of the classroom
teaching/learning if they attempt these various aspects.
197
References
Abbot, M. G., Caccavale, T. S., & Stewart, K. (2007, Fall). Cognitive Benefits of
Language Learning. (G. L. Duke Tip, Interviewer)
Abiy Yigzaw (2005). Effects of Teacher Mediation on Students Approaches to Reading.
Unpublished PhD Thesis. AAU.
Alderson, J. C., Clapham, C. & Wall, D. (1995). Language Test Construction and
Evaluation. Cambridge: Cambridge University Press.
Alem Eshete. (1974). The Pre-war attempt to promote the use of the English language in
the education system of Ethiopia in place of French. The Ethiopian Journal of
Education , 6 (1), 65-84.
Allwright, D., & Bailey, K. M. (1991). Focus on the Language Classeroom An
Introduction to Classroom Research for LAnguage Teachers. Cambridge :
Cambridge University Press.
Ashcroft, K. (1996). Series Introduction. In K. Ashcroft, & D. Palacio, Researching into
Assessment and Evaluation in Colleges and Universities (pp. 1-26). London:
Kogan Page.
Atai, M. R., & Meratzadeh, B. (n.d.). The impact of Self, Peer, and Teacher Evaluation
on Iranian EFL Students'Writing Performance. Retrieved october 15, 2009, from
http://www.paaljapan.org/resources/proceedings/PAAL10/pdfs/atai2.pdf
Awol Endris (1999). Conceptions of Language Teaching and Learning and Classroom
Decision-making. A case Study of two High Schools. Unpublished PhD Thesis.
AAU
Bachman, L. F., & Palmer, A. S. (1996). Language Testing in Practice: Designing and
Developing Useful Language Tests. Oxford: Oxford University Press.
Bachman, L. (1990). Fundamental Considerations in Language Testing. Oxford: Oxford
University Press.
198
Baye Ashebir. (2006). Assessment Practice of EFL Teachers in Teacher Education:
Alternative Methods in Focus (The case of Dessie CTE 10+3 Programme. MA
Thesis . Addis Ababa University.
Berg, B. L. (2001). Qualitative Research Methods for the Social Sciences. Boston:
Pearson Education Campany.
Black, P. (1999). Assessment, Learning theories and Testing Systems. In P. Murphy
(Ed.), Learners, Learning and Assessment (pp. 118-134). London: Paul Chapman
Publishing.
Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for
Learning: Putting it into practice. New York: Open University Press.
Blaxter, L., Hughes, C., & Tight, M. (2006). How to Research. New York: Open
University Press.
Bloxham, S., & Boyd, P. (2007). Developing Effective Assessment in Higher Education:
a practical guide. New York: McGraw-Hill Education.
Boud, D. (1997). The Move to Self-assessment: liberation or a new mecanism for
opperation? Education-Line , 10-23.
Boud, D., & Brew, A. (1995). Developing a Typology for Learner Self Assessment
Practices. Research and development in Higher Education , 18.
Boud, D., & Falchikov, N. (2007). Introduction: assessment for the longer term. In D.
Boud, & N. Falchikov (Eds.), Rethinking Assessment in Higher Education:
Learning for the longer term (pp. 3-13). London: Routledge.
Boud, D., & Falchikov, N. (1989). Quantitative Studies of Student Self-Assessment in
Higher Education: A Critical Analysis of Findings. Higher Education , 18 (5),
529-549.
Brown G. with Bull, J., & Pendlebury, M. (1997). Assessing Student Learning in Higher
Education. London: Routledge.
199
Brown, G. T. (2004). Teachers' Conception of Assessment: implications for policy and
professional development. Assessment in Education , 11 (3), 301-318.
Brown, G. T., & Hirschfeld, G. H. (2008). Students' Conception of Assessment: Links to
Outcomes. Assessment in Education: Principles, Policy and Practice , 15 (1), 3-
17.
Brown, H. D. (2003). Language Assessment Principles and Classroom Practices.
London: Longman.
Brown, S., & Knight, P. (1994). Assessing Learners in Higher Education. London:
Kogan Page.
Bunts-Anderson, K. (2004). Teachers’ conceptions of language learning: out-of-class
interactions. Proceedings of the Independent Learning Conference 2003.
Butler, Y. G. (2006). On-Task versus Off-Task Self-Assessments among Korean
Elementary School Students Studying English. The Modern Language Journal ,
90 (4), 506-518.
Butterfield, S. (1995). Educational Objectives and National Assessment. Buckingham:
Open University Press.
Bygate, M. (1987). Speaking. Oxford: OUP.
Central Statistics Agency. (2007). Statistical Abstract. Addis Ababa.
Chambers, F. (1997). What Do We Mean by Fluency. System , 25 (4), 535-544.
Chamot, A. U.(2004). Issues in Language Learning Strategy Research and Teaching.
Electronic Journal of Foreign Language Teaching , 1 (1), 14-26.
Chamot, A. U. (2005). Language Learning Strategy Instruction: Current Issues and
Research. Annual Review of Applied Linguistics , 25, 112-130.
200
Chappuis, S., & Stiggns, R. J. (2004/5). Classroom Assessment for Learning.
Educational Psychology , 206-209.
Chen, Y. m. (2008). Learening to self-assess oral performance in English: A longitudinal
case study. Language Teaching Research , 12 (2), 235-262.
Chen, Y. (2005). Self Assessment of Oral Performance in the EFL University Classroom.
Retrieved January 13, 2009, from
http://fllcccu.ccu.edu.tw/conference/2005coference_2/download/C10.pdf
Cheng, W., & Warren, M. (2005). Peer Assessment of Language Proficiency. Language
Testing , 22 (1), 92-121.
Clapham, C. (2000). Assessment and Testing. Annual Reveiw of Applied Linguistics , 20,
147-161.
Clark, E. V. (2004). How language acquisition builds on cognitive development. Trends
in Cognitive Sciences , 8 (10), 472- 478. avilable @ http://fias.uni-
frankfurt.de/~triesch/courses/cogs1/readings/Clark04.pdf.
Cohen, L., Manion, L., & Morison, K. (2000). Research Methods in Education. London:
Routledge& Falmer.
Commission of the, European Communities. (2007). Schools for the 21st Century.
Commission Staff Working Paper. Brussels.
Cook, V. (2001). Second Language Learning and Language Teaching. New York:
Hodder Arnold.
Coombe, C. (2002). Self-Assessment in Language Testing: Reliability and Validity Issues.
Retrieved February 2009, from Karen's Linguistics issues:
http://www3.telus.net/linguisticsissues/selfassess.html
Creswell, J. W. (2007). Qualitative Inquiry and Research Design choosing among five
approaches. London: Sega Publication.
201
Creswell, J. W., & Plano Clark, V. L. (2007). Designing and Conducting Mixed Methods
Research. Thousand Oak: SAGE Publication.
Cummins, J., & Yee-Fun, E. M. (2007). Academic Language: What Is It and How Do We
Acquire It? In J. Cummins, & C. Davison (Eds.), International Handbook of
English Language Teaching Part I (pp. 797-810). New York: Springer Sciences-
Business Media, LLC.
Dagne Tiruneh. (2009). The exploration of the Practice of Continuous Assessment by
EFL Instructors: the case of Jimma Teachers' College. MA Thesis . Addis Ababa
University.
Dann, R. (2002). Promoting Assessment as Learning improving the learning process.
London: Routledge/Falmer.
Davis, D. C. (1971). Model for a Humanistic Education The Danish Folk Highschool.
Columbus: Charles E. Merrill Publishing Company.
Dawson, C. (2002). Practical Research Methods: A user-friendly guide to mastering
research techniques and projects. Oxford: HowtoBooks.
Deakin-Crick, R., Lawson, H., Sebba, J., Yu, G., & Harlen, W. (2005). Retrieved
september 13, 2009, from
http://eppi.ioe.ac.uk/EPPIWebContent/reel/review_groups/assessment/assessment
_protocol5.pdf
Derebssa Dufera. (2006). Quality of Teaching and Learning in Ethiopian Higher
Education: Tension between Tradition and Constructivest Teaching Approaches.
summary of Proceedings of the Conference on the Future Direction of Ethiopian
Higher Education (pp. 127-135). Addis Ababa: Addis Ababa University Press.
Dickinson, L. (1987). Self-instruction in Language Learning. Cambridge: Cambridge
University Press.
Dickinson, L., & Carver, D. (1980). Learning How to Learn: Steps towards Self-
Direction in Foregign Language Learning in Schools. ELT Journal , 35 (1), 1-7.
202
Dornyei, Z. (2007). Research Methods in Applied Linguistics. Oxford: Oxford University
Press.
Ellis, R. (2009). The Differential Effects of Three Types of Task Planning on the
Fluency, Complexity and Accuracy in L2 Oral Production. Applied Linguistics ,
30 (4), 474-509.
Falchikov, N. (2005). Improving Assessment through Student Involvement Practical
Solutions for aiding learning in higher and further education. London:
RoutledgeFalmer.
Falchikov, N., & Goldfinch, J. (2000). Student Peer Assessment in Higher Education: A
Meta-Analysis Comparing Peer and Teacher Marks. Review of Educational
Research , 70 (3), 287-322.
Freeman, D. (1996). Redefining the relationship between research and what teachers
know. In M. B. Kathleen, & D. Nunan, Voices From the Language Clasroom (pp.
88-117). Cambridge: CAmbridge University Press.
Freeman, D., & Richards, J. C. (1993). Conceptions of Teaching and the Education of
Second Language Teachers. TESOL Quarterly , 27 (2), 193-216.
Gardner, H. (1999). Assessment in Context. In P. Murphy (Ed.), Learners, Learning and
Assessment (pp. 90-117). London: Paul Chapman Publishing in association with
The Open University.
Geeslin, K. L. (2003). Student Self-Assessment in the Foreign Language Classroom: The
Place of AuthenticAssessment Instruments in the Spanish Language Classroom.
Hispania , 86 (4), 857-868.
Given, L. M. (Ed.). (2008). The Sage Incyclopedia of Qualitative Research Methods
(Vol. 1). Los Angeles: SAGE Publications Inc.
Greene, J. C., Kreider, H., & Mayer, E. (2005). Combining Qualitative and Quantitative
Methods in Social Inquiry. In B. Somekh, & L. Cathy (Eds.), Research Methods
in the Social Sciences (pp. 274- 282). London; Thousand Oaks: Sage Publications.
203
Harris, M. (1997). Self-assessment of Language Learning in Formal Setting. ELT
Journal , 51 (1), 12-21.
Havnes, A., & McDowell, L. (2008). Introduction: Assessment dilemmas in
contemporary learning cultures. In A. Havnes, & L. McDowell (Eds.), Balancing
Dilemmas in Assessment and Learning in Contemporary Education (pp. 3-14).
New York: Routledge and Taylor & Francis Group.
Heal, J. (2003). Mind, Reason and Imajination :Selected Essays in Philosophy of Mind
and Language. Cambridge: Cambridge University Press.
Heaton, J. B. (1990). Writing English Language Tests. London: Longman.
Housen, A. & Kuiken, F. (2009). Complexity, Accuracy, and Fluency in Second
Language Acquisition. Applied Linguistics , 30 (4), 461-473.
Hughes, A. (2003). Testing for Language Teachers. Cambridge: Cambridge University
Press.
Irons, A. (2008). Enhancing Learning through Formative Assessment and Feedback.
London: Routledge.
Jafarpur, A., & Yamini, M. (1995). Do Self-Assessment and Peer-rating Improve with
Training? RELC Journal , 26 (1), 63-85.
James, M., & Pedder, D. (2006). Beyond Methods: assessment and learning practices and
values. The Curriculum Journal , 17 (2), 109-138.
James, M., McCormick, R., Black, P., Carmicheal, P., Drummond, M.-J., Fox, A., et al.
(2007). Improving Learning how to learn: Classrooms, Schools and Networks.
London: Routledge.
Knapper, C. K., & Cropley, A. J. (2000). Lifelong Learning in Higher Education.
London: Kogan Page.
204
Knight, P. (2007). Grading, Classifying and Future Learning. In D. Boud, & N. Falchikov
(Eds.), Rethinking Assessment in Higher Education: Learning for the longer term
(pp. 72-86). New York: Routledge.
Kreber, C. (2003). The Scholarship of Teaching: A Comparison of Conceptions Held by
Experts and Regular Academic Staff. Higher Education , 46 (1), 93-121.
Kwok, L. (2008). Stuents' Perceptions of Peer Evaluation and Teachers' Role in Seminar
Discussions. Electronic Journal of Foreign Language Teaching , 5 (1), 84-97
retrived from @ http://e-flt.nus.edu.sg/v5n12008/kwok.htm.
Kwon, E. S. (2004). A New Constructivist Learning Theory for Web-Based Design
Learning with Its Implementation and Interpretation for Design Education.
Unpublished PhD Dissertation. Ohio State University
Larsen-Freeman, D., & Long, H. M. (1991). An introduction to Second Languagr
Acquisition Research. New York: Longman.
LeBLANC, R., & Painchaud, G. (1985). Self-Assessment as a Second
LanguagePlacement Instrument. TESOL Quarterly , 19 (4), 673-687.
Lewkowicz, J. A., & Moon, J. (1985). Evaluation: A Way of Involving the Learner.
Evaluation: Pratical Papers on English LAnguage Education , pp. 45-79.
Lim, H. (2007). A Study of Self- and Peer-assessment of Learners' Oral Proficiency.
CamLing , 169-176.
Little, D. (2005). The Common European Framework and the European LAnguage
Portfolio:involving learners and their judgements in the assessment process.
Language Testing , 22 (3), 321-336.
Logan, S., & Moore, N. (2004). Implementing Learner Training from a teacher's
Perspective. Proceedings of the Independent Learning Conference.
Luoma, S. (2004). Assessing Speaking. Cambridge: Cambridge University Press.
205
Luoma, S., & Tarnanen, M. (2003). Creating a Self-rating Instrument for Second
Language Writing: from idea to implementation. Language Testing , 20 (4), 440-
465.
Mackey, A., & Gass, S. M. (2005). Second Language Research: Methodology and
Design. London: Lawrence Erlbaum Associates.
Makasha Kasaye. (2005). An Exploration of the Task Designing Procedure of EFL
Teachers in Ethiopia. A case study. Unpublished PhD Thesis . Addis Ababa
University.
Marczyk, G., DeMatteo, D., & Festinger, D. (2005). Essentials of Research Design and
Methodology. New Jersy: John Wiley & sons, INC.
Marshall, H. H. (2004/5). Cultural Influences on the Development of Self-concept:
Updating Our Thinking. Educational Psychology: Annual edition , 68-71.
Marton, F. (1981) Phenomenography- Describing Conception of the World around Us.
Instructional Science 10, 177-200
Marzano, R. J., Pickering, D. J., & McTighe, J. (1993). Asessing Student Outcommes
Performance Assessment Using the Dimensions of Learning Model. Alexanderia:
Association for Supervision and Curriculum Development.
McDonald, B., & Boud, D. (2003). The Impact of Self-Assessment on Achievement: The
Effect of Self-Assessment Training on Performance in External Examinations.
Assessment in Education , 10 (2), 209-220.
McNab, C. (1989). Language Policy and Language Practice. Implementation Dilemmas
in Ethiopian Education. Stockholm: Institue of International education, University
of Stockholm .
Medel-Añonuevo, C., Ohsako, T., & Mauch, W. (2001). Revisiting Lifelong Learning for
the 21st Century. Retrieved June 2010, from www.unesco.org/education/uie •
206
mehrang, F., & with Rahimpour, M. (2010). Investigating Effects of Task Structure on
EFL Learner’s Oral Performance. English Language Teaching , 3 (4), 10-17.
Mekonnen Disasa. (1998). Investigating Methods of Training for Developing Students'
Skill for Academic Oral Work. Focus on Social Science Students AAU.
Unpublished PhD Thesis . Addis Ababa University.
Mika, S. (2006). Peer-and Instructor Assessment of Oral Presewntations in Japanese
Universaity EFL Classrooms: A pilot Study. Retrieved may 2009, from
http://dspace.wul.waseda.ac.jp/dspace/bitstream/2065/11344/1/13M.Shimura.pdf
Miller, L., & Ng, R. (n.d.). Peer Assessmewnt of Oral LAnguage Proficiency. Retrieved
January 28, 2009, from http://sunzi1.lib.hku.hk/hkjo/view/10/1000076.pdf
MoE. (2005). Education Sector Development Program III. Addis Ababa: MoE.
MoE. (2008). Review of the Ethiopian Education Training Policy and Its Implementation.
Addis Ababa.
MoE. (2003). Teacher Education System Overhaul (TESO) Handbook. Addis Ababa.
Nunan, D. (1988). Learner-Centred Curriculum. Cambridge: Cambridge University
Press.
Nunan, D. (1992). Research Methods in Language Learning. Cambridge: Cambridge
University Press.
Nuru Mohammed. (2000). Feedback in the EFL Classroom An Exploration of Its Role in
the Communication of Teacher Expectation. PhD Thesis . Addis Ababa: Addis
Ababa University.
Omelcheva, M. (2004, February 19). Self and Peer Evaluation in Undergraduate
Education: Are Promises Worth Risking the Perils? Retrieved June 2009, from
http://www.allacademic.com/meta/p117476_index.html
207
Oxford, R. L. (1990). Language Learning Strategies What Every Teacher should Know.
Boston. HEINLE & HEINLE Publishers.
Oxford, R. (1994). Language Learning Strategies: An Update. Retrieved February 13,
2009, from Online Resources: Digests:
http://findarticles.com/p/articles/mi_pric/is_199410/ai_1412535941/col1
Pajares, M. F. (1992). Teachers' Beliefs and Educational Research: Cleaning up a Messy
Construct. Review of Educational Research , 62 (3), 307-332.
Partnership for 21st century skills. (2008). 21st Century Skills, Education &
Competitiveness: a resource and policy guide Retrieved December 11, 2009, from
http://p21.org/documents/21st_century_skills_education_and_competitiveness_gu
ide.pdf
Peirce, B. N., Swain, M., & Hart, D. (1993). Self-Assessment, French Immersion, and
Locus of Control. Applied Linguistics , 14 (1), 25-41.
Pierson, C. A., & Beck, S. S. (1997/8). Performance Assessment: The Reality that
Influence the Reward. Educational Psychology Annual Edition , 225-229.
Race, P., Brown, S., & Smith, B. (2005). 500 Tips on Assessment. London:
RoutledgeFalmer.
Reagan, T. (1999). Constructivist Epistemology and Second/Foreign Language
Pedagogy. Foreign Language Annals , 32 (4), 12-25.
Richards, J. C., & Rodgers, T. S. (2001). Approaches and Methods in Language teaching.
Cambridge: Cambridge University Press.
Roever, C., & Powers, D. (2005). Effects of Language of Administration on a Self-
Assessment of Language Skills. Retrieved May 13, 2009, from www.ets.org/toefl:
http://www.ets.oreg/Media/Research/pdf/RM-04-06.pdf
Ross, J. A. (2006). The Reliability, Validity and Utility of Self-Assessment. Practical
Assessment, Research & Evaluation , 11 (10), 1-132.
208
Ross, J. A., & Starling, M. (2005). Effects of Self-Evaluation Training on Achievement
and Self-Efficacy in a Computer-Supported Learning Environment.
Toronto: Paper presented at AERA.
Ross, J. A., Rolheiser, C., & Hogaboam-Gray, A. (1998). Effects of Self-evaluation
Training on Narrative Writing. Toronto: Paper presented at the Annual
Meeting of the Canadian Society for the study of Education.
Ross, S. (1998). Self-assessment in Second Language Testing: a meta-analysis and analysis of
experiential factors. Language Testing, 15(1), 1-20.
Rubin, J., Chamot, A. U., Harris, V., & Anderson, N. J. (2007). Intervening in the use of
strategies. In A. D. Cohen, & E. Macaro (Eds.), Language Learner Strategies (pp.
141-160). Oxford New York: Oxford Univesity Press.
Sarantakos, S. (2004). Social Reearch. Palgrave: McMillan.
Schilling, J. (2006). On the Pragmatics of Qualitative Assessment: Designing the process
for content analysis. European Journal of Psychological Assessment , 22 (1), 22-
37.
Sebba, J., Deakin-Crick, R., Lawson, H., Yu, G., Harlen, W., & Durant, K. (2008).
Systematic review of research evidence of the impact on students in
secondaryschools of self and peer assessment. London: EPPI-Centre SSRU.
Segers, M., Dochy, F., & Cascallar, E. (2003). The Era of Assessment Engineering:
Changing Perspectives on Teaching and Learning and the Role of New Modes of
Assessment. In M. Segers, F. Dochy, & E. Cascallar (Eds.), Optimising New
Modes of Assessment: In Search of Qualities and Standards. Dordrecht: Kluwer
Academic Publisher.
Seliger, H., & Shohamy, E. (1989). Second Language Research Methods. Oxford: Oxford
University Press.
Shing, L. W., & Fai, H. S. (2007). Conceptions of Assessment of Mainland China
College Lecturers: A Technical paper Analyzing the Chinese Version of COA III.
The Asia Pacific-Education Researcher , 16 (2), 185-198.
209
Silashi Argaw. (2007). The Perception and Implementation of Continuous Assessment
(the case of Debub Ethiopia College of Teacher Education . MA Thesis . Addis
Ababa University.
Sinclair, B., & Ellis, G. (1992). Survey: Learner Training in EFL Course Books. ELT
Journal , 46 (2), 209-225.
Skehan, P. (2009). Modelling Second Language Performance: Integrating Complexity,
Accuracy, Fluency and Lexis. Applied Linguistics , 30 (4), 510-532.
Smith, K. (2000). Nagotiating Assessment with Secondary-school Pupils. In P. B.
Michael, & A. Littlejohn (Eds.), Classroom Decision-making: Negotiation and
Process syllabus in Practice (pp. 55-62). Cambridge: Cambridge University
Press.
Stern, H. (1992). Issues and Options in Language Teaching. Oxford: Oxford University
Press.
Stevick, E. W. (1990). Humanism in Language Teaching. Oxford: Oxford University
Press.
Struyven, K., Dochy, F., & Janssens, S. (2003). Students' Perception about New Modes
of Assessment in Higher Education: a review. In M. segers, F. Dochy, & E.
Cascallar (Eds.), Optimising New Modes of Assessment: In Search of Qualitie and
Standerds. Dordrecht: Kluwer Academic Publishers.
Struyven, K., Dochy, F., & Janssens, S. (2005). Students’ perceptions about evaluation
and assessment in higher education:a review. Assessment & Evaluation in Higher
Education , 30 (4), 331-347.
Sweet, D. (1993, September). Consumer Guide. Retrieved May 21, 2010, from
Archived Information:
http://www2.ed.gov/pups/OR/ConsumerGuides/perfasse.html
Tan, H. K., Teo, C. T., & Ng, C. S. (2011). Variation in Students’ Conceptions of Self-
Assessment and Standards. Education Research International , 1-9.
210
Tan, K. (2007). Conception of Self-assessment: What is needed for long-term? In D.
Boud, & N. Falchikov (Eds.), Rethinking Assessment in Higher Education:
Learning for the longer term (pp. 114-127). London: Routledge.
Tang, C. (1994). Assessment and student learning: Effects of Modes of Assessment on
Students’ Preparation Strategies. In G. Gibbs (Ed.), Improving Student Learning:
theory and Practice (pp. 151-170). Oxford: Oxford Brookes University, the
Oxford centre for Staff Development.
Taras, M. (2001). The Use of Tutor Feedback and Student Self-assessment in Summative
Assessment Tasks: towards transparency for students and for tutors. Assessment
and Evaluation in Higher Education , 26 (6), 605-614.
Taras, M. (2003). To Feedback or Not to Feedback in Students Self-assessment.
Assessment and Evaluation in Higher Education , 28 (5), 549-565.
Taras, M. (2002). Using Assessment for Learning and Learning for Assessment.
Assessment and Evaluation in Higher Education , 27 (6), 501-510.
Taye Regassa. (2008). English as a Tool for Quality Education: A Study of Its Role in the
Reading Comprehension of First Year Students at College of Social Sciences,
Addis Ababa University. Proceedings of the National Workshop on LAnguage,
Culture and Development in Ethiopia; Good Governance and Development in
Ethiopia; Higher Education, Democracy and Development in Ethiopia. The
Ethiopian Chapter of OSSREA (pp. 181-193). Addia Ababa: Addis Ababa
University Press.
Teddlie, C., & Tashakkori, A. (2009). Foundations of Mixed Methods Research:
Integrating Quantitative and qualitative Approaches in the Social and
Behavioural Sciences. London: SAGE Publication Ltd.
Todd, R. W. (2002). Using Self-assessment for Evaluation. English Teaching Forum , 40
(1), 16-19.
211
Topping, K. (1998). Peer Assessment between Students in Colleges and Universities.
Review of Educational Research , 68 (3), 249-276.
Topping, K. (2003). Self and Peer Assessment in School and University: Reliability,
Validity and Utility. In M. Segers, F. Dochy, & E. Cascallar (Eds.), Optimising
New Modes of Assessment: In Search of Qualitie and Standerds. (pp. 55-88).
Dordrecht: Kluwer Academic Publishers.
Tsui, A. B. (1996). Reticence and anxiety in second language learning. In M. B.
Kathleen, & D. Nunan, Voice fron the Language Classroom (pp. 145-167).
Cambridge: Cambridge Universaity Press.
Tudor, I. (1996). Learner-centredness as Language Education. Cambridge: Cambridge
University Press.
Vandeyar, S., & Killen, R. (2007). Educators’ Conceptions and Practice of Classroom
Assessment in post apartheid South Africa. South African Journal of Education,
27 (1), 1-15.
Van lier, L. (1989). The Classroom and the Language Learner. London: Longman.
Wallace, S. (2007). Teaching, Tutoring and Training in the lifelong Learning Sector.
Exeter: Learnint Matters.
Weir, C. J. (1990). Communicative Language Testing. New York: Prentice Hall.
Wenden, A. (1991). Learner Strategies for Learner Autonomy. Englewood Cliffs, New
Jersey: Prentice Hall.
Wen-ming, Y., & Xiao-zhen, X. (2008). Self Assessment in Second Language Learning.
US-China Foreign Language , 6 (5), 20-24.
White, E. (2009). Student Perspectives of Peer Assessment for Learning in a Public
Speaking Course. Asian EFL Journal , 33, 1-55.
212
Wiggins, G. (1998). Educative Assessment: Designing Assessment to Inform and Improve
Students' Performance. San Francisco: Jossey Bass.
Willey, K., & Gardner, A. (2009). Changing Student’s Perceptions of Self and Peer
Assessment. Proceedings of the Research in Engineering Education Symposium.
Queensland: REES.
Williams, M., & Burden, R. L. (1997). Psychology for Language Teachers A Social
Constructivist Approach. Cambridge: Cambridge University Press.
Winterbottom, M., Brindley, S., Taber, K., Fisher, L., Finney, J., & Riga, F. (2008).
Conception of Assessment: trainee teachers' practice and Values. The Curriculum
Journal , 19 (3), 193-213.
Winterbottom, M., Taber, K. S., Brindley, S., Fisher, L. G., Finney, J., & Riga, F. (2008).
Understanding differences in trainee teachers' values and practice in relation to
assessment. Teacher Development , 12 (1), 15-35.
Woods, D. (1996). Teacher Cognition in Language Teaching. Beliefs, decision-making
and classroom practice. Cambridge: Cambridge University Press.
Wragg, E. C. (2001). Assessment and Learning in the Secondary School. London:
Routledge/Falmer.
Yalew Endawok. (2004). Teachers' Beliefs, Knowledge and Practrice of Learner-centred
Approaches in Schools of Ethiopia. The Ethiopian Journal of Education , XXIV
(2), 17-41.
Yin, R. K. (2011). Qualitative Research from Start to Finish. London: The Guilford
Press.
Yonas Adaye. (2008). The Role of English in Quality Education in Ethiopia: Focus on
Primary School English Teachers' Perception of Communicative Approaches.
Proceedings of the National Workshop on Language, Culture, and Davelopment ;
Good Governance and Development; and Higher Education , Democracy and
213
Development in Ethiopia: the Ethiopian Chapter of OSSREA, (pp. 143-180).
Addis Ababa.
Zariski, A. (1996). Student peer assessment in tertiary education: Promise, perils and
practice. In Abbot, J and Willcoxson L. (Eds) Teaching and Learning Within and
Across Disciplines. Proceedings of the 5th Annual Teaching and Learning Forum
(pp. 189-200). Perth: Murdoch University retrived from
http://lsn.curtin.edu.au/tlf/tlf1996/zariski.html.
Zergahun Chufamo. (2007). An Investigation of Implementation of Continuous
Assessment by EFL Instructors. A case of Hossana College. MA Thesis . Addis
Ababa.
214
6 Appendices
Appendix 1: Students’ and instructors’ interview schedule
1. What does it mean by assessment?
2. What do you think is the main purposes of classroom assessment
3. Who should do the assessment?
4. How do you describe learners’ self-assessment?
5. Do you think that students should be involved in assessment scheme, i.e., developing
assessment criteria and standards, and marking their own work? why/why not?
6. If you (students) are allowed to assess your/their own performance, do you think you
(students) assess the work honestly and realistically, why/why not?
7. Do you feel that engaging students in the evaluation of your/their own learning outcomes
has any benefits and/or problems? If yes, what are the benefits (advantages) and/or
problems (disadvantages)?
215
Appendix 2
Instructors’ interview Transcript
Appendix 2A: Instructor A
Interviewer: good afternoon. Thank you for giving me your time for this interview. As I told you last
time I’m doing a research for my PhD study. So, the very purpose of this interview is to gather data
for the research. We discuss some issues of assessment in general and students’ self assessment in
particular. Basically in our discussion there is no any wrong or right response; I just want you to tell
me whatever you feel I mean your opinion and thoughts genuinely. What is important is your
genuine response that makes the study valid. And I want you to be sure that any information I get
will be used only for my research. Therefore, just feel free and respond to my questions. Thank you
again.
To begin with general ideas, what do we mean by assessment in general and particularly classroom
assessment, and why is it for? What is the main purpose of classroom assessment?
Ins: first of all I would like to thank you for having me as one of your interviewees for your research.
Having said this, when we come to the meaning and purpose of classroom assessment, when we say
assessment it is the method of assessing evaluating students’ performance. When we come to the
purpose of assessment it does have a number of purposes. The first one is to check whether the
students really understand what we teach in the classroom that is one of the main purposes of
assessment the second is as I think it helps to identify the standard of our students because
identifying academic standard of our students helps us to help them differently because students do
have different standards: there are high performing students, low performing students and medium
performing students by using assessment we can identify them then we can give some tutorial
classes for those performing at low level. This is the second purpose of assessing students. Thirdly,
the purpose of assessing is it helps students to improve themselves. In a sense when we assess
students we have to give them feedback and based on the feedback or they get from the instructor
students can improve their error based on the feedback they get from the instructors. So, for me
these are the three purposes that lead us to assess our students.
Interviewer: ok you’ve mentioned three purposes of classroom assessment: one is to check their
understanding the second is to put them into categories and the third one is to help them improve
their learning. If you are asked to put them according to their priority, which one do you prioritise?
Ins: ok for me the first one is the first priority; just I assess to check whether they learnt or not what I
taught them. This is my prior purpose.
Interviewer: good, coming to my next question, who do you think should do the assessment? just to
make it clear, conventionally or as we know, it is the teacher who carries out assessment; but who
do you think is responsible for classroom assessment?
216
Ins: for me both the teacher and the students should take responsibility of assessment. That can be done
in a number of ways. That is there are a number of assessment ways. That is one. Secondly the
students should also evaluate themselves to know at what level they are performing, they have to
know themselves. Are they performing well, are they underperforming or are they performing at
medium level. Each and every student should evaluate himself. I don’t know how we can let them
evaluate themselves since I’m not in it practically, but basically I believe both the teacher and the
students should be involved in assessment.
Interviewer: good, now if we say that students should be involved in assessment, in what way can they
be involved?
Ins: students assess themselves in different ways for example when they take a test or assignment they
can say whether they have done well or not when they listen to a lecture or attend class they
evaluate how much they have understood or how difficult the topic was, so students participate
these and other ways.
Interviewer: ok that is one. The other way of involving learners in assessment is through learners’
self-assessment. if you are asked to describe learners’ self-assessment, how do you describe it?
What is self-assessment for you?
Ins: ok self-assessment of students, it is a difficult question in fact since I’m not practicing it, but
when we say students should assess themselves they should have some benchmark or some criteria
set by the teacher or in consensus with the students. So based on the already established criteria that
both the teacher and students set that means they can commonly create some criteria based on
which students should be assessed. After setting criteria I think students can evaluate himself in
terms of the already set criteria. I think that is what we mean by self-assessment.
Interviewer: you’ve just mentioned that self-assessment means students participate in setting criteria.
In the process of assessment what comes after setting criteria is marking students work or
performance. My question here is do you think students should be involved in setting the criteria
and giving mark to their own work or performance, especially when it is a kind of subjective
assessment like oral or writing performance? Can they and should they be allowed to do these?
Ins eeem, I don’t think students should evaluate their tests their assignments because that could lead
them to be biased for themselves. Even about setting criteria in practical world it is not practicable,
but if we really need to make them evaluate themselves we have to do that.
Interviewer: good. You said that student should not be allowed to mark their own test and
assignments. Why you said that? Can you explain the reason?
Ins: the reason why I said that is it can lead them to bias they could favour themselves while they are
marking.
Interviewer: do you mean they do not mark honestly?
Ins: sure. That is what I mean. That is my fear.
Interviewer: what do you think is the cause of that dishonesty?
217
Ins for me the main reason is that if we generally see students’ current performance, they are not
performing well. Generally when we take the overall students’ performance, they are not
performing good. Since all students are not performing good, what they want to do is they search
for ways they can get marks to get good grade. So their poor educational background their poor
performance may lead them to favour themselves as I think this is the main thing or driving force.
Interviewer: so you mean it cannot be valid if students are allowed to give marks for themselves.
Ins: it cannot be valid among all students by all students, but you can find few genuine students
specially those who are good students who are confident to their performance. They can genuinely
mark their work or performance, but those who are medium or poor performers can favour
themselves. There is high probability to favour themselves.
Interviewer: ok may be may last question. You said that students should be involved in some aspects
of the assessment process. Is there any benefit or advantage for the students and the teacher, and
any problem?
Ins: yes, the advantage is that it develops transparency in grading between the students and the teacher
and that will help minimise complaints that will be raised at the end of the course when the grade is
done because if the grade is done transparently every student knows how they have been marked
how they have been graded so there will not be any complaints by the students at the end. This will
be one of the main advantages. Secondly if we can practically do that, students might feel
committed to their education. It would increase students’ commitment. I guess they feel committed
to their work because they are going to evaluate their work clearly in front of other students and
teacher and they would be ashamed of scoring less. Therefore they will be motivated to do better.
So sense of commitment would be developed by that sort of assessment.
Interviewer: Any problem it has besides its invalidity?
Ins: I have already mentioned the problem of bias or selfishness. Apart from that I don’t see any
disadvantage or problem. Its advantage outweighs its problem.
Interviewer: thank you. I have finished my questions, but if you have anything to say or comment?
Ins: from its very concept it is a good way of assessment. if students self assess, it would be nice, but it
doesn’t practically work. What I suggest is if we believe in its importance its relevance, there
should be some mechanism devised to implement such kind of assessment because it has much
advantage and fewer disadvantages. For that disadvantage we have to search for solution (way) of
avoiding the invalidity of the marks given by students. to let them assess themselves or to avoid the
invalidity we can put some framework in which the students can think to mark certain questions.
That means we can jot down some key points so that the students are guided by that points. We can
encourage learners’ self-assessment in such a way.
Interviewer: once again thank you for your cooperation and time.
218
Appendix 2B: Instructor B
Interviewer: good afternoon. Thank you for your cooperation to give me this interview for my study. The
very purpose of this interview is to gather data for the research. We discuss some issues of
assessment in general and students’ self assessment in particular. Basically in our discussion there is
no any wrong or right response; I just want you to tell me whatever you feel I mean your opinion
and thoughts genuinely. What is important is your genuine response that makes the study valid.
And I want you to be sure that any information I get will be used only for my research. Therefore,
just feel free and respond to my questions. Thank you again.
Let me start with a general question. What do we mean by assessment in general and particularly
classroom assessment for you, and why is it for? What is the main purpose of classroom
assessment?
Ins: alright, assessment is a technique by which we evaluate the performance and participation of students
in the class. Most of the time especially in our today time the main purpose is to check whether the
objectives set before were achieved or not. There could be two types: either at the end to check the end
result or the progress. So we can see in these two ways.
Interviewer: so you mean the main purpose of classroom assessment is to check whether the objectives
were achieved either at the end or while the teaching is in progress.
Ins: sure that is it but not only that it may also help the instructor it may help the teacher himself
sometimes to device the methodology mechanism if the mechanism devised is not of that
much effect.
Interviewer: well, coming to my next question related to this is that who do you think should carry out the
assessment?
Ins: ok, assessment should be carried out by the teacher. It can also be carried out by students, also by the
leader. Program leader can assess students. Sometimes parents can assess the students because there is
connection between parents and the institutions. So most of the time the teacher should assess in the
classroom based on their achievement based on their work day to day work. Students can also assess
each other. Sometimes parents can also be involved, and the program leader, department head and the
faculty heads can assess students.
Interviewer: good, you’ve said that students can take part in the assessment; you mentioned that they can
ass each other. What about themselves? Can they and should they assess their own work, performance?
Ins: self-assessment, yes possible, but it needs something. Guidelines are needed; the teacher should
prepare guideline for assessment. Students have to get the target of assessment, first they have to
change their attitude towards assessing themselves because they do not honestly assess themselves.
Interviewer: now you said that students can assess themselves, i.e., they can do self-assessment if they get
guidelines. Ok, if you are asked to describe students’ self-assessment how do you describe it? What is
learners’ self-assessment for you?
219
Ins: I can say self-assessment is the way one or somebody can check how much s/he has carries out certain
activity in a given time with the limited resource and also based on the objectives set. So if one person
assess themselves they have to focus on these points it is easy for them to check their attempts to reach
the target.
Interviewer: ok, what are the procedures in learners’ self-assessment what does it involves?
Ins: yaa, checking their results, checking their end results or progress and check if they have achieved
their target objectives. If the teacher gives them the guidelines, they can do this.
Interviewer: ok, the next question related to this is that to assess their students, instructors first design the
task for the assessment then set the criteria for marking the work or performance especially when it is
subjective assessment like for example writing a paragraph or critical reading comprehension or oral
performances. Then based on those criteria, she or he scores or gives marks to the work. Now, my
question is do you think students should participate in this assessment process? I mean can and should
they take part in designing the tasks, setting criteria and giving mark to their own work?
Ins: of course emmmm here there are certain difficulties because it was not practiced in our cases in our
situation but if we try to involve students in how to assess themselves even as you said they can set
their own criteria and pass through that and involved in that they can do though there are difficulties in
the practical aspect. For example when students are made to assess themselves they don’t honestly
assess themselves.
Interviewer: thank you, if we say students should be involved in the assessment scheme do you see any
advantages and or disadvantages I mean benefits and problems of learners’ self-assessment?
Ins: yaa, there are advantages. Basically we are focusing on student-centred method of learning, so when
students try to assess themselves they will also learn their mistakes, their strong sides and weak sides,
they will identify and learn by themselves this is one thing. Another when they assess themselves when
a person assesses himself or another, she or he identify mistakes and just consult the teacher or any
expert. So it can develop confidence within the person or within the student who assess himself or
herself. So it can develop confidence. It can facilitate learning also self learning. I can say this is
advantage. And disadvantage could be it takes time for the teacher to set criteria or guiding the students
needs additional time. And also many students may not honestly assess themselves. So this could be
the disadvantage.
Interviewer: ok, may be my last question, you said students may not be honest; it means when the students
mark their own work that mark may not be valid. Why do you think?
Ins: one thing is basically they may lack background the target of assessment. Another thing is the
students lack knowledge of the subject they learn. And another, naturally, human being likes himself so
they need to give high marks so they hide their weak sides and try to seem correct in all aspects. So this
is the weak side of self-assessment. anyway, if students believe in self-assessment and correctly focus
on the guide line provided, it is very important for learning.
Interviewer: thank you I have finished my questions, but if you have anything to add?
220
Ins: yaa I want to add something. That is most of the course, for example we are offering different course
for different departments common courses or major courses in English department. When we prepare
lesson or when we design some activities, there is no self-assessment part. Also in continuous
assessment when the general guideline is given from the faculty or department level, teachers’
assessment, group work and test it says most of the time, but no self-assessment is included in that one
in the course outline in the lesson plan also in whatever planned there. So if self-assessment is added as
one more technique of continuous assessment, it will improve students’ learning in students-centred
method. That is what I want to add.
Interviewer: thank you very much again.
221
Appendix 2C: Instructor C
Interviewer: good afternoon. Thank you for your cooperation to give me this interview. To introduce
myself, I’m Rufael Disasa, a PhD student at Addis Ababa University. I’m doing a research in
assessment. So, the very purpose of this interview is to gather data for the research. We discuss
some issues of assessment in general and students’ self assessment in particular. Basically in our
discussion there is no any wrong or right response; I just want you to tell me whatever you feel I
mean your opinion and thoughts genuinely. What is important is your genuine response that makes
the study valid. And I want you to be sure that any information I get will be used only for my
research. Therefore, you just feel free and respond to my questions.
Ok, let’s begin by talking about assessment in general. What do you think is or are the main
purposes of assessing students in classrooms?
Ins: it is or has multipurpose not only one purpose. The first purpose of I think I mean of assessment is
I mean it enables to check whether the students understand or not especially the objectives it could
be either general or specific and not only that the teacher also give grades at the end of the course it
is not only instrument which used to check whether the students understand or not but also it also
enable the teacher himself sometimes to device even the method if the method used is not effective.
An another importance of assessment might be it helps the students to share idea among themselves
these are I mean the importance of assessment
Interviewer: thank you. My next question related to this is that who should do the assessment. I mean
whose role whose responsibility to carry out the assessment?
Ins: I think it is clear I mean it is obvious it should be the teacher who gives the course. The course
instructor knows the general and specific objectives of the course and he has to check continuously
whether he has achieved his objectives. The instructor also must report the students’ grade at the
end of the semester or course. For this reason he is responsible for assessing his students. He is also
professionally trained. Anybody cannot do it. It is difficult for others to I mean who are not
involved in teaching the course they do not know the level of the students which content they have
covered. So this is clear as I think.
Interviewer: ok, let me go to my next question about learners’ self-assessment. What do we mean by
students’ self-assessment how do you describe it?
Ins: it is one way of assessment. Teacher’s assessment is formal one and learners’ self assessment is
another one. It is the way students evaluate themselves. They always check whether they understand
a topic or what they learned. A student can say which course is easy and which one difficult for
him. For example, when they study they can evaluate whether he or she has understood it. Again
after doing an assessment task exam or a test he or she can check how much he or she has answered
and missed. After the exam also they evaluate themselves by comparing one’s result with that of
classmate’s how well they have done the exam therefore students always make self-assessment
Interviewer: thank you. The other point I want to raise is that is it possible and appropriate to involve
the students in the assessment process. To make clear to assess students, the instructor or teacher
first prepares or designs the assessment task it can be a test or group assignment or anything. Then
222
he or she sets criteria for marking especially if the assessment is performance that needs subjective
scoring like writing a letter or paragraph like prompt or impromptu speech or dialogue the teacher
prepares criteria and then marks scores the performance using the criteria. My question is do you
think it is possible and necessary to involve learners in this procedure of assessment, that is in
setting criteria and giving mark formally?
Ins: ok, just before saying yes or no one thing is it should be seen in two ways. I mean as I understood
self-assessment is very important but if I mean students’ background should be considered. Students
are not aware of what self assessment is and its criteria it is better to be assessed by somebody else.
if the students are trained of the criteria in that case it is important but if in the case students are not
aware of self assessment they do not have good idea of assessment it will be problematic even it
might be destructive.
Interviewer: you mean students cannot participate in developing the criteria and marking of their own
performance?
Ins: ok, I mean this self assessment yes they should be involved in developing criteria. Again there
thing should be I mean here again here I mean the students background should be under question
that means his capability in developing if they have good understanding on the criteria of evaluation
in that aspect I mean he can be involved but students’ background is under question and need to be
considered.
Interviewer: good, if you allow your students to score or give mark to their own performance do you
think it will be valid I mean acceptable?
Ins: no here no doubt to say that they exaggerate their mark. The students want to get high mark good
grade, so they do not put the right mark they will be biased. Therefore the teacher should intervene
the teacher should supervise. if the students are left free without any intervention such exaggeration
happens. As the assessment is subjective the students see it from their aim there should be
intervention otherwise it is problematic
Interviewer: thank you. May be my last question If we try to use self assessment in classroom does it
have any benefit and or problem? in what ways?
Ins: for sure it helps. People internalize their mistake as they view than when people assess them so, it
is of high importance.
Interviewer: I have finished my question. Once again thank you very much for your cooperation.
Thank you.
Ins: you’re welcome
223
Appendix 2D: Instructor D
Interviewer: welcome. First of all thank you for giving me your time for this interview. I’m Rufael
Disasa, PhD student at Addis Ababa University. I’m doing a research in the area of classroom
assessment. So, the very purpose of this interview is to gather data for the research. And I want to
assure you that the information I get from you will be used only and only for my research.
Therefore, you just feel free and respond to my questions. We discuss some issues of assessment in
general and students’ self assessment in particular. Basically there is no any wrong or right answer;
I just want you to tell me whatever you feel I mean your opinion and thoughts about the points we
raise. What is important is your genuine response that makes the study valid.
Let’s start by talking about the purpose of assessment. What do you think is or are the main purposes
of learners’ classroom assessment in general?
Ins: well, beginning from the main purpose of assessment, as to me I feel that assessment is a means or
a tool that to check for the students their current performance might be or past experience might be
or plan it may help you to check whether students are engaged or they are going to be engaged in
some sort of activity, whether they understand or not the portion you taught them. So I think it is a
means or a tool for checking students understanding. You also grade the students achievement that
is grades because to decide they promote or fail
Interviewer: thank you, and who do you think should do the assessment? Who is responsible to
evaluate the students?
Ins: well, for me it is the teacher. Assessment is the part of teaching process. After teaching a portion
of the course the teacher need to check the effectiveness of his teaching. Therefore it is the teacher
who should carry out the evaluation.
Interviewer: ok, now let’s talk about students’ self-assessment. What do we mean by students’ self-
assessment, how do you describe it?
Ins: ok, em... students’ self-assessment as to me is what students do for themselves. A student can
assess or evaluate him/herself by thinking what he can do and cannot do. For example, when he
study he can evaluate how well he understand and which topic is difficult for him. They can also
evaluate themselves after doing assignments or taking a test. When they see the result they can
evaluate how much they have done. A student usually evaluates himself after getting the result of
the test by comparing with their classmates. So self assessment is the way students check their
ability and learning before and after they take tests and exams.
Interviewer: thank you. A relate question is that should students be involved in the assessment scheme
formally? To make clear, in the assessment process the teacher devices the assessment format may
be a test or assignment or class work. Then s/he will set the criteria for marking especially for
subjective assessments like writing a paragraph or speaking, debate like that, and then marks the
performance using the criteria. My question is that is it possible and appropriate to involve students
in doing these things of this assessment process? Why or why not?
224
Ins: in the assessment process I think that students should be involved in the assessment because might
be in my feeling is that when the teacher sets criteria you know he might gear to his own way might
be biased but if he or she engages students in setting the criteria of assessment it is good so I think
students should be involved in setting criteria of assessment together with the teacher. This might
help them to take responsibility of learning
Interviewer: ok, you said that students should participate in setting criteria. What about the marking?
Should they be allowed to give mark to their own performance?
Ins: this is very difficult. If students assess themselves you know everybody is selfish by nature you
know, so their aim is to score high mark to achieve high grade so the basis of their aim might be a
head of misleading might occur while they assess themselves but it is possible to avoid this
misleading if a kind of means of controlling is set if a kind of means is set this might be solved but
as I said first it is not acceptable.
Interviewer: thank you, ok, may be my last question. Do you think learners’ self-assessment, if they
participate in the assessment, has any academic benefit or any additional problem other than what
you have mentioned, i.e., its being not valid?
Ins: yes of course, as I mentioned earlier, it helps them to check their academic status where they are.
In addition when the teacher provides any kind of assessment activity students have got you know
both opportunity at the same time they assess themselves and they get a kind of opportunity to learn
from that activity. So, it helps
Interviewer: Ok, how does it help or in what way do they learn from their self assessment?
Ins: first, they check whether or not they understood the activity or what they are told to do. Second
they practice for example if it is a speaking or a writing task they practice speaking or writing.
Interviewer: I have finished my question. Thank you once again for your time and cooperation.
Thank you.
Ins: thank you
225
Appendix 2E: Instructor E
Interviewer: good morning, thank you for coming and your willingness to giving me your time for this
interview for my research on classroom assessment. So, why we are here I mean the only purpose
of this interview is to gather data for the research. And I assure you that the information I get from
this discussion will be used only and only for my research. Therefore, you feel free and just say
whatever you feel I mean your opinion and thoughts about the points we raise. We discuss some
issues related to assessment in general and students’ self-assessment in particular. You know there
is no any wrong or right answer; I just want you to express whatever you think is or should be the
case. What is important is your genuine response that makes the study valid.
Ins: Alright
Interviewer: Ok let’s start by talking about the purpose of assessment. What do you think is or are the
main purposes of classroom assessment in general?
Ins: as to me assessment is a way of finding information about students’ performance behaviours
interests their participation in teaching learning process so that you give grade for the performance.
So in my opinion the main purpose of assessment is to identify the students strengths and weakness
and to know their academic status whether they top middle or low and then to give remedies on the
weak area
Interviewer: thank you the next point is who should do the assessment? I mean who is responsible to
for the assessment practice?
Ins: well for me the teacher it must be the course instructor. Because the teacher has the objectives and
the criteria he or she has to check whether or not the students understood it whether or not the
objectives are achieved to he or she knows when and how to assess also to give grade is also his
responsibility part of his job as a professional so he has to take continuous assessment of his
students this is one. It is difficult for others to do this I mean they do not know which portion is
covered how the method of the delivery for example the department can support by giving guidance
but do not do the assessment by itself the instructor knows the level of the students what and how
he taught which content they have covered. So, I think this is clear.
Interviewer: ok, now my next question is about students’ self-assessment. What do we mean by
students’ self assessment, how do you describe it?
Ins ok, em... for me students’ self-assessment is students evaluating themselves student assess or
evaluate themselves always they think for example, when they study they evaluate how much they
understand and which topic is difficult for them. They can also evaluate themselves while and after
doing assignments or taking a test. From their result of test exam or any assignment they can
evaluate how much they have done. each student is always evaluating him or herself. This is
students’ self-assessment
Interviewer: ok, let me raise a question related to students’ self assessment. You have said that the
teacher is responsible to carryout learners’ assessment.
226
Ins: yea, that is right.
Interviewer: but is it possible and is it necessary to involve learners in the assessment scheme? should
students assess themselves formally? To make it clear when the instructor is to make assort of
assessment he or she first thinks of the kind of assessment task whether it should be a sort of quiz or
class work or assignment then he or she thinks of the criteria against which she evaluates I mean
marks or scores the performance of the students on the task or activity to be carried out by the
students especially if the kind of assessment needs subjective scoring like for example writing a
letter, or developing a paragraph or speaking like dialogue or debate and others he should set clear
marking criteria and lastly she or he give marks to the students performance. My question here is
should students be involved in the setting of criteria and marking of their own performance work?
Ins: no in my opinion they should not they can’t participate if it is necessary there should be some
criteria prerequisite premedication steps for example in such kind of universities example Wollega
university there is continuous assessment you know the teacher assess students in that case if you let
the students to assess themselves the students are always in quest of marks they are searching for
mark even without any effort any endeavour or task they simply want to earn marks they just
simply are eager to get marks passing marks passing grades I don’t want to complain about the
administration or the legislation or anything but the policy let them to do so I know that so if you let
them assess themselves do they assess really? Unbiased? They cannot be honest at least you have to
be honest for yourself but you cannot be honest in search of marks you want to pass your course
you want to be confident when you get out of the campus you have to get job to live your life you
have to get your livelihood so in educational level the first criteria is you have better mark or better
grade when you where in school so hirers the job providers search you by your mark they look at
only the paper so the main aim or criteria for job provider is what is on the paper so they look at the
paper if they see good grade they select you by the grade not with your real potential anything else
so involving students in the assessment is not that much necessary and important because students
do not know that the aim of learning is a quest for knowledge they think learning is getting mark to
get job but if students know learning is not just getting mark but quest for knowledge it may not be
difficult to involve students in the assessment.
Interviewer: thank you, may be my last question is but does it help or not. I mean if we try to let the
students to participate in the setting of criteria and marking of their performance does it have any
benefit and or problems?
Ins: yea just I have mentioned one problem the students cannot be honest in giving mark for
themselves if not why should we bother it is just wasting time. But the benefit is students may be
active not passive and improve their participation this is the advantage
Interviewer: I have finished my questions thank you very much once again for your time and
cooperation thank you
Interviewee: Thank you
227
Appendix 3 Students’ Pre-intervention interview transcript
Interviewer: first of all I would like to thank you for giving me your time for this interview. To
introduce myself, I’m Rufael Disasa, a PhD student at Addis Ababa University. I’m doing a research in
the area of classroom assessment. So, the very purpose of this interview is to gather data for the
research. And I want to assure you that the information to be obtained from this interview will be
used for this research only; therefore, you just feel free and respond to my questions. We discuss
some issues of assessment in general and students’ self assessment in particular. Basically there is no
any wrong or right answer, I want you just to tell me whatever you feel I mean your opinion and
thoughts about the points we raise. And I hope you understand that your genuine I mean true
answer is very important for the validity of the research result. I also want to inform you that we can
express our idea using any language we are comfortable with if we want to. ማለት ሃሳባችንን በምመቸን
ቋንቋ መግለጽ እንችላለን ።
My first question is on some general ideas about assessment. if you are asked to say what assessment is and why it is for, how do you describe it, why it is done? What is assessment and what is the purpose of assessment? Shall we start from you?
S1 yes, assessment means evaluating ourselves or oneself when you do anything or if we take an
example or if we take our activity to see the value of our own self or evaluating you yourself that
means when you see the result or product of your job or your activity you see how the clarity of that
your job or how it can be available or how it can be accepted in front of the evaluator or to evaluate
ourselves
S2 that is important for the teacher because by taking assessment he can identify students who are
weak and strong, who have the knowledge. Unless he evaluates he cannot understand our potential
whether we understand his lecture whether it is clear or not without evaluation he did not
understand. Even in one class he may give lecture method for us on the same topic the same idea he
can understand which students have high capacity and which students have low capacity and which
students are medium without assessment he cannot identify it is important for the teacher also to
give grade for the students and when student make mistake or don’t understand his lecture the
teacher correct students and teach again
S3 : the purpose of assessment is during the teaching process established that means when the
teacher teaching students he evaluates which or among the students who are working hard to
differentiate who is weak in his work and his study evaluate whether they are who record the higher
mark he want to know when he is teaching who is hard working and to evaluate teach from one
grade to another grade students they need to be assessed and then evaluate to pass or progressive
purpose they pass from one session to another this is the main reason and whether the students
have understood what they have been taught before and to prepare themselves for the study
purpose
228
S4 okay thank you very much for giving me this chance to share my idea concerning assessment and
from the very beginning assessment means it is nothing but the way of evaluating students and when
we say the purpose of assessment the main purpose of assessment is nothing but evaluating or
measuring performance the work etc. of the students within the classroom during the process of
teaching learning and also the purpose is it helps to hold the students towards the work for example
in case of our university or Wollega university the process of teaching learning is continuous
assessment and due to this we have different tests and assignments throughout the week due to
this reason we visit library in order to answer the assignments and also in order to study for our day
to day quizzes if no continuous assessment students don’t study always they go to towns playing
therefore in a simple way or simple manner it helps to hold the students towards their work.
Interviewer እሽ ሌላ፣ ጥያቄዉን ለማሰታወስ ያህል አሴስመንት ምንድን ነዉ? ዓላማዉስ? ለምንድን ነዉ
መምህራን ተማሪዎችን ክፍል ዉስጥ የምገመግሙአቸዉ? ጥቅሙ ምንድን ነዉ?
S5 first of all thank you very much to give me this chance to explain what we mean by assessment.
Assessment means as he tried to define assessment is evaluating activity of any student while they
are learning their course and also the purpose of assessment is to evaluate the students depending
on their activities in the classroom and to search how the student attending their class during their
day to day life in the classroom not only in the classroom also in their dormitory. Another in order to
answer the questions which is given for the students they share different ideas among the group to
understand or to answer the questions which their teacher gives for them . it has own many
advantage or many purpose because it helps to change their mind in day to day activities search
different books to different assignments and study for the test given for them in day to day lesson.
S6 First of all thanks to give me this chance assessment means as I think evaluating someone how
much the person understands the given activity or the given topic, giving activity and evaluating the
students day to day. Another the purpose of assessment is to develop or increasing students ability in
the whole direction means speaking ability reading ability and also writing ability in classroom and
also the part of continuous assessment in giving assignment is the most important for developing the
students’ ability because among the students searching the assignment their ability increase from
time to time.
Interviewer okay gara biraan naminni yaada qabu yoojirate?
S7 assessment is shortly the way of evaluating that means I myself I’m student while the teacher is
teaching or giving lecture while he is attending is he attending or not attending the given lecture in
the classroom. Therefore the main purpose of assessment is to know where I am or where s/he is in
terms of the teaching learning process.
Interviewer: okay any additional or different idea? Ok let me go to my next question. Who do think
should do the assessment? whose role is assessment? ግልፅ ለማድረግ ሰስሰ ማድረግ ያለበት ማን ነዉ? የማን ድርሻ
ነዉ ብላችሁ ታስባላችዉ?
229
S2: ok thank you again the assessment in the classroom or during the teaching learning process is
conducted by both the students and the teacher in the classroom. For instance, when the teacher
gives a class work for example in case of writing skill the teacher gives different assignments to
students to do in the class and then after they write whatever work given to them the students
evaluate themselves through peer editing process means they evaluate themselves in the class by
sharing or exchanging their work with each other and also the evaluated material will be given to the
teacher for additional evaluation. So in our current kind the assessment is engaged by both teacher
and students and also other staff members.
S8 : ok I would like to thank our instructor conducting the research. When I proceed to my opinion
since assessment is one part of effective teaching learning process it must be done both by the
instructor and the students and also department.
Interviewer: any additional or different idea? no. do you agree with these guys? በነዚህ ሃሳብ ሁሉም
ይስማማል? The next question is ok you have said that assessment should be done by both the teacher
and the students. ok how or in what way do students do the assessment? can you explain how you
the students do assessment?
S5: ok the students can do assessment by when the teacher give some certain topic to do for the
students the teacher must give instruction because if there is no instruction the students cannot do
what they are going to do therefore the instructor must give the way the students can do or can
participate in the form of assessment.
S8 ok when I say assessment is done by both the teacher and students I mean that students can
evaluate themselves. It is obvious that instructors always evaluate students
Interviewer: ok more idea? I mean answer? Ok you have said that students can assess themselves.
That means it is self-assessment. a question related to this is what is self-assessment? tell me what
self-assessment is. How do you describe it? ግልፅ ለማድረግ ሰልፍ አሰሰመንት ወይም ተማሪዎች ራሳቸዉን መገምገም
ማለት ምን ማለት ነዉ እንዴት ይገለጻል?
S2 when we say self-assessment it means the method of assessment or the type of
assessment in which a person evaluate his work by his own without offering or giving to
other person.
Interviewer: can you give me any example?
S8 ok when we say self-assessment it is the way in which students can evaluate his or herself.
Example I myself can write one paragraph and edit that means I evaluate myself how the punctuation
the grammar is correct that means I evaluated myself
230
S6 self-assessment means a method of testing ourself or method of evaluating oneself means how I
can do something or how I done the previous test or I have done the topic on the assignment given
for me by this way students evaluate him or herself.
S7 self-assessment means it is assessment by which we evaluate ourselves for example if we go to
library we read our exercise books or references after that we may take some question or we
may ask ourselves what I know about this question already read it and what type of question.
By this without the involvement of others we ask ourselves and we can evaluate ourselves so
this is what we call evaluating ourselves
S1 self assessment means as I think evaluating ourselves for example if we do or if we are ordered
to do something or an activity that activity may be taken by another body and we will do and that
activity and come to evaluate ourselves. at that stage when we evaluate ourselves the acceptability of
our activity that we have done and how it can be acceptable and is as it is expected or as that activity
will be accepted or to identify error or to separate which is bad action from the good ones
Interviewer: any person to add or any different idea? Ok the next question is do you think learners
should participate in the assessment process? Look, in the process of assessment the teacher (the
assessor) first design the task which the students carryout it can be an objective kind of assessment
like multiple choice or matching items, or it can be a writing task or sort of oral performance. in the
case of the objective types the teacher just prepares answer key but In the case of the writing or the
speaking performance the assessor sets or prepares criteria by which he scores the performance and
then s/he marks the work using the set criteria. ማለት ምንድን ነዉ ፣ መጀመሪያ ፈተናዉን ያወጣል ወይም ተማሪዎቹ የሚሠሩትን አከቲቪቲ ያዘጋጃል ለምሳሌ የሚፀፍ ነገር ከሆን የፅሑፉን ዓይነት ይወስናል፣ ርዕሶች ያዘጋጃል። ቀጥሎ እንዴት
እንደምታረም ክራይቴሪያ ያዘጋጃል። ከዚያ ያርማል ክራይቴሪያዉን በመጠቀም። then my question is do you think
students should be involved in this process should they participate in designing the task, setting the
criteria and should they score their own work? ተማሪዉ ክራይቴሪያ በማዉጣትና ለሰራዉ ስራ ማርክ በመስጠት
ህደት ዉስጥ መሳተፍ ይችላሉ ወይ መሳተፍስ አለባቸዉ ወይ what do you think?
S4 ok thank you again evaluating himself for students is good because since certain criteria is given
for her or him depending on that criteria s/he can evaluate himself but on the designing of the
criteria it is not fair to give chance for or allow students to design criteria for evaluating because
as I think for example there may be inclination for instance if I am asked to design criteria for
evaluating a writing thing and if I am good at handwriting that means if the appearance of my
written thing is well attractive and if I commit many grammatical errors and punctuation and the
like I give the highest point for the neatness of the work and give low point for the other
problems in order to be beneficiary therefore such problem might occur during the designing of
the criteria by students so it is not fair to give a chance for students to design criteria.
Interviewer: what about the marking?
231
S5 this is also obvious that it is not fair because for example there is no person no one give a little
mark for himself since there is no awareness on such practice or since there is idea that for
example if a teacher allow students to do certain task and evaluate for themselves they think as
the teacher take the mark as permanent record due to this reason in most cases almost all
students give high mark for themselves during the self-assessment. So it is not fair not good.
S8 ok as he said giving mark for ourselves is not fair for example last week we have done certain
topic in that we gave mark to ourselves in that marking I may give high mark for myself for
benefit as he said so it is difficult to give mark to ourselves why when task given to us I may do it
how I know it and I consider myself this is right I do not know my error so since I don’t know my
error where my punctuation grammar error is I don’t know so I may think this is correct this is
right so I must get due to that reason giving mark to ourselves is not fair.
Interviewer: additional or different idea? ጥያቄዉን ለማሰታወስ ተማሪ ክራይቴሪያ በማዉጣትና ለራሱ ሥራ ማርክ
በመስጠት መሳተፍ ይችላል አይችልም አለበት የለበትም ለምን?
S5 wanti ani jechuu barbaadu gaaffii kiratera ittiin of madaalu ykn ulaagaalee barataan ittiin of
madaalu baasuuf hindanda’aa barataan jedhuuf akkasumas immoo barataan offsaaf qabxii
nikennaa jedhu irratti kun fudhatama qabaa hinqabuu jedhu irratti fakkeenyaaf akkuma isaan
kaasuu yaalanitti barataan hojii isaa ofiin of madaaluu irraa kan hafe qabxii ofii kennuu fi
akkasumasmoo ulaagaalee ittiin of madaalan sana baasuun barataadhaaf hindanda’mu
sababnisaas ulaagaa sana kan baasuu danda’uuf qabxii sana immoo ulaagaa sana keessatti
madaalee kennuu kan danda’u barsiisaa sana ykn instraktatara sana ta’uutu irrajiran jedha ani
akka Yaada kootiitti sababnisaa barataan of madaaluu keessatti barataadha waan ta’eef qabxii
ga’aa argachuu barbaada waldorgomna waanta’eef qabxii ga’aa argachuu waan barbaannuuf
hundi keenya qabxii xiqaa ofii hinlaannu ulaagaaleedhumasanarra dabarree qabxii ol’aanaa ofii
laanna fayyadamuu sababa barbaannuuf jechaadha kanaaf ulaagaalee sana baasuummoo
barsiisaan utuu jiru kan madaaluu danda’u ulaagaalee sana kan baasuu danda’u barsiisaa isa
barataa san madaalu malee ulaagaa sana baasuun barataadhaaf hindanda’mu. Akkasumasmoo
qabxii ofii laachuun barataadha dhugummaa hinqabaatu maaliif barataan yoo qabxii sana ofii
laate akkuma barsiisaan fuudhee qabxii sana isaaf galmeessee kaa’u itti fakkaata waan ta’eef
qabxii dhugaa ofii hinlaatu.
S8 akkumanni jechuu yaale barataan ulaagaalee qabxiin ittiin kennamuu baasuufi ulaagaalee kana
irratti hundaa’ee qabxii ofii laachuu danda’a yoo jenne haqa ta’uu hindanda’un jedha ani
sababnisaas ammayyuu Yeroo ingliffaan haasofnu dogongora qabna uttun dogongora Koo
beekee dogongora hinuumun ture Yeroon qabxii ofii kennus dogongora koo hinbeeku sababan
ta’ef akkamittin qabxii ofii kenna? Ammayyuu dogongoruma sana akka waan inni sirrii ta’eettii
qabxii ofii kenna Garuu namni biraan barataa biraan ykn barsiisaan dogongora ilaaluu nidanda’a
Kana malee hamma namni biraan hinilaalletti ofiikootii wantan hojjedhe sirriidha jedheen
hojjechaa ture kanaaf barataan ofii qabxii kennuun qulqulluu miti.
232
S3 አሁን ማርክንግ ክራይቴሪያ በተማሪዎች መስጠት የተለመደ ነገር አይደለም ደግሞም ለመስጠትም አዌር አይደለንም ደግሞም
ክራይቴሪያ በተማሪዎች መሰጠቱ ትክክል አይደለም ፐላሰ ደግሞ ለራሳችን ማርክ የመሰጠት ጉዳይ ደግሞ ፌር ላይሆን ይችላል አሁን እኔ የተወሰነ ነገር ሠርቼ ያንን ኤሬሬን አይደለም የማየዉ ሁሉንም ነገር አሟልቼ እንደሠራሁ የማይገባኝን ለራሴ ልሰጥ እችላለሁ ችግሮቼን አላያቸዉም ማለት ነዉ ስለዚህ ማርክንግ ክራይቴሪያም ሆነ ማርክ በተማሪዎች መሰጠት የተለመደም አይደለም ትክክልም አይመስለኝም
Interviewer እሽ ሌላ ተጨማሪ ወይም የተለየ ሃሳብ? Ok may be my last question ተማሪ ራሱን እቫሉኤት ብያደርግ
ማለትም ክራቴሪያ ብያወጣና ማርክ ብሰጥ ያለዉን አንዳንድ ችግሮች አንስታቿል ሌሎች ተጨማሪ ችግሮች አሉ? ደግሞ ችግሮቹ
እንዳሉ ሆነዉ የተማሪዉ ራስን እቫሉኤት ማድረግ ጥቅም አለዉ? ለተማሪዉም ሆነ ለአስተማሪዉ የሚሰጠዉ ጥቅም አለ?
S5 galatoomaa ofiisaaf barataan of madaaluun faayidaa qabaamoo isa jedhame irratti faayidaa qaba
sababnisaa barataan beekumsasaa ittiin guddifachuuf gargaara wanta barate sana hammam
akka hubate of madaalee fooyyeffachuuf carraaqa fakkeenyaaf barataan waatokko dubbisee
ulaagaa sanarra dhaabbatee waan dubbise sana bifa ofiisaatiin deebisee barressu hammam akka
hubate arga kanaaf faayidaa qaba jedheen Yaada.
S4 itti dabalee barataan ofii of madaaluun faayidaa qabaa isaa jedhuuf iddoon itti faayidaa qabu jira
iddoon itti faayidaa hinqabnes jira. Fakkeenyaaf amma yoo kaanee qixa waa dubbisuu bifa
layibirarii deemanii waa dubbisuutiin yoo ilaalle fakkeenyaaf namni tokko kitaaba fuudhee
seenaa haata’uu waa’ee waan tokko haata’uu hinqo’ataa hindubbisa. Erga qo’atee booda kitaaba
sana ykn materiyaalii sana cufee gaaffiilee adda addaa baasuu danda’a. Gaaffiilee adda addaa
erga baasee booda ofii isaa deebii itti laata ykn immoo yaadota jiran barreessuu yaale kana
booda wanti sun sirrii ta’eef ta’uu dhiisuusaa kitaaba sana ykn maateriyaalii sana bane
walbiraqabee ilaaala dogongorri jiraannaan maal godhaa nisiirreeffata. Salf asseesimentiin ykn
ofiin of madaaluun qixa kanatti gaariidha akkanni hinirraannffannee isa gargaara iddoo itti
dogongoree sana hinhiraanfatu. Garuu iddoon itti faayidaa hinqabuu jedheen yaadu fakkeenyaaf
spookenii keessattilee ykn dandeettii barressu keessatti fakkeenyaaf barataan keeyyata tokko
akka barreessuuf yoo kennameef keeyyata sana gaafa barressu akkuman jalqaba kaasee turetti
beeka jedheeti kan inni barressu waan beeka jedhu barreessa hinbeeku namni jedhu hinjiru.
Achi keessaa immoo dogongorri baayyeeen jiraachuu danda’u. kana booddee ofiisaatiin ivaalu’et
akka of godhu kiraateeriyaa kennameef sana irratti hundaa’ee jechaadha yoo ta’ee abbaan
ammoo dogongora isaa hinbeeku sirriidha jedheen barreesse waan ta’eef giraamarii irratti qabxii
guutuu ofii kennu danda’a paankchu’eeshinii irrattilee akkasumma beeka jedhee waan ka’uuf.
Kanaaf nannoo barreeffamaa nannoo hojii ofiin uumamu keessatti ykn kiri’eetiive work warra
ta’an keessatti ofiin of madaaluun qulqullina hinqabu jedheen Yaada
Interviewer: thank you. Anybody to add? Ok I have finished my questions but if you have anything
to say as the last word in general or have comment or suggestion about whatever we have been
taking so far you can say anything.
S1 kanan jechuu barbaadu faayidaan ykn bu’aan of madaaluu akkuma inni dubbate kanneen akka
waa dubbisanii namaa galuuf dhiisuu ilaaluu irratti of fooyyessuf waan nugargaarruf gaariidha
233
garuu waan akka barreeffamaa keessatti ulaagaalee irratti hundaa’anii qabxii ofii kennuu
keessatti rakkisaa natti fakkaata.
Thank you again
234
Appendix 4 Students’ post intervention interview transcript
Interviewer: first of all I would like to thank you for your willingness and devoting your time to give
me this interview. I don’t think I should introduce myself and tell you, the purpose of this
interview. I mean it is obvious. But I want to reassure you that the information to be obtained
from this interview will be used for this research only; therefore, as you did in the previous
discussion, you just feel free and respond to my questions. We discuss some issues of assessment
in general and students’ self assessment in particular. You also know that there is no any wrong or
right answer, I want you just to tell me whatever you feel I mean your opinion and thoughts about
the points we raise. And I hope you understand that your genuine I mean true answer is very
important for the validity of the research result. Also remember that we can use any language we
are comfortable with. ማለት እንዳለፈዉ ጊዜ አሁንም ሃሳባችንን ለመግለጽ በምመቸን ቋንቋ መጠቀም
እንችላለን ።
I start with my first question. what do you think is the main purpose of classroom assessment?
When teachers assess students in classroom what is their main purpose for doing it?
S1 when we say assessment it is used for developing students activity when there is assessment
now assessment is continuous when there are assessment students study their books and exercise
book what they learned or what they got from the lectures or from other indirectly help to
developing their knowledge. They don’t forget what they learn in and there is strong interaction
between students then to develop experience to interpreting and exchanging their idea the purpose
of assessment in general is the main one is to develop students’ knowledge. The teacher want to
develop the students language skills speaking writing etc. he uses assignments class works and
others to help students
S2 ok The main purpose of assessment is it makes the students active enough and also makes
students search different reference and also the main purpose of assessment is to maximise
students grade point and in continuous assessment most students are advantages to own pass mark
this the main purpose assessment.
S3 to begin from its definition as we discussed in the last weeks sessions the process and way of
evaluating someone’s performance work and the like and the main purpose of classroom
assessment in general is to improve the ability and also the performance of the students and the
other purposes are also to give grade to see weak and strong students to give feedback these are
the main purposes of assessment. if the students do not score good mark the teacher help by
tutorials or revision and also the teacher knows whether the students get the knowledge and
understand the topic he teach by giving different assessments.
S4 also as the other students tried to raise the purposes of assessment are many to check the strength
of students and their weakness and also as they said assessment is to give grade and the main
reason why the teacher assess students is to modify the ability or capacity of students and the other
thing is to get knowledge ability to develop our schemata this is the main purpose of assessment.
235
when you see your mark is less or not good as your classmate you work hard to get more in the
next time
S5 assessment in my opinion it helps us as we learn from our mistake not to repeat our mistake and
then assessment is to increase our ability and also its purpose is to teach us what we can do and
what we can’t do the teacher also give mark and store that mark and give grade at the end of
semester As I think assessment in the classroom is to develop students’ skills improving by
writing reading so on ማለት ልጆቹን አቅማቸዉን ለመቀየርና ለማሳደግ ከፍ ለማድረግና ሌላ በተጨማሪ ለተማሪዎቹ
ግሬድ ይሰጣል አይደለም አሴስመንት ይጠቀማል and to know which student is weak and which is
hardworking assessment is important and you remember if you get good mark you are happy ማለት የተሸለ ለማግኘት ደግሞ ትሰራለህ
Interviewer: additional idea or any different idea? Ok my next question is who should do the
assessment in the classroom? ማለት አሰስ ማድረግ ያለበት ማን ነዉ? በተለመደዉ አሰራር ተማሪን የምገመግም
አስተማሪዉ ነዉ ግን ስታስቡ ማን ነዉ መገምገም ያለበት?
S5 thank you as you said who should do the assessment the responsibility to do the assessment is both
teacher and students or both the instructor and the students
S3 your question is who should do the assessment and my answer is the assessment should be done by
both the teacher and the students. the advantage of doing assessment for students is since their
future work or since evaluating might be one of their future work for example if that person or
those persons become teacher in their future they are going to evaluate students in their teaching
learning process. Therefore from participating in evaluation or assessment students learn how to
evaluate somebody
Interviewer: ok you said that assessment should be done by both the teacher and the students. But
how or in what way can the students participate in the assessment? In what way they do
assessment? እንዴት ወይም በምን መልክ ነዉ ተማሪዎች በአሰሰመንት ዉስጥ የምሳተፉት?
S6 ok students participate in assessment in doing what the teacher assigned them to do and another is
by evaluating themselves or their classmates assignments for example in continuous assessment is
done by myself I evaluate myself my own assignment I can evaluate according to we learned last
time last week.
S5 in the assessment the students can discuss with the teacher and creates some criteria for doing that
assessment and depending on that criteria students also score their own work or their friends
work. The students also participate in doing assessment in different ways for example by taking
different assignment s in the form of group or individual and taking test to be evaluated. So the
responsibility or the person who have the responsibility to do assessment is both teacher and
students
236
S2 ok as my friend said students participate in assessment in different ways these are by doing
different assignments and participating in presentation and by evaluating their own work or by
evaluating other students work their presentation as we evaluated our presentation last time.
Interviewer: any person to any idea? Ok you have said that students participate in assessment by
evaluating their own work and others work the means by self-assessment and by peer assessment.
Now let me focus on self-assessment. What is self-assessment for you? How do you describe it?
Can you tell me the procedures in self-assessment? ራስን እቫሉኤት ማድረግ ወይም ሰልፍ አሰሰመንት ስንል ምን
ማለታችን ነዉ እንዴት ነዉ ተማሪዉ ሰልፍ አሰሰ የሚያደርገዉ?
S3 ok self-assessment is nothing but the way you evaluate your own performance depending on the
designed criteria may be by both the teacher and the students first by participating in the class
discussion with the teacher to design criteria and doing the assignment or activity in the class or at
dorm and marking for yourself for example last time when we learned self-assessment we
participated by designing criteria and giving mark to our own presentation. This is self
assessment.
S7 as you said self-assessment is to evaluate ourselves how we did something or to identify capacity
of doing that activity to determine our knowledge identify our error good or bad and self-
assessment should be taken by individuals from the designing of criteria by discussing in class and
giving mark for my own performance.
Interviewer: anyone to add or have different idea? Ok let’s move to the next point. You said
that self-assessment means students participate in designing criteria and give mark to their
own performance. do you think that it is appropriate for students to participate in setting
criteria and marking their own work? Should and can they do that? Do you think the mark
would be acceptable? ማለት ተማሪ ክራይቴሪያ በማዉጣትና ለራሱ ስራ ማርክ መስጠቱ ተገቢ ነዉ ወይ
የሚሰጠዉ ማርክስ ተቀባይነት ይኖረዋል ወይ?
S7 as you asked the students should participate or should not participate in making criteria for self-
assessment as I think to evaluate themselves the students should be participate to makcriteria
which they do in the future time or the students should be involved in setting criteria which is used
to evaluate what they are going to do because it helps him to know that criteria and work hard to
do as the criteria is needed. I also support that is good appropriate for students to give mark
depending on the criteria designed by the teacher and student because they follow that criteria and
cannot cheat the teacher by giving high mark.
S5 as I think setting criteria and giving mark based on the set criteria is not appropriate because
students don’t know how to set a criteria or to evaluate their work because students have no
awareness about criteria or self-assessment last week we have learned about setting criteria about
self-assessment now it is good for us to give mark for ourselves because we know the advantage
and disadvantage. If they know or have awareness about setting criteria and giving mark for
themselves it is possible and it is appropriate it is acceptable but one thing behind that if they
237
don’t have awareness about setting criteria and giving mark it is problem and the other one is
some students may be selfish or make themselves get good mark not depending on that criteria so
it is not appropriate for them to set criteria and marking for themselves
S3 my idea is opposite from other and my answer is yes. Students should participate in setting criteria
for evaluation here it is possible to avoid bias the teacher also help and give direction to set the
appropriate criteria. but it is not fair for the students to give mark for themselves because some
students can be honest but some students may not be honest they wish to get good grade which is
not according to their ability and knowledge.
S5 but that is good for students If we make bias we cannot know our weak side and strong side. If we
neglect the sense of selfishness we can evaluate ourselves depending on our work. If I give simply
high mark to get A or B for myself that is unnecessary it is beyond my capacity I should learn
from my mistakes. If I give something equivalent to my capacity I also try to learn from the
mistakes and improve my capacity.
Interviewer: ok any idea? Ok I have one more question to finish. Now some of you said that
students should participate in setting criteria and marking of their own work and some others have
the opposite idea or opinion and you give different reasons. My question related to this is does
self-assessment have any advantage importance and problems for both the students and the
teacher?
S8 it is important I support by depending on the given criteria any student can evaluate himself
whether or not followed the given activity done in the class or other assignments and so on. To
participate in these activities there should be self assessment. Therefore as I think it is very
important.
S1 I think self-assessment is not reliable every person doesn’t want to get low mark means there is no
honesty due to this every student is not equal to get mark everybody want to get high mark. If s/he
is honest that is good focus on his ability his knowledge to evaluate himself if depend on self
confidence that is good but all students do not put depending on the criteria due to this it is not
good students participate in marking.
Interviewer: does it have any advantage despite this problem of reliability, or any additional
problem?
S1 yes it has advantage it develops the knowledge ability and experience of the students so it is
necessary for students to became active
S8 when they evaluate themselves they don’t correctly evaluate themselves they consider themselves
as good writers so they can’t estimate their ability and they may be affected that they can’t get
their errors and correct it they can’t get experience can’t get the knowledge. If they do the
activity and accept as it is right or as it is correct one they can’t correct themselves.
238
S4 ok before as we said both students and teachers should be involved in the assessment and also how
students can participate or involved in assessment by doing self-assessment we have said. and
also students can set criteria and mark for themselves as we last time discussed and set criteria
for presentation and we give mark for ourselves by looking the video. Setting criteria and
marking has advantage or disadvantage you have said. As I think it has so many advantages
students set criteria after that they mark their own performance in this case since they
implement the set criteria they learn many things from that criteria. For example I don’t have
seen any such activity before. After you asked to do starting that research I saw many things
for example we participate in some activity and after that we give mark for ourselves I learned
many things from this how the criteria is already set depending on the criteria when I consider
my performance I considered myself where is my lack where is my strength where is my
weakness. In that case now I could get many ability and learn many things from this through
process I hope I will change it due to this reason it has many advantages. If I don’t did this
activities I don’t know this criteria and how I can improve my weakness. Now I get self
confidence. Due to that reason the students setting criteria and giving mark for their
performance is crucial to understand their weakness and improve their knowledge. It help us to
correct the wrong grammar, the vocabulary of speaking and also how to correct writing
problems. The disadvantage is if students have many classes in the week they have no time to
look at the video and give mark for themselves.
Interviewer: anyone to add or have any different idea? Ok I have finished my questions but if you
have anything to say if you have any comment or anything you like to add on whatever we
have been discussing
S3 ok as a conclusion I want to say self-assessment is advantageous for students because as you tried
to tell us during that teaching of self-assessment it is must for students to have certain criteria
they read or become aware of that criteria and for the future performance they try to modify
their weakness depending on the set criteria and also make themselves competent enough for
the other work. For example criteria for evaluating one performance may be fluency, grammar
and the like. When a certain student sees these criteria developed to evaluate the oral
performance it is must to develop his grammar ability speaking ability and prepare himself for
the better work for the future. So self-assessment is advantageous for the students. Again,
when we go the teaching profession for example we become teacher we can teach in high
schools and we must evaluate the students. If I don’t learn how to make criteria and mark I
can’t evaluate my students so it has opportunity to learn how to assess students
S6 first when you participate in self assessment it is good I became happy because I don’t see
this kind of teaching learning process another advantage is when the teacher give me
low mark I am not angry because we know the criteria and I may give wrong mark not
following the criteria but the teacher depend on the criteria so I accept. I also get the
ability to assess my students if I become teacher I can prepare the assessment criteria
and how to assess my students this is also advantage
239
S2 I say it have many advantage when you discuss about the criteria we learn we know what we must
do at the end the last objectives of the topic or the course so I know what must I do which book
I must read or I ask able students to fulfil that criteria that objectives after I give mark for
myself using the criteria, I can check where is my problem my error so I try to correct it for the
next time by this process my speaking ability or writing ability develop and improve so I have
self confidence
Interviewer: thank you so much again for your cooperation to help me a lot of time you gave me.
Thank you.
240
Appendix 5: Questionnaire filled out by students
Thank you for taking time to fill out this questionnaire. It is designed to collect data on students’ self-
assessment and it is used for research purpose only. Please genuinely indicate the degree to which each
statement applies to your thoughts and feelings about engaging in self-assessment of your English oral
performance by circling one of the numbers corresponding to each of the following statements. There
is no right or wrong statement. The degree of your agreement to the statements is expressed as:
Strongly agree (SA) Agree (A) undecided (UD) Disagree (DA) strongly disagree (SDA)
S. No Statements
1. The main purposes of classroom assessment is
to check whether or not students have mastered
what they have learned
2. The main purposes of classroom assessment is
indentifying the strengths and weakness of the
learners
3. The main purposes of classroom assessment is
helping learners to improve their learning
4. Students should participate in assessing their
own work
5. The main purposes of classroom assessment is
providing feedback to students about their own
performance
6. Learners’ self-assessment means students
marking their own test or assignment using
answer keys provided by the instructor
7. The main purposes of classroom assessment is
grading and categorising students
8. Self-assessment means participating in the
process of developing criteria and standards for
scoring and then marking one’s own
performance
9. The main purpose of classroom assessment is
checking whether or not the course objectives
are achieved
SA A UD DA SDA
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
241
S. No Statements
10. The main purpose of classroom assessment is
providing the teacher with information for
making ongoing decisions about her/his
teachings
11. One of the students’ role in the teaching learning process
is participating in the whole
process of assessment
12. Self-assessment means checking how well you
understood a topic like while you are reading a
material or listening to a lecture
13. If I assess my own work, I would be able to
recognize my errors easily
14. It’s the lecturer’s job to evaluate students’
work
15. Assessing my own work makes me feel
responsible for my learning
16. Being involved in assessment of my own
learning outcomes helps me to improve my
language skills and knowledge
17. If I am to mark my own work I would get
bored
18. Assessing my own performance has no
importance to the improvement of my language
skills and knowledge
19. Learners’ assessing their own work helps to
minimise cheating and misbehaving
20. I consider the real assessment to be assessment
that is done by the teacher.
21. Self-assessment enables me to understand the
quality of performance expected of me
SA A UD DA SDA
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
242
S. No Statements
22. Students can assess their own performance
honestly
23. It is difficult for students to understand the
assessment criteria
24. Assessing my own work makes me trust the
instructor’s marking
25. Learner’ SA is an important part of learning
teaching process
26. Students’ self-assessment can count to the final
grade
27. Self assessment makes me spend much of my
time on studying and working hard
28. Students’ self-assessment is just adding burden
to both teachers and students
29. Assessing myself makes me think what and
how much to do to achieve the learning
objectives
Thank you once again
SA A UD DA SDA
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
243
Appendix 6 Questionnaire filled out by instructors
Thank you for taking time to fill out this questionnaire. It is designed to collect data on
students’ self-assessment and it is used for research purpose only. Please genuinely ndicate the
degree to which each statement applies to your thoughts and feelings about engaging students
in self-assessment of their own English oral performance by circling one of the numbers
corresponding to each of the following statements. There is no right or wrong statement. The
degree of your agreement to the statements is expressed as: Strongly agree (SA) Agree
(A) undecided (UD) Disagree (DA) strongly disagree (SDA
S. No Statements
1. The main purposes of classroom assessment is to
check whether or not students have mastered what
they have learned
2. The main purpose of classroom assessment is
indentifying the strengths and weakness of the
learners
3. The main purposes of classroom assessment is
helping learners to improve their learning
4. Students should participate in assessing their own
work
5. Learners’ self-assessment means students marking
their own test or assignment using answer keys
provided by the instructor
6. Learners’ self-assessment is a technique of
assessment where students participate in setting
criteria and scoring of their own performance
7. The main purposes of classroom assessment is
checking learners’ progress against the course
objectives
SA A UD DA SDA
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
244
S. No Statements
8. The main purposes of classroom assessment is
providing feedback to students about their own
performance
9. Learners’ Self-assessment means the learner
checking how well s/he understood a topic like
while reading a material or listening to a lecture
10. The main purposes of classroom assessment is
grading and categorising students
11. The main purposes of classroom assessment is
getting information for the teacher on the way s/he
teaches
12. Participating in assessment scheme helps students
to improve their approaches to learning
13. If students assess their own work, they would be
able to recognize their errors easily
14. Learners are most likely to provide inflated
information on their own performance
15. Involving learners in assessing their own learning
outcomes is a good way of providing feedback on
their own performance
16. Assessing their own work makes students feel
responsible for their learning
17. Getting students to assess their own work is a way
of solving cheating problems and disruptiveness
18. It’s the instructor’s job to evaluate students’
performance
19. Assessment is one aspect of the learning process in
which students should take part
SA A UD DA SDA
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
245
S. No Statements
20. Participating in assessing quality of their
performance has little effect on improving students
learning
21. I consider the real assessment to be assessment that
is done by the teacher
22. Assessing themselves makes learners more aware of
what they need to know in the subject
23. Learners’ self-assessment enables them to
understand the quality of performance expected of
them
24. Students assess their own performance honestly
25. If students are to assess their own work, they would
get bored
26. Learners’ self-assessment is an important part of the
teaching learning process
27. It is difficult for students to understand the
assessment criteria
28. Assessing their own work improves the social
climate of the teaching learning process
29. Students’ self-assessment can count to final grade
30. Assessing themselves makes them think what and
how much to do to achieve the learning objectives
31. Learners’ self-assessment is just putting burden on
students and teachers
32. learners’ self-assessment is the way of increasing
students’ participation in learning activities
SA A UD DA SDA
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
Thank you once again
246
7 Appendix 7 Training Material (learners’ Handout)
Learning to Self-Assess
_________________________________________________________________
A Material Prepared For Training Students on Learners’ Self-assessment of
English Oral Performance (as part of research instrument)
Feb. 2011
Addis Ababa
247
Table of content
Topics page
Classroom Self-assessment
Overview……………………………………………………………………………………. 1
Learning Objectives………………………………………………………………………….1
Introduction
Objectives………………………………….………………………………………………..2
Activity I: Expectation and learning mode………………………………………….2
Section 1 Meaning and purpose of assessment
Activity I: Meaning and purpose of Assessment……………………………………3
Activity II: Who should do the assessment? ..............................................................4
Activity III: Self-assessment......................................................................................5
Section 2: Identification of components of oral performance
Objectives .......................................................................................................................6
Activity I: Major Constructs of Oral Skill........................................................................6
Activity II: Specific Elements of Oral Skill.....................................................................8
Activity III: Description...................................................................................................8
Section 3: Developing Criteria
Objectives......................................................................................................................10
Activity II: Scoring Techniques.....................................................................................10
Activity III: Writing the Rubrics....................................................................................12
248
Section 4: Applying the Criteria and Feedback
Objectives............................................................................................................................14
Activity I: Assessing others............................................................................................15
Activity II: Assessing own oral performance.................................................................15
Section 5: Setting Goals and Action Plans
Objectives............................................................................................................................16
Activity I: Reflection..................................................................................................16
Activity II writing goals and action plans..................................................................16
249
Classroom Self-assessment
Overview
From the learner-centred approach point of view, self-assessment is the major component of
learning process which promotes active learning. It is an assessment technique which enables
learners to participate in the assessment process including setting assessment criteria and
scoring their own performance.
The purpose of this training is to create and/or increase your awareness on issues in self-
assessment and develop your ability of assessing your own English language oral performance
so that you participate in the assessment process meaningfully. This is believed to help you
improve the quantity and quality of your learning of the course.
This material is prepared for the trainees (students) to be used during the training. It is
organised into five sections that will be integrated with the Spoken English course contents.
These sections are I) introduction II) identifying elements of oral performance, III) setting
criteria, IV) applying the criteria and giving feedback and V) setting goals and action plans.
Each section has objectives and contains activities and tasks that need to be done individually,
in pair or small group.
The training will take the form of participatory and active learning methods. During the
sessions, you are expected to be active participant and interact in pair and small group
discussions to carry out the activities and tasks. However it does not mean that the suggested
activities and tasks are hard and fast rules; the instructor could use her/his own ways that s/he
finds appropriate.
The material could be covered within seven to nine hours (an hour/week) of additional time to
schedule of the course.
Learning Objectives The overall objective of this training is firstly to help you understand the meaning of self-
assessment and its significance in improving learning outcomes and secondly help you to
develop your ability and skills of assessing your own oral performance fairly and realistically.
Specifically, by the end of the training you will be able to:
250
understand the main purposes and benefits of SA
develop criteria for assessing English oral performance
determine quality of their own performance responsibly
design strategies for improving their own learning outcomes
Introduction (1 hr) This section introduces the training by providing the general objectives, what is included in
and how you would go about it.
Objectives:
At the end of this section you will:
explain the main objectives of the training
discuss and agree on the procedures and modes of learning proposed
Activity Expectation and learning mode
Task 1 What do you expect to gain from this training?
1. The instructor will introduce you to the training. Listen to the instructor and
look at the table of content of your material
2. Think about and list down the knowledge and skill you expect and/or want
to get from the training.
3. Join the person by your side and compare your lists. If you find differences,
discuss and improve your list if you like.
4. Look at the list of objectives on page 2. Are there any similarities between
your list and the list in the learning objectives?
5. Discuss it with the instructor
Task 2 How do you like the training to be carried out
1. Try to remember and list the active learning techniques you know and used
in classrooms. Which of them are interesting and which are not?
2. Form a group of three or four and make one comprehensive list of active
learning methods
251
3. Discuss the advantage and disadvantages of each technique and suggest
how the problems can be minimised to be used during this training
4. Share your idea with the whole class
Section 1 Meaning and Purposes of Assessment
Objectives:
At the end of this section you will:
Explain meaning of assessment
list the main purposes of classroom assessment
identify the main features (process) of assessment
discuss who should assess learners’ work
define SA
tell the benefits and problems of SA
Activity I Meaning and purpose of assessment
Task 1 What is assessment?
1. How do you define assessment? Following are some persons’ description
of assessment. Read the statements and put or X mark in the box to say it
is correct or wrong. If you say it is wrong (marked it X), reason out.
a. Assessment is measuring the ability and skills of students
b. Assessment means tests or exams we take
c. Assessment is the process of gathering information about
students’ learning behaviour
d. Assessment is evaluating the quality of students’ performance
e. Assessment includes designing task/performance, setting criteria,
to determine the quality of the work using the criteria and
judging the value
2. Identify one that best applies to your sense, or write your own or improve
one of the above statements.
3. Share your definition in pair and then as a class
252
Task 2 Why assessment?
1. What do you think is the instructors’ purpose of assessing their students?
Below are what some people think about the purpose of assessment. Read
the sentences and show the extent to which you agree with the people by
writing SA (strongly agree), A (agree), DA (disagree) or SDA (strongly
disagree) in the boxes. If you have two or more the same responses e.g.,
you strongly agree to two or three statements, try to rank them.
a. assessment is done primarily to grade students achievements
b. Instructors do assessment to accomplish their duty
c. To provide feedback to students on their learning
d. To maintain students’, teachers’ and schools accountability
e. To check effectiveness of a program
f. To identify good and poor students
g. To guide students improve their approach to learning
h. To let students identify their own strengths and weaknesses
i. To check whether or not the objectives of the course are achieved
j. To keep students working hard
k. To evaluate the effectiveness of the instructors
2. Share your response in a group of three or four. Give reasons for your
responses. Afterwards, you will share and discuss it with the whole class
Activity II: Who should do the assessment?
Task 1 Your experience
1. Think back about your experience of assessments in schools. Who are
involved in carrying out assessment?
2. Share your experience with the persons by your side and then with the
class
Task 2 Vote on your feet
Who do you think should be involved in the process of assessing your
performance? Four parties are suggested to be involved in assessing a student’s
work: teacher, school/external body, student him/herself, peers.
253
1. Express your opinion about who should assess students’ performance by
moving to the corner of the room that represents the party you vote for. If
you want to vote for two parties, draw a line between corners and stand at
the middle ground, and if you fail to decide, remain seated.
2. Explain the reason of your choice and listen to others’. If you change your
mind during the activity, e.g., after listening to others’ reasoning, you can
change your place and move to another corner.
3. Think about assessing your own work or performance, how do you feel?
Activity III: Self-assessment
Task 1 Describing Self-assessment
In task 1, activity II of this section, you have described assessment as a process of
gathering information on students learning; and it includes setting criteria and
using the criteria to measure the quality of the performance. Having this in mind,
1. Individually, try to describe self-assessment
2. In a group of four or five, share your descriptions and write the
comprehensive one
3. Share it with the whole class
Task 2 Beliefs about self-assessment
How do you think about assessing your own learning outcomes (work or
performance). Think about its advantage and disadvantage, problems and benefits.
Below are statements of the beliefs held by persons about self-assessment
1. Form a group of four or five and see if you agree or disagree with each of
the statements and why. You also can add your own beliefs that is not
mentioned
a. Assessment is the duty and responsibility of the teacher, so students
should not be involved in assessment
b. Assessing oneself helps to clearly identify weaknesses and strengths
of oneself
c. Assessing oneself is difficult and time consuming
d. Teachers/school do not trust students assessing themselves
254
e. Students do not want to assess their own or their classmates’ work
f. Students cannot assess themselves because they cannot be sure
about what is correct and what is wrong
g. Teachers are not willing to involve students in the assessment
process
h. Involving students in the assessment of their own performance
motivates them to make effort to improve their result
i. Students do not assess themselves realistically and trustfully, they
are selfish
j. Students cannot do self assessment because they do not have the
skill, ability and experience of assessing
k. Self-assessment can minimise students’ complaints about grading,
and this improves student-teacher relationship
l. Doing self-assessment is doing the job of the teacher
m. If given the opportunity and do practices, students are willing and
able to assess their own performance
2. Now discuss and decide whether or not you want to use self assessment in
your spoken English II classroom. If you decide to use, think how the
problems can be alleviated and the disadvantages can be minimised
3. Share your solutions with the whole class
Section 2: Constructs of Oral Performance (1 ½ hr.)
Objectives
At the end of this section you will be able to
describe characteristics of good English oral performance
identify the specific linguistic elements (language skills, sub skills and competences)
that are constructs of English oral performance
identify the non linguistics components of English oral performance
describe each of the specific linguistic and non linguistic elements to be assessed in
oral performance
255
Activity I: Major constructs of oral skill
Task 1 Your perception of good and poor oral performance
You must have listened to variety of English oral performances, e.g., instructors
lecture, classroom presentations, interviews, conversations, chatting friends, etc.
both physically and on TV and radio.
Try to remember one oral performance you liked and another one you did not like.
1. Think about why you liked the one and did not like the other. Jot down your
reasons
2. Join in a group of three and describe the performances you liked and did not
like and explain your reasons for liking and disliking
3. In your group list down the things you think are characteristics of
good/effective English oral performance.
4. Pick a person to present your suggestion to the class
Task 2 Linguistic skills: Accuracy, Fluency and Complexity
Fluency, accuracy and complexity are the major linguistic constructs (features) of
an oral performance. Fluency refers to the ability to produce spoken language with
ease; or the ability to produce continuous speech easily without causing
comprehension difficulties or a breakdown of communication.Accuracy is the
degree of correctness or deviation from the grammatical norm (error). A person’s
ability to use more advanced rich and diversified linguistic elements is referred to
as complexity; for instance, the person’s ability to use longer sentences; variety of
words to express the same idea, etc.
1. Now in a group of three or four members, discuss and try to understand
what fluency, accuracy and complexity means. For instance, if a speaker
uses many grammatically wrong sentences, is it a problem of fluency,
accuracy or complexity? And what about if s/he uses short, disconnected
and simple sentences?
256
2. Which of these elements do you think is more important and which is not?
Rank them according to their degree of importance you think and explain
your reason.
3. Present your result to the class. If the members cannot agree unanimously,
one can keep his/her own position and explain to the class.
Task 3 Non-linguistic skills
1. Think of doing an oral task. In addition to the above mentioned linguistic
skills, what other skills you need to have to do effective oral performance?
2. Try to recall your knowledge of writing skills course and list down the non-
linguistic skills required for good/effective written/oral performance.
3. compare your list in pairs, discuss and improve it
4. volunteer to read your list to the class
5. listen to the others’ and improve yours, if needed
Task 4 Strategic Skills: compensation
If you feel difficulty while you are doing oral performance what can you do?
For example if you lack appropriate word or expression what measure do you
take? Discuss in your small group
Task 5 Strategic skills: managing the stage
In addition to the strategy you use to fill (compensate) your linguistic skills,
what other strategy you need to use to help your interlocutor or audience
understand you easily? Does your voice matter? What about your manner?
Discuss in your small group.
Activity II: Specific Elements of Oral Skill
As you discussed above, speaking is one of the major skills that involves the linguistic skills
(accuracy, fluency, and complexity) and other non-linguistic skills. Oral skill requires the
knowledge of these elements and the ability to assemble and use these components during
performance.
1. Think by yourself the specific language elements and other competencies you should
have to accomplish certain oral performances e.g., classroom presentation, interviews,
257
public speaking, etc. For example, is it necessary to know words and different
expressions? Do you need the skill of organizing your idea before speaking?
2. Think as many of language elements as you can and list them down.
3. Which of them do you think are related to fluency and which are related to accuracy?
4. In addition to the language element, what other additional techniques you need to make
effective performance? List them down under different column.
5. Make a group of four or five and compare your lists. Refine and produce one
comprehensive list of the elements. Then, try to put the items on your list in order of
importance as you feel
6. Voluntarily, Present your result to the class
Activity III: Description
Task 1 Identify errors
Below are extract from records/transcripts of conversations between persons in different
situations. In a group of three or four, listen/read each of the extracts and:
1. See if there is any error. Jot down/underline area where you think has error.
2. Identify the kind of the error committed; for example is it error of verb tense or the
subject and verb agreement (grammar), wrong use of word (vocabulary), irrelevant
idea (content), disordered ideas (organization) or any other problem you feel.
3. Which of the errors cause problem of understanding and which are acceptable?
4. What do you think is the cause of the error, can you correct it?
5. Compare your answers with other groups and present to the class
Two students are doing their mathematics homework sitting in open place in a campus
A: Look how fat that woman is
B: Yes, she’s very fat. Even she can’t walk properly, lots of water coming out of her
face
258
A: She walks little and stands now and then
B: When she gets tired, she puts her two hands on her waist
A: Now stop staring at her; let’s do our business
B: Sure, where are we?
A: We’ve started solving problem number 2
B: oh, this question is very heavy, we can’t solve it
A: ok, we’ll ask friends to help us, now let’s go to the next,
They are in a cafe talking about personal things
A: Where are your parents?
B: Nekemte. My father worked in the zonal education office since 2007 and my mom
work in the hospital
A: i see..i mean.. graduating wanna go back there?
B: of course, you see... i wish i teach in Wollega university, but.... i can not be
confidential
A: what, why you need to be confidential? I mean i did’n get you
B: er.... i mean, you know... to be assigned to teach in a university, i should have to
graduate with very good GPA ...must be above... may be 3.5 can i score this then?
I’m not sure.....
Bola has applied for a job and is being interviewed by a recruiting committee
Chairperson: welcome Bola, we’ve seen your CV and we want to ask you some
questions. Feel free to answer our questions. Good luck!
Interviewer: can you tell us briefly the effects of HIV/AIDS on the society?
Bola: ok thanks, hiv aids, yes, hiv aids is a disease. It is emmm.., it is caused by virus. It is
abbreviation for human immune virus. Aids... hiv aids has many effects. It has no
vaccination or treatment. It kills young and adults who are productive. Therefore ....
therefore ..emmm it causes the decrease of production. Parents and relatives spend their
money and time to take care of the patients. It is transmitted in many ways. many ways
259
emm... for example unsafe sexual practice, traditional practices. The traditional practice
are circumcision, polygamy etc. we must stop aids thank you
Task 2 Describe the errors
The instructor will guide you to form groups and s/he assigns you to work on one of the
elements or aspects of English oral performance.
1. Explain what is meant by that element. For example what does it mean by grammatical
error, problem of content, problem of vocabulary or organization, etc.?
2. Illustrate by giving your own examples
3. Find a group that is working with the same element as yours and discuss your answers.
Improve if you need
4. select a spokesperson to present the result of your discussion
Section 3: Developing Criteria (1 ½ hr.)
Objectives
At the end of this section you will be able to:
identify the holistic and the analytic scoring techniques
describe each of the scoring techniques
develop rubrics against which each of the elements of oral performance could
be assessed following the analytic scoring method
Activity I: Scoring Techniques
Task 1
1. If you are asked to mark and score your classmates’ paper of objective types
test (m.choice, t/f, or matching items) how do you go about it? Share ideas with
persons by your sides and tell to the instructor.
2. Do you score subjective tests, e.g., written performances like a paragraph or an
essay and oral performance like speaking on a topic or interviews in the same
way as you do for the objective type? Why/why not? Discuss in pairs
Task 2
260
Instructors were asked the techniques they use to score written and oral
performances. The instructors’ responses are grouped in to four
Group A: Error counters
These instructors count any kind of error committed in the work done and deduce it from the
total mark assigned to the work
Group B: Holistics (impressionists)
This group reads or listens to/watches the performance as a whole and then give value based
on the impression they derived from the work as a whole
Group C: Analysts
Group C use a marking scheme. That means they first identify the specific aspect of language
they want to assess. Next, they decide the weight of each aspect. Then, they write rubrics
(criteria) which helps them to decide the amount of mark or value they should give for each
aspect
Group D Purpose concerned
The last group gives mark based on the purpose of the task performance. Like group B, this
group also reads or listens to/watches the performance as a whole and decide the extent to
which the performance achieved its purpose. For example, if the purpose of a speaker/writer is
to show the differences and similarity of two things by comparing and contrasting, the value is
given based on how effectively s/he achieved this purpose.
Form a group of three or four and read all the scoring techniques used by the groups of
instructors.
1. Discuss the strength and weakness/advantages and disadvantages of
each technique
2. If you were to choose one among the four, which one do you opt for?
Why?
3. Present the result of your discussion to the class
Activity II: Writing the rubrics
Now you are going to assess your classmates’ and your own oral performance using
the analysts scoring technique (analytic scoring). Remember that in task 1, activity II
of section I, you have described assessment as a process of gathering information on
261
students learning; and it includes setting criteria and using the criteria to measure the
quality of the performance. Therefore, you start the assessment procedure by
developing the marking scheme. Marking scheme is simply a form that guides you on
what and how to assess. It contains the elements and aspects of the performance to be
assessed, the weighting it counts for and detailed description (rubrics) of the
performance that helps you to decide the value that particular performance deserves.
1. Form group following the instructor’s instruction
2. Look at the list of specific elements of oral performance you developed in
activity II of section 2. The instructor will assign you to work on one of these
oral skills
3. Assume that you are going to score a performance out of 100%. Decide the
weight (percentage) you allot to the item you are assigned to work on
4. Decide the number of standards (levels of quality) you want to specify.
5. Decide the value of each standard (level of quality).
For example if you are working with grammar and, you decided to set three
standards, you may set the descriptor of the highest quality as ‘very few
grammatical errors, clearly understandable’ and the least as ‘ lots of
grammatical error, very difficult to understand’. Then if the weight you
allotted for grammar is 20%, it means that highest quality performance
deserves 20% and the least quality performance may deserve 5% or less. You
may also use intervals like 15% to 20%, 10% to15% etc.
6. Now using the form below, develop a marking scheme for the component of
oral skills you are assigned to work on.
7. When you finish, find a group working on your element and compare your
marking schemes. Discuss and make any improvement
8. Now the two groups have got one marking scheme. Pick a spokesperson and
present your scheme to the class. You may consider comments and suggestions
from the class and improve your rubric.
9. Remember that all the schemes developed by the class for different elements
will be combined on a single form and used to assess your classmates and your
own oral performance.
262
Sample Working Form for a group Form A : Performance/Outcome: ___________________________Name (ID): _________
Date: ______________________Rated by: Self __ __ teacher ____________
Form B: Performance/Outcome: ___________________________Name (ID): __________
Date: ______________________Rated by: Self __ __ Teacher ______
Section 4: Applying Criteria, giving and using Feedback (3 hrs.)
Objectives
After this section you will:
Practice scoring their peers’ and their own English oral performances using the
rubrics (scoring guide) they have developed
Give feedback to each other
Element weight descriptors points score comment
Grammar (tense, agreements, etc)
20%
Very few errors, clearly understandable
17-20
Few grammatically correct utterances, very difficult to understand
2-5
Components Weight
descriptors point
s
Score comment
Content
(relevance and
adequacy)
(30%)
Has little or no relation and limited
ideas
5
very relevant and comprehensive
information
30
263
Use feedback from peers and instructor to improve their scoring
Activity I: Assessing others
Task 1 Scoring a video recorded performance
The instructor will play you a video recorded performance. Watch the video for
the first time.
1. Get the marking form ready and
2. With the instructor, decide on which elements of the performance you
want to assess and the weighting you allot to each element
3. Watch the video again and score the performance focusing only on the
kill you want to score. After you scored you can write comment about
the performance in the space provided
Task 2 Feed back
1. Compare your score in pair or group of three and discuss on the
differences: justify yours and listen to the others’. Improve if you are
convinced.
2. Listen to the instructor’s scoring and compare to yours. Is it very
different or slightly different?
4. Watch the video again and see if you have anything to improve.
5. Do you think that it is possible to avoid differences in such kind
(subjective) assessment? Why?
Task 3 Scoring classmate’s oral performance
Now you are going to score your classmates’ performance. Get the scoring
form ready (the instructor may provide you with the form)
1. With the instructor, decide the elements to be assessed and the
weighting
2. Attend to your classmates’ performance and score it
Task 4 Peer Feedback
1. Join in groups of three or four and compare and contrast your scores:
discuss on your difference and give reasons to justify. You can improve
your scores if convinced during the discussion
264
2. Listen to the instructor’s scoring and compare with yours. You can ask
questions or justification if you are not satisfied.
Activity II Assessing own oral performance
Task 1 Scoring
Now you score your own oral performance by watching the video record. Get
the form ready and discuss with the instructor what aspect of the performance
you score. Then watch and score. You could watch the video twice.
Task 2 Feedback
The instructor will show you her/his score of your performance. Compare it
with yours and see the differences and similarities. Ask any question or
justification, if you need. Remember that the main purpose of assessment is to
help you improve your learning, and you are involved in assessing yourself to
contribute to this purpose. The teacher is also doing this by indicating your
strengths and weaknesses so that you devise your own strategy (way) of
improving your learning.
Section 5: Setting Goals and Action Plans (1 hr.)
Objectives
At the end of this section, you will be able to:
identify their strengths and weaknesses of their oral performances
write a major and two/three specific goals to improve their weakness
describe the actions they will take to achieve the goals
Activity I: Reflection 1. Think back your self assessment and reflect on what you have done well
and what went wrong. Write down as many specific points as you can about
your strengths and weaknesses.
For example,
I have put my ideas in logical orders but the expressions I used to move
from one to the other were not appropriate.
265
I used relevant ideas but I failed to express the idea using the right
words
My grammar is good but I usually misuse the modal verbs like should,
would, have to, etc.
2. Select two or three of your weaknesses that you need to improve soon
3. Move in the class and find a student/s who has/have got the same problem
as yours. (if there are many with similar problem, you can break into groups
of three or four)
Activity II writing goals and action plans
In your group or pair, discuss on your problems
a) What do you think is/are the source of the problem
b) to what extent you need to improve (your goal)
c) what things they need to do to improve it (action)
Write down your goals and actions in specific clear and achievable
sentences
For, example,
Goals:
- I will improve my knowledge and use of perfect tenses
- I will improve the way I conclude my speech and
Action plans:
- I read grammar books on perfect tenses
- I practice using perfect tenses in the appropriate situation with friends
- I discuss with Mr. X on smart ways of concluding a speech, presentation,
interview, etc.
- I practice with my classmates
266
Appendix 8 Training Material (Instructor’s Guide)
Learning to Self-Assess
_________________________________________________________________
A Material Prepared for Training Students on Learners’ Self-assessment of
English Oral Performance (as part of research instrument)
Feb. 2011
Addis Ababa
267
Table of content
Topics page
Classroom Self-assessment
An Overview…….…………………………………………………………………………. 1
Training Objectives…….....………………………………………………………………….1
Some General Tips……………………………………………….………………..…………2
Introduction
Objectives……..……………………………………………………………………………..3
Activity I: Expectations and learning mode …..………………….………………….3
Section 1 Meaning and purposes of assessment
Activity II: Meaning and purpose of Assessment……………………………………4
Activity III: who should do the assessment? ………………………………………..4
Activity IV: Self-assessment........................................................................................5
Content information............................................................................................................5
Section 2: Identification of components of oral performance
Objectives ..........................................................................................................................6
Activity I: Effective English oral performance...................................................................7
Activity II: Specific Elements of Oral Skill........................................................................7
Activity III: Description......................................................................................................8
Content information............................................................................................................9
Section 3: Developing Criteria
Objectives..........................................................................................................................9
Activity I: Scoring Techniques..........................................................................................10
Activity II: Writing the Rubrics.........................................................................................10
268
Content information..……………………………………………………………………11
Section 4: Applying the Criteria and Feedback
Objectives..............................................................................................................................14
Activity I: Assessing others..............................................................................................14
Activity II: Assessing own oral performance...................................................................15
Content information...............................................................................................................16
Section 5: Setting Goals and Action Plans
Objectives.............................................................................................................................16
Activity I: Reflection....................................................................................................16
Activity II writing goals and action plans....................................................................16
269
Classroom Self-assessment
I. An Overview
From the learner-centred approach point of view, self-assessment is the major component of
the teaching learning process which promotes active learning. Self-assessment (SA) is an
assessment technique which enables learners to participate in the assessment process including
the setting of assessment criteria and scoring their own performance. However, learners need
some training and exercise before they are involved in the actual assessment.
This training is designed for second year university students majoring in English to help them
understand issues related to SA and develop their ability of assessing their own English
language oral performance fairly and responsibly. It is aimed at enabling them participate in
the whole process of assessment i.e., making decision about the explicit criteria of
performance and making judgement about quality of the performance.
This trainer’s guide is intended to help trainers (instructors) to accomplish their role of
facilitation successfully. It is organised into five sections that will be integrated with the
Spoken English course contents. These sections are I) introduction II) identifying elements of
oral performance, III) setting criteria, IV) applying the criteria and feedback and V) setting
goals and action plans. Each section has objectives and contains activities and tasks. Where it
is necessary, content information is provided at the end of sections so that you can refer to it.
It is intended to be presented using participatory and interactive methods like brainstorming,
pair or small group discussion, vote on your feet, etc. However it does not mean that the
suggested activities and procedures are hard and fast rules; the instructor could use her/his
own ways that s/he finds appropriate. It could be covered within seven — nine hours (an
hour/week) of additional time to the schedule of the course.
II. Training Objectives
The overall goals of this training is to create and/or increase the students’ awareness on the
meaning and significance of self-assessment in improving students learning outcomes, and to
270
develop their ability and skills of assessing their own oral performance fairly and realistically
so that they can participate in the assessment process meaningfully.
Specifically, by the end of the training the students will be able to
understand the main purposes and benefits of SA
develop criteria for assessing English oral performance
determine quality of their own performance responsibly
design strategies for improving their own learning outcomes
Some General Tips
The role of the trainer (instructor) is to organise and facilitate
situations in which the students learn by themselves, not to tell
everything
Students learn better when the situation is conducive and the tasks are
interesting, not monotonous
Always try to begin a session in soothing funny like jokes or other
icebreakers
Many of the tasks in this material require students to do in pairs or
groups. Group works are effective techniques, need care
Use different ways of group formation like alphabets,
birthdates interval numbers etc. so that a student gets chance
of meeting different groups
Round and supervise group discussions
Though students should learn by themselves, they obviously
need your help, so during group discussions, visit groups,
participate and contribute to their discussion. (even some
groups may need you badly)
271
During the whole class discussion, encourage all groups, not
only the volunteers
Introduction (1 hr)
This introductory part is intended to lay ground by providing the students with an overview of
the training. It gives the general objectives, what is included in and how they would go about
it.
Objectives:
At the end of this section the students will:
explain the main objectives of the training
discuss and decide on how to go about it
Activity I: Expectations and learning mode
This activity is to arose the interest and curiosity of the learners by letting them
aware of what and how thing happen during the training so that they feel
responsible and set up their mind from the starting
Task 1 expectations
Distribute the student’s material
Introduce the topic (Classroom Self-assessment) and mention that you are
discussing the topic of SA for some weeks together with the contents of the
course to help them learn how to self-assess.
Ask what they expect to gain from the training
Guide them through the task and finally explain the general and specific
objectives of the training
Task 2 Learning mode
In this task, help the learners to remember a number of active learning
techniques and identify their pros and cons
During the whole class discussion, focus on how group discussion can be
used effectively
272
Section 1: Meaning and Purpose of Assessment
Objectives:
At the end of this section the students will:
Explain meaning of assessment
list the main purposes of classroom assessment
identify the main features (process) of assessment
discuss who should assess learners’ work
define SA
tell the benefits and problems of SA
Activity I: Meaning and Purposes of Assessment
Guide them through tasks 1 and 2
Ask some volunteers their responses and reasons for thinking that way
Direct students attention to statements d & e to get the intended meaning of
assessment and statements c, g, h & i for purpose of assessment
Activity II: Who should do the assessment?
Before this session, write these four words/phrases: teacher, school/external body,
student him/herself, peers, on separate sheets of paper and post it in the four
corners of the room.
Tell the students that this activity asks them to express their opinion about who
should assess students’ performance
In task 1, let them share ideas and experiences about who usually carry out
assessment. In task 2, Point out that each of the four corners of the room
represents a party suggested to assess learners’ work, and every one express her/his
idea about who should assess learners learning outcomes by moving to the corner
that represents the party
Make clear that one can vote for two parties by drawing a line between the corners
and standing at the middle ground between the two corners; and if one could not
decide, s/he remains seated.
273
Explain that after the students have moved to the area of their choice, you will ask
for them to explain why they feel that way.
Tell participants that they can change their mind and move their position at any
time during the activity, for example, after listening to others’ reasoning.
Have the students return to their seats for general discussion. Ask them whether or
not they like to self-assess, why/why not.
Summarise that different parties can do assessment for different purposes, but in a
classroom assessment where the main purpose is to improve learning, learners
should participate in the assessment process and assess themselves and peers to
improve the quality and quantity of their learning outcomes.
Activity III: Self-assessment
Task 1 describing self-assessment
Remind them the definition given to assessment. Ask students to describe self-
assessment based on that definition first individually then in their groups
Let them discuss and write down their description
Ask two or three volunteer groups to present their answers to the class, and
give opportunity for the others if they have anything to add
Pool their ideas together and finally guide them to see that self-assessment
means participating in the whole process including setting standards,
developing criteria and evaluating one’s own performance using the criteria
responsibly; and this has a number of advantages of which the major one is the
improvement of learning.
Task 2 beliefs about self-assessment
Mention that in this task they are going to express their belief about self-
assessment
In their group, guide them to read all the statements and see whether or not they
agree with the statements
Let them explain their reasons and discuss as whole class
274
Summarise that some of these beliefs come from our experience of the
conventional role of the teacher and students and direct their attention to the
positive aspects of self-assessment and ways of minimizing the problems.
Content information
1. The main objective of the training is to help learners participate in the assessment
process of their own English oral performance realistically and fairly. It does so by
enabling them to identify and define the major and specific constructs of oral
performance, develop the assessment criteria and apply it. It also helps them to practice
recognising weaknesses and setting action plans for improving their weaknesses
2. Assessment is a systematic procedure for obtaining information to describe or better
understand about learners’ learning outcomes to make judgments and decisions about
characteristics of the learners or programs
3. Assessment can be carried out by teachers/instructors, schools or external bodies for
different purposes like diagnosis, placement, promotion, certification, etc. However,
the main aim of classroom assessments is to improve the quality and quantity of
students’ learning outcomes. To achieve this purpose of classroom assessment,
students must be involved in the whole process
4. Typical process of assessment includes making decisions about the explicit criteria and
standards of performance expected and making judgments about the quality of the
performance in relation to these standards.
5. SA can be defined as a form of assessment in which students are involved in
identifying standards and/or criteria to apply to their work and making judgements
about the quality of their own performance and the extent to which they have met the
standards.
6. SA helps learners to be clear with the training objectives and the standards of
performance expected of them. This is believed to have many benefits. Researches
have shown that it improves cognitive and non cognitive learning outcomes. For
example, it develops higher order cognitive skills, fosters deep approaches to learning,
develops reflective skills, sharpens critical abilities, increases autonomy, increases on-
task behaviour, increases commitment to subsequent performance and increased
participation reduces distrust and disruptive behaviours
275
7. Problems related to SA are students’ lack of skill and ability of assessing and
unfairness in assessing themselves. The other problems are related to learners’ and
teachers’ conceptions of conventional roles of learners and teachers. These problems
can be solved by training and through experience.
Section 2: Constructs of oral performance (1 ½ hr.)
Objectives
Completing this section the learners will be able to
describe characteristics of good English oral performance
identify the specific linguistic elements (language skills, sub skills and
competences) that are constructs of English oral performance
identify the non linguistics components of English oral performance
describe each of the specific linguistic and non linguistic elements to be
assessed in oral performance
Activity I: Effective English oral performance
Guide them through task 1
Invite volunteer groups to tell the class what they think are characteristics of
good/effective English oral performance.
Write their expressions on the board; they may use words like interesting,
attractive, understandable, etc.; or may use some specific characteristics like
correct grammar, fluent speech, related ideas, well organised etc. to describe a
good/effective oral performance. This much is enough for this task, and to lead
them to the next task, explain that oral performance basically involves two major
skills; linguistic and non-linguistic. The linguistic skill includes fluency, accuracy
and complexity.
Guide them working through task 2,
During the whole class discussion help them to understand the skills and
competences the terms fluency, accuracy and complexity refers to. If need be,
explain briefly with examples.
276
Make clear that all the three elements are equally important for effective oral
performance, but you do not focus on complexity in this training.
In task 3, 4 and 5 help them to remember the non-linguistic skill (organisation and
content) they learned in their writing course and help them to understand the
compensation strategies and delivery strategies (they could use different
expressions to mean content, organisation and/or delivery technique).
Activity II: Specific Elements of Oral Skill
Remind them that speaking is one of the major skills that involves other specific
elements, and in this activity, they are going to identify and explain these elements
During the class discussion, make columns with the heading: fluency accuracy,
organization, content, strategies.
Invite volunteer groups to read out their list. When they read an item, ask under
which column to put, and ask the class whether they agree or not.
Regarding the rank of importance, explain that all are still equally important
because there cannot be an oral performance devoid of one or the other element.
However, the attention to be give to each element depends on contexts and
purposes. When friends chat about personal matters, for instance, things like
grammatical accuracy, relevance of content and sequence of ideas are not usually
attended to. In assessment also you do not give equal attention to all the elements;
some may be ignored and some may be given a little attention depending on the
purpose of that particular assessment.
Activity III: Description
Task 1 Identifying errors
To help student do this task, it would be better if you get the conversations
recorded and students listen them rather than reading. Anyway, adjust the
kind of help you provide then to the format you are able to use.
Guide the learners to answer questions 1 to 4 for all the three extracts (A to
C) turn by turn beginning from A
277
Write their answers for each extract, guiding them to the appropriate
answers. In the first extract (A) the major problem is lack of vocabulary.
For example, “ lots of water coming out of her face” is to say she is
sweating. “ is very heavy” is to say difficult. In B the problem is grammar;
e.g., “worked in the zonal education office since 2007” is to say has been
working in the zonal education office since 2007. This makes understanding
difficult because the listener is confused whether or not the father is still
working in the hospital. In C the major problems are a) irrelevance of
information (content), b) disorganised ideas and c) lack of fluency
(unnecessary repetition of words and taking unnecessarily longer pause).
Task 2 Describe the error
Divide the class in to groups so that the number of groups equals twice the
number of specific elements of oral skill: grammar, vocabulary, content,
organisation, fluency and strategies.
assign one item to two groups by lottery system or any way you like
guide them to work through the task and provide assistance where they
need
when a group presents its discussion of an element to the whole class,
encourage all the other groups to participate and share ideas
summarise and write the description of each element clearly on the board so
that everyone understands or copy down
Content information
1. Fluency can be defined as the ability to produce spoken language with ease; or the
ability to speak with a good but not necessarily perfect command of intonation,
vocabulary ,and gram mar ;or the ability to produce continuous speech without
causing comprehension difficulties or a breakdown of communication.
2. Accuracy is the degree of correctness or deviation from the norm (error)
3. The specific elements of oral performance are content, organisation, vocabulary,
fluency, pronunciation, grammar, strategy of delivery and timing
278
- Content is the relevance to and adequacy or comprehensiveness of ideas
raised in relation to the requirements of the task performance
- Organisation is the coherence and cohesion of the ideas
- Vocabulary is the appropriateness and correct use of lexical items
- Grammar is the degree of accuracy (acceptability) of the linguistic
elements (verbs, nouns)
- Pronunciation is the intelligibility and consistency with which words and
phrases are said
- Delivery strategy includes eye contact, confidence, manners, position, use
of non verbal languages, voice quality (loud enough) etc.
- Timing is the ability of budgeting the time available properly and using it
4. Compensation strategies include using ellipsis, rephrasing, repetition,, fillers and
hesitation devices, borrowing words and phrases from other languages they know,
or engaging the listener in collaborative meaning-making.ies of delivery for oral
task
Section 3: Developing Criteria (1 ½ hr)
Objectives
At the end of this section the students will be able to:
identify the holistic and the analytic scoring techniques
describe each of the scoring techniques
develop rubrics against which each of the elements of oral performance could
be assessed following the analytic scoring method
Activity I: Scoring Techniques
Guide them to answer questions 1 and 2 in task 1. Their response to
number 1 should be something like “prepare answer key and decide on the
weight of each item then mark using the key”.
279
Elicit their response to number 2 and let the be aware that it is impossible
to score subjective assessments like written and oral performance in the
same way we score the objective ones because it is very difficult to prepare
a precise answer key. Therefore we need different scoring techniques.
In task 2, get them read the four suggested procedures thoroughly and have
clear idea. You may need to explain each scoring technique before they
answer the questions
Listen to their response and write the strengths/advantages and weaknesses/
disadvantages on the board and discuss if need be.
Ask which one they choose and why.
Mention that you (the class) are going to use the analytic technique. Reason
out if they are not convinced.
Activity II: Writing the Rubrics
tell the students that in this activity you (the class) develop marking scheme
they use to score oral performances of their classmates and their own
because they follow the analytic technique of scoring
group the class into a number that is twice the number of elements of oral
skill
assign an element to two groups by lottery system or the way you like
tell them that they are going to set assessment criteria (marking scheme) for
the element they have got
show them to use the form provided
guide and help them to decide the weight (percentage) they should allot to
that specific element out of the total mark (100%) out of which the whole
performance is evaluated; the number of standards (levels of quality) and
the value of each level. Note that each element can have different weight
and number of standards from the other. For the start they may use three
standards (levels) and may move to five or more standards as they get
experienced
280
help them by providing appropriate expressions like few errors, lots of
hesitations, many redundancies, lack conclusion, in appropriate word, etc.
and remind them not to use general descriptors like good, excellent
let them join a similar group and compare their scheme, discuss and
improve
Let each group present its marking scheme and guide the class to discuss
and agree on the rubrics developed for each element so that they are shared
as a class and to be used in SA
Merge all the forms for different components into one and have all the class
copy it and inform them that they will use it throughout the rest sessions
Content information
1. The students may use some general terms like interesting, attractive, comprehensible,
fluent speech, etc.; or may use some specific characteristics like correct grammar,
related ideas, well organised etc. to describe a good/effective oral performance. This
much is enough for this activity.
2. Generally there can be four techniques of scoring an oral performance: error counting,
impressionistic (holistic), primary trait scoring and analytic. In the error counting
method, the marker counts the number of errors in the texts and deduct from the total
marks assigned to the work. It ignores the different constructs and sub-skills and fails
to take in to account of the kind of error to be or not to be emphasised (Heaton, 1990;
Hughes, 2003).
The impression (holistic) method involves awarding marks to students’ work based on
the total impression of the performance as a whole. Though this method is good for
efficiency, marking large number of composition, it is vulnerable to the markers’ bias.
It also ignores the aspects of error to be focused on and it is difficult for students to get
feedback (Heaton, 1990; Hughes, 2003; Brown, 2003).
The primary trait scoring involves focusing on a single primary purpose of the
performance. For example, if the purpose of the oral performance is to argue, it is
marked based on how well the performer achieved this purpose
281
The analytic method involves the use of marking scheme that helps the marker to
identify the language aspects and skills required to be exhibited by the testee. It also
guides on the amount of mark to be given for each aspect element. Therefore, it is
relatively reliable than the others and helps to give feedback to students on their
performance
3. Rubric ( scoring guide) is a set of criteria used to discriminate effectively between
quality of different performances
Working Form for a group
Form A
Performance/Outcome: ___________________________Name (ID): __________________
Date: ______________________Rated by: Self __ _ Teacher _____
Form B
Performance/Outcome: ___________________________Name (ID): __________________
Date: ______________________Rated by: Self __ __ Teacher _____
Section 4: Applying the Criteria, giving and using Feedback (3 hrs.)
Element weight descriptors points score comment
Grammar
(tense,
agreements, etc)
20%
Very few errors, clearly
understandable
17-20
Few grammatically correct
utterances, very difficult to
understand
2-5
Components Weight
descriptors points Score comment
Content
(relevance and
adequacy)
(30%)
Has little or no relation and limited ideas
5
very relevant and comprehensive
information
30
282
Objectives
After this section the students will be able to:
Practice scoring their own English oral performances using the rubrics (scoring
guide) they have developed
Give feedback to each other
Use feedback from peers and instructor to improve their scoring
Activity I: Assessing others
Task 1 scoring
get a video record of an interactive or extensive oral performance e.g.,
speech, presentation, dialogue, interview, etc. let them watch first to give
the hint of what kind of English oral performance they are going to score
check that each student have got the scoring form
decide, with the whole class, which elements should be included in the
scoring and which should be excluded and the weighting of each element
depending on the type of task performance you have got and/or the
objective of the assessment
let them watch the video and score the performance using the form
task 2 Feedback
get the students join in groups of three or four and compare and contrast
their scores
let them discuss on their difference and give reasons to justify or improve if
convinced or keep it until you replay the recording
when they have done it show them your scoring and let them compare and
contrast it with their own
replay the recording again and let them improve their scoring if they need
see the number of students whose scoring is nearly similar, a bit different ,
and very different from yours
let everyone raise the areas of differences and; you explain yours and they
too. One can also improve her/his scoring during this discussion
283
explain that as this is subjective judgement, it is difficult to avoid differences and
have the same score; but the very important thing is to avoid personal bias about
the assessee and concentrate on the criteria of the assessment
assign an individual or a pair oral task to be performed next session
Task 3 Scoring classmates’ oral performance
Get the students ready with their scoring guide and inform them that they are
going to assess their classmates’ performance
Decide on what to be assessed and the weighting of each element
Have all the students do the performances you assigned the last session in turn and
let the students assess the performance of their classmates. You can divide the
class into groups of assessors in the way that a student gets a chance of scoring at
least three performances
video record each performance so that the performers score their own performance
later
you also score the performance using these same scoring guide
let them get feedback from peers and you as done in task 2 above
Activity II: Assessing own oral performance
Task 1 scoring
Get each student watch video of her/his own performance and score it. Let him/her
watch the video once more if s/he needs
You also score every student’s performance
Task 2 Feedback
In one-to-one format, show your scoring of the student’s performance and let
him/her compares it with his/her self-assessment
Discus on the differences and give feedback to each of their self-assessment
Remind them that the very purpose of self assessment is improving one’s own
learning
provide the students with the opportunity to get ample feedback by repeating this
activity II of this section
284
Content information
In an assessment it may not be necessary either to include all the elements of oral
performance or give equal emphasis. The element which should be included in the
criteria and the amount it should count depends on the objective/s of the assessment. For
instance, if the objective of the assessment is to see how well students can select relevant
information and organise it, then content and organisation would be the main criteria.
The others can be ignored or assigned to count lesser percentage of the assessment
value.
8 Section 5: Setting Goals and Action Plans (1 hr.)
Objectives
At the end of this section, the students will be able to:
identify their strengths and weaknesses of their oral performances
write a major and two/three specific goals to improve their weakness
describe the actions they will take to achieve the goals
Activity I: Reflection have the students to think back their self assessment and identify
- what they have done well
- what they failed to do
Individually, let them list down as many specific points about the good and poor
performance. Help them to specify clearly.
Guide them to decide on two or three specific problems that they need to improve
soon
let them find and join classmates with the same problem (if there are many with
similar problem, they can break into groups of three or four)
Activity II writing goals and action plans
In their group or pair, guide them to discuss and
- Identify the source/s of their problems
- to what extent they need to improve it (goals) and
285
- what things they need to do to improve it (action)
let them write down their goals and actions. Guide and help them to write their
goals, and plans in specific, clear and achievable expressions. You can add to the
examples given in the student’s material
286
Appendix 9: Marking Scheme for students
Performance/outcome: ___________________________________ID: _____________
Marked by Self
Elements
wei
gh
t
Descriptors
score
Content
(relevance
and
adequacy)
4pts
Has no any relation 0
Has little relation and limited ideas 1
Has some relevance but inadequate information 2
Relevant and sufficiently adequate 3
very relevant and entirely adequate information 4
Organisation
(Lead-in,
finishing,
coherence,
cohesion)
4p
ts
Has no introduction (lead-in) and/or conclusion (ending), ideas are not logically
sequenced and there are no or few appropriate cohesive devices and transition signals, no
clear main and supporting details
0.5
Not easily recognisable (weak) introduction and conclusion, serious problem with logical
order of ideas, some appropriate cohesive devices and transition signals, few supporting
ideas
1.25
Clear lead-in and ending, some problems with sequencing ideas, not fully supported
ideas and a few inappropriate cohesions and transitions
2.5
Interesting introduction and finishing, very few errors in sequencing, clear and fully
developed main and supporting ideas, appropriate use of cohesion and transition
4
Grammar
(tenses,
pluralisation,
agreements,
etc.)
5pts
Lots of unintentional grammatical errors, difficult to understand 1
Considerable grammatical errors but understandable 2
A few grammatical flaws, but easily understandable 3
Rare or few unintentional errors 4
Virtually no or very rare unintentional error (native- like accuracy) 5
287
Fluency
(ease of
production)
3pts
Frequent hesitation, many unintentional incomplete sentences, long pauses and
regressions, perceived difficulty in production difficult to understand
0.5
Occasional hesitation, a few unintentional fragments and pauses, less ease of producing
sentences (speech), but understandable
1.25
Few hesitation and effortless production, few fragments and unintentional pauses, little
regressions, easy to understand
2.15
Virtually no any fragment, no unintentional pause, effortless (easy) production of
sentences
3
Vocabulary
(choice &
appropriaten
ess)
4pts
Very limited vocabulary, frequent misuse of lexical items, too wordy, no idea of
register
0.5
inadequate vocabulary and poor expression of idea, frequent use of inappropriate
words
1
Adequate vocabulary, but some misuse of words, lack of variety 2
Occasional use of inappropriate terms, lack of some idea of register 3
Precise and variety use of words, concise and good register 4
Intelligibility
3pts
Frequent and consistent mispronunciation, very much influenced by mother
tongue, difficult to get meanings
1
A few mispronunciation , a little effect of L1 leading to occasional problem of
comprehension
2
Few mispronunciation, almost no problem of understanding, though marked
foreign accent
3
Strategy of
Delivery 2p
ts
Little eye contact, facial expressions and other body languages, inappropriate
voice level and sitting or standing posture, unnecessary mannerism
0.5
eye contact fixed at a point, insufficient body language, inappropriate voice level
a few mannerism
1
Consistent and appropriate facial expressions and other body, appropriate voice
level and sitting or standing posture few or no mannerism
2
Adapted from: The Associated Examining Board Test in English for Educational Purpose: Assessment Criteria for the Oral
Test (Weir, 1990)
288
Appendix 10 : Marking scheme for instructor
Performance: ________________________________
The student’s code
Name
Topics assigned
Construct
wei
gh
t Descriptors
poin
ts 1 2 3 4 5 6 7 8 9 10 11 12 13
Content
(relevance
and
adequacy) 4pts
Has no any relation 0
Has little relation and limited
ideas
1
Has some relevance but
inadequate information
2
Relevant and sufficiently adequate 3
very relevant and entirely
adequate information
4
289
1 2 3 4 5 6 7 8 9 10 11 12 13
Organisation
(Lead-in,
finishing,
coherence,
cohesion)
4pts
Has no lead-in and/or ending,
ideas are not logically
sequenced and there are no or
few appropriate cohesive
devices and transition signals,
no clear main and supporting
details
0.5
Not easily recognisable (weak)
lead-in & finish, serious
problem with logical order of
ideas, some appropriate
cohesive devices and transition
signals, few supporting details
1.25
clear lead-in and ending, some
problems with sequencing
ideas, not fully supported ideas
and a fewappropriate
cohesions and transitions
2.5
Interesting introduction and
finishing, very few errors in
sequencing, clear and fully
developed main and
supporting ideas, appropriate
use of cohesion and transition
4
290
1 2 3 4 5 6 7 8 9 10 11 12 13
Grammar
(tenses,
pluralisation,
agreements
etc.)
5pts
Lots of unintentional grammatical
errors, difficult to understand
1
Considerable grammatical errors
but understandable
2
A few grammatical flaws, but
easily understandable
3
Rare or few unintentional errors 4
Virtually no or very rare
unintentional error (native- like
accuracy)
5
Fluency (ease
of production)
3pts
Frequent hesitation, many
unintentional incomplete
sentences, long pauses and
regressions, perceived difficulty
in production difficult to
understand
0.
5
Occasional hesitation, a few
unintentional fragments and
pauses, less ease of producing
sentences (speech), but
understandable
1.
2
5
Few hesitation and effortless
production, few fragments and
unintentional pauses, little
regressions, easy to understand
2.
2
5
Virtually no any fragment, no
unintentional pause, effortless
(easy) production of sentences
3
291
1 2 3 4 5 6 7 8 9 10 11 12 13
Vocabulary
(choice &
appropriate
ness) 4pts
Very limited vocabulary, frequent misuse of
lexical items, too wordy, no idea of register
0
inadequate vocabulary and poor expression of
idea, frequent use of inappropriate words
1
Adequate vocabulary, but some misuse of
words, lack of variety
2
Occasional use of inappropriate terms, lack of
some idea of register
3
Precise and variety use of words, concise and
good register
4
Intelligibilit
y
3pts
Frequent and consistent mispronunciation, very
much influenced by mother tongue, difficult to
get meanings
1
A few mispronunciation , a little effect of L1
leading to occasional problem of comprehension
2
Few mispronunciation, almost no problem of
understanding, though marked foreign accent
3
Strategy of
Delivery
2pts
Little eye contact, facial expressions and other
body languages, inappropriate voice level and
sitting or standing posture, unnecessary
mannerism
0.
5
eye contact fixed at a point, insufficient body
language, inappropriate voice level a few
mannerism
1
Consistent and appropriate facial expressions
and other body, appropriate voice level and
sitting or standing posture few or no mannerism
2
292
9 Appendix 11 Students interview Transcript (1
st round pre-intervention)
Interviewer: thank you for your cooperation for this interview. To introduce myself, I’m Rufael
Disasa I am student at Addis Ababa University. I’m doing a research in the area of assessment. So, the
purpose of this interview is to collect data for the research. And I want to assure you that the
information to be obtained from this interview will be used for this research only; therefore, you just
feel free and respond to my questions. We raise some issues of assessment in general and students’ self
assessment in particular. Basically there is no any wrong or right answer, I want you just to tell me
whatever you feel I mean your opinion and thoughts about the points we raise. And I hope you
understand that your genuine I mean true answer is very important for the validity of the research
result. we can also use our native language and express our idea if we want to. ማለት ለምሳሌ እኔ ሃሳቤን
ለመግለጽ በተመቸኝ ቋንቋ እጠቀማለሁ ሃሳባችንን ነጻ ሆነን በምመቸን ቋንቋ መግለጽ እንችላለን ።
To begin with, what do you think is the main purpose of assessment or evaluation? I mean why you
think a teacher assess your work? Shall I begin with you?
S1: the purpose of assessment is during the teaching process established that means when the teacher
teaching students he evaluates which or among the students who are working hard to
differentiate who is weak in his work and his study evaluate whether they are who record the
higher mark he want to know when he is teaching who is hard working and to evaluate teach
from one grade to another grade students they need to be assessed and then evaluate to pass or
progressive purpose they pass from one session to another this is the main reason and whether
the students have understood what they have been taught before and to prepare themselves for
the study purpose
Interviewer: thank you, any additional idea? Yes,
S2: the teacher wants to check which student is strong and which student is weak. He can see the
successfulness of the learning process by continuous assessment
Interviewer: thank you, okay you have something to add?
S3: Additionally my classmates more said something there the teacher is to assessing the students for
improving the students how to students understand how to students come to the class how to
students work hard to differentiate which one is study hard which one is lazy which one is by
depend on other etc are differentiated the teacher to assess this and evaluate students and score
the grade and knowledgeable of the course
Interviewer: thank you, ok you.
S4 that is important for the teacher because by taking assessment he can identify students who are
weak and strong, who have the knowledge. Unless he evaluates he cannot understand our
potential whether we understand his lecture whether it is clear or not without evaluation he did
293
not understand. Even in one class he may give lecture method for us on the same topic the same
idea he can understand which students have high capacity and which students have low capacity
and which students are medium without assessment he cannot identify
Interviewer: thank you all; let me raise a question related to this. Who do you think should do the
assessment?
S2: the teachers should do the assessment because he is the profession he knows whatever he teaches
the level of the students etc. So the teacher it is his job
S5: the teacher or the department can do assessment but the course instructor should prepare it he must
prepare grade at the end so he should make preparation for grading for example the department
may not know how the students learned the ability of the students but the instructor knows
Interviewer: thank you. Let me go to my next question. What is self assessment? How do you describe
it?
S3: ok, this assessment is just the process of evaluating the progress and achievement and self
assessment can be the base for all assessment and the key for success because before we sit for
the exam it is better to evaluate yourself whether you can or not that is good as I think
Interviewer: thank you, anyone who has anything to add? Okay,
S6: in my opinion self assessment is very essential to students the students when you every time
assessing students, students must do self assessment to developing their confidence and to more
knowledgeable to do anything to self confidence it is very important to developing and their self
confidence from time to time everything is not expected from the teacher it is mostly expected
from the student
Interviewer: anything to add or any different idea? Ok, one question related to this is, you just said that
assessing is the job of the teacher. Do you think that students can be involved in assessing themselves?
I mean, for example the teacher gives an assignment, a test or exam and then give mark for your work.
The teacher also sets the criteria against which your work is marked. I mean, when the assessment is
not objective type like multiple choice, true/false or matching item, but subjective type, for example,
when assessing speaking or writing a paragraph the teacher decides on the criteria. The criteria may be
the content, the grammar, vocabulary, etc. It is also the teacher who marks your work using the criteria.
Do you think that students should be involved or participate in the assessment process? Is it appropriate
to do so? If yes how? I mean in what ways? If not, why? What is your reason? ግልፅ ለማድረግ እኛ
አንደምናዉቀዉ እና የተለመደዉ አሠራር ምንድን ነዉ ፈተናዉን የመያወጠ ፈተናዉ የምታረምበትን ክራተርያ ወይም መሥፈርት
የምያወጣ መምህሩ ነዉ ክራተርያ ወይም መሥፈርት ስባል ለምሳሌ ፓራግራፍ ስታረም ይህ ፓራግራፍ ይህንን ይህንን እና ይህንን
ነጥቦች ካሟላ ይሀን ያህል ማርክ ይህንን ይህንን እና ይህንን ነጥቦች ካሟላ ግን ይህንን ይህንን ካላሟላ ይሀን ያህል ማርክ ብሎ
መወሰን ማለት ነዉ እና የኔ ጥያቄ ተማርዉ በዚህ ሂደት በዚህ ዉሳኔ መሳተፍ ይችላል ወይ? መሳተፉስ አግባብ ነዉ ወይ? ሁለት
ጥያቄ ነዉ
S1: well I would like to say students can assess for example now teacher may assess me it is possible
but I myself I can assess myself in which way when I say this for example I know even if I am
294
not telling to someone intentionally I know that I’m studying well where I’m the position where
when I get the result of my tests and look at my mark, I can evaluate myself when I’m mistaken
when the subject is difficult which portion I can evaluate myself in case of exam the part which
is become difficult for me to understand I know I can revise the subject matter and then in case
of competition may be the students fellow of my friends when they are studying I know who is a
head of me in his position I can make competition with them by evaluating myself by assessing
myself always emmm…. I can assess myself true my result position this can make me to become
strength in the whole part I can do by this way I suppose it is better not to rely on the teacher
assessment only we ourselves can assess ourself to progress our activity of our learning process.
Emmm..... but I do not show or tell to my fellow students
S4: we ourselves can determine our capacity or our knowledge depending on how we are reading or
answering that question and where we are. This is evaluating ourselves. Not only the teacher can
we evaluate ourselves. You evaluate yourself by looking at the mark you get.
Interviewer: ok, thank you. Let me clarify my idea. What you are saying is what you do informally by
yourself isn’t it? But do you think that you should be allowed, formally, to mark your own work, give
mark for their own work and this mark should count to their final grade? For example, I’m your
instructor, I prepare an assessment task you do the task, and I allow you to score give mark to your
own work. Do you think that this is possible and it is appropriate?
S7: I think this one is not appropriate way because everybody is not doing justice as the teacher do
because he has already been qualified and has professional of doing this in teaching learning
process he knows the way but now many students they may mark for the questions that may be
wrong they mark right because they want good grades in this case may be difficult for students
to assess themselves truly in case of exam
Interviewer: thank you, ok do you share his idea or have a different one
S2: yes it is not possible why, that students depend on only teacher they are thinking the teacher is
helping me always they say but it is important for the students to practice work hard more
writing more speaking and more vocabulary but for me it is not important for students to give
mark for themselves and evaluate itself
Interviewer: do you mean that you do not think that students should mark his/her own work?
S2: yes the teacher knows how to give mark. This purpose of learning is to gain knowledge and to
develop my skills and to create new thing. In this case if I evaluate myself this is a problem there
is a teacher the work of the teacher and the students are different even though they have relation
teacher have already qualified is well trained how to evaluate and check the course but the job of
the students is learning getting knowledge and develop their skills and the students they have no
such criteria and it is competition
Interviewer: anyone who shares this idea or have something different
S5: yes it is good to evaluate ourselves i.e., to evaluate how we can answer the question and now the
time is now student centred and it needs self-assessment so that self-assessment is better before
295
sitting on the exam, not after the exam, the exam should be evaluated by another, but before the
exam we can evaluate ourselves how can answer the questions how to prepare for the exam that
is good for the students
S3: yes it is impossible for students to mark their own because the teacher is already qualified and has
profession to correct you. Students their purpose is to learn and ask questions on the subject
matter not marking tests. The purpose of the teacher is to correct you. In case of for example you
write a paragraph or essay in case of thesis, the supporting details, the grammar, vocabulary all
these must be corrected by the teacher the purpose of the teacher is to correct but the students
cannot correct for themselves they cannot mark
Interviewer: why?
S3: because people are selfish they do not do reality they are selfish they put high mark for themselves
in case of exam right now for example if you give exam out of ten you can differentiate who put
high mark or ten out of ten for themselves if we write right now and after we write we put mark
for our own I cannot put for myself because by applying my own knowledge and my position I
put ten out of ten I cannot put seven or five or six out of ten because I need mark and not the
ability or that I can do. I put more than that. Therefore in that process they cannot mark their
own exam the teacher is responsible to mark.
Interviewer: thank you. Ok you.
S1: when I am student i need high mark. There is self evaluation when we do something for example
speak or write I evaluate my grammar or vocabulary but it is impossible to mark for students
እዚሀ ላይ አንድ ነገር አለ ራስ በራስ መፈተን ሳይሆን ሰዉ ስፈትንህ ነዉ አቋምህን የምታዉቀዉ እንጅ ራስህን አታዉቀዉም
አሁነ እኔ ዊኪ ሳይድ እና ስትሮንግ ሳይድ አለኝ እኔ በራሴ ግን ዊኪ ሳይድ ትቼ ስትሮንግ ሳይድ ይዤ መሄድ ስለምፈልግ ያንን
ሰልፊሽነቴን ይዤ በሰዉ ላይ ሱፒርየር መሆን እፈልጋለዉ እንጅ ያንን አቁሜ እዚህ ላይ ችግር አለብኝ አልልም ግን
አስተማርዉ ከሆነ አዎ እዚህ አከባቢ ዊኪ ሰለሆነ ስትሮንግ አርገዉ ይላል always study work hard an doing
everything. That is teacher evaluation is important
Interviewer: thank you. You have an idea?
S7: yea, on this, most of the time most of the students want to get better results so that as I think we
want to build on the result of the record to put up because አሁን እዚህ ላይ ማለት ነዉ ብዙ ልጆች
በኮንፍደንስ ራሳቸዉን እቫሉኤት ኣያደርጉም ትኩረታችዉ ዉጤት ላይ ነዉ። ዉጤት ማግኘት ላይ ነዉ ። ይህ ነገር to
develop to improve my own their own development and skill that is better but because of many
students want to get good results on the exam or group works so forth ስለዚህ ማለት ነዉ ግን
ኮንፍደንሱን ወይም እንዴት ያንን ተረድቻለሁ ለማለት ለራሳቸዉ ማርክ ቢሠጡ ጥሩ ነበር ግን ብዙ ተማሪዎች ትኩረታቸዉ
ዉጤት ማግኘት ላይ ብቻ ስለሆነ እኔ በኔ በኩል ማለት ነዉ it is good in one direction and it is problem in
other direction
Interviewer: anyone to add or have an idea? Okay,
S6: the students cannot participate
Interviewer: why?
296
S6: he has not experience to writing, or speaking or other skills or developing criteria, but the teacher
for the classroom take for example the teacher gives pair work, or group work , knows every
participation classroom participation, so the teacher should develop the criteria and mark the
assignments and tests
Interviewer: so, you mean that it is only the teacher who should develop the criteria but students cannot
participate in doing this?
Ss: yes
Interviewer: Good; look let me ask you one more question may be my last question. Do you think that
participating in the process of assessing your performance that means deciding the criteria and marking
of your own performance has any benefit I mean advantage or disadvantage? For example, you are
given a speaking assignment and you are asked to decide on the points to be given for the different
performances, and then to evaluate or give marks using the criteria you agreed on. Does this have any
advantage or disadvantage?
S5: on this way it has importance because since we are campus students may be after some years we
graduate from this campus and we become teachers, so we must able on giving mark to our
students and such like, for example, if we get high school so the potentiality of grade nine to
twelve students is very high so if I not evaluate within campus myself እዚሀ ላይ ማለት ነዉ እዚህ
ዉስጥ ሆኜ ራሴን መገምገም የማልችል ከሆነ ተማሪዎቼን እቫሉኤት ለማድረግ እዚህ ዉስጥ ልምድ ካላገኜ ማለት ነዉ ወጥቼ
ልጆችን ማስተማር ላይ ወይ የሆነ ኮሌጅ እገባለሁ እዚያ ላይ ተማርዎቼን አሰስ ላማድረግ እዚህ ዉስጥ የበለጠ አስተማሪ
እቫሉኤት ቢያደርግ በጣም ጥሩ ነዉ sometimes it is necessary for students to self assess because of if
you went out from this you may became true in every place and every school so እዚህ ዉስጥ ራስን
ለመገመት ራሳችንን እቫሉኤት ለማድረግ እዚህ ዉስት ብሠጥ ጥሩ ነዉ ይህ አጋጣሚ የበለጠ ይሆናል ነዉ የምለዉ ለወደፊት
S2: something to say when we are given the criteria to mark for our own we can focus on and then we
can mark for our own we can record or we can evaluate ourselves with whatever we are dealing
with for example we are to write and then after writing I am given the criteria to mark for my
own I can evaluate for my own I can do it but in case of competition and I repeatedly say people
are not justice in case of competition students do not put reality for their mark it is difficult for
students to evaluate themselves in case of exams but they can evaluate themselves intentionally
they know the position where they are they can differentiate their weak side and strong side by
checking the criteria but in case of competition really it is difficult to evaluate your mark for
your own in my opinion.
Interviewer: whatever it is I mean whether the person be biased or not, but do you think the practice by
itself has any advantage or disadvantage in what way?
S3: no there is advantage you can get some knowledge from there by evaluating when we are given the
criteria to check up and avoid in case of difficulties in case of strength where own ability is to
check up I can check my paragraph by following the rule how to write how to check the first
phase the second phase supporting details and conclusion when I read all these I can evaluate
myself so there is no harm it has advantage
297
S1: the purpose of assessment is removing your mistake to remove your weak side rather than
developing if the person is biased for himself he cannot remove his mistake it is not for himself
to store mark by covering his mistake is this case it is not good to evaluate himself may be
disadvantage
Interviewer: thank you. Anyone who want to say anything? Okay, thank you very much once again for
giving me you time to answer my questions and helping me to gather data for my pilot
research. Thank you
298
10 Appendix 12
Students interview Transcript (post intervention non-treatment group)
Interviewer: welcome and thank you for your cooperation to give me this interview. I’m Rufael
Disasa I am student at Addis Ababa University. I’m also a staff of the English Department of this
university. I’m doing a research in the area of assessment. And the purpose of this interview is to
gather data for my research. And I want you to be sure that any information I get from this interview
will be used for this research only; no one is allowed to see it without your permission therefore you
feel free We raise some issues of assessment in general and students’ self assessment in particular.
there is no any wrong or right things or answer to my question I want you just to tell me whatever you
feel your opinion and what you believe about the points we raise. And I hope you understand that your
genuine I mean true answer is very important for the of the truthfulness research result. we can also use
our native language and express our idea if we want to. ማለት ለምሳሌ እኔ ሃሳቤን በእንግልዝኛ ለመግለጽ
ካስቸገረኝ በተመቸኝ ቋንቋ አማርኛ ወይም ኦሮምኛ እጠቀማለሁ ሃሳባችንን ነጻ ሆነን በምመቸን ቋንቋ
መግለጽ እንችላለን ።
Let’s start our discussion with talking about purpose of classroom assessment. what do you think is the
main purpose of classroom assessment or evaluation? I mean why you think a teacher assess your
work? Ok you start.
S1: the purpose of assessment is to know good students and weak students when the teacher teaching
students he evaluates which are working hard who is weak in his work and his study in the
classroom and the teacher give grade give continuous assessment to give final grad out of 100 to
pass from one semester to another this is the main reason
Interviewer: thank you, any additional idea? Yes,
S2: the teacher check how much the students understood or not understood what he taught to check
which student is strong and which student is weak. He can see the successfulness of the learning
process by continuous assessment
Interviewer: thank you, okay you have something to add?
S3: ok something there the teacher is to assessing the students to know the students how students
come to the class how to students work hard to differentiate which one is study hard which one
is lazy which one is by depend on other etc are differentiated the teacher to assess this and
evaluate students and give the grade of the course
Interviewer: thank you, ok you.
S4 that taking assessment is important for the teacher he can identify which students have high
capacity and which students have low capacity and which students are medium by giving
continuous assessment he can identify
299
Interviewer: thank you; the next question is Who do you think should do the assessment? who is
responsible for doing assessment?
S2: the instructor must give the assessment because he teaches the course he knows the difficult and
the students So the teacher should do assessment
S1: the department or the teacher can do assessment but it must be prepared by instructor he must give
grade at the end of the semester so he should give continuous assessment for grading for
example the department do not know which topics the students learned and the strength and
ability of the students but the instructor knows
Interviewer: thank you. Let me go to my next question. What is self assessment? How do you describe
it?
S5: ok, self-assessment is the process of assessing the strong and weak side of myself. self assessment
is good for all assessment and important because for example before you take a test or exam
you evaluate yourself whether you studied and understand what you read that is good as I think
Interviewer: thank you, anyone who has anything to add? Okay,
S3: self assessment is very essential to students the students in my opinion, students always must do
self assessment. after the test or exam they must think how the exam was difficult or easy which
question to developing their ability to do anything to self confidence it is very important to
developing and their self confidence from time to time everything is not expected from the
teacher it is mostly expected from the student
S4 self-assessment is when you compare yourself to your classmate what is your ability low high or
medium you check the result of your test with the your friends this is must because you work
hard to get good mark for the next time
Interviewer: anything to add or any different idea? Ok, the next question is related to this now you said
that the teacher should do the assessing because it is his job. But do you think that students can
participate in the assessment process? I mean, for example the teacher gives an assignment, a
test or exam and then give mark for your work. The teacher also sets the criteria against which
your work is marked. I mean, when the assessment is not objective type like multiple choice,
true/false or matching item, when it subjective type, for example, when assessing speaking or
writing a paragraph the teacher decides on the criteria. The criteria may be the content, the
grammar, vocabulary, etc. It is also the teacher who marks your work using the criteria. Do you
think that students should be involved or participate in the assessment process? Is it appropriate
to do so? If yes how? I mean in what ways? If not, why? What is your reason? ግልፅ ለማድረግ እኛ
አንደምናዉቀዉ ምንድን ነዉ ፈተናዉን የሚያወጣ እና ፈተናዉ የምታረምበትን ክራተርያ ወይም መሥፈርት የምያወጣ
መምህሩ ነዉ ክራተርያ ወይም መሥፈርት ስባል ለምሳሌ ፓራግራፍ ስታረም ይህ ፓራግራፍ ይህንን ይህንን እና ይህንን
ነጥቦች ካሟላ ይሀን ያህል ማርክ ይህንን ይህንን እና ይህንን ነጥቦች ካሟላ ግን ይህንን ይህንን ካላሟላ ይሀን ያህል ማርክ
ብሎ መወሰን ማለት ነዉ እና የኔ ጥያቄ ተማርዉ በዚህ ሂደት በዚህ ዉሳኔ መሳተፍ ይችላል ወይ? መሳተፉስ አግባብ ነዉ
ወይ?
300
S1: yes in my opinion I say students can assess the teacher assess me that is must but I myself I can
assess myself in which way for example if I am not telling to someone intentionally I know that
I’m I am understanding well or not where I’m the position when look at my mark result of my
tests, I can evaluate myself when I’m mistaken myself the part of exam which is difficult for
me to understand
S3: we ourselves can determine our capacity or our knowledge depending on how we are reading or
answering that question and where we are. This is evaluating ourselves. Not only the teacher can we
evaluate ourselves. You evaluate yourself by looking at the mark you get.
Interviewer: ok, thank you. Let me clarify my idea. What you are saying is what you do informally by
yourself isn’t it? አሁን ያልነዉ ነገር ወይም አሰራር ተማሪዉ በየግሉ የሚያደርገዉን ነዉ ነገር ግን አሁን እንደልኩኝ ክራተሪያ
በማዉጣቱና በማረሙ ላይ ተማሪዉ ይሳተፋል ወይ መሳተፍ ይችላል ወይ? Do you think that this is possible and it is
appropriate?
S3: as I think it is not right because the student cannot do as the teacher because he has professional
of doing this in teaching learning process he knows the way but students they may give mark for
the questions is wrong they want grades in this case not appropriate for students to assess
themselves in this way
Interviewer: thank you, ok similar idea or any different one?
S4: this way is not possible why, that students must learn from teacher the teacher is teaching and
giving assessment but it is important for the students to study more to develop ability writing
more speaking and more vocabulary but for me it is not important for students to give mark for
themselves and evaluate itself
Interviewer: do you mean that you do not think that students should mark his/her own work?
S2: yes in this case it is not appropriate if I evaluate myself I give the top mark for myself because it is
competition and students are selfish we don’t use the criteria but give high mark.
Interviewer: anyone who shares this idea or have something different
Interviewer: thank you. Ok you.
S1: we assess ourself always there is self evaluation when we do something for example speak or write
I evaluate my grammar or vocabulary but it is impossible to mark for students ምንድን ነዉ እዚህ ላይ
አንድ ሰዉ ለራሱ ዝቅተኛ ማርክ አይሰጥም ስለዚህ ሰዉ ስፈትንህ ነዉ እንጅትክክለኛ ዉጤትህን የምታዉቀዉ ራስህን
አታዉቀዉም አሁን ለምሳሌ እኔ ደካማ እና ጠንካራ ሳይድ አለኝ እኔ በራሴ ግን ደካማ ጎኔን ማሳየት አልፈልግም ስትሮንግ
ሳይድ ይዤ መሄድ ስለምፈልግ እዚህ ላይ ችግር አለብኝ አልልም ግን አስተማርዉ ከሆነ ትክክለኛዉን ነገር ስለምነግረኝ
እሱን አይቼ ለማስተካካል እሞክራለሁ ማለት ነዉ
Interviewer: thank you. You have an idea?
S5: አሁን ማለት ነዉ ብዙ ልጆች ራሳቸዉን በትክክል አሰስ ማድረግ አይፈልጉም ዉጤት ብቻ ይፈልጋሉ ። ዉጤት ማግኘት ላይ
ነዉ ። ስለዚህ ማለት ነዉ ብዙ ተማሪዎች ትኩረታቸዉ ዉጤት ማግኘት ላይ ብቻ ስለሆነ እኔ ማለት ነዉ ተማሪዎች
ለራሳቸዉ ማርክ መሰጠታቸዉ አሰቸጋሪ ነገር ነዉ ብዬ አስባለሁ
301
Interviewer: ተጨማሪ ወይም የተለየ ሃሳብ ካለ
Interviewer: Good; one more question may be my last question. Do you think that participating in the
process of assessing your performance that means deciding the criteria and marking of your own
performance has any benefit I mean advantage or disadvantage? For example, if you are given a
speaking assignment and you are asked to decide on the points to be given for the different
performances, and then to evaluate or give marks using the criteria you agreed on. Does this have any
advantage or disadvantage?
S2: in my opinion it is has importance when we are given the criteria to mark for our own we can use
the criteria when we study or doing assignments to understand whether our answers are correct
or not and then we can mark for our own we can record or we can evaluate ourselves I can
evaluate for my own I can do it but in case of competition and I say people are biased in case of
competition students do not put true mark for their mark it is difficult for students to evaluate
themselves in case of exams but they can evaluate themselves intentionally they know their
weak side and strong side by checking the criteria but in case of competition it is not appropriate
to give mark for your own.
Interviewer: whether the person be biased or not, ተማሪዉ ትክክለኛ ማርክ ባይሰጥም በእንደዚህ አይነት ራሱን አሰስ
ማድረጉ በራሱ ጥቅም ወይም ጉዳት አለዉ ወይ do you think the practice by itself has any advantage or
disadvantage in what way?
S3: it has advantage we learn knowledge from there by evaluating when we do the criteria and check
where our ability can check my ability of writing paragraph or speaking topics by using the
criteria how to write how to check organisation supporting details and conclusion when speak
my topic information relationship of my ideas etc.
S1: but in my opinion if we don’t remove your mistake than developing if the person is putting wrong
mark he cannot remove his mistake by forgetting his mistake is this case it is not advantage to
evaluate himself it is disadvantage
S5: I think it has importance because may be we graduate from this university and become teachers, so
we must able to assess our students and such like, for example, if I not learn evaluating myself
in this university I cannot እዚሀ ላይ ማለት ነዉ እዚህ ዉስጥ ሆኜ ራሴን እንዴት አሰስ ማድረግ ካልተማርኩ
ተማሪዎችን እቫሉኤት ለማድረግ አልችልም ስለዚህ ወጥቼ ልጆችን ማስተማር አሰስ ላማድረግ ጥሩ ነዉ እዚህ ዉስጥ ግን
አስተማሪ እቫሉኤት ቢያደርግ በጣም ጥሩ ነዉ
Interviewer: thank you. Anyone who want to say anything? Okay, thank you very much once again for
giving me you time to answer my questions. Thank you
11 Declaration
I, the undersigned, declare that this thesis is original product of my work and has not been
presented at any university in the same or different form to obtain any degree other than that
for which I am now a candidate, and that all sources used for the thesis have been duly
acknowledged.
Sign_________________________
Rufael Disasa Worabu
June 2013
Top Related