Alternative Assessment Tools in ELT

F. Ilke Buyukduman Istanbul Sehir University

ilkebuyukduman@sehir.edu.tr

AbstractConventional assessment methods usually fall behindassessing students’ communicative skills as well as theirinvolvement in their own evaluation. In order to promotestudent engagement in the assessment and evaluationprocess, alternative assessment tools play a key role.Such tools are more student centered; they emphasizeintegrated skills and the process of learning. The aim ofthis paper is to review and present some alternativeassessment tools being utilized at Istanbul SehirUniversity English Preparatory Program. The outcomes ofthe study reveal that students perceive the alternativeassessment tools as useful. However, further follow upstudies are needed to measure the predictive value ofthese tools, such as how students perform in theirfaculties displaying the skills taught and assessed inthe Prep Program.

Key words: Assessment, alternative assessment, Englishlanguage testing, constructive assessment tools, ESL/EFL.

1

1. IntroductionAssessment is the systematic process of collecting,analyzing, and acting upon data related to studentlearning (Huba & Freed, 2000). The aims of assessmentcould be summarized as follows:

To determine whether the intended learning outcomesof the course have been achieved,

To provide feedback to students, teachers,administrators, curriculum designers and parents onstudents’ learning,

To enable and encourage students to analyze theirperformance according to the outcomes of theassessment,

To inform all stakeholders on student attainment,and to take necessary actions accordingly.

Evaluation, on the other hand refers to “the process ofmonitoring progress during and after instruction” (Nelson& Price, 2007, p.65). Therefore, questions regarding assessment, evaluation,and instruction could be viewed as in chart 1.

Chart 1. Questions Regarding Assessment, Evaluation andInstruction (McIntyre, M., et.al., 2010)

Traditional assessment in English language teachinginvolves tests of all kinds (multiple choice, gap fill,cloze tests, matching, true/ false/ no information,arranging paragraphs/ sentences/ words, transformation,etc.) essay writing and oral assessment (question-answeror interview with the examinee, creating a dialogue on agiven situation, debating, etc.) Such tests are usually

2

not interactive and not based on real life experiences.To illustrate, students are almost never asked totransform an active sentence into passive form in reallife, but in paper and pen tests they are. Tests andexams are direct evidence of students’ academicachievement and thus they are believed to be moreobjective, valid and reliable. In fact, these are thebasic fundamentals of any assessment. Any assessmentmethod should be reliable, valid and relatively easy tooperate (practical and functional). Although traditionalpaper and pen tests can cover a lot of materialreasonably well, are fair, effective in assessingdeclarative knowledge of content and easier to constructand administer, they have some weaknesses. Some of theseweaknesses are that they require a lot of forethought;they are less effective in assessing procedural knowledgeand creative thinking. Besides, construction of goodhigher-level recognition items in traditional tests israther difficult.

Many language teachers would agree that the ultimate goalof language teaching is to enable students to use thetarget language in meaningful, real life situations. Ifthe aim is to develop communicative language skills, thenstudents’ performance using those skills should betested. As Weir (1990) states “a communicative approachto language teaching is more likely to be adopted whenthe test at the end of a course of instruction is itselfcommunicative. A test can be a powerful instrument foreffective change in the language curriculum”. Therefore,the content of tests has become broader with moreemphasis on sub-skills involved in listening, speaking,reading and writing. In the communicative era, tests tendto focus on learners’ ability to extract meaning and toconvey messages in both written and oral forms. Tasktypes become more life-like, and grading rubrics focus onaspects of communicative language use as opposed to thetraditional criteria concerned with the accurate commandof language (Hedge, 2000). When alternative assessment tools are used, the focus ison assessing the process of learning, higher-orderthinking skills, knowledge construction and ability toapply the knowledge in flexible contexts rather than an

3

assessment of task completion and factual knowledgethrough standardized tests (Jonassen, 1991). Alternativeassessment could include but may not be limited toperformance assessment, authentic assessment andportfolio assessment. Performance assessment is eitherproject-based or problem-based. Students demonstratetheir knowledge and/ or comprehension throughperformance. Such assessment includes projects,interviews, presentations, constructed response questionsthat require critical thinking as opposed to prescribed,predictable responses, demonstrations, etc. Authenticassessment calls for application of concepts to real lifeexpectations. According to Wiggins (1989) in authenticassessment the criteria for meeting the course objectivesare given greater attention than the criteria applied tothe traditional assessment approach. The criticaldifference between performance and authentic assessmentis students’ self-assessment. Students are expected topresent and defend their work to demonstrate mastery.Portfolio assessment, on the other hand, is thecollection of many iterations of a paper or a projectthat show student development and growth. These areprocess-oriented portfolios. Product oriented portfoliosfocus on the best work of the student in a particularlearning unit or discipline or concept.

In a university where the medium of instruction isEnglish, the goal of the English preparatory program isto enable students to follow their departmental lecturesin the target language and take notes, to have them writeresearch assignments and answer exam questions, to takepart in class discussions, to deliver presentations on agiven topic, to read lengthy texts and derive meaning,and so on. Therefore, the instruction along with theassessment has to be in line with the expectations ofwhat students are expected to do in their departments.That is the reason why in Turkey the English preparatoryprograms of all universities where English is the mediumof instruction strive to achieve this aim. The comparisonbetween traditional assessment and alternative assessmentcould be summarized as in table 1.

Table 1. Traditional assessment versus alternativeassessment

4

Traditional Assessment Alternative AssessmentOne-shot, standardizedexams

Continuous long termassessment

Timed, multiple choiceformat

Untimed, free responseformat

Decontextualized testitems

Contextualized communicativetasks

Scores suffice forfeedback

Individualized feedback andwashback

Norm referenced scores Criterion-referenced scoresFocus on right answer Open-ended, creative answersSummative FormativeOriented to product Oriented to processNon-interactiveperformance

Interactive performance

Fosters extrinsicmotivation

Fosters intrinsicmotivation.

Istanbul Sehir University, English Preparatory Program isone of those programs whose aim is to raise the Englishcompetency of high school graduates to a level where theycan handle coursework in their faculties to be held inEnglish. The instructional program lends itself to thisend. However, it is usually a challenge to adoptalternative assessment tools instead of traditional paperpen tests because the Turkish educational system ismainly based on exams. At the completion of secondaryschool, students wishing to be placed in better highschools have to take a national exam and at thecompletion of high school there is another national examto be placed in universities. Having high stakes exams onmind, students have to go through a paradigm shift whenthey start the English preparatory programs ofuniversities, especially when the assessment system isdifferent from what they are accustomed to throughouttheir prior educational practices.

2. The StudyAt Istanbul Sehir University, English PreparatoryProgram, the program goals are assessed through analternative assessment approach. The tools utilized for

5

this purpose are video projects, oral presentations,group discussions, process writing and vocabulary logs.

The program is comprised of five levels: Elementary, Pre-Intermediate, Intermediate, Upper-Intermediate and Pre-Faculty. It is a modular program, each module lastingseven weeks (approximately 35 days). Video projects areonly for lower levels, namely for elementary and pre-intermediate students. This assessment requires studentsto record themselves speaking on a given topic orperforming a task. There is a set rubric to grade thetask. Process writing is writing two drafts on a promptprovided. The first draft is corrected using errorcorrection codes and students receive detailed oralfeedback during tutorials. Then, they write the seconddraft. Both drafts are graded using a rubric. Oralpresentation starts in the pre-intermediate level andcontinues until the last level, pre-faculty. The natureand the requirements of the task vary depending on thelevel. However, the rationale stays the same: havingstudents deliver a presentation in a logical order havingdone research and collected data and using visuals(usually PowerPoint). The duration of the presentationgradually increases according to the level starting with5 minutes in pre-intermediate, ending with 15-20 minutesin pre-faculty levels. Group discussion also starts as ofpre-intermediate and ends in upper-intermediate. Itrequires the students to discuss in groups of 3-4 on agiven topic. The topics vary according to the objectivesof the program for each level. For instance, in pre-intermediate level, students are expected to read gradedreaders and discuss the plot and characters, whereas inupper intermediate level, they are expected to readacademic journal articles provided for them and discussthe ideas in them. Finally, the vocabulary logs are inall levels and the expectation is to have the studentskeep track of their own vocabulary building by recordingat least 15 words per week in a meaningful manner, thatis by writing the synonym, definition, collocation andforming a sentence of their own. All these assessmenttools have a separate rubric, which is shared with thestudents both electronically and physically. Theycomprise of 60 percent of their overall assessment.

6

For this particular study, the perception of studentstowards these assessment tools has been analyzed.

2.1 MethodThe method used in this study is quantitative. Theresponses were gathered from 474 participants thatstarted the English preparatory program from one of thefour levels. For this particular study, the correlationbetween student perceptions and the level they havestarted from has not been analyzed. Rather, the datagathered was evaluated holistically regardless of thelevel based expectations and perceptions.

2.2 Research QuestionsThe research questions of the study are:1. To what extent are the alternative assessment toolsused to evaluate the speaking skills of studentsperceived as useful?- Video projects- Oral presentations- Group discussions2. To what extent are the alternative assessment toolsused to evaluate the writing skills of studentsperceived as useful?- Process writing3. To what extent are the alternative assessment toolsused to evaluate the vocabulary building of studentsperceived as useful?- Vocabulary logs 2.3 Participants of the StudyThe participants of the study are the students ofIstanbul Sehir University School of Languages EnglishPreparatory Program in 2012-2013 academic year. Thestudents were grouped according to the levels theystarted the program, namely elementary (ELE), pre-intermediate (PIN), intermediate (INT), and upper-intermediate (UPP) levels. The reason for this divisionwas due to the different alternative assessment toolsutilized for each level. The summary table regarding the participants is asfollows:

Table 2. Participants of the Study

7

ELE PIN INT UPP TotalN % N % N % N % N %177 37.3

5167

35.23

75 15.82

55 11.6

474

100

2.4 Data AnalysisThe questionnaire used in this study consists of threeparts. The first part is designed to collect data aboutthe alternative assessment tools used so as to evaluatetheir speaking skills as perceived by the participants.The second part is the writing skills and the last partis regarding their vocabulary building. The data weregathered using a 5-point Likert scale from Very Useful(5) to Not Useful At All (1). This part consisted offive items: video project, oral presentation, groupdiscussion, process writing and vocabulary logs.Underneath each part, participants were asked to writetheir comments related with the relevant assessmentcomponent in a box provided for them.

Data were analyzed using Microsoft Excel program. Thefrequencies and percentages of student responses weretaken and used for data analysis.

3. Findings3.1 Alternative Assessment of SpeakingThe data regarding the students’ perceptions on theusefulness of video projects as a tool for alternativeassessment is given in table 3.

Table 3. Perceived usefulness of video projectsf %

Totally Useless 47 12.11Useless 43 11.08No idea 125 32.22Useful 111 28.61Very Useful 62 15.98

8

TOTAL 388 100

About the perceived usefulness of video projects, thehighest percentage belongs to ‘no idea’ (32.22%). Thereason for this could be that video projects are onlyutilized in elementary and pre-intermediate levels.Because the participants of the study are from alllevels, intermediate and above starters obviously do nothave any idea. Yet, when we add up the percentages of‘useful’ and ‘very useful’ (44.59), it could be concludedthat nearly half of the entire group believes that theyare useful. The ones who believe they are ‘totallyuseless’ and ‘useless’ add up to 23.19%. Further analysisis needed to understand why they think video projects areuseless. A qualitative research only on the ones thathave been exposed to the video projects could give a morethorough result on this assessment tool.

As regards to the perceived usefulness of oral presentations, the frequencies and percentages of studentresponses are presented in table 4.

Table 4. Perceived usefulness of oral presentationsf %

Totally Useless 48 10.21Useless 44 9.36No idea 44 9.36Useful 203 43.19Very Useful 131 27.87TOTAL 470 100

Table 4 clearly shows that students regard oralpresentations as a useful assessment tool with 27.87%believing they are ‘very useful’ and 43.19% ‘useful’. Asoral presentations are used in all levels except forelementary, students have a good idea what they are andthey can assess their value better compared to videoprojects. The ones that have no idea (9.36%) are probablythe ones that cannot really decide whether they areuseful or not. For this assessment tool, studentsdistinctively have positive ideas as the percentage of

9

those who think they are ‘totally useless’ and ‘useless’add up to 19.57%.

In table 5, the perceived usefulness of group discussionscan be seen.

Table 5. Perceived usefulness of group discussionsf %

Totally Useless 33 6.98Useless 48 10.15No idea 53 11.21Useful 231 48.84Very Useful 108 22.83TOTAL 473 100

Student responses on the usefulness of group discussionsas an assessment tool are definitely on the positive sidewith a total of 61.67% students that think they areeither ‘useful’ or ‘very useful’. The ones that believegroup discussions are ‘totally useless’ and ‘useless’ addup to 17.03%, the lowest among the three assessment toolsused for speaking. The reason for this difference couldbe because students are aware of the fact that thisassessment requires them to use skills in an integratedway and it has the highest predictive value for theirfaculty courses. Interacting with their peers from thefaculty, they must have heard that discussions take placequite frequently. Thus, the majority believes that thistool teaches them towards the most needed skill for theirfuture studies.

Students in general believe that the alternativeassessment tools utilized are useful. The most useful

10

according to them is the group discussions, oralpresentations follow it and the least useful is regardedas video projects. However, in the process of cultivatingtheir speaking skills, video projects play an importantrole, but they cannot be expected to evaluate it.Probably they think it is not a realistic, down-to-earthassessment method. Whereas for oral presentations andgroup discussions they know they will be expected toexecute similar tasks in their departmental studies. 3.2 Alternative Assessment of WritingThe perception of students on process writing as an alternative assessment tool is in table 6.

Table 6. Perceived usefulness of process writingf %

Totally Useless 17 3.62Useless 26 5.54No idea 49 10.45Useful 208 44.35Very Useful 169 36.03TOTAL 469 100

A thorough look at table 6 tells us that a good majority of the students believe that process writing is a useful component of the alternative assessment system. When the percentages of ‘very useful’ and ‘useful’ have been added80.38% of responses are on the positive side, while only 9.16% think they are either ‘totally useless’ or ‘useless’. Therefore, it is possible to conclude that process writing is regarded as a useful assessment tool.

3.3 Alternative Assessment of Vocabulary BuildingThe data gathered concerning the perceived usefulness of vocabulary logs have been presented in table 7.

Table 7. Perceived usefulness of vocabulary logsf %

TotallyUseless 100 21.19Useless 83 17.58No idea 60 12.71

11

Useful 142 30.08Very Useful 87 18.43TOTAL 472 100

Regarding the perceived usefulness of vocabulary logs,48.51% of the students think they are either ‘useful’ or‘very useful’. However, the percentage of students thatthink they are ‘useless’ or ‘totally useless’ is notnegligible. It adds up to 38.77%. It could be concludedthat students have ambiguities about this assessmentcomponent. More in-depth analysis is needed to analyzethe uncertainty as regards to vocabulary logs.

4. ReferencesHedge, T. (2000). Teaching and learning in the language classroom.

Oxford: OUP.Huba, M. E. & Freed, J. E., (2000) Learner-centered assessment

on college campuses: Shifting the focus from teaching to learning.Boston: Allyn and Bacon.

Jonassen, D. (1991). Evaluating constructivist learning.Educational Technology, 31 (9), 28-32.

McIntyre, M., Degraffenreid, D., Nishball, L. & Vilders, B. (2010). Assessment and evaluation of L2 learning. TESOL 410. Retrived from: h ttp://www.wce.wwu.edu/Resources/CIRCLE/Presentations/ assessevaln.pdf [March, 27th, 2013].

Nelson, K.L. & Price, K.M. (2007). Planning effective instruction.Blmont, CA: Thomson Wadsworth.

Weir, C.R (1990). Communicative language testing. New York:Prentice Hall.

Wiggins, G. (1989) "A True Test: Toward More Authenticand Equitable Assessment," Phi Delta Kappan, 70, 9 (May).

12