How Do Faculty Make Formative Use of StudentEvaluation Feedback?: A Multiple Case Study

Yuankun Yao & Marilyn L. Grady

Published online: 31 May 2006# Springer Science + Business Media, LLC 2006

Abstract This multiple case study explored faculty formative use of studentevaluation feedback. Results were based on interviews of ten education facultymembers at a midwestern research university and reviews of relevant documents. Itwas found that end of semester student evaluation, originally used for summativeevaluation, was also used by faculty for teaching improvement purposes. Facultycared for student evaluation feedback and used discretion in using the feedback toimprove their teaching. Faculty also experienced anxiety and tensions in the contextof a mandated summative evaluation. Variations were found in faculty use ofevaluation feedback, as a result of difference in background and experience.

Keywords Student evaluation of teaching .

Formative use of evaluation feedback . Summative evaluation

Student evaluation of teaching is widely used in higher education. Research hasgenerally identified student evaluation as a reliable measure of teaching. Evidence isless conclusive about the validity of student evaluation (Algozzine et al., 2004;Marsh, 1987; Olivares, 2002; Shevlin et al., 2000). Student evaluation is alsoconsidered biased (Cramer & Alexitch, 2000). Research has identified a correlationbetween student ratings and grade inflation (Eiszler, 2002; Sojka et al., 2002).Wolfer & Johnson (2003) found student ratings on a standardized teachingevaluation instrument to be severely negatively skewed and truncated. Problemswith student evaluation have prompted many to search for alternative measures ofteaching effectiveness, such as peer evaluation, small group interviews, and portfolioevaluation (Fenwick, 2001; Saroyan & Amundsen, 2001). Such alternative measures,however, have problems of their own, such as the low level of reliability during datacollection, and the large amount of time needed to implement the evaluation.

A purported use of student evaluation is for faculty members to improve theirteaching, although personnel decision-making is more often the reason behind

J Pers Eval Educ (2005) 18: 107–126DOI 10.1007/s11092-006-9000-9

Y. Yao (*)3300 Lovinger, Central Missouri State University, Warrensburg, MO 64093, USAe-mail: yyao@cmsu1.cmsu.edu

M. L. Grady128 Teachers College Hall, University of Nebraska-Lincoln, Lincoln, NE 68588, USAe-mail: mgrady@unlserve.unl.edu

student evaluation (Marincovich, 1998). In a longitudinal study of faculty ratingsover a period of 21 years, Murray et al. (1996) found that student evaluationresulted in improved ratings. A more recent study by Kember et al. (2002), however,found teaching improvement in only four out of 25 departments over a period of 3to 4 years. According to Marsh & Roche (1993), faculty members provided withstudent evaluation feedback and consultations received significantly higher ratingswithin a year than faculty members who did not receive any feedback orconsultations. The study also found more improvement in ratings for targeted areas,the teaching dimensions that initially received relatively low ratings by students butwere perceived by the faculty participants as critical to their success of teaching.

There are many factors that seem to have an impact on faculty use of studentevaluation feedback to improve their teaching. According to Centra (1993),teaching improvement occurs only if a faculty member knows how to make changesand is motivated to do so. Even if faculty members know how to interpret studentevaluation results, they may not know what to do in order to improve their teaching(Jacobs, 1987). They regard the quality of student evaluation and institutionalsupport for faculty professional development as essential for them to successfullyimprove their teaching (Frost & Teodorescu, 2001). The summative use of studentevaluation, if mandated by an institution, is likely to generate Bfear, damagedrelationships and self-doubt, ... (and) technicist concepts of teaching’’ (Johnson,2000, p. 433). Student evaluation feedback is also perceived to be useful only incertain dimensions of teaching. According to a survey by Schmelkin et al. (1997),student feedback on interaction with students was considered by faculty members asmost useful, followed by feedback on grading practices, global ratings of instructorand course, and course structure. Spencer & Flyr (1992) found that faculty memberswere more likely to use student feedback to make changes regarding handoutmaterials, number of assignments, and presentation habits such as pacing and verbalpauses. Contrary to the Schmelkin et al. study, Spencer and Flyr found facultymembers were least likely to use student feedback to make changes in their gradingpractices. Nor did they use the feedback to make much change in their basicphilosophy of education.

Evaluation data, if properly produced, should be used for both summative andformative purposes. In a large measure, formative use of the evaluation data shouldbe the ultimate purpose, since it will directly lead to the improvement of a programor the people in the program. Student evaluation of teaching in higher education isno exception. The formative use of student evaluation by faculty members toimprove their teaching is imperative given that most faculty members have neverreceived any formal training in pedagogy when they started their teaching career.

Studies on faculty use of student evaluation for teaching improvement so far havelargely been based on quasi experiments and self-reported surveys. Most quasi-experimental studies examine whether faculty members are able to improve theirratings after a period of intervention with student evaluation feedback andconsultations on how to use the data. It is not clear from such studies, however,whether or how faculty members actually make an effort to use student evaluationto improve their teaching. The self-reported surveys provide one way to examinehow faculty members respond to student evaluation feedback or perceive theusefulness of the feedback based on a pre-determined set of behaviors orperceptions. It is often not clear from the survey studies, however, why facultymembers respond to and make use of student evaluation feedback in certain ways.

108 J Pers Eval Educ (2005) 18: 107–126

In addition, a survey may not capture the complex pictures involved in faculty use ofstudent evaluation to improve their teaching. In particular, the potentially importantcontextual variables tend to be left out in a survey study.

The present study used the multiple case study approach to interview ten educa-tion faculty members in an attempt to recreate their experiences of responding toand making use of the feedback to improve their teaching. Special attention waspaid to the context of a summative evaluation environment and other relatedvariables that may have important implications for faculty formative use of studentevaluation feedback.

Materials and Methods

To investigate faculty formative use of student evaluation feedback, a multiple casestudy approach was adopted for the study. The goal of this type of qualitativeresearch is to arrive at an understanding of Bthe complex interrelationships amongall that exists’’ (Stake, 1995, p. 37). The qualitative researcher takes Ban ongoinginterpretive role’’ (p. 43) in an attempt to explore Bunanticipated as well as expectedrelationships’’ (p. 41). A case study is Ban exploration of a Fbounded system_ or acase over time through detailed, in-depth data collection involving multiple sourcesof information rich in context’’ (Creswell, 1998, p. 61). According to Stake, animportant criterion for selecting cases in a multiple case study is maximum learning,which can be accomplished through the examination of unusual cases. In a multiplecase study, the representativeness of a case is not a high priority.

Context of Study

The study took place in the School of Education at a public research I university in amidwestern state. There were five departments and approximately 130 facultymembers in the School of Education.

As part of the university requirement, the faculty members in the School ofEducation were required to go through an annual evaluation, which Bcommunicates(to a faculty member) areas of progress and strength, and alerts the faculty memberto performance deficiencies at the earliest possible time’’ (Midwest University, 1989,p. 18). Every class in the School of Education was required to have an end-of-semester student evaluation, including classes offered during the summer andthrough distance learning (Midwest University, School of Education, 1997a). Asidefrom annual evaluations, faculty members applying for tenure and promotion wererequired to provide summaries of student ratings and comments for all courses. Inaddition, a post-tenure review process was recently introduced at the university. Theprocess was targeted at those faculty members who had received tenure but haddemonstrated obvious deficiences in their profession (Midwest University, 1998).

Although student evaluations were required for personnel decisions such asannual reviews and tenure and promotion, the School of Education also expected itsfaculty members to collect and use student evaluation feedback to improve theirteaching performance. BThe ability (of a faculty member) to gather and usefeedback to improve performance,’’ according to the Official Papers of the School ofEducation, Bis essential to teaching effectiveness’’ (Midwest University, School ofEducation, 1997b, p. 1). The faculty members were encouraged to work with their

J Pers Eval Educ (2005) 18: 107–126 109

colleagues to support their own professional development. The School of Educationmade several workshops and other professional development opportunities avail-able to its faculty members.

Student Evaluation of Instruction

The individual departments in the School of Education were responsible fordeveloping their own student evaluation forms. All the forms contained Likert scaleclose-ended questions and open-ended questions, although the wording of thequestions and the number of questions varied with the individual departments. Inone instance, there were a total of 37 close-ended questions in a student evaluationform. The questions covered both specific areas of teaching, such as facultypresentation of the content, command of content knowledge, and facilitation ofdiscussion, and general topics such as the overall value of the course and the overalleffectiveness of the faculty member in teaching the course. There were also sevenopen-ended questions for students to comment on the strengths and weaknesses ofthe course, and the potential areas for improvement, what students liked and dislikedabout the course, evaluation of the textbook, and comparison with other courses.

One of the departments used several different forms, which were used toaccommodate the different types of courses that were available in the department.In addition to the required end of semester student evaluation, the education facultymembers were encouraged to use midterm student evaluation. The universityTeaching Center had a midterm evaluation form that was available for faculty touse. The faculty members were also free to develop their own instruments for themidterm student evaluation.

The end of semester student evaluation was administered in the last week ofclass, either before or after the final examination was given. The instructor wouldpick a student to administer the evaluation. The instructor would leave theclassroom when the evaluation was being conducted. Students in a class would begiven a bubble sheet to record their response to each of the close-ended questionsand respond to the open ended questions. The students would place their completedevaluations in a special envelope, which would be collected at the end of theevaluation by the student administering the evaluation and taken to the departmentsecretary. The bubble sheets would be passed on to the Teaching Center forscanning and reporting. The student responses to the open ended questions wouldbe typed by the secretary. The feedback faculty members received from studentevaluation included both numeric ratings for the Likert-scaled questions andresponses to the open-ended questions. The numeric ratings were summarized interms of frequency distribution, mean, median, and variance. The responses toopen-ended questions were typed and summarized in such a way that studentanonymity was ensured.

Interviews

The data for this multiple-case study was mainly collected through interviews. Tenfaculty members from the School of Education were chosen as participants in theinterviews. The School of Education became the site for the study for convenienceconsiderations, because at the time of study the researchers were either studying orworking within the school, which helped with the data collection process.

110 J Pers Eval Educ (2005) 18: 107–126

The criterion for the selection of participants was that participants should comefrom diverse backgrounds, so that maximum information would be obtained from thestudy. Diversity in faculty background was defined as variation in faculty seniority(tenure status, academic rank, and years of teaching), gender, and home department.The principal researcher obtained information regarding faculty names, academicrank, department affiliations, and campus phone numbers through the official Schoolof Education website and the university phonebook. The researchers developed aninitial list of potential participants in the study by including faculty members whodiffered in terms of seniority, gender, and home department, and contacted eachperson on the list by phone. The list included ten faculty members from a total of 130faculty in the School of Education, so that each department had about two facultymembers represented in the study. (Due to the different sizes of the individualdepartments, not every department had two faculty participants in the study. SeeTable 1 for details). Since representativeness was not a major criterion for theselection of participants, there was not necessarily a balance in terms of seniority,gender, and department affiliation among the participants of the study. The totalnumber of participants was limited to ten so that the qualitative data for the studywould be manageable. The associate dean of the School of Education was contactedfor verification of the faculty information on the list. Except for one instructor whodeclined to participate and hence was replaced by another faculty member from thesame department, all other faculty members on the initial list agreed to participate.Table 1 summarizes the background information of the ten participants.

The interviews were conducted in each participant’s office. A consent form waspresented to each participant before the interview started. Permission was sought totape-record each conversation. During each interview, the principal researcher ledthe participants through the interview questions, although most of the time theparticipant was encouraged to expand on topics that were mentioned previously.Occasionally the principal researcher made notes during the interviews. Eachinterview lasted for about 1 h and 30 min on the average. Except for two cases wherethe interviews were separated into two sessions, most interviews were completed inone session

Table 1 Background information of participants

Case Gender Affiliated

department

Tenure status Academic

rank

Approximate number

of years teaching

at current university

A Male One Tenured Associate professor 15

B Male One Tenured Full professor 25

C Female Two Tenure-track Assistant professor Less than 5

D Male Two Tenured Full professor 25

E Male Three Tenured Full professor 15

F Male Four Tenured Associate professor 25

G Female Four Non-tenure track Instructor Less than 5

H Male Four Tenured Associate professor 25

I Male Five Non-tenure track

(part time)

Lecturer Less than 5

J Male Five Non-tenure track Senior lecturer Less than 5

J Pers Eval Educ (2005) 18: 107–126 111

An interview protocol that contained semi-open questions was used during theinterviews. The following questions and follow-up questions were asked during theinterviews:

1. What has been your teaching experience?2. What is your philosophy of teaching? Do you think teaching can be improved?

If so, how do you think it can be improved? How would you characterize yourapproach to teaching and your teaching effectiveness? What has been yourexperience in improving your teaching?

3. What do you think of student evaluation of teaching? What has been yourexperience with student evaluation of teaching?

4. What kind of feedback do you receive after the evaluation is completed? Howuseful do you think the feedback is for your teaching improvement?

5. Are there any requirements or expectations from the department or the Schoolof Education for you to deal with the feedback? If so, what are they? What doyou think of these requirements and expectations?

6. What do you think you can do specifically with the feedback that you receivefrom student evaluation for the purpose of improving your teaching? What doyou actually do with the student evaluation feedback? Why do you respond tothe feedback in that way?

7. What teaching areas do you think are most likely to be affected by faculty use ofstudent evaluation feedback? What specific effects are they? What specificteaching areas in your teaching have been affected by your use of studentevaluation feedback? What areas are you trying to target right now as you usestudent evaluation feedback?

8. What barriers have you encountered in your use of student evaluation feedbackto improve your teaching? What suggestions do you have that may enable youto make more effective use of student evaluation feedback?

In order to encourage free conversations, the questions that were actually askedvaried to some extent from the ones originally prepared. Sometimes a differentfollow-up question would be asked, depending on the response of the interviewparticipant. The resulting in-depth conversations were expected to lead to findingsthat might not have been anticipated.

A coding scheme was established to facilitate the analysis of the individualinterviews, so as to identify patterns in faculty response to and use of studentevaluation feedback. There were three levels of the coding scheme. The first levelwas represented by a word or phrase that summarized a potentially importanttheme, the second level the letter Q followed by a number, indicating the interviewquestion involved, and the third level a randomly selected capitalized letterrepresenting a particular reply to the interview question that provided support forthe theme. For example, if a participant commented that student evaluation shouldbe valued since students constituted the primary constituency of education, thecomment would be coded as BValue Q7A,’’ meaning that the theme value wassupported by A (the concept of constituency) when the participant responded to theseventh interview question.

The recorded interviews were transcribed and read through by the researchersseveral times, on different occasions. The coding scheme was used to identifypatterns of interview responses and important themes. The initial phase of analysis

112 J Pers Eval Educ (2005) 18: 107–126

focused on each individual case. In the second phase of the analysis, the themesfrom the individual cases were compared to see if common themes emerged fromthe different cases. Sometimes in the process of comparing the individual cases theresearchers went back to the individual cases to see further similarities and dif-ferences across the cases. When common themes were initially identified, the re-searchers again went back to the individual cases to see if the themes really appliedto them.

Documents

In order to provide information regarding the background for the study, and toverify the findings from the interviews, the researchers obtained copies of variousdocuments within the School of Education. The researchers were particularlyinterested in any information regarding the institutional policy for faculty evaluationand promotion and tenure process, sample student evaluation instruments, andstudent evaluation feedback form. Such information was expected to give the re-searchers the frame of reference during the interviews and help the researchers tounderstand the responses of the interviewees.

The documents were obtained through the various administration offices withinthe School of Education, and the individual faculty members who participated in theinterviews. The websites for the university and the School of Education alsoprovided some useful information for the study.

Limitations of Study

A limitation of the study was that it only used faculty interviews and documentreviews. Due to the sensitive nature of the topic, some of the participants might notfeel free to reveal their true thoughts. Future research is needed to validate thefindings of the study. Also, the results were based on a multiple case study ofeducation faculty at a mid-western research university. Although the participantswere selected for maximum learning, their experiences or perceptions may not begeneralized to faculty from other disciplines or institutions.

Results

Based on the results of the interviews and the review of relevant documents, eachcase was analyzed by itself and then compared to one another. The following is asummary of each individual case with regard to the faculty member’s response toand use of the student evaluation feedback to improve teaching.

Case A

Dr. A was a tenured associate professor in Department One, with 16 years ofuniversity teaching experience behind him, 14 years of which were spent at thecurrent university. Dr. A was once a recipient of the Distinguished Teaching Awardfrom the School of Education. He felt he was a better teacher than he used to be,which he attributed mainly to experience. He said he had taught the courses oftenenough to know when to say certain things, in what order, and how to say them.

J Pers Eval Educ (2005) 18: 107–126 113

Dr. A said evaluation by people other than his students did not mean much tohim, because one couldn’t understand the culture and the feeling of a class by justcoming in and doing a one-time only video-taping of a teacher. He said studentsknew who was a good teacher and who was not. He had used mid-semesterevaluations and then stopped using them, because they did not tell him anythingdifferent from what he got at the end-of-semester student evaluation. Dr. Aidentified two conditions that he felt should be met in order for student evaluationfeedback to be helpful: first, there was a need for dramatic changes of instructionalapproaches; second, the professor was willing to make the changes. Since heconsistently got good evaluations from students, the feedback only served toreinforce him in his teaching approach.

In general, there were only minor changes Dr. A made on the basis of studentfeedback. Among the things students said about his teaching, he was most interestedin student comments on his availability, his enthusiasm, and his willingness to help.He said he was less interested in feedback about his teaching style, which he feltcame with experience. Neither would he change much about the course planning orassessment used in class. Dr. A said students should not drive the content of acourse, which should be solely determined by the professor. He said instructorsshould look at the basic trend in student feedback over a long period of time, ratherthan an individual comment for a particular time.

Dr. A did not feel there was sufficient accountability in the university. Everybodygot raises, he said. Instructors who got consistently good evaluations were not re-warded as much as they deserved, whereas modestly effective teachers and teacherswho made some improvement in their teaching got all kinds of rewards. There werealso faculty members who did not care about teaching evaluations, even if they didnot do well in their teaching. He felt the administrators were not willing to Btake thestands they need to improve teaching.’’

Case B

Dr. B, a tenured full professor from Department One, had been teaching at theuniversity for 27 years. Dr. B believed that teaching could be improved, becausemost university professors were never taught how to teach. He felt it was a challengefor him to make students learn as much as he wanted them to learn while still havinga good time learning. Dr. B used to reach out to the best students in his class. Hesaid his teaching evaluation used to be Bbimodal,’’ with quite a number of peopleenjoying the class and an equal number of people hating the class. This had led himto pay his attention to students who did not seem to enjoy his class.

In terms of student evaluation, Dr. B said he mainly used the end-of-semesterstudent evaluation to obtain feedback about his teaching. He also asked students inhis undergraduate class to write down their comments about the course on a card inthe middle of a semester. Based on this feedback he would make changes for thesecond half of the semester. Dr. B felt the mid-semester evaluation was more useful tohim, because it provided him with information about the process, especially if thefeedback was confirmed by the rest of the class. Dr. B said he took the end-of-semester student evaluation feedback seriously. He would be concerned if his ratingswere not above the mean. He would look carefully at student comments, to see if whatwas buried in what they have said was some good advice on how to make a class moreeffective. Based on the feedback from students, Dr. B would make some small

114 J Pers Eval Educ (2005) 18: 107–126

modifications in his class. Sometimes the adjustment was made in the amount ofcontent in a course, even though the change would not be substantial. He said he didnot feel comfortable having to change his teaching style, because he felt studentsmight not be able to learn as much as he would like them to, although sometimes hedid make such changes to accommodate their desires. In general, he did not feel hewas able to get much information from the end of semester student evaluation.

Dr. B said sometimes students did not see the relevance of what they werelearning and might give negative evaluations, even if they learned much from theinstructor. In particular, he said the low performing students could easily getfrustrated in class and as a result would give poor evaluations of their professors. Hefelt students nowadays expected more entertainment and interest from theinstructor than they did 30 years before. Dr. B said many of his students complainedabout not being allowed to eat or drink, or wear hats in class. They also complainedabout the difficult tests for the class. Occasionally he accommodated student needs.He said he wanted the students to be happy. In one class he switched fromexaminations to papers, much to the pleasure of his students. Dr. B said there werecertain things he would not change. BJust because they complain about the fact thatI ask them to take the hats off in my class, I am not going to change that.’’ Therewere times he felt the need for Bholding to my principles.’’

Dr. B mentioned the potential for faculty members to get high evaluations bymaking the course easier for the students, or by administering the evaluation a fewdays before the final exam, when most of the students who did not like the classwere usually absent. He also perceived a tension between the responsibility of thestudent and that of the teacher. He said it would be difficult for the teacher tobecome what every student wanted him to be. Students should also take theirresponsibility in the process, such as being willing to ask questions. He wanted to seethe students reminded of their responsibility to learn, and to participate in theteaching and learning process, he said.

Dr. B said sometimes he did not know how to interpret or utilize studentevaluation feedback. He felt the help from the Teaching Center was not adequate.

Case C

Dr. C, a tenure-track assistant professor in Department Two, was in her third yearof teaching at the university. She felt she had more confidence about her teachingnow than 3 years before. She said she used to worry if what she said in class madesense to her students. She said her lectures had Bmore flow’’ than in the beginning.

Dr. C regarded evaluation as an integral component of teaching. BI think it’s howwe become better teachers,’’ she said. She felt a good instructor should pay attentionto student expectations, because students were the primary customers of education.Dr. C said it was important for the teacher to be sensitive to what was going on withthe students.

Dr. C said she adapted the open-ended questions originally designed for the end-of-semester evaluation for her mid-semester evaluations. Most of the times, onlystudents happy with the courses turned in the mid-semester evaluations. For bothtypes of evaluation feedback, she would read every comment, and tried to think if itmade sense. She would make changes if there were good points made by thestudents. She said the information from the numeric feedback and writtencomments was consistent. In terms of what was most likely to be affected by

J Pers Eval Educ (2005) 18: 107–126 115

student evaluation feedback, Dr. C mentioned her lecture style, or the way thematerials were presented, and the amount of time allowed for assignments.Normally she got good evaluations from students. Part of the reason was that allthe courses were required for the students, and the course materials were allrelevant and interesting. The majority of complaints Dr. C got from students wereabout the workload. Students complained about too much reading. Dr. C felt thiskind of assignment was necessary for a graduate course. She also insisted onrequiring her students in one class to turn in a paper. Dr. C felt that requirementwas essential to their learning how to write in their profession. She said she wouldnot change things that are integral to students’ professional development.

When she first started teaching, Dr. C was anxious to see the overall instructorgrade from student evaluation. She said she no longer attached so much importanceto it. Dr. C also had a fairly good sense of how things were going now. She saidfaculty members should have an open mind when reading student evaluations. Theyshould examine their behaviors and the context under which the feedback waswritten, and try to see if this was something that could be changed, and should bechanged. Dr. C also felt it important to look at student evaluations globally.

Case D

Dr. D, a tenured full professor from Department Two, had been teaching at theuniversity for about 25 years. Dr. D believed everybody’s teaching could beimproved. He said he always tried to find ways to help students understand theconcepts better. He was generally satisfied with the results of his teaching, althoughhe said one could always do better. He usually received lower ratings fromundergraduate students than graduate students. He said undergraduate studentstended to find his class hard to understand.

Dr. D said student evaluation was useful in helping him improve his teaching.The end-of-semester student evaluation provided the major means for him to obtainstudent feedback about his teaching. The department where Dr. D was workingused multiple forms of end of semester student evaluations to accommodate theneeds of different courses. Dr. D said the form he used for end of semester eval-uation was originally developed by himself, although the department had toofficially approve it. He would look at student feedback when he was preparing acourse for a new semester. He found the feedback especially useful when he wantedto try something new, such as introducing a new book for a course. He felt thenumeric rating was useful in its own way, for example when comparing the relativeusefulness of two textbooks. The written comments to him were more useful,because they explained why students liked or disliked certain aspects of his teaching.Dr. D said he tried to search for patterns in student evaluations. He would try tofigure out what the problem was if several students were complaining about thesame thing. One should not take every comment from each student as a basis forchange. Some students might have misperceived things, he said.

Dr. D sometimes made changes in the way he presented the content in class. Hesaid he did not want to make changes about what examinations he should use. Insuch areas he would have his own judgment, even though students complainedabout them. He said he tried to strike a balance to satisfy both students’ needs andhis own. He said he was willing to accommodate student needs. At the same time, hewanted his class to be sufficiently challenging, or Bintellectually respectable.’’

116 J Pers Eval Educ (2005) 18: 107–126

Dr. D was once the chair of his department and served on the annual reviewcommittee for a number of years. He said the mean scores could be misleading,because most faculty members in his department got high evaluations. In the pastfew years, his evaluations had become fairly predictable, so that the feedback wasnot so useful. As a result, in recent years he would only look at the feedback to see ifthere were any surprises.

Dr. D said student evaluation provided him with a rare chance to see the diversityof student views. It helped him to find ways to make the course valuable to studentswith different backgrounds. At the same time, he was concerned about studentevaluation being the main source of evidence and heavily weighted in personneldecisions. He said student evaluation could constitute a threat to academic freedom.Occasionally he felt the pressure to ease up a little and lower his standards to makestudents happy. He said the tenure system had given him the courage to stick to hisstandards, even if that would hurt his evaluation results.

Case E

Dr. E was a tenured full professor who had been teaching in Department Three forabout 15 years. He considered himself somewhere in the middle in terms of histeaching effectiveness. He often asked students to take the evaluation home so thatthey would have more time to respond to the questions. He said he got strikinglydifferent evaluations from students. Some of his students were really appreciativeand responsive in his classes, while others were rather impatient or frustrated.

Dr. E felt he cared about student evaluation at a normal level. He said he did notthrow them in the trash. Dr. E would look at the evaluations when he was preparingfor the annual reviews. He appreciated student feedback that was informative, eventhough it might be negative. He felt sometimes the feedback was not specificenough. He did not like comments that were not relevant to teaching, even thoughthey were positive. He liked to emphasize classroom discussion and dialogue. Dr. Esaid he did not care what students had to say about this. Even if 99% of the studentswanted him to lecture, he would not make a change. Dr. E said he would payattention to student feedback concerning whether students understood the class ornot, and find ways to help students understand better. He would also pay attentionto student feedback on the textbook and assignments and make appropriate changesif necessary. He said that over the years student evaluations had become quitepredictable. As a result, he said that he now might pay somewhat less attention tothe feedback than he did initially.

Dr. E said that students were qualified in evaluating their professor to the extentthat they sat through the course every week. Students needed to have the chance toexpress their thoughts of the experience they had. On the other hand, he wasconcerned about those students who had not done their part in trying to understandwhat was taught in class. He felt the lack of maturity and subject knowledge mightlimit students’ ability to evaluate their teacher properly. There were also studentswho were not serious about evaluations, because they thought the evaluations wouldnot make much difference to a professor anyway. Dr. E said student evaluationwould be meaningless if faculty members took student evaluation as a popularitycontest. The purpose of student evaluation was to improve teaching, not a means forgetting at faculty members who graded students down or for getting back at studentswho did not do the assignments.

J Pers Eval Educ (2005) 18: 107–126 117

Case F

Dr. F, a tenured associate professor in Department Four, had been teaching at theuniversity level for about 25 years. He had used a number of things to improve histeaching, one of which was student evaluation. Dr. F said he did not always getpositive evaluations from students. In some semesters he had much work to do andhad to give up on something, he explained. He wished he could ignore studentevaluation feedback when it arrived in his mailbox, because he was not sure what itwas going to be. In reality, he would take it out and read through it quickly. Then hewould put it down for a day or two before looking at each one item more closely.Sometimes he would pull out evaluations for the same course from previous yearsand make comparisons. Dr. F would be particularly interested in student opinionsregarding teaching materials or textbooks that remained unchanged for a number ofyears.

Dr. F said he did not act on every piece of feedback students provided about histeaching. There were requirements that students found Btedious’’ yet he foundappropriate. For instance, he always encouraged students to ask questions andengage in classroom discussions. BThat’s just who I am and that’s how I teach.’’ Hesaid student evaluation mostly had an impact on the way he delivered content to thestudents, whereas how he conducted his lessons would stay much the same. Hewould also rely on student evaluation to learn if there was too much information onthe slides, or if the photocopies used in class were legible. He said studentevaluation feedback became particularly useful when he wanted to introduce newtextbooks in a course.

Dr. F felt the majority of students were not serious about student evaluation ofteaching, although there was a small group of students who really took the task atheart. He said that most students became weary of doing so many evaluationstoward the end of each semester. Sometimes student evaluation of teaching did notreally reflect a teacher’s teaching ability or skill, but more of the dissatisfaction withthe grade that the students got, or just a personality difference, or a disagreementwith some of the requirements in the class.

Dr. F gave a score of three on a 1–5 point scale in terms of the usefulness ofstudent evaluation in general. He said it was useful in providing an opportunity forstudents who were shy and reticent to express their opinion about his teaching. Hewould be concerned if student evaluations were used heavily in such matters as payincreases, promotions, and tenure decisions. Using feedback from students who areyoung adolescents as a sole determinant of a teacher’s effectiveness was, said Dr. F,Bsomewhat risky.’’

Case G

Instructor G was in her third year teaching as a non-tenure track instructor inDepartment Four. Instructor G used mid-semester student evaluation and pre- andpost-instructional surveys to seek additional information about her teaching. Shewas not comfortable with student evaluations in her first year of university teaching.However, she felt the feedback did tell her clearly how she could improve herteaching the next time.

Instructor G said when she first came to the university, she was emotionallyattached to the feedback from students. Now she felt more secure in her teaching

118 J Pers Eval Educ (2005) 18: 107–126

and student evaluation no longer had that personal impact on her, even though itwas still important to her. If she was in a good mood, Instructor G said she wouldtake a look at the feedback upon its arrival. She would not make a snap judgmentbased on the first look. She would take it out a week later and make further analysisof it. She would go back to the feedback several times before she made any newplans. She said she would have a more objective view by the third or fourth time.She liked to use a comparison grid to study areas where she was not particularlystrong. Normally she would ignore both the glowing comments and those terriblynegative comments, and try to look at the consistent patterns. The way shepresented the information was the area that was most affected by student evaluationfeedback. She said the information she presented in class was quite dry, so how itwas presented to the students would make a big difference to the students.

Instructor G felt student evaluation should not be the only measure of aninstructor’s teaching effectiveness. Students should not be the sole governing factorin that evaluation process. She said students sometimes liked things just becausethey were easy, rather than being valuable for their education. She said sometimesadministrators took student evaluation of teaching too seriously.

Case H

Dr. H had been teaching as a tenured associate professor in Department Four forover 20 years. He felt he was much more effective in teaching than he was manyyears ago.

Dr. H said he used mid-semester student evaluation as well as end of semesterevaluation to get feedback from students. He paid particular attention to studentfeedback about assignments, day-to-day aspects of the course, and the way hepresented information in class. Sometimes he made compromises concerning the useof discussion in class, which students preferred, but he found this would limit thetime for him to adequately cover the topics. On one occasion, student feedbackmade him realize that he provided too much information during a limited timeperiod. In general, he valued student feedback about the mechanics of theinstructional process instead of the content, since the latter was often prescribedas part of a larger curriculum.

Compared to the beginning of his career, Dr. H said now he probably paid moreattention to student evaluation feedback, because he felt there was more emphasison teaching recently. However, he did not feel student evaluation was a majorinfluence on his teaching. Neither did he feel it affected his relationship withstudents. BI might go three or four semesters where the input doesn’t affect thingsvery much and then once in a while it does,’’ he said. Dr. H felt sometimes studentsdid not necessarily have in their own experience a big enough picture to interpretthe importance of content. He said student evaluation should not be the only way ofevaluating teaching. He thought it would be helpful to have some outsideperspectives on how to interpret the evaluation.

Case I

Dr. I was a part-time, non-tenure track instructor in Department Five. He used tobe an administrator in a public school system before he came to work at the currentuniversity 2 years ago. Dr. I thought it was important for a college professor to learn

J Pers Eval Educ (2005) 18: 107–126 119

something from students. He said too often faculty members thought they had allthe answers. He felt he had become more effective in his teaching than he used tobe. Dr. I said student evaluation objectively reflected student perception of aninstructor’s teaching. The feedback deserved the instructor’s attention because thestudents were trying to validate or invalidate the teaching process. BYou might feelthat you are accomplishing what you set out to accomplish. But until you hear fromthe individuals that are directly involved, you really don’t know.’’

Dr. I said student evaluation of his teaching had been positive. He tried to beprofessional in his reaction to student evaluation feedback, especially if the feed-back was negative. Dr. I felt his experience as an administrator and consequently anevaluator of other people’s performances had helped him to accept criticism. Hesaid in the beginning he was Ba little more hesitant, scared, and apprehensive’’ aboutwhat students would say about his teaching. He still thought it would be nice ifstudents liked him, but he was now more concerned about whether they werelearning, or whether they were enjoying what they were doing, rather than how theyviewed him personally.

Based on student evaluation feedback, Dr. I had adopted the seminar approachin his teaching. He found the seminar approach more invigorating for the students.He felt it was important to have a good number of students to validate what wasbeing said in the evaluation process, so as not to misuse Bisolated opinions.’’ Dr. Ialso felt it important that students understand the importance of the evaluationprocess and be clear about their responsibility in the process. Sometimes studentsfelt uncomfortable listening to his views and beliefs as an African American. Hesaid if the instructor had certain values and beliefs that he was not willing tocompromise, he should either help students to incorporate similar values in theprocess or tell students that Bwe need to agree to disagree.’’

Case J

Dr. J was an adjunct non-tenure track senior instructor in Department Five. Beforehe joined the faculty at the university 3 years ago, he had been a fulltimeadministrator at a community college for about 20 years. In spite of this experience,Dr. J regarded himself as a novice in teaching. He said he was still in the learningprocess. Dr. J felt student evaluation of teaching had come a long way. At the sametime, it had not received sufficient attention by everybody. He said his experiencewith student evaluation had been limited. He regretted not having a formalevaluation of his own, referring to the fact that the formal evaluation that he did sofar was the one required by the department. He felt he could have done more interms of systematically collecting student evaluation feedback.

Dr. J said he was not sure if the end of semester student evaluation feedback hadbeen helpful for him to improve his teaching. Although many written commentswere positive, he felt they were just individual comments. BI don’t know how muchweight you (should) give to the one comment out of a whole class,’’ he said. Studentevaluation feedback did give him a general sense of whether he was going in theright direction. He said he would love to look at the positive ones, and would have ahard time living with a few negative comments from students. Dr. J said he wasscared to open the envelope containing student evaluation feedback. Althoughgenerally he felt good about his teaching, he said he was not sure what his studentswere going to say about his course. He said he would let it lie on the desk for a while

120 J Pers Eval Educ (2005) 18: 107–126

before he opened it. In terms of what aspects of his teaching were most likelyaffected by the feedback, Dr. J mentioned the mechanical aspects, such as theamount of reading that was needed, the way materials were presented, the pace ofthe course, and the number of questions asked in class. He said the content of thecourse was seldom commented on.

Dr. J said the number of students who responded to the evaluation in his web-based course was small. This was not a concern in his face-to-face course, where theinstructor could almost force the students to fill out an evaluation. He felt theinstruments used for course evaluations were as good as they could be. However, hesaid an ideal survey instrument should incorporate things that were specific to thecourse, the instructor, and the students. He said that was an area where he couldseek feedback on his own.

Discussion

While each individual case presents a unique picture of faculty response to studentevaluation feedback, it is also important to examine the multiple cases together tosee if there are themes that are common across the cases.

The discussion of the themes involves a comparison of senior and junior facultymembers, since the researchers felt that faculty members’ response to the studentevaluation of teaching was possibly linked to their seniority, especially tenure status,if the evaluation was directly used in personnel decisions. There were altogether sixsenior faculty members who participated in the interviews. They came from four ofthe five departments. All of them had tenure, were either full professors or associateprofessors, and had been working in the School of Education for approximately 15years or more. All of them were male. The four junior faculty members ranged fromnon-tenure track lecturers or instructors to tenure track assistant professors. All ofthem had less than 5 years of faculty appointment in the School of Education. Two ofthem were female faculty members.

Common Themes

An examination of the multiple cases within the same context revealed commonthemes of the study. The variations among the cases helped to maximize ourunderstanding of the themes (Stake, 1995). Through the analyses of the narratives,anxiety, attention, discretion, and tension were identified as common themes.

Anxiety

It was clear from the individual cases that the participants of the study experiencedvarious degrees of anxiety about their student evaluation results. Several partic-ipants mentioned being nervous when the envelope containing the results of thestudent evaluation arrived.

Although both senior and junior faculty had some anxiety with student evaluationfeedback, faculty members in the beginning of their teaching careers particularlyexperienced nervousness or apprehension at the evaluation results. Instructor G, forinstance, was emotionally attached to the evaluation results in the beginning.

Senior faculty members were not completely free from anxiety, especially if theyhad something new in their teaching. In general, however, student evaluation

J Pers Eval Educ (2005) 18: 107–126 121

feedback had become quite predictable for them. Experience had helped them todevelop a more professional attitude toward student evaluation results. Perhapsmore importantly, as one participant implied, the tenure system, which providedthem with academic freedom, also provided them with a sense of security.

This finding made intuitive sense, given that end of student evaluation results weredirectly linked to the annual review and tenure and promotion processes. Mid-semester evaluation was not used for summative purposes. However, if the resultsseemed to converge with results from end of semester student evaluation, it would stillworry faculty members, unless they were able to fix the problems within the secondhalf of the semester, which could be difficult to accomplish most of the times. Forsenior faculty members having already secured their tenure, negative feedback mightstill be a bother, especially if a post-tenure review system was in place, as was the casein the School of Education. The pressure for them, however, was substantiallyreduced, thanks to the tenure system. Their experience with student evaluation hadalso helped them to take a more objective stance towards the evaluation results.

Attention

In spite of various concerns about student evaluation of teaching, the participantsseemed to have paid attention to student evaluation feedback and tried to make useof the feedback to improve their teaching. Nobody seemed to put the feedback Binthe trash.’’

Junior faculty members tended to spend more time looking at the feedback andfind the feedback more useful than senior faculty members. For the latter, thefeedback was primarily used to confirm their teaching approach. However, in termsof the degree of attention paid to student evaluation feedback, seniority did notseem to be a major factor. Some tenured senior faculty members were still payingattention to the student evaluation feedback, to the extent that they voluntarily usedmid-semester student evaluation. When the mid-semester evaluation did notprovide any information different from end of semester evaluation, they sometimesstopped using it.

The attention faculty members paid to the feedback might be understood in thecontext of summative evaluation at the university. The mandated use of studentevaluation feedback for personnel decision-making seemed to be a sufficient mo-tivator for junior faculty members to pay attention to feedback from students.However, this did not seem to explain why some of the tenured faculty memberswere still using mid-semester evaluation to gather student feedback. According tothe Official Papers of the School of Education, the post-tenure review was onlyreserved for extreme cases. In general, there was not much pressure for tenuredfaculty members to constantly scrutinize their teaching behaviors. The finding thatat least some tenured faculty members continued to seek student feedback on avoluntary basis seems to suggest that something other than pressure fromsummative evaluations might be motivating some faculty members to pay attentionto student evaluation feedback.

Discretion

Almost every participant talked about the need to use discretion when utilizingstudent evaluation feedback. They were generally open to student feedback about

122 J Pers Eval Educ (2005) 18: 107–126

mechanical aspects of their teaching but reluctant to change aspects of their teach-ing practice that they had valued over a long period of time, such as the teachingstyle and the course content. Senior faculty in particular felt they had better judg-ment than their students about certain aspects of their teaching. An exception mightbe made in the case of junior faculty members. Being at the formative stage of theirteaching career, they seemed to be more open to all kinds of student feedback.

Faculty also showed discretion in the way they read student evaluation feedback.Several participants mentioned they wanted to look at the whole picture and triednot to rely on isolated opinions. The individual comments could Brupture your ego,’’as one participant put it. They could also misrepresent the real situation. As severalparticipants pointed out, some students might use the evaluation opportunity to getback at their professor. As a result, faculty members would try to validate individualstudent comments before they implemented any change in their teaching. The needto use such discretion was mentioned by both senior and junior faculty.

The discretion faculty members exercised in interpreting and making use ofstudent evaluation feedback seemed to agree with their view that student evaluationhad some limitations, although at the same time they also felt there was value in thefeedback.

Tension

Several participants alluded to the tension they felt between student needs andfaculty requirements. Although they attempted to accommodate student needs, theyfound not all student needs were academically justifiable. There were students whomight just want to have a good time in class. They did not like difficult examinationsor a large amount of homework. If the faculty members did not satisfy studentneeds, they might not get good evaluation results. If they did, it would be a violationof their professional ethics. More often the tension was due to a difference betweenthe professor and the students in their views about teaching. As one participantpointed out, students may not necessarily know what was best for them. Eventhough the feedback truly reflected their thinking of teaching, it could be flawed ormisinformed.

Tension could also arise from potentially different treatments of student feed-back in formative and summative evaluations. On the one hand, faculty tended touse their professional judgment to find valid evaluation information to improveteaching. On the other hand, administrators might use all the information studentsprovided in annual review and in tenure and promotion materials. Although facultycould justify their disregard of certain student feedback with good reason, theycould feel powerless during personnel evaluations.

Tension also existed between faculty need for support for teaching improvementand the support they were able to receive. There were two types of supportmentioned by the participants in the study, administrative support in the form ofrewarding good teaching and using valid measures of teaching, and institutionalsupport in the form of advice and consultation on how to interpret and use studentfeedback to improve their teaching.

Although the tensions seemed to be felt by both senior and junior faculty, juniorfaculty members were most likely to fall victim to such tensions. They tended to beless sure of their teaching, and less likely to be upfront with the students regardingrequirements for a class. It would result in a harmful professional experience for

J Pers Eval Educ (2005) 18: 107–126 123

junior faculty if the feedback they relied on was seriously flawed. Senior faculty,however, were more likely to rely on their professional experience to determine theworth of student feedback. The tenure system also in general enabled them to stickto their standards.

Overarching Theme

Taken together, the case study results found that faculty members paid attention toand valued student evaluation feedback and were in general interested in makinguse of the feedback to improve their teaching, although their use of the feedbackwas affected by a number of factors, such as the quality of the feedback and theexistence of a mandated evaluation process. It was clear from the study that the useof student evaluation for personnel decisions such as promotion and tenure was notthe only reason behind their attention to student evaluation. Some faculty membersvoluntarily sought student evaluation feedback, even though they felt studentevaluation feedback was not perfect. It was possible that a respect for the teachingprofession and a care for student learning were among the reasons why faculty paidattention to and made use of the student evaluation feedback for teachingimprovement. The findings seem to lend support that faculty are likely to improvetheir teaching practice based on the feedback from student evaluations, since theyhave the motivation for improving their teaching.

Faculty members also experienced anxiety when receiving student evaluationfeedback. Much of the anxiety seemed to be linked to the summative use of thefeedback by administrators to make personnel decisions. Although some amount ofanxiety due to the concern of summative evaluation might be healthy, too much of itmight be harmful for faculty formative use of the evaluation feedback. A potentialarea for future research would be to find out how the anxiety resulting from thesummative use of student evaluation can be turned into positive use of the feedback,so that faculty remain committed to the use of student evaluation feedback, withoutgenerating any fear about the process (Johnson, 2000).

Although faculty members paid attention to and to some extent valued studentevaluation feedback, they also used discretion when they made use of the feedbackto improve their teaching. Such discretion was consistent with the general facultyperception that student feedback was valuable in certain aspects of teaching(Schmelkin et al., 1997). Faculty members also did not completely agree on theexact areas where the feedback from students would be most useful, even thoughthe general consensus seemed to be that the daily, mechanical aspects of teachingwould most likely benefit from student feedback. The difference in faculty opinionssees to echo the disparity in findings in the literature (Schmelkin et al., 1997;Spencer & Flyr, 1992). A potential explanation is that the impact of studentevaluation feedback on different teaching dimensions could vary across subjectareas, and may also vary with faculty experience.

Faculty members also experienced tensions in the process of student evaluation,between the requirements for the students and the responsibilities for them, betweenthe need of formative student evaluation and summative student evaluation, andbetween the need for support and availability for support. The tension betweenformative and summative use of the student evaluation feedback within the School ofEducation tended to reduce the potential effectiveness of the data for facultymembers to improve their teaching (Johnson, 2000). The impact was particularly

124 J Pers Eval Educ (2005) 18: 107–126

strong for junior faculty members, who did not have the necessary experience or thepower to dismiss student feedback that they might have doubt about. The tensionbetween the need for support and the availability for support has also received muchattention in recent years (Frost & Teodorescu, 2001; Marincovich, 1998). Withoutsuch support, teaching improvement is less likely to occur, even if faculty have themotivation to utilize student evaluation feedback. The tensions to a large extentrepresent the complex interrelationships of the context of study.

In conclusion, faculty use of student evaluation feedback for the improvement oftheir teaching operated in a complex system, with external variables such as po-tential pressure from summative use of student evaluation, support from admin-istrators, colleagues, and university Teaching Center, and internal variables such asfaculty tenure status and teaching experience, faculty perceptions of teaching andperceptions of student evaluation feedback, and their general attitude toward theirprofession. Such variables are intricately interrelated to each other. The interplay ofthe variables could produce tensions, which could have negative impact onformative use of student evaluation feedback. On the other hand, if all the variablesare clearly identified and properly addressed, the worthy cause of formative studentevaluation can be eventually accomplished, to the benefit of both the facultymembers and the students.

References

Algozzine, B., Beattie, J., Bray, M., Flowers, C., Gretes, J., Howley, L., et al. (2004). Studentevaluation of college teaching. College Teaching 52(4), 134–141.

Centra, J. A. (1993) Refflective Faculty Evaluation San Francisco, California: Jossey-BassCramer, K. M., & Alexitch, L. R. (2000). Student evaluations of college professors: identifying

sources of bias. Canadian Journal of Higher Education 30(2), 143–164.Creswell, J. W. (1998). Qualitative Inquiry and Research Design: Choosing Among Five Traditions.

Thousand Oaks, California: SageEiszler, C. F. (2002). College Students’ Evaluations of teaching and grade inflation. Research in

Higher Education 43(4), 483–501.Fenwick, T. J. (2001). Using student outcomes to evaluate teaching: a cautious exploration. New

Directions for Teaching and Learning 88, 63–74.Frost, S. H., & Teodorescu, D. (2001). Teaching excellence: how faculty guided change at a research

university. Review of Higher Education 24(4), 397–415.Jacobs, L. C. (1987). University Faculty and Students’ Opinions of Student Ratings. Bloomington,

Indiana: Bureau of Evaluative Studies and Testing. (ERIC Document Reproduction Service No.ED291291).

Johnson, R. (2000). The authority of the student evaluation questionnaire. Teaching in HigherEducation 5(4), 419–434.

Kember, D., Leung, D. Y. P., & Kwan, K. P. (2002). Does the use of student feedback questionnairesimprove the overall quality of teaching? Assessment & Evaluation in Higher Education 27(5),411–425.

Marincovich, M. (1998). Ending the Disconnect Between the Student Evaluation of Teaching and theImprovement of Teaching: A Faculty Developer’s Plea (Report No. NCPI-1-02). Standford,California: National Center for Postsecondary Improvement. (Eric Document ReproductionService No. ED428590).

Marsh, H. W. (1987). Students’ evaluations of university teaching: research findings, methodologicalissues, and directions for future research. International Journal of Educational Research 11, 255–389.

Marsh, H. W., & Roche, L. (1993). The use of students’ evaluations and an individually structuredintervention to enhance university teaching effectiveness. American Educational ResearchJournal 30(1), 217–251.

J Pers Eval Educ (2005) 18: 107–126 125

Midwest University (1989). Guidelines for evaluation of faculty: annual evaluations, promotion, andtenure. Retrieved May 5, 2000, from http://www.???.edu/svcaa/policies/faculty/ tenureguide.html.

Midwest University (1998). Post-tenure review policy. Retrieved May 6, 2000, from http://www.???.edu/svcaa/polocies/posttenure.html.

Midwest University, School of Education (1997a). Promotion and Tenure Document.Midwest University, School of Education (1997b). Official Papers.Murray, H. G., Jelley, R. R., & Renauld, R. D. (1996, April). Longitudinal Trends in Student

Instructional Ratings: Does Evaluation of Teaching Lead to Improvement of Teaching? Paperpresented at the Annual Meeting of the American Educational Research Association, NewYork.

Olivares, O. J. (2002). A conceptual and analytic critique of student ratings of teachers in the USAwith implications for teacher effectiveness and student learning. Teaching in Higher Education8(2), 233–245.

Saroyan, A., & Amundsen, C. (2001). Evaluation university teaching: time to take stock. Assessmentand Evaluation in Higher Education 26(4), 341–353.

Schmelkin, L. P., Spencer, K. J., & Gellman, E. S. (1997). Faculty perspectives on course andteacher evaluations. Research in Higher Education 38(5), 575–592.

Shevlin, M., Banyard, P., Davies, M., & Griffiths, M. (2000). The Validity of student evaluation ofteaching in higher education: love me, love my lectures? Assessment & Evaluation in HigherEducation 25(4), 397–405.

Sojka, J., Gupta, A. K., & Deeter-Schmelz, D. R. (2002). Student and faculty perceptions of studentevaluations of teaching: a study of similarities and differences. College Teaching 50(2), 44–49.

Spencer, P. A., & Flyr, M. A. (1992) The formal evaluation as an impetus to classroom change: mythor reality? Riverside, California. (Eric Document Reproduction Service No. ED349053).

Stake, R. E. (1995). The Art of Case Study Research. Thousand Oaks, California: Sage.Wolfer, T. A., & Johnson, M. M. (2003). Re-evaluating student evaluation of teaching: the teaching

evaluation form. Journal of Social Work Education 39(1), 111–121.

126 J Pers Eval Educ (2005) 18: 107–126