ASSESSING INSTRUCTIONAL QUALITY OF THAI MATHEMATICS TEACHERS PARTICIPATED IN

LESSON STUDY AND OPEN APPROACH Duanghathai Katwibun, Nutjira Busadee

Chiang Mai University, Centre of Excellence in Mathematics, CHE, Si Ayutthaya Rd., Bangkok, 10400, Thailand

Duanghathai.k@cmu.ac.th, nnuutt37@hotmail.com

This study is part of a three-year professional development project from 2009-2011, supported by the Center for Research in Mathematics Education (CRME). The study aimed to assess the instructional quality of Thai mathematics teachers who had participated in the project by implementing lesson study and open approach. The Instructional Quality Assessment (IQA) toolkit was used as a tool of measurement. The findings revealed that the instructional quality of the teachers was at an overall good level. The teachers’ instructions and assignments were compounded with active participating, asking and answering with reasons, and engaging in complex mathematical thinking and dynamic classroom discussions. There was a significant correlation between the observed instructions and mathematics assignments. The outcomes provide a basis for mathematics teachers to emphasize on their improvement efforts. Keywords: Instructional Quality Assessment (IQA), Lesson Study, Open Approach, Professional Development, Mathematics Teachers

INTRODUCTION

During the past 20 years, Thailand has experienced a rapid change from a predominantly agriculture-based to a market-driven economy. In order to support this change, high-levels of knowledge and skills including competencies in the new technologies of employees are needed. Similar to many other countries, new expectations for citizens are increased such as being life-long and self-regulated learners, as well as having the ability to readily adapt to changing situations. As new human capabilities are recognized, aspects of the Thai education system especially the teaching and learning approach and educational management practices have changed dramatically (Office of Commercial Services, 2002).

In 1999, the Thai Government enacted the National Education Act (Office of the National Education Commission, 1999) with the attention of improving the quality of education as well as supporting it with processes that create citizens with the high-level of knowledge and skills. Along with its attention, the Act has emphasized on the significance of teachers’ professional development as they are responsively involved to the quality of teachers. Since the release of the National Education Act of 1999, several efforts have been established to promote teacher quality or effectiveness in teaching mathematics, as mathematics

5138

achievement is becoming a prerequisite for many careers in the market-driven economy. Many projects and studies on professional development have been conducted. Yet, only few of the studies focused on assessing the quality of instruction.

In fact, researchers revealed that student achievement can be most influenced by the quality of instruction (Sanders & Horn, 1994). Additionally, the instructional quality that teachers provide for students is varied, even within the same school. These could have an effect on academic progress in different rates for students with similar demography and academic performance (Rowan, Correnti, & Miller, 2002). Moreover, researchers have recently developed an assessment tool to measure the quality of instruction: The IQA toolkit (e.g., Boston & Wolf, 2006; Junker et al., 2004; Matsumura et al., 2006).

Thus, the purpose of this study was to assess the instructional quality of Thai mathematics teachers who had participated in a three-year professional development project by implementing the lesson study and open approach, during 2009-2011.

CONCEPTUAL FRAMEWORK

Studies on the development of IQA, that Matsumura and her associates conducted, provide us a tool to monitor the quality of teaching in mathematics classrooms (Boston & Wolf, 2006; Matsumura et al., 2006). The IQA has been developed to evaluate the quality of instruction based on dimensions that are observable and considered to be key indicators of student achievement: the quality of classroom discussion (Accountable Talk), the rigor of learning activities (Academic Rigor), and the expectations communicated to students for the quality of their work (Clear Expectations). For Accountable Talk, it’s focused on students’ participation in the discussion, teachers linking students’ contributions to each other, students linking each other’s contributions, teacher asking for evidence or requesting students to be explicit in their thinking, and students to provide evidence or explain their thoughts. For Academic Rigor, it involves the potential of the task, the implementation of the task, and students’ discussions of the mathematical concepts following the task. For Clear Expectations, the emphasis is on clarity and detail of expectations, academic rigor in teacher’s expectations, and students’ access to expectations (Matsumura et al., 2006). Four-point scales (1 = poor and 4 = exemplary) were used in the IQA toolkit to evaluate the quality of observed lessons and assignments (Matsumura et al., 2006). Matsumura et al. (2006) conducted a pilot study of the IQA where they explored the reliability and potential validity of the ratings of observed instruction and the quality of teachers’ assignments with student work. The findings revealed a moderate to high level of reliability. In mathematics, the quality of teachers’ observations and assignments were significantly correlated.

Boston and Wolf (2006) illustrated the development of the Academic Rigor in Mathematics (AR-Math) rubrics of the IQA and reported the findings from a small pilot study in 2003. In assessing the reliability of the AR-Math rubrics, results indicated that the rubrics were able to capture the differences in students’ opportunities to learn mathematics in each context. In addition, researchers suggested that the IQA toolkit can serve as a tool not merely for measuring instructional quality, but also for promoting professional development.

5139

Focusing on lesson study, it is “a Japanese form of professional development that centres on collaborative study of live classroom lessons...” (Lewis, Perry, Murata, 2006, p.1). It provides a mean for teachers to look at their own practice (Stigler & Hiebert, 1999).

Lesson study is…built into the work week. In pursuing lesson design (What is the mathematics we consider central? How do we plan to have students approach it? What are students likely to do, and how will we react to it?) (Schoenfeld & Kilpatrick, 2008, p. 8).

The lesson study cycle, implemented in this study, involves three main phases: collaborative lesson planning, collaborative lesson observation, and collaborative reflection on the observed lesson.

Regarding the open approach, it was rooted in Japan through the research on the evaluation of higher-order thinking in mathematics education, using open-ended problems as a theme in the early 1970s (Becker & Shimada, 1997). Later on, Nodha (2000) suggested a teaching method called the “open approach” that engaged students to solve non-routine open-ended problems. The problems require various means to solve or have several correct answers. Sawada (1997) illustrated five advantages of using open-ended problems. First, it allows students to articulate their thoughts more freely and participate in classrooms’ activities more actively. Second, it provides students greater opportunities to operate their mathematical knowledge and skill more widely. Third, it lets individual student to solve the problem in his/her own meaningful ways. Fourth, it offers students rational experiences. Finally, it gives students a chance to sense the fulfilment of discovery and the approval of their classmates.

Lesson study and open approach has been adapted and implemented in Thailand since 2002 in attempting to improve teacher education program (Inprasitha, 2010). This study was part of the three-year professional development project from 2009-2011 that carried out in six schools in four provinces in Northern Thailand by the Faculty of Education, Chiang Mai University.

MEDTHOLOGY

The study was conducted at an elementary level (grade 1 to 6) with 12 participating teachers, who were nominated by the six schools (two teachers from each school). Data collection (from classroom observations, teachers’ assignments, and students’ work) and the usage of the IQA toolkit were based on and adapted from research by Matsumura et al. (2006).

In this study, the three phases of the lesson study cycle were implemented and completed in the schools every one or two weeks, for the whole academic year. Collaboration in each phase of the cycle was emphasized. Additionally, the implementation of the open approach in this study involved four steps, including launching open-ended problems, self-learning through investigating and solving problems, whole-class sharing and discussing the findings, and summarizing based on connecting students’ ideas.

In the second semester of the academic year 2011, one full lesson (50-60 minutes) was observed for each teacher. During the observation, two raters (the researchers) made field

5140

notes providing additional data to validate their rates. After the observation, each rater independently scored and used the IQA rubrics of Accountable talk, Clear Expectations, and Academic Rigor. In addition, selected mathematics assignments (four per teacher) and students’ work (four per assignments: selected by the teachers as medium to high quality) were gathered and scored by the same raters. Before ratings, the raters learned about the IQA rubrics, and examples of observed lessons and assignments that were considered as high or low quality. After ratings, the raters discussed on their ideas and reached a consensus score, if there was any disagreement on classroom observations or scoring teachers’ assignments. In addition, video recordings of the observed instructions and lesson plans of the assignments were collected as additional data to verify the ratings. In data analysis, the consistency of rating scales and the inter-reliability were computed. Mean and standard deviations (SD) for each dimension was reported, and Pearson correlation between observed instructions and mathematics assignments was presented.

RESULTS

In this study, the purpose was to assess the instructional quality of Thai mathematics teachers who had participated in a three-year professional development project in implementing lesson study and open approach. The researchers used the IQA toolkit to capture the quality of teachers’ Accountable Talk, Academic Rigor, and Clear Expectations. Results showed Cronbach’s alpha of r = 0.92 for the IQA observation measure and r = 0.97 for the assignments. The inter-reliability or the level of inter-rater agreement for most rubrics ranged from a moderate to high level. Exact scale-point agreements averaged 82.9%. In addition, there was a significant positive correlation between the observed instructions and the mathematics assignments (r = 0.95, p ‹ 0.001).

Descriptive statistics were calculated to illustrate the characteristics of the observed mathematics instructions (see Table 1) and assignments (see Table 2). Overall, for the observed instructions, the average quality of the observed lessons was a good level (mean = 3.00). Teachers regularly provided students’ opportunities to participate in dynamic classrooms. The students had a chance to participate in investigating and solving high potential tasks and students were regularly asked for reasons to justify their contributions and explain their thoughts. Teachers’ linking with students’ contributions was recorded at a good level. The students often provided accurate and appropriate evidence for their claims or explained their ideas with reasoning. However, students’ linking their contributions to each other was at a fair level. Focusing on the Academic Rigor, all three dimensions, including the potential of the task; the implementation of the task; and the students’ discussions following the task, were approached to an outstanding level. These meant that the observed instructions were enriched with using tasks that required a high-level of thinking, maintaining high-level demands of the task as it was implemented, and having students discuss and express their thoughts following the task. Regarding Clear Expectations, teachers showed high expectations for students’ learning through higher-level thinking skills such as classifying, comparing, and analysing. Nevertheless, teachers did not often explanations for their expectations for the quality of students’ work or contributions. In addition, teachers appeared to pay less attention to providing clear explanations, or discussing criteria for students’ work.

5141

For mathematics assignments, the average quality of collected assignments was almost at a good level (mean = 2.95). In fact, all four dimensions of Academic Rigor were approached to an exemplary level. The assignments’ potential and the engagement of students were involved in investigating the nature of mathematical concepts, procedures, and/or mathematical relationships, using multiple strategies or representations. However, mathematical generalizations were not sufficiently emphasized in the assignments and students’ engagements. Students provided sufficient explanations of their methods and thoughts in their written work. Regarding Clear Expectations, teachers showed high expectations for students’ learning through exploring and understanding; but they barely provided or communicated the criteria for the quality of the students’ work as well as the clarity and detail of their expectations with their students.

Table 1: Quality of the observed mathematics instructions (N= 12 observations, Range 0-4)

Rubric Scales Mean (SD)

Accountable Talk:

Participation 3.50 0.67

Teacher’s linking

Students’ linking

Teacher’s asking

Students’ providing

Academic Rigor:

Potential of the task

Implementation of the task

Students’ discussions following the task

Clear Expectations:

Clarity and detail of expectation

Academic rigor in teacher’s expectations

Communications of teachers’ expectations

3.25

2.75

3.08

3.50

3.58

3.75

3.58

1.33

3.67

1.00

0.75

0.87

0.90

0.52

0.79

0.62

0.51

0.78

0.49

0.00

Table 2: Quality of the mathematics assignments (N= 48 assignments, Range 0-4)

5142

Rubric Scales Mean (SD)

Academic Rigor:

Potential of the task 3.73 0.41

Implementation of the task

Rigor in students’ responses to the task

Academic rigor in teacher’s expectations

Clear Expectations:

Clarity and detail of expectation

Communications of expectations

3.79

3.54

3.35

1.00

1.00

0.32

0.61

0.52

0.00

0.00 DISCUSSION AND CONCLUSION

In summary, the results of this study demonstrate that the instructional quality of Thai mathematics teachers, who are involved with lesson study and open approach, are over all at a good level. In the observed instructions, students obviously gained opportunities to participate in dynamic classroom activities and discussions. They were motivated to learn through high-order-thinking tasks and encouraged to provide explanations and reasons in asserting their contributions. Teachers had high expectations for students’ learning; but they did not focused on providing clear expectations and criteria for students’ work. For the assignments, they had the potential to promote complex mathematical thinking and using multiple strategies or representations. However, mathematical generalizations were not sufficiently emphasized. Students provided adequate explanations of their methods and thinking in their written work. Above all, the data suggested that teachers who had been involved with the professional development project implementing lesson study and open approaches appeared to be able to create a moderate to high quality of instructions or teaching effectiveness. Besides, the findings could serve as a basis for teachers to focus on their development efforts as it revealed the keys elements of the in-service teachers’ practices based on lesson study and open approach.

Regarding to the instructional quality, the quality of Academic Rigor dimension (for both of the observed instructions and assignments) were at high level. About 93% of the tasks were of a high quality (scored a 3 or a 4), this is not surprising as a matter of fact that the nature of an open approach focuses on student-centered learning (especially mathematical thinking); meanwhile, lesson study emphasizes teachers’ learning of their teaching profession (Inprasitha & Loipha, 2007). However, the Clear Expectations dimension for both of the observed instructions and assignments were at a poor level. Apparently, the teachers have not yet recognized their roles in providing and discussing the expectations the teachers have on

5143

the students, and the criteria for the quality of the students’ work. This should be a point for the teachers to be more aware of the clarity and communication of their expectations and the criteria.

As the findings suggested that the classrooms implementing lesson study and open approach provided opportunities for students to learn mathematics with complex thinking and understanding, lesson study and open approach have recently gained more attention among mathematics teachers and educators. Still, there is a need for further research on mathematics classrooms and professional development that implement lesson study and open approach in order to develop an in-depth understanding of this area.

Regarding to the use of IQA toolkit, the researchers (the raters) newly employed the toolkit; however, the inter-reliability results were at a high level. In fact, this could be explained by the researchers’ involvement with classroom observations during the three-years of the professional development project. Frequently, the researchers observed open-approach classrooms, then, discussed and reflected on learning and teaching practices in the lesson study cycles. Using the IQA toolkit to observe the classrooms guided the researchers to focus in more specific dimensions. Being familiar with the classroom observations and the classroom context, might benefit the researchers to employ the IQA toolkit with less difficulty.

Last but not least, as the IQA could be implemented by various purposes and users: a) to monitor instructional quality across schools by researchers or administrators; b) to facilitate teachers reflection on their own teaching practices and plan learning activities through instructional coaches and principals, and c) to do self-assess teaching practices by teachers (Matsumura et al., 2006), further research on the usage of the IQA toolkit with different purposes in mathematics classrooms is still needed.

Acknowledgment

The researchers would like to thanks Dr. Tipparat Nopparit for her comments. References Becker, J. P., & Shimada, S. (Eds.) (1997). The open-ended approach: A new

proposal for teaching mathematics. Reston, VA: National Council of Teachers of Mathematics.

Boston, M., & Wolf, M. (2006). Assessing Academic Rigor in Mathematics Instruction: The development of the Instructional Quality Assessment Toolkit. (CSE Tech rep. No. 672). Los Angeles: Center for the Study of Evaluation

Inprasitha, M. (2010). One feature of adaptive lesson study in Thailand: Designing learning unit. In CheonSoo, C., SangGa, L., & YoungHan, C. (Eds.) Proceeding for the 24th Korean National Meeting of Mathematics Education.

5144

(pp. 193-206). Dongkook University Gyeongju, Korea Society of Mathematics Education.

Inprasitha, M., & Loipha, S. (2007). Developing student’s mathematical thinking through lesson study in Thailand. Progress report of the APEC project: HRD 02/2007. CRICED: Japan.

Junker, B., Matsumura, L. C., Crosson, A., Wolf, M. K., Levison, A., Weisberg, Y., & Resnick, L. (2004). Overview of the Instructional Quality Assessment.

Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.

Lewis, C., Perry, R., Murata, A. (2006). How should research contribute to instructional improvement? A case of lesson study. Educational Researcher, 35(3), 3-14.

Matsumura, L. C., Slater, S. C., Junker, B., Peterson, M., Boston, M., Steele, M., et al. (2006). Measuring reading comprehension and mathematics instruction in urban middle schools: A pilot study of the Instructional Quality Assessment. (CSE Tech rep. No. 681). Los Angeles: Center for the Study of Evaluation

Nohda, N. (2000). Teaching by Open-Approach method in Japanese mathematics classroom. p. 39–53. In T. Nakahara and M. Koyama (Eds.) Proceedings of the 24th Conference of the International Group for the Psychology of Mathematics Education. 23–37 July. Hiroshima, Japan.

Office of Commercial Services. (2002). Teacher development for quality learning: The Thailand education reform project. Brisbane, Australia. Retrieved May 16, 2011, from http://www.edthai.com/publication/0005/fulltext.pdf

Office of the National Education Commission. (1999). National Education Act of B.E. 2542. Bangkok: Author. Rowan, B., Correnti, R., & Miller, R. J. (2002). What large-scale, survey research tells us

about teacher effects on student achievement: Insights from the prospects study of elementary schools. Teachers College Record, 104(8), 1525-1567. Retrieved May 16, 2011, from http://www.cpre.org/images/stories/cpre_pdfs/rr51.pdf

Sanders, W., & Horn, S. (1998). Research findings from the Tennessee Value-Added Assessment System (TVAAS) database: Implications for educational evaluation and research. Journal of Personnel Evaluation in Education, 12(3), 247-256.

Sawada, T. (1997). Developing Lesson Plans. In J. Becker, & S. Shimada (Eds.), The open-ended approach: A new proposal for teaching mathematics. (p. 23-35). National Council of Teachers of Mathematics.

5145

Schoenfeld, A. & Kilpatrick, J. (2008). Toward a theory of proficiency in teaching mathematics. In D. Tirosh &T. Wood (Eds.).The International Handbook of Mathematics Teacher Education Voume 2: Tools and Processes in Mathematics Teacher Education (pp. 321-354). Rotterdam: Sense Publishers.

Stigler, J.W. & Hiebert, J. (1999). The teaching gap: Best ideas from the world’s teachers for improving education in the classroom. New York: Summit Books.

5146