Journal of Educational Psychology Copyright t998 by the American Psychological Association, Inc. 1998, Vol. 90, No. 2, 261-278 0022-0663/98/$3.00

Does Concept-Oriented Reading Instruction Increase Strategy Use and Conceptual Learning From Text?

John T. Guthrie, Peggy Van Meter, Gregory R. Hancock, Solomon Alao, Emily Anderson, and Ann McCann

University of Maryland College Park

The authors define reading engagement as the mutual support of motivations, strategies, and conceptual knowledge during reading. To increase reading engagement, a collaborative team designed a year-long integration of reading/language arts and science instruction (Concept- Oriented Reading Instruction, CORI). The authors compared students who received this instruction to similar students who received traditionally organized instruction aimed toward the same objectives. A path analysis showed that CORI had a positive effect on strategy use and text comprehension for students at Grades 3 and 5 when accounting for past achievement and prior knowledge. CORI also had a positive, indirect effect on conceptual knowledge mediated by strategy use, and this instruction facilitated conceptual transfer indirectly through several paths simultaneously. The findings are discussed in relation to a growing literature on instructional contexts for motivated strategy use and conceptual learning from text.

The term engagement in reading refers to the motivated use of strategies and conceptual knowledge during reading. These processes operate dynamically, increasing over time. The engaged reader is in a continual process of activating and extending his or her conceptual understanding. As the individual reads, he or she acquires core knowledge that lies at the heart of the domain (Alexander, Schallert, & Hare, 1991; Chi, DeLeeuw, Chiu, & Lavancher, 1994). For example, conceptual knowledge of a topic such as adapta- tion includes information about species, habitats, life cycles, and evolutionary change. Particular propositions about a species and its habitat are related to each other, and broad principles form a network of understanding (Alexander, Jetton, & Kulikowich, 1996; Chi et al., 1994). From our perspective, conceptual transfer occurs when the student uses conceptual knowledge to address a new question or to solve a problem.

Engaged readers use and regulate strategies to enhance their conceptual understanding. Using prior knowledge (Anderson & Pearson, 1984), and posing questions as they explore their environment, engaged readers are involved in a process of searching (Singer & Donlan, 1982). They read multiple texts, examine a variety of documents, and extract critical details (Guthrie, Weber, & Kimmerly, 1993). As engaged readers succeed in searching, they use strategies for integrating information (Dole, Duffy, Roehler, & Pearson, 1991) from expository and narrative texts (Graesser, Gold-

John T. Guthrie, Peggy Van Meter, Gregory R. Hancock, Solomon Alao, Emily Anderson, and Ann McCann, Department of Human Development, College of Education, University of Mary- land College Park.

Correspondence concerning this article should be addressed to John T. Guthrie, Department of Human Development, College of Education, University of Maryland, College Park, Maryland 20742- 2109. Electronic mail may be sent to jg76@umail.umd.edu.

ing, & Long, 1991). Engaged learners also use strategies for communicating and representing their understanding that may entail drawing, charting, note taking, and composing either in narrative, expository, or persuasive rhetorical structures (Flower et al., 1990; Harris & Graham, 1992).

Learning these strategies is strongly associated with the motivational attributes of attention, effort, and desire to understand (Pintrich & Schrauben, 1992). For example, Meece, Blumenfeld, and Hoyle (1988) showed that task mastery goals, which refer to the desire to learn the content of a lesson or unit, predicted the self-reported use of such learning strategies as self-questioning, self-monitoring, and integrating new information with prior knowledge. Guthrie, Van Meter, et al. (1996) reported that intrinsic motivations for reading gathered through an in-depth interview corre- lated highly with students' searching, extracting, and integrat- ing as they studied multiple texts and illustrations (r = .8 for Grade 5; r = .7 for Grade 3). Pintrich and DeGroot (1990) found that intrinsic value (i.e., interest in a course topic) predicted the availability and self-regulation of cognitive strategies including goal setting, self-monitoring, and sum- marizing. Motivation to learn about a topic also contributes to conceptual understanding. Schiefele (1996) and Grolnick and Ryan (1987) showed that a stimulating and involving topic was understood more meaningfully than an uninterest- ing topic after prior knowledge was controlled. In addition, intrinsic motivation has been found to be related to other reading outcomes, such as school grades (Gottfried, 1985; Sweet, Guthrie, & Ng, 1998), cognitive engagement in the classroom (Skinner & Belmont, 1993), and the amount and frequency of reading activity (Wigfield & Guthrie, 1997).

Following our formulation of reading engagement pro- cesses, we next developed principles of instruction to increase reading engagement. First, we placed strategy instruction in a rich context of real-world interactions and opportunities for self-directed learning. On the basis of

261

262 ~trTm~tE ET AL.

previous research, we provided strategy instruction in read- ing with modeling, coaching, and guided practice (Pressley, Harris, & Marks, 1992). To generate long-term needs for using the strategies, we provided "hands-on" science learn- ing activities in which students generated their own personal questions about life science around a theme. We expected that these questions and the interest aroused by the science interaction activities would motivate students to learn and to use reading strategies. Real-world interaction and self- directed learning activities were designed to support intrin- sic motivation for reading. Teachers' strategy instruction emphasized that reading with strategies would help students answer their questions and compose their new knowledge in the forms of posters, classroom publishing activities, and videos. To provide a social milieu for strategy learning, teachers organized students to work in teams as well as individually. Collaborative teams discussed both the concep- tual content and the reading strategies they were learning.

We expected that the combined principles of strategy instruction, real-world interaction, self-directed learning, and collaboration around a conceptual theme would be a facilitative context for increasing reading engagement. De- veloped in a long-term researcher-teacher collaboration in one classroom (Guthrie, McGough, Bennett, & Rice, 1996), these principles were refined into the program of Concept- Oriented Reading Instruction (CORI) in a study examining growth of reading engagement in multiple classrooms in multiple schools (Guthrie, Van Meter, et al., 1996). Although these prior studies documented the increase of motivations, cognitive strategy use, and amount of reading in CORI classrooms, they did not compare CORI with an alternative form of instruction, which was a major purpose of this investigation. We attempted to make a cross-program com- parison of CORI with one traditional form of instruction to complement the previously observed within-programs in- creases of reading engagement. We then discuss the prin- ciples, which are more fully presented by Guthrie and Alao (1997).

Principles of CORI

The purpose of CORI is to optimize the development of reading engagement. Our instructional goal was to create classroom environments in which reading engagement was sustained for extended periods of time. We expected that this context would lead to long-term generalized engagement in reading, in which students approach new learning situations with more motivated strategy use than students who have not experienced the instruction.

Conceptual Theme

CORI classrooms were organized around broad, interdis- ciplinary themes to enable students to gain a conceptual understanding that was flexible, transferrable, and informed by multiple genres. A conceptual theme is a set of principles that defines a knowledge domain (Alexander, 1992) and can be understood through multiple texts and genre (Hartman, 1995). Conceptual learning includes several types of acquisi-

tion consisting of particular features, propositions, and principles in a domain (Chi et al., 1994). Students at the highest level of conceptual learning acquire explanatory principles for phenomena and events (Alexander et al., 1996). This knowledge is active and not inert. It can be transferred to new situations to solve problems.

Real- World Interaction

The classroom contexts provided ample opportunity for actual, direct sensory experiences with phenomena that were relevant to the conceptual theme. "Hands-on" science activities were provided to give students the experience of seeing, hearing, feeling, and smelling objects and events that were conceptually related. For example, students from one of the classrooms went on a cricket hunt in the backyard of the school. Such activities were designed to support question- ing, reading, thinking, and writing about the conceptual theme of adaptation, especially species-biome relationships. Direct, sensory science interactions serve two vital purposes in the development of engagement. First, they are exciting. Students are attentive and energized. This arousal is not only enjoyable and entertaining, but this contextualized, situ- ational interest can be linked to book reading (Linn & Muilenburg, 1996; Schraw, Bruning, & Svoboda, 1995). A second purpose of real-word interaction linked to the conceptual theme is to provide the opportunity for student questioning. Ross (1988) concluded from a recta-analysis of experiments that hands-on activities enhance the quality and number of questions and the recall of relevant knowledge. In our experience, if instruction supports students in answering their conceptual questions, students become engaged.

Self-Direction

The classrooms were structured to be autonomy support- ive (Deci, Vallerand, Pelletier, & Ryan, 1991). Teachers provided opportunities for student input into the questions that guided teaching in several ways. Students set subgoals for their own reading and writing. Students were free to choose books for learning, within the conceptual theme, although the options were initially limited to a few key choices. Broader options were constructed as students showed the capacity for productive use of them. Such support for student choice in reading and writing is motivat- ing for first graders (Turner, 1995), fifth and sixth graders (Skinner & Belmont, 1993; Sweet et al., 1998) and college students (Pintrich & Schrauben, 1992). In CORI, autonomy support was directed at (a) conceptually significant issues; (b) avenues for learning that are meaningful (e.g., book reading vs. a computer search in a given 2-day period); and (c) alternatives for expressing knowledge that had visible consequences (e.g., writing a poster vs. writing an informa- tional story).

Collaboration

CORI teachers supported students in working together toward understanding the conceptual theme, gaining cogni-

CONCEPT-ORIENTED

five strategies, and learning how to communicate effectively in groups. Students participated in varied social structures that included individual work, partnerships, small teams, and whole-class activities. Two prominent formats were literature circles, in which groups formed an interpretation of literary work (Almasi, 1995), and idea circles, in which students pooled diverse information sources to gain concep- tual understanding from multiple texts (Guthrie & McCann, 1996). Among the most widely researched variables in cooperative learning is positive group interdependence (Johnson, Johnson, & Stanne, 1989; Smith, 1991). In this condition, students must coordinate their efforts, share information, contribute specialized expertise, and build on each other's thinking. When students in a group adopt goals that involve conceptual learning (Meloth & Deering, 1994), students are superior in learning science concepts and strategies for reading. In addition, interpersonal skills, such as listening, taking turns, speaking in a suitable voice, and encouraging full participation, can be encouraged, which enhances text interpretation and enjoyment of group work (Almasi, 1995; Johnson et al., 1989). Collaboration in CORI was consistent with these findings.

Strategy Instruction

Teachers provided the support students needed to acquire strategies for learning and expressing conceptual knowledge through a variety of genres. Strategies included using prior knowledge (Anderson & Pearson, 1984), searching for information (Guthrie et al., 1993), comprehending informa- tional text (Dole et al., 1991), interpreting literary text (Graesser et al., 1991), and self-monitoring (Baker & Brown, 1984). After appraising students' knowledge of these strategies, teachers provided modeling, small-group discus- sion, peer modeling, whole-class discussion, and individual self-evaluation depending on the amount of explicitness that students required. In addition to strategies for reading, strategies for writing were taught, including brainstorming, planning, drafting, revising, and self-regulating (Harris & Graham, 1996). For all strategies, teachers provided model- ing, small-group discussion, guided practice, and revision that led to publication in the classroom.

Self-Expression

Several classroom characteristics contribute to its self- expressive quality. Teachers must provide ample time for students to think, plan, write, and revise (Oldfather & McLaughlin, 1993). Instruction affords writing in a wide range of alternative genres and topics. Such "open" tasks are more motivating than "closed" activities, which are highly defined and constrained (Turner, 1995). If students are empowered to be self-expressive, they tend to view knowledge as contextual, experience themselves as creators of knowledge, and value both subjective and objective strategies for knowing (Belenky, Clinchy, Goldberger, & Tarule, 1986). CORI classroom contexts contained the expectation that students would define the topic, the form of the text, the audience, and the collaborative structures for

R~ADINC INSTRUCTION 263

communicating their understanding of the conceptual theme to others.

Coherence

Teachers created coherence by linking the activities, materials, and contexts, which enabled students to make connections. By emphasizing integration of reading and content, teachers enabled students to perceive the connec- tions among (a) real-world experience and reading, (b) strategies for reading and knowledge about a particular topic, and (c) scientific and literary texts. As Lipson, Valencia, Wixson, and Peters (1993) reported, and as Pate, McGinnis, and Homstead (1995) suggested, the rationale for integrated instruction is usually to enhance the learning of content and metacognitive strategies. They also noted that coherence (or integration of the curriculum) provides a more interesting, meaningful way to teach and attain the main goals of the curriculum. Although relatively few empirical studies have been conducted to examine whether varying degrees of coherence in fact benefit students, a number of educational researchers (Brophy & Alleman, 1991) and curriculum specialists (Beane, 1995) advocate coherence.

Purposes of the Study

The study can be discussed in terms of Figure 1. At the most general level, we were interested in whether the instruction, grade level, and student background variables would predict the reading engagement processes. The paths in this model are numbered. For example, Path 1 denotes the effect of instruction on strategy use. In this study, instruction was dichotomized into either CORI or traditional instruc- tion, and a positive prediction indicated a positive advantage for CORI students compared with traditional students. Path 2 refers to the effect of instruction on conceptual knowledge. Path 3 refers to the effect of instruction on conceptual transfer. Continuing in the model, Paths 4--6 refer to the effects of grade level on strategy use, conceptual knowledge, and conceptual transfer, respectively. The next block consist- ing of Paths 7-9 refers to an Instruction × Grade interaction, which may predict the reading engagement processes.

Past achievement in literacy and science was represented in Paths 10-12. We introduced the indicator of past achieve- ment for purposes of statistical control to examine the predictions related to instruction more rigorously. In addi- tion, we introduced a measure of prior knowledge of the topic of the performance assessment. Paths 13-15 represent the prediction of prior knowledge on strategy use, concep- tual knowledge, and conceptual transfer, respectively. Prior conceptual knowledge was a task-specific indicator of student background, whereas past achievement was a gen- eral academic indicator. Prior conceptual knowledge and past achievement were used for statistical control and were not central to our theoretical formulations in this study. In the figure, these student characteristics are presented in the same column with instruction and grade because they were treated simultaneously in the structure equation modeling.

264 G t r r m ~ ET ~ .

Instruction and Student Background

Instruction

Grade

Instruction x Grade

Past Achievement

~ Prior Conceptual Knowledge

Reading Engagement Processes

~~ 3 2 v Strategy Use

47 6 116 18 w Conceptual Knowledge

10 ,

Conceptual v [ Transfer

Figure 1. Path model.

The effect of strategy use on conceptual knowledge is represented by Path 16, and the effect of conceptual knowledge on conceptual transfer is represented by Path 17. In addition, strategy use may be expected to predict concep- tual transfer directly through Path 18 in the model. Note that we use the terms predict and effect interchangeably. In path analysis, the basic data set is a covariance matrix among all the variables, and a path coefficient is basically a prediction of one variable (variable y) by another variable (variable x) with all others in the model controlled. Variable x predicts variable y significantly if the path coefficient is significant. Given a significant path coefficient, variable x can be said to have an effect on variable y if there is a theoretical justification or if variable x was antecedent to variable y in time. In this model, we fulfill both conditions for using the term effect. For example, instruction was theoretically expected to increase strategy use, and instruction occurred for 5 months before the measure of strategy use was administered. Consequently, in this study it means the same to state that instruction predicted strategy use as to state that instruction had an effect on strategy use.

Our first hypothesis was that instruction would predict strategy use. That is, we expected that Path 1 would be statistically significant while controlling for the effects of grade, Instruction X Grade interaction, past achievement, and prior knowledge on strategy use.

Our second hypothesis was that instruction would predict conceptual knowledge indirectly through strategy use. That is, we expected that the instruction would predict strategy use and that strategy use would predict conceptual knowl- edge. In the model, the flow of predication would begin with Path 1 and continue to Path 16. We wanted to examine this indirect prediction of instruction on conceptual knowledge

holding constant grade, Instruction x Grade interaction, past achievement, and prior knowledge.

Third, we hypothesized that grade level would predict strategy use, conceptual knowledge, and conceptual trans- fer when instruction, Instruction x Grade interaction, past achievement, and prior knowledge were controlled. That is, in the model we expected that Paths 4-6 would be significant.

Fourth, we hypothesized that instruction would predict conceptual transfer indirectly. That is, we did not expect that the direct prediction of instruction on conceptual transfer shown in Path 3 would be significant. Rather, we expected that instruction would predict strategy use (Path 1), which would predict conceptual knowledge (Path 16), which would predict conceptual transfer (Path 17). This is one indirect path from instruction to conceptual transfer. A second indirect path from instruction to conceptual transfer includes Paths 1 and 18, which we expected to contribute significantly. A third indirect effect consists of Paths 2 and 17. We intended to test all these indirect effects combined.

Finally, we expected that past achievement and prior knowledge would predict most of the engagement processes. However, we did not form specific hypotheses for them but instead used these factors as controls for examimn" g the effects of instruction and grade on reading engagement processes.

Method

Participants

Teachers and schools. Three schools bordering a large mid- Atlantic state metropolis participated in this study. Each school had

CONCEPT-ORIENTED READING INSTRUCTION 265

a multicultural population consisting of approximately 55% Black, 22% White, 15% Hispanic, and 7% Asian or other. Two schools were Chapter 1, and one school had a mainstream program for orthopedicaily disabled students. The schools were nominated by the district supervisor of reading as being likely to benefit from an integrated curriculum for low-achieving students, and the princi- pals were pleased to participate. All the teachers in all the schools volunteered for the study and were willing to teach in either CORI or traditional classrooms. The teachers at each grade in each school who were most similar in age, teaching experience, educational background, and management expertise were assigned to CORI and traditional classes. They were aged 41-50 years; they had 20-24 years of experience; and their educational backgrounds were all bachelor's degrees plus 45 hr.

Teachers participated in a summer workshop for 10 half days to plan CORI for the year. The four teachers, two reading specialists, one university faculty member, and one graduate student developed the instructional goals to be consistent with school and district objectives. They designed student activities and teaching strategies to attain the goals using CORI principles. The project grant contributed resources (trade books and manipulables) needed to implement the CORI principles into the classroom practices. During the year, these teachers met 1 day per month to discuss instruction, describe how they were implementing the principles, and exchange instructional techniques.

Classrooms and students. Third-grade classrooms in all schools were self-contained, with approximately 30 students in each at the beginning of the year. Third-grade students who completed the full year and the performance assessment consisted of 48 students in two CORI classrooms and 42 students in traditional classrooms located in the same schools. A total of 40 fifth-grade students in two classrooms completed the year of COP, I, and a total of 42 fifth-grade students in two traditional classrooms completed the year and the performance assessment. The students consisted of approximately equal number of boys and girls with 5-6 special education students mainstreamed into each classroom. A total of 172 students were included in the analyses. Grades 3 and 5 were used to permit us to compare strategy use, conceptual learning, and conceptual transfer at these two grade levels. We were interested in whether CORI would be equally beneficial for primary and intermediate students.

The three schools in the study were located in a traditionally low-achieving district. In the year of the study, one school had Comprehensive Tests of Basic Skills (CTBS) reading scores for Grade 3 at the 29th percentile, and Grade 5 Metropolitan Achieve- ment Tests (MAT) reading achievement was at the 23rd percentile. The second school had a CTBS reading percentile at 39 for Grade 3 and Metropolitan Achievement Tests reading at the 36th percentile for Grade 5. The third school was at the 42nd percentile for Grade 3 and the 35th percentile for Grade 5. Students in the CORI classrooms and traditional classrooms were representative of the schools' populations. We attempted to implement CORI for stu- dents who were low achievers from lower income schools in a school district that was demographically and culturally diverse. We believe there is an urgent need for instructional innovation and research with this population.

Design

This study contained two instructional conditions, CORI and traditionally organized basal and science instruction. CORI was implemented in four classrooms: two third-grade and two fifth- grade classrooms. Classrooms within each school were selected for comparison to the CORI classrooms on the basis of students, teachers, and school settings. The measures of strategy use and

conceptual knowledge were based on a performance assessment administered to all eight classrooms in the first half of April 1995. Preassessments consisted of teachers' ratings of students' levels of English and language arts and science obtained in the fall of 1994.

Instructional goals. The CORI and traditional programs were both directed toward identical goals for English and language arts and science. Teachers in traditional classrooms were working toward the same instructional goals as teachers in the CORI classrooms. Their emphasis on the skills of text comprehension, reading and writing multiple genres, and integrating information was equally high. The means to reaching the goal (e.g., the instructional design) were the basal and science textbook in the traditional classrooms. The means were the instructional principles of CORI in the experimental classrooms. Although the objectives were constant across the classrooms, the methods and materials differed in the CORI and traditional classrooms. In English and language arts, the objectives consisted of interpreting stories, comprehending expository texts, locating and integrating informa- tion from multiple texts, summarizing, self-monitoring, writing personal narratives, composing informational reports, and writing poetry. In science, the common goals included understanding the life cycles of plants and animals, describing important adaptations of animal species, understanding cycles of weather and seasons, collecting data, interpreting graphs and tables, and interpreting data from classwide projects. More complex, higher order goals were adopted for fifth-grade than for third-grade students. Instruction was adjusted to meet the needs of all learners, including approxi- mately 5 students in each Grade 3 CORI classroom who entered the year reading at a 1.5 grade level.

CORL In the framework for CORI, teachers first identified a conceptual theme for instructional units to be taught for 16-18 weeks in the fall and spring. The themes selected by third-grade teachers consisted of the adaptations and habitats of birds and insects. The third-grade units in the spring consisted of weather, seasons, and Earth formations. Fifth-grade units in the fall related to life cycles of plants and animals, and the spring units emphasized Earth science, including the solar system and geological cycles.

At the beginning of each unit, students engaged in observational and hands-on science activities both outside and inside the classroom. Third and fifth graders participated in activities such as collecting and observing crickets, constructing spider webs, dissect- ing owl pellets, and building weather stations. Within each activity, students personalized their learning by composing their own questions as the basis for observing, reading, and writing. These questions generated opportunities for self-directed learning. Stu- dents chose their own subtopics, found particular books, selected peers for interest-based activities, and constructed their goals for communicating to others.

To enable students to answer their questions, teachers helped them to use the library, find information books, locate information within expository texts, and use a diversity of community re- sources. Direct strategy instruction was provided to help students integrate across information sources including texts, illustrations, references, and human experts. In addition to informational texts, teachers introduced novels, folklore, and poetry related to the unit's conceptual theme. The last phase emphasized communicating to others. Having gained expertise in a particular topic, students were motivated to speak, write, discuss, and display their understanding to other students and adults. In both third- and fifth-grade class- rooms, students made posters, wrote classroom books, and com- posed extended displays of their knowledge. One class made a videotape of its weather unit, providing a lesson on weather prediction and explanation for the rest of the school. A more thorough description of the CORI framework with an accompany- ing literature review is provided elsewhere (Guthrie & Alao, 1997).

2 6 6 GtrrHRm EX AL.

Traditionally organized instruction. The teachers in the tradi- tional classrooms followed their usual pattern of using the teachers' guide and the sequence of content and activities in the McGraw- Hill basal program for both Grades 3 and 5. Students answered the end of unit questions, and were provided materials that matched their reading levels. Science content in the third- and fifth-grade basal-reading classrooms was directed at similar objectives as the CORI classrooms. Topics of adaptation, life cycles, weather and seasons, and solar systems were taught. Addison-Wesley textbooks and materials were used in all traditional classes. In all the schools, traditional teachers were frequent visitors to the CORI classrooms, adopting some texts and teaching approaches used in the CORI classrooms. This sharing may have led to an underestimate of the distinctiveness of the CORI program and to a conservative estimate of the instructional effects.

Instructional Questionnaire and Intervention Check

To determine the extent to which the principles of CORI were implemented, we developed an instructional questionnaire collabo- ratively with both the CORI and traditional teachers in the study. The research team wrote 99 items that addressed each of the seven dimensions. The response format was a Likert scale ranging from 1 (very true of my class) to 4 (not at all true of my class). Other items asked the teachers to estimate the time invested in an instructional practice with choices ranging from almost everyday to once a month or less. The questionnaire was administered to the partici- pants in the spring of the academic year. Fourteen items were excluded from analyses because of missing data. Six additional items were withheld because they showed low variability.

This instructional questionnaire contained seven bipolar scales. For example, the scale on direction of instructional control ranged from student directed (more CORI-like) to teacher and program directed (more traditional). Items in this scale were coded to reflect either an emphasis on student direction or an emphasis on teacher direction. Questionnaire data are displayed in Appendix A. Reliabili- ties were generally adequate, except for written expression.

Correlations with instructional type were all significant, except for written expression. A positive correlation indicated that CORI teachers emphasized the left side of the bipolar dimension and traditional teachers emphasized the right side of the dimension. In other words, CORI classrooms were characterized by real-world interactions, conceptual orientation, student direction, collabora- five social structures, explicit strategy teaching, and integration of language arts with science (shown in Appendix A). The traditional classrooms were characterized by more abstract representation of real-world experiences, factual knowledge learning, teacher direc- tion, individual work, implicit strategy learning, and departmental- ized language arts and science. This confirmed the distinctions between CORI and traditional instruction on six of the seven dimensions.

Past Achievement

To construct a measure of students' general level of literacy and science background at the beginning of the year, teachers per- formed ratings. The teachers and the reading specialist rated each child in the basal and CORI classrooms using the cumulative folder containing standardized test scores, previous grades, and school history. Each student was placed into a quintile (top fifth to bottom fifth) for language arts and a separate quintile for science back- ground. Grade-level expectations were used for reference sepa- rately for Grades 3 and 5. These past achievement assessments

were used as controls in some of the statistical analyses to adjust the traditional and CORI students.

Performance Assessment of Reading and Language Arts

In view of the association between motivation and cognitive variables in learning, Blumenfeld (1992), Corno and Kanfer (1993), and Meece et al. (1988) have used the term cognitive engagement to refer to the joint functioning of these factors. Consistent with this usage, we use the term reading engagement to refer collectively to the processes of motivation, cognitive strategy use, and conceptual knowledge during reading. In this study, we examined reading engagement in a complex, multitext, conceptual learning task. Strategies of searching, comprehending, and note taking were prominently required. The use of these strategies was highly correlated with intrinsic reading motivations in a previous study (Guthrie, Van Meter, et al., 1996), which is consistent with an extensive literature (Pintrich & Schrauben, 1992). Consequently, we did not measure motivation directly in this study, but we inferred student levels of reading engagement from performance on the strategy use task.

The performance assessment was designed to measure five aspects of reading engagement. The assessment was administered during a 1-week period in each traditional and CORI classroom by the teachers. All students took the assessment in the same 3-week period at the beginning of April 1995. Students worked approxi- mately I hr daily for 5 days. The assessment consisted of a learning context with seven tasks. The tasks are described here, and rubrics for coding performance are presented in Appendix B. The assess- ment was introduced with an observational activity. Students were given a colored picture of a pond with fish, insects, birds, and trees. With a partner, students discussed all the things they observed in the picture for 5 rain.

Prior conceptual knowledge. After viewing a picture, students worked independently to write their knowledge of the topic before entering the next phase. Students were asked to explain how ponds are different from deserts and were aided in the task by being asked, "What is a pond like? What is a desert like? How are they different?" All students were given sufficient time to finish their writing.

Strategy use. Students were next given an opportunity to search for ideas and information about ponds and deserts. Their task was to explain the differences between ponds and deserts. Students were given a 46-page booklet containing 14 sections with 1-5 pages of information in each. A table of contents, index, and glossary were provided. Four of the sections were not directly relevant to the question, and 10 were directly useful. Of the 10 relevant sections, 5 were appropriate for third-grade students, and 5 were more likely to be appropriate for fifth-grade students.

Students were given a log to fill out during the search activity. In the log, they were asked to present which section they selected, their reasons for choosing this information, and their notes on what they learned from the section. Students were given an unlimited amount of space to fill out their log. The search took place during a 1-hr period on 2 different days, allowing students as much time as they thought they needed to complete the activity. Students who finished early were permitted to rest or read a book of their choice.

New conceptual knowledge. First, students were asked to draw a picture to show how ponds are different from deserts. Students were asked to label the important parts and given ample time for the activity. This produced a drawing that was coded on the drawing rubric. After completing the drawing, students were given a writing task with the following instructions, "Write an explanation of how ponds are different from deserts." Asked to write about the important parts of ponds and deserts and use science ideas in their

CONCEPT-ORIENTED READING INSTRUCTION 267

explanations, students were given unlimited time for this activity. This produced a writing score that was scored on the writing rubric. After the writing task, students were given six specific questions to probe their knowledge of these biomes more deeply. The questions addressed particular animals, their habitats, and their adaptations to the habitat.

Conceptual transfer. Students were asked to perform a task that required them to transfer their knowledge from the previous learning activities. The instructions were as follows: "Suppose some people drink all of the water from a pond one day; would the pond now be like a desert? Please explain your answer." Students were not permitted to refer to their previous drawings, writings, or texts. They were given ample time to complete the activity and encouraged to think extensively as they performed the activity.

Text comprehension. Text comprehension tasks were designed to be similar to reading activities, which were prominent in the traditional classrooms. The objectives of story comprehension and informational text comprehension were emphasized in these class- rooms to be responsive to district guidelines. Two text types with accompanying questions were administered as the last part of the performance assessment. Although scores on the two tasks corre- lated at a low level (r = .16), consistent with previous research (Graesser et al., 1991), these tasks were combined to represent a general construct of text comprehension. Although we wanted to examine instructional effects on a robust measure of text compre- hension, we also analyzed the tasks separately.

For story comprehension, students were provided a narrative of approximately 1,000 words. An age-appropriate, authentic short story was given for each grade level. Students answered three questions, the first addressing recall of a particular event, the second addressing the author's perspective on one character, and the third addressing the theme of the literary selection. For informational text comprehension, students were given an exposi- tory text containing a prose, a diagram, and an illustration. They were given instructions to read this material and answer four questions. The first question required the students to integrate information across all three forms of information in the task. The next three questions asked students to address the illustration, the text, and the diagram, in that order. Space was given for each answer separately.

Performance Assessment Coding Rubrics

Coding rubrics were conslructed to classify the students' re- sponses on each task of the performance assessment (see Appendix B). To build the rubrics, we started with the writing task. We sorted the responses of 25 students holistically into six relatively higher and lower categories. Next, we identified the critical qualities of each level. The validity of the rubric was judged by comparing it with the rubric of Chi et al. (1994), which represented conceptual knowledge of the circulatory system. Like the Chi et al. rubric, higher levels of knowledge on our rubric contained more informa- tion about physical features, functions, relations among functions, and principled understanding than lower levels. This rubric was used for prior knowledge, drawing, and transfer.

A rubric for the search-comprehend-integrate task was con- structed in a similar manner to the writing rubric. Twenty-five student responses were sorted holistically by one rater into six equal-interval categories. The rater identified prototypes for each category and defined the characteristics of each category. A second rater attempted to sort these responses based on the definitions and the prototypes. Iterations of rating and discussing continued until the two raters concurred on definitions and prototypes.

To examine interrater agreement for each rubric, an independent graduate student rated 25 student responses according to the definitions and prototypes. The interrater agreements for adjacent

and exact coding into these categories respectively were as follows: prior knowledge (96% and 84%); search, comprehend, and inte- grate (92% and 84%); drawing (96% and 88%); writing (96% and 80%); and conceptual transfer (92% and 80%). Rubrics also were used for scoring text comprehension. Interrater agreements were as follows: informational text comprehension, 100% and 85%; narra- tive interpretation, 95% and 90% for third grade and 100% and 85% for fifth grade.

Measurement Characteristics of Constructs

The rubrics were designed to have equal-interval scales. We constructed the rubrics to have an equal moun t of knowledge difference reflected from Levels 2-3 as Levels 3-4, for example. Note that in absolute terms, no psychological scale is fully interval-to-ratio (Peterson, Kolen, & Hoover, 1989). That is, in a measure of history knowledge or intelligence, there is no true zero point, according to Snow and Lohraan (1989). Consequently, the numbers are not literally in a ratio (e.g., a 140 IQ is not twice the level of a 70 IQ in actual mental functioning). In our rubrics, we did not have a zero point. However, the intervals were judged to be equal by three types of professionals: a science education professor, a science teacher, and us. Consequently, the numbers generated from the rubrics were used in the path modeling analyses.

Constructs for analysis of reading engagement processes as represented in Figure 1 were built from the performance assess- ment. First, the construct of strategy use was derived from the score on the rubric for searching-comprehending-integrating. This score was the sum of three separate subscores: (a) number of relevant selections of material recorded in the log, (b) quality of the notes takes from the selections, and (c) the reasons for choosing the selections. These three parts of the search rubric correlated significantly, and they had a Cronbach's alpha reliability of .73.

The construct of conceptual knowledge included in Figure 1 was the sum of the scores on the drawing and writing rubrics. This construct had a Cronbach's alpha reliability of .65 based on these two scores. We subjected the measure of conceptual knowledge to an analysis of content validity. The science education professor working with the project determined that the materials available to the learner, the question, and the open-ended writing opportunity followed by the prompted response opportunities gave us a sensitive indicator of the depth of conceptual understanding about this science topic (Holliday, Alvermann, & Yore, 1994).

To measure conceptual knowledge with maximally sensitive procedures, we computed two scores. In addition to the general question, we asked students six follow-up questions after their writing and drawing activity. We constructed a prompted score by placing students on the writing rubric after accounting for the follow-up questions. Analysis of the prompted scores showed that students attained significantly higher rubric levels on prompted than on the original, general question, t(221) = 7.94,p < .0001. To examine whether the advantage varied by grade or instructional program, we conducted a 2 (grades) × 2 (types of instruction) analysis of variance on the difference of writing and prompted scores. No significant effects were observed. Therefore, we con- cluded that the prompts improved all scores irrespective of instructional condition or grade level. Therefore, the prediction of instruction and grade on prompted and original responses would not differ. As a result, we used the original writing scores in subsequent analyses because it was simpler and easier to replicate.

Past achievement was measured by teachers' ratings of the students' background in literacy and science at the beginning of the year. These ratings had a Cronbach's reliability of .82. The construct of prior knowledge was task specific, in contrast to the past achievement measure, which was global. One indicator of the

268 GtrrnRm ET AL

reliability of this measure was that it correlated .47 (p < .01) with the writing measure. The prior knowledge question and writing question were the same: "What is the difference between a pond and a desert?" Because only the search-comprehend-integrate task intervened, this correlation is like a test-retest reliability indicator for the prior knowledge test.

Conceptual transfer was a construct intended to indicate whether the knowledge gained in the search-comprehend-integrate activity was usable in a different situation. Because the task contained one part, we did not have a measure of its reliability. However, the conceptual knowledge measure correlated with conceptual transfer at .45 (p < .01), which suggested that the measure cannot be extremely unreliable.

Procedures

Students in the CORI classrooms received a full academic year of this teaching, and the comparison students received traditional instruction throughout the year. Teachers had full responsibility for all instruction and participated in a monthly workshop led by the university researcher to share progress and address problems in implementing the principles. The performance assessment was given by the teachers in the first half of April. Traditional students performed comfortably on the performance assessment, finding it to be an enjoyable activity. Coding of the performance assessment was completed by graduate students and one of us.

Results

Summary data are presented in Table 1. The instructional column contains information for traditional and CORI conditions at Grades 3 and 5. The reading engagement processes consisted of strategy use, conceptual knowledge, and conceptual transfer. The student background variables consisted of past achievement and prior knowledge. It is evident that CORI students at Grade 3 were higher in strategy use (M = 4.69) than traditional students at Grade 3 (M = 3.64) and that CORI students at Grade 5 were higher in strategy use (M = 5.40) than traditional students at Grade 5 (M = 4.00). A similar pattern appeared for new conceptual knowledge. For prior conceptual knowledge, CORI students at both grades were higher than traditional students, making the use of prior conceptual knowledge an important control in the path model. Past achievement was mixed, with traditional Grade 3 being higher than CORI Grade 3 but CORI Grade 5 being higher than traditional Grade 5. The advantage of CORI on text comprehension is apparent. Generally, the individual components of these main con- structs (e.g., the relevance of selections, reasons for selec- tions, and quality of notes) followed the pattern of the tested constructs.

The correlations of the variables are presented in Table 2. The variables of strategy use, conceptual knowledge, concep- tual transfer, past achievement, and prior knowledge were coded such that a higher score represented a higher perfor- mance by the student on the construct. Instruction was coded 1 for CORI and 0 for traditional reading instruction. With this coding, a positive correlation of .44 (p < .01), as indicated in Table 2, showed that students receiving CORI attained higher scores in strategy use than students receiving traditional instruction in this zero-order correlation. The

construct of grade was coded 1 for Grade 5 and 0 for Grade 3. In Table 2, the correlation of .18 (p < .05) indicated that students in Grade 5 received higher scores on strategy use than students in Grade 3 in the zero-order correlation. The Instruction × Grade interaction term, its coding, and its relationship to other variables are described later in the Results section. From Table 2, it is apparent that a construct such as strategy use correlated positively with both past achievement, which was .28 (p < .01), and prior knowledge at .20 (p < .05), indicating that it is important to control for these variables in an analysis of the effects of educational conditions such as instruction and grade level on strategy use. We retained both of these variables in the path model because they correlated moderately (r = .23, p < .05) with each other and it did not seem sensible to collapse them together. Grade did not correlate with past achievement partly because achievement was a within-grades rating. Grade did not correlate with prior conceptual knowledge partly because all students were relatively low at a rubric level of 2.0-2.5 on a rubric scale of 1--6. Furthermore, grade did not correlate with new conceptual knowledge. This may be attributable, however, to the Grade × Instruction interac- tion, in which CORI students at Grade 3 were high in new conceptual knowledge.

A path analysis was conducted to analyze the predictions of instruction and grade level on reading engagement processes including strategy use, conceptual knowledge, and conceptual transfer. In this context, structural equation modeling provided a full examination of the instructional, grade, and student characteristics related to reading engage- ment processes. We expected that the instructional treatment of CORI would increase strategy use among students and that strategy use would increase new conceptual knowledge acquisition. In addition, we wanted to examine the extent to which the instructional treatment of CORI increased new conceptual knowledge acquisition directly, irrespective of its effect on strategy use. We expected that students in this treatment might learn to form relational concepts and principled understanding that are valuable for conceptual knowledge apart from the strategies that they learned. Likewise for conceptual transfer, we wanted to examine the extent to which instruction increased transfer both indirectly through the strategy use and conceptual knowledge paths as well as directly without mediation. We wanted to control both of these instructional effects for past achievement and prior knowledge. Furthermore, we wanted to examine whether the relationships would be the same at different grade levels and a possible Grade X Instruction interaction. Structural equation modeling is capable of assimilating all of these issues into one analysis. The alternative of a multivari- ate analysis of variance (MANOVA) does not permit simultaneous examination of all of these relationships. Although several MANOVAs could have been conducted to address most of the issues in this study, we expected that structural equation modeling would represent our data and theoretical formulations the most comprehensively.

The structural equation modeling package EQS (Bentler, 1995) was used to analyze the covariance matrix of the variables in Table 2, and missing data were handled through

CONCEPT-ORIENTED READING INSTRUCTION 269

Table 1 Means and Standard Deviations for Instruction, Grade, Student Background, and Reading Engagement Processes

Traditional instruction

Variable Grade 3 Grade 5 Grade 3

CORI

Grade 5

Strategy use M 3.64 4.00 4.69 5.40 SD 1.39 1.29 0.99 1.10

Relevance M 3.93 4.31 5.46 5.48 SD 1.39 1.44 1.50 1.52

Reasons M 4.10 4.38 4.71 4.65 SD 1.27 0.94 0.92 1.10

Notes M 3.88 4.24 4.75 5.60 SD 1.27 1.27 1.00 1.01

New conceptual knowledge M 4.92 5.32 6.18 7.57 SD 2.05 1.72 2.19 2.08

Draw M 2.53 2.53 2.95 3.54 SD 1.16 0.99 1.38 1.17

Write M 2.39 2.80 3.23 4.03 SD 1.26 1.02 1.22 1.34

Text comprehension M 13.73 13.61 13.52 14.61 SD 2.45 1.98 2.77 2.67

Story comprehension M 6.46 4.67 6.00 5.90 SD 1.41 1.44 1.62 1.61

Informational text comprehension M 7.02 8.65 7.42 8.71 SD 1.87 1.51 1.54 1.47

Past achievement M 6.51 6.65 5.77 7.06 SD 2.60 1.87 2.56 2.56

Literacy level M 3.23 3.35 2.87 3.31 SD 1.49 0.89 1.20 1.49

Science level M 3.28 3.30 2.90 3.75 SD 1.32 1.18 1.55 1.27

Prior conceptual knowledge M 2.03 2.12 2.57 2.53 SD 0.77 0.77 1.06 0.85

Note. CORI = Concept-Oriented Reading Instruction.

pairwise deletion. Maximum-likelihood estimation was used to fit the path model, which rests on the assumption of conditional multivariate normality. Skewness and kurtosis values were inspected for all variables modeled, showing no extreme departures from univariate normality. Although univariate normality does not ensure multivariate normality, maximum-likelihood estimation has been shown to yield reliable parameter estimates when univariate normality is present (Bollen, 1989).

To examine the hypotheses of the study, we computed all possible direct and indirect effects (see Table 3). The statistical significance of these effects was evaluated unilat-

erally using an alpha level of .05; total effects, which are the sum of direct and indirect effects, do not have significance levels inferrable from EQS output. Note that goodness-of-fit statistics, commonly accompanying structural equation mod- els, do not appear in our analyses. Just as is the case in multiple regression models, our path model was just identi- fied (i.e., it had as many parameters to estimate as there were unique values in the covariance matrix); thus, the model had 0 dfs, rendering perfect goodness of fit by default. Because the purpose was not to compare models but to examine paths within a model using maximum controls, the just-identified structural equation model was considered appropriate.

270 GUTHRIE ET AL.

Table 2 Correlations of lnstruction, Grade, Student Background, and Reading Engagement Variables

Variable I 2 3 4 5 6 7 8

1. Strategy use 2. Conceptual knowledge .45** - - 3. Conceptual transfer .34** .45* - - 4. Past achievement .28** .39** .30** - - 5. Prior knowledge .20* .46** .29** .23* 6. Instruction .44** .38** .21" - .04 7. Grade .18" .17 .24* .15 8. Instruction × Grade .39** .39** .43** .13

.26** - . 0 1 - . 0 2 - -

.13 .56** .58**

*p < .05. **p < .01.

All the study's hypotheses can be discussed in terms of the paths (1-18) in Figure 1, as indicated in the introduction. Our first hypothesis was that instruction would have a positive effect on strategy use, represented by Path 1 in Figure 1. As can be seen in Table 3, instruction had a direct effect on strategy use, with a statistically significant path coefficient of .748. This path value is a standardized partial regression weight, indicating that instruction was highly predictive of strategy use even when the associations of strategy use with grade level, Grade x Instruction interac- tion, past achievement, and prior knowledge were con- trolled. Specifically, the CORI group was still higher in strategy use than the traditional instruction group when accounting for potential differences on the other predictor

variables. These data appear to be consistent with the study's first hypothesis, which was that instruction would predict strategy use when the variables of grade, Instruction x Grade interaction, past achievement, and prior knowledge were controlled. However, there was a significant Instruc- tion X Grade interaction requiring us to qualify the conclu- sion, which is presented later.

Our second hypothesis was that instruction would have an indirect effect on conceptual knowledge. Using Figure 1, we expected that instruction would predict strategy use (Path 1) and that strategy use would predict conceptual knowledge (Path 16). As Table 3 indicates, this indirect path coefficient of instruction on conceptual knowledge was .135 (p < .05). This path coefficient represents the product of Path 1 and

Table 3

Path Coefficients for Model of Reading Engagement Processes

Effects variable Direct effects Path no. Indirect effects Total effects

Strategy use

Instruction .748* 1 - - .748 Grade .487* 4 - - .487 Instruction × Grade -.346* 7 - - - .346 Past achievement .287* 10 - - .287 Prior conceptual knowledge -.021 13 - - -.021

New conceptual knowledge

Instruction .226 2 .135" .361 Grade .117 5 .088* .205 Instruction × Grade .053 8 -.062 -.009 Past achievement .255" 11 .052* .306 Prior conceptual knowledge .303* 14 - .004 .299 Strategy use .180" 16 - - .180

Conceptual transfer

Instruction - .409" 3 .209* -.201 Grade -.295 6 .129* - . 166 Instruction × Grade .646* 9 - .061" .585 Past achievement .072 12 .118 * .190 Prior conceptual knowledge .156" 15 .065 * .221 Strategy use .169* 18 .041 .210 New conceptual knowledge .228* 17 - - .228

Note. Path numbers are referenced to Figure 1. Dashes indicate that no indirect route was possible in the path analysis. *p < .05.

CONCEPT-ORIENTED READING INSTRUCTION 271

Path 16. That is, in Figure 1, Path I (.748) multiplied by Path 16 (.180) equaled .135. A way of stating this relationship is to say that the effect of instruction on conceptual knowledge was mediated by strategy use. Therefore, the second hypoth- esis that instruction would predict conceptual knowledge indirectly through strategy use was consistent with the data.

The direct effect of instruction on conceptual knowledge is represented by Path 2 in Figure 1, with a path coefficient of .23 (ns). This effect was assessed while controlling for grade, Instruction × Grade interaction, past achievement, and prior knowledge and included a control for the mediated effect of instruction on conceptual knowledge through strategy use. Although instruction had a significant zero- order correlation of .38 (p < .01) with conceptual knowl- edge, the direct effect was not significant when controlling for other variables in the model.

The third hypothesis was that grade level would directly predict strategy use, conceptual knowledge, and conceptual transfer when the variables of instruction, Instruction × Grade interaction, past achievement, and prior knowledge were accounted for. As shown in Table 3, grade level had a significant direct effect on strategy use (Path 4), with a path coefficient of .487 (p < .05). Grade level did not, however, directly predict conceptual knowledge. Grade level showed a negative direct effect on conceptual transfer (Path 6), with a path coefficient of - .295 (p < .05). In all these path coeffÉcients, the variables of instruction, Instruction × Grade interaction, past achievement, prior knowledge, strat- egy use, and conceptual knowledge were controlled. There- fore, the evaluation of the third hypothesis was mixed. With other variables in the model controlled, grade level had a positive direct effect on strategy use, but grade did not predict conceptual knowledge, and grade had a negative direct effect on conceptual transfer. Note that grade had a significant indirect effect on conceptual knowledge, with a path coefficient of .088 (p < .05), and a significant indirect effect on conceptual transfer, with a path coefficient of . 129 (p < .05).

The fourth hypothesis was that instruction would have an indirect effect on conceptual transfer through conceptual knowledge and strategy use. This hypothesis can be dis- cussed with reference to Figure 1. We expected that instruc- tion would predict strategy use (Path 1), that strategy use would predict conceptual knowledge (Path 16), and that conceptual knowledge would predict conceptual transfer (Path 17). This is one set of paths from instruction to conceptual transfer. There is a second set of paths that consists of instruction predicting strategy use (Path 1) and strategy use predicting conceptual transfer (Path 18). A third indirect set of paths consists of instruction predicting conceptual knowledge (Path 1) and conceptual knowledge predicting conceptual transfer (Path 17). Each of these three sets of paths is an indirect route from instruction to conceptual transfer. The hypothesis of indirect prediction in the EQS path modeling program sums these three indirect routes and evaluates the significance of the combination of all three sets. As shown in Table 3, the indirect effect of instruction on conceptual transfer was .209 (p < .05). This indicated that compared with traditional instruction, CORI

increased conceptual transfer by increasing strategy use and conceptual knowledge, which was consistent with the fourth hypothesis.

Although the data were consistent with the first hypoth- esis, that instruction would predict strategy use, we also tested the effect of the Instruction × Grade interaction on strategy use. As Table 3 indicates, this effect was significant at - .346 (p < .05). To examine the nature of this interac- tion, we plotted the data and illustrated them in Figure 2. The data in the plot were based on the standardized regression coefficients generated by EQS, representing the relative levels of strategy use for Grade 3 and Grade 5 students who received CORI and traditional instruction. These relative cell means were based on the standardized path coefficients for total effects and the coding scheme for instruction, grade, and the Grade X Instruction interaction. As Figure 2 indicates, CORI students were higher than traditional stu- dents at both Grade 3 and Grade 5, but the advantage for CORI students was more pronounced at Grade 3 than at Grade 5. In other words, the positive effect of instruction on strategy use was present for both grade levels, but it was stronger for Grade 3 than Grade 5. This suggests an important qualification that the effect of instruction on strategy use was stronger for Grade 3 than Grade 5 students.

As Table 3 indicates, the direct effect of the Instruction X Grade interaction on conceptual transfer was significant, with a path coefficient of .646 (p < .05). To examine this interaction, we plotted the relative levels of conceptual transfer for students in CORI and traditional instruction at Grades 3 and 5 using standardized path coefficients for total effects from Table 1 in Figure 3. Note that some of the coefficients are negative, and a constant of .21 was added to generate a more readable figure with positive numbers. As Figure 3 shows, the total effects of instruction on conceptual transfer were positive for Grade 5 and negative for Grade 3. In conceptual transfer, CORI students were higher than

(D

v

oo

1.2

1.0

.8

.6

.4

.2

.0

CORI x - - - x Traditional o

o . . X . . o - ° X o . ° o ° /"

Grade 3 Grade 5

Figure 2. Total effects of instruction and grade on strategy use. CORI = Concept-Oriented Reading Instruction.

272 GtrrnRIE ET AL.

0.6

0.5 ._>

"-" 0.4

e -

~ 0.3 I--

~ 0.2 t -

O

0 0.1

0.0

CORI x---x Traditional o-

X

0" Y I - - I

Grade 3 Grade 5

Figure 3. Total effects of instruction and grade on conceptual transfer. CORI = Concept-Oriented Reading Instruction.

traditional students in Grade 5, but traditional students were higher in Grade 3. In addition, it carl be noted in Table 3 that the direct effect of instruction on conceptual transfer was negative at - .409 (p < .05). On conceptual transfer, tradi- tional students were higher than CORI students when the variables of grade, Instruction X Grade interaction, past achievement, prior knowledge, strategy use, and conceptual knowledge were accounted for. Another way of stating this is that when we took account of the relatively high levels of CORI students on strategy use and conceptual knowledge, the advantage of Grade 3 CORI students on conceptual transfer was not only eliminated but reversed. These findings do not alter the support for the fourth hypothesis, which was that instruction would indirectly predict conceptual transfer through strategy use and conceptual knowledge constructs.

To examine whether there would be a difference between CORI and traditional students on text comprehension, we conducted analyses of covariance (ANCOVAs) separate from the path model. In the first analysis, the dependent variable was text comprehension, which consisted of the scores on the narrative and informational tasks combined. The ANCOVA was a 2 (instructional conditions) × 2 (grade levels), with past achievement as the covariate. There was a significant effect for instruction, F(1,121) = 5.24, p < .02, indicating that CORI students were higher than traditional students. The interaction between grade and instruction was not statistically significant. There was no grade effect, which was expected because the texts were selected to be develop- mentally appropriate for third and fifth graders. Conse- quently, the grade comparison was not statistically meaning- ful, although the instructional comparison and the test of the interaction were valid. This result shows that the CORI students were higher than traditional students on a conven- tional measure of text comprehension.

As the narrative and informational comprehension tasks were not highly correlated, we analyzed them separately with 2 (instruction) × 2 (grade) ANCOVAs, with past achievement as the covariate. For narrative, instruction had a significant effect, F(1, 131) = 5.00, p < .03. There was a significant interaction between instruction and grade, F(1, 131) = 9.50, p < .002, indicating that the CORI students were higher than traditional students in Grade 5 but that the traditional students were slightly higher than CORI students in Grade 3. For the information task, there were no significant effects. However, trends on the information task favored the CORI group, which may have contributed to the significant instruction effect (p < .02) for the combined text comprehension measure.

Discussion

In this study, we formulated a model of the effects of instructional conditions and student background characteris- tics on reading engagement processes including strategy use, new conceptual knowledge, and conceptual transfer. The model was intended as a framework for examining the specific hypotheses of the study, which related primarily to the effects of instructional conditions on the reading engage- ment variables. The path analyses yielded three important findings. First, the effect of instruction on strategy use was significant. Elementary school children who received CORI were more likely to learn and use strategies for gaining knowledge from multiple texts than students in traditional instruction that included a basal reader supplemented by children's literature. Children demonstrated their strategies in a performance assessment in which they searched mul- tiple texts for information, read informational texts, dia- grams, and illustrations, and took notes from the texts to answer a broad conceptual question. We observed an interaction between instruction and grade level, which indicated that although CORI was beneficial for students at both grade levels, the advantage was more apparent for third graders.

These findings are particularly substantive in view of the fact that we controlled past achievement in reading and language arts and science as well as prior domain knowledge on the content of the performance assessment that was used to measure strategy use. Although other studies have shogen the benefit of explicit instruction on strategy learning (see Hiebert & Raphael, 1996, for a review), this study added important controls for past achievement and prior knowl- edge in the content domain in which the strategies were displayed. In addition, the strategies required to succeed in this performance assessment included searching multiple books, comprehending a range of different difficulty levels, taking notes, integrating information, and performing the processes of complex search (Guthrie et al., 1993; Mosen- thai, 1996), which have not been extensively examined in instructional research.

The second finding was that CORI conditions had a positive effect on conceptual knowledge through the mediat-

CONCEPT-ORIENTED READING INSTRUCTION 273

ing variable of strategy use. In other words, this instruction increased students' ability to use a range of strategies, and students who were more adept in using these strategies gained more conceptual knowledge in the performance assessment than students who were less proficient in the strategies. The instructional conditions (e.g., the contrast between CORI and traditional instruction) did not have a direct effect on conceptual knowledge. The direct route from instruction to knowledge was not significant. However, the indirect route from instruction through strategy use to conceptual knowledge was significant. This finding occurred when past achievement and prior knowledge were statisti- cally controlled. Because the measure of prior knowledge was an indicator of conceptual understanding in the content domain of the performance assessment, the benefit of instruction can be regarded as a benefit for conceptual learning. The dependent measure of knowledge was con- trolled for prior knowledge levels, making the outcome variable a conceptual learning construct. It is reasonable to say that CORI facilitated the students in the processes of conceptual learning through the use of a set of cognitive strategies for understanding multiple texts.

Our third finding was that the instructional conditions of CORI had a positive, indirect effect on conceptual transfer. The measure of conceptual transfer consisted of a problem- solving task that followed the children's expression of their conceptual knowledge through drawing and writing. This effect, however, was not direct, but it was indirect through strategy use and conceptual knowledge. The path of most importance was from instruction to strategy use to concep- tual knowledge to conceptual transfer. Students who were given CORI instruction were more likely than traditional students to gain strategies of searching, comprehending, and integrating information from multiple texts. Students with these highly developed strategies were likely to form conceptual knowledge that included physical features, rela- tions among features, and high-level principles. Finally, students with these advanced forms of conceptual understand- ing, including specific information, relational information, and principled understandings, were most likely to solve the conceptual transfer task problem. In other words, instruction did not influence transfer directly, but it did promote transfer indirectly by enabling students to acquire productive strate- gies that enabled them to build a strong conceptual under- standing that was useful in the transfer task. The direct effect of instruction on transfer interacted with grade level such that CORI was higher than traditional Grade 3 but that traditional was higher than CORI for Grade 5. This interac- tion is reflected in the total effects in Figure 3, but it does not contradict the previous interpretation of indirect effects.

We concur with Reynolds, Sinatra, and Jetton (1996) that strategy learning is situated and that rich contexts will facilitate strategy acquisition and use. We have attempted to explicate a productive classroom context by describing seven design principles: (a) conceptual themes, (b) real- world interaction, (c) self-direction, (d) collaboration, (e) strategy instruction, (f) self-expression, and (g) coherence.

Evolved from classroom-based collaborative inquiry (Guth- tie, Van Meter, et al., 1996), these principles are consistent with the theoretical literature (Guthrie & Alao, 1997), but they must be maintained in a delicate balance (Guthrie et al., in press). We have not attempted to quantify these principles or to attribute the acquisition of reading engagement pro- cesses to any one of them. Rather, we suggest that they are a systemic, dynamic collection of classroom attributes.

Instructional interventions for reading engagement (e.g., the joint functioning of strategies, knowledge, and motiva- tions in reading) have been examined in several recent studies. These investigations include the following: the report by Au and Asam (1996) of instruction to increase ownership of literacy; the discussion by Brown (1997) of learning communities to support the growth of transferable knowledge; the experiment of Morrow, Pressley, Smith, and Smith (1997) to increase science knowledge and reading motivation; the curriculum of Calfee (1994) organized around a set of instructional principles; the study by Collins (1993) on teaching thinking within a reading framework; and Bruning and Schweiger's (1997) account of integrated instruction for reading, writing, and inquiry.

Because of the complexity of the instruction, the measure- ment issues, and challenges of experimental or statistical control for extraneous variables, these researchers have examined different aspects of the problem. In this limited space, we cannot represent the studies fully, but we suggest that the studies variably contain the following characteris- tics: (a) long-term intervention, lasting I year or more (Au & Asam, 1996; Brown, 1997; Calfee, 1994); (b) mainstream curriculum implementation (Au & Asam, 1996; Calfee, 1994); (c) performance assessment of strategy use and knowledge acquisition from text, beyond self-report data (Au & Asam, 1996; Brown, 1997; Brtming & Schweiger, 1997; Calfee, 1994; Morrow et al., 1997); (d) comparison of experimental intervention to an alternative form of instruc- tion (Collins, 1993; Morrow et al., 1997); (e) preintervention assessments, or pretests as controls (Morrow et al., 1997); and (f) positive instructional effects on reading engagement- related variables (Au & Asam, 1996; Brown, 1997; Bruning & Schweiger, 1997; Collins, 1993; Morrow et al., 1997). Although each of these studies contains several of the characteristics, none includes all of them.

In this study we incorporated all the characteristics of instructional research that were present collectively in the previous studies. We built on these previous investigations by including all of the listed characteristics into our design, data collection, and report. The findings of this investigation suggest that long-term strategy instruction was effective in a public school setting. Made on performance tasks that demand the motivated use of reading strategies and prior conceptual knowledge, the comparisons in our study suggest that instruction can increase the processes of reading engage- ment in natural classroom situations. This conclusion contrib- utes to the growing knowledge base that integrated reading instruction formed around principles, such as those in CORI, has verifiable benefits for students' acquisition of reading

274 GUTHRIE ET AL.

strategies and conceptual knowledge acquisition capabili- ties. However, there are limitations to this study that indicate that this conclusion must be viewed cautiously and points to further research.

Several limitations to this study should be noted. First, we did not attempt to compare this set of principles (CORI) with a different set, such as those proposed by Ames (1992) or Stipek (1996). Rather, we attempted to explore the multiple effects of one complex set of instructional principles. Constraints on the inferences of such a study should be noted. The evidence reported here indicates that students in one set of classrooms (CORI) showed higher literacy engagement and conceptual learning than students in an- other set of classrooms (traditional). Similar to other research- ers (see Purcell-Gates, Mclntyre, & Freppon, 1995; White- hurst et al., 1994), we asked the empirical question of whether the set of intervention characteristics appeared to increase various aspects of student development. Purcell- Gates et al. (1995) noted that their literature-based class- rooms differed from skills-based classrooms in the amount of reading aloud, book exploration, literature discussion, self-chosen writing, and use of narrative texts. Whitehurst et al. (1994) reported an early literacy intervention that in- eluded small-group reading, tutoring, video training for teachers, altered books, sound training, worksheets, and extension activities. Similar to these authors, we attribute our effects to a composite of the instructional components (in our case, the seven principles), and the texts. Like Whitehurst et al. (1994), our teachers were comparable in age, experience, and teaching expertise. Also like White- hurst et al., we used covariates to remove initial differences as sources of the intervention effect. We attribute the higher level of literacy engagement in our results to a composite that includes the CORI principles and the texts used in the classrooms. We have not yet examined which components of this composite are most influential. Almost certainly, the principles operate as a system (Salomon, 1991) and this system warrants further research. The system, however, is not indeterminate. Classrooms characterized by these prin- ciples can be reliably identified (from teacher questionnaires as evidenced in this study) and the effects of these contexts on students' reading engagement and conceptual learning are predictable. Consequently, the results of this study suggest an important role for this contextual composite in increasing engagement in reading. They also suggest that further theoretical and empirical explication of these relation- ships are warranted.

A second limitation is that the performance assessment and text comprehension measures were experimenter con- structed. It might be argued that the measures of strategy use, conceptual knowledge, and conceptual transfer as dependent variables in the model were highly similar to the learning activities being provided in the CORI classrooms. To a certain degree, this is true, and to examine whether the benefit of CORI extended beyond these reading engagement variables, we examined text comprehension of CORI and traditional students. Consisting of reading an authentic story

and answering both reproductive and reflective questions, the narrative interpretation task was strongly emphasized in the traditional classrooms. To form a text comprehension construct, the narrative task was combined with a conven- tional informational text comprehension task in which students read and responded to a free-recall question. CORI students exceeded traditional students on this text compre- hension measure, showing that the reading strategies were not limited to those of the performance assessment, which resembled the activities in the CORI classrooms.

A third limitation is that the path model was exploratory. We did not attempt to compare different structural equation models to identify a best fitting model because we did not have serious alternative theories as a rationale for compari- son. Rather, we explored the significance of several paths and sets of paths within one model (see Figure 1). We conducted within-models hypothesis testing rather than across-models theory evaluation. The number of participants in the study was marginal but acceptable for the path analysis we reported. The students in the study were members of intact classes. Although two measures were used for statistical control, the group membership of partici- pants could decrease the within-groups variance to an unknown degree. Furthermore, the measures of constructs were designed for this investigation. The degree to which they represent true interval scales was not established, and their measurement properties have not been examined with studies of multimethod-multitrait scaling and generalizabil- ity (Cronbach, Linn, Brennan, & Haertle, 1977), which would be desirable. The optimal procedure to address these limitations is replication, which is needed.

Fourth, in this analysis we have used the word effect to refer to the contribution of instructional and background variables to reading engagement processes. Although causal attributions should not be exaggerated in this study, the term effect seems justified because the instruction was a con- structed (manipulated) variable and grade level was a selection (agelike) variable. Past achievement was measured in the fall, whereas the performance assessment was con- ducted in the spring. Prior knowledge was measured at the outset of the performance assessment before students had an opportunity to read and gain conceptual knowledge. Conse- quently, all these variables were antecedent to the occur- rence of the reading engagement processes. It seems reason- able to make cautious causal attributions to these variables. During the performance assessment, students read from multiple books for approximately 2 days and the strategy- use measure was based on this learning activity. After that activity, students participated in a drawing and writing task to display their knowledge. Because the strategy use was antecedent to the writing and drawing, it is plausible to make causal attributions from performance on the writing and drawing tasks to performance on the task in which strategy use was measured. In addition, the conceptual transfer task was given last, and it is reasonable that students who were able to write and draw about their conceptual understanding would have stated knowledge in a form that it could be used

CONCEPT-ORIENTED READING INSTRUCTION 275

in the transfer activity. Consequently, it seems plausible to make an attribution of causal influence from the success on the drawing and writing task, which measured conceptual knowledge, to success on the conceptual transfer task.

Future investigations should address each of these limita- tions. Furthermore, studies of integrated instruction for reading engagement should address the issue of measuring and describing the context of instruction. The principles of CORI, as well as other sets, could be quantified and examined as dimensions of context. Researchers could attempt to isolate the effects of each principle. However, the principles are probably synergistic. Consequently, inter- actions among p'rmciples should be captured in future investigations. Becausereading engagement is a system of components (e.g., motivational, cognitive, and conceptual aspects of reading), it is reasonable that a system of contextual components will be needed to increase reading engagement in predictable ways. Mapping the network of relations between these two systems of reading engagement and instructional context seems to be a worthy research challenge.

References

Alexander, E A. (1992). Domain knowledge: Evolving themes and emerging concerns. Educational Psychologist, 27, 33-51.

Alexander, E A., Jetton, T. L., & Kulikowich, J. M. (1996). Interrelationship of knowledge, interest, and recall: Assessing a model of domain learning. Journal of Educational Psychology, 87, 559-575.

Alexander, E A., Schallert, D. L., & Hare, V. C. (1991). Coming to terms: How researchers in learning and literacy talk about knowledge. Review of Educational Research, 61, 315-343.

Almasi, J. E (1995). The nature of fourth graders' sociocognitive conflicts in peer-led and teacher-led discussions of literature. Reading Research Quarterly, 30, 314-351.

Ames, C. (1992). Achievement goals and the classroom motiva- tional climate. In D. Schunk & L Meece (Eds.), Student perceptions in the classroom (pp. 327-349). Hillsdale, NJ: Erlbanm.

Anderson, R. C., & Pearson, P. D. (1984). A schema-theoretic view of basic processes in reading. In R. Barr, M. L. Kamil, & E Mosenthal (Eds.), Handbook of reading research (pp. 255-291). New York: Longman.

Au, K. H., & Asam, C. L. (1996). Improving the literacy achievement of low-income students of diverse backgrounds. In M. E Graves, E van den Brock, & B. M. Taylor (Eds.), Thefirst R: Every chiM's right to read (pp. 199-223). New York: Teachers College Press.

Baker, L., & Brown, A. L. (1984). Metacognitive skills and reading. In R. Barr, M. L. Kamil, & E Mosenthal (Eds.), Handbook of reading research (pp. 353-394). New York: Longman.

Beane, J, A. (1995). Curriculum integration and the disciplines of knowledge. Phi Delta Kappan, 76, 616-622.

Belenky, M., Clinchy, B., Goldberger, N., & Tarule, L (1986). Women's ways of knowing: The development of self, voice, and mind. New York: Basic Books.

Bentler, P. M. (1995). EQS structural equations program manual. Encino, CA: Multivariate Software.

Blumenfeld, E C. (1992). Classroom learning and motivation: Clarifying and expanding goal theory. Journal of Educational Psychology, 84, 272-281.

Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.

Brophy, J., & Alleman, J. (1991). Activities as instructional tools: A framework for instructional analysis and evaluation. Educa- tional Researcher, 20, 9-23.

Brown, A. L. (1997). Transforming schools into communities of thinking and learning about serious matters. American Psycholo- gist, 52, 399-414.

Bruning, R., & Schweiger, B. (1997). Integrating science and literacy experiences to motivate student learning. In L T. Guthrie & A. Wigfield (Eds.), Reading engagement: Motivating readers through integrated instruction (pp. 149--168). Newark, DE: International Reading Association.

Calfee, R. (1994). Critical literacy: Reading and writing for a new millennium. In N. J. Ellsworth, C. N. Hedley, & A. N. Baratta (Eds.), Literacy: A redefinition (pp. 19-39). Hillsdale, NJ: Edbaum.

Chi, M. T. H., DeLeeuw, N., Chin, M., & Lavancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439-477.

Collins, C. (1993). Strategy instruction in a literature-based reading program. Elementary School Journal, 94, 139-151.

Corno, L., & Kanfer, R. (1993). The role of volition in learning and performance. In L. Darling-Hammond (Ed.), Review of research in education (pp. 301-341). Washington, DC: American Educa- tional Research Association.

Cronbach, L. J., Lima, R. L., Brennan, R. L., & Haertle, E. H. (1977). Generalizability analysis for performance assessments of student achievement or school effectiveness. Educational and Psychological Measurement, 57, 373-399.

Deci, E. L., Vallerand, R. J., Pelletier, L. G., & Ryan, R. M. (1991). Motivation and education: The self-determination perspective. Educational Psychologist, 26, 325-346.

Dole, J. A., Duffy, G. G., Roehier, L. R., & Pearson, P. D. (1991). Moving from the old to the new: Research on reading comprehen- sion instruction. Review of Educational Research, 61, 239-264.

Flower, L., Stein, V., Ackerman, J., Kantz, M. J., McCormick, K., & Peck, W. C. (Eds.). (1990). Reading to write: Exploring a cognitive and social process. New York: Oxford University Press.

Gottfried, A. E. (1985). Academic intrinsic motivation in elemen- tary and junior high school students. Journal of Educational Psychology, 77, 631-645.

Graesser, A., Golding, L M., & Long, D. L. (1991). Narrative representation and comprehension. In R. Barr, M. L. Kamil, E Mosenthal, & E D. Pearson (Eds.), Handbook of reading research (Vol. 2, pp. 171-205). New York: Longman.

Grolnick, W. S., & Ryan, R. M. (1987). Autonomy in children's learning: An experimental and individual difference investiga- tion. Journal of Personality and Social Psychology, 52, 273- 288.

Guthrie, J. T., & Alao, S. (1997). Designing contexts to increase motivations for reading. Educational Psychologist, 32, 95-107.

Guthrie, J. T., Cox, K., Anderson, E., Harris, K., Mazzoni, S., & Rach, L. (in press). Principles of integrated instruction for engagement in reading. Educational Psychology Review.

Guthrie, J. T., & McCann, A. D. (1996). Idea circles: Peer collaborations for conceptual learning. In L. Gamhrell & J. Almasi (Eds.), Lively discussions! (pp. 87-106). Newark, DE: International Reading Association.

276 GtrrnmE ~ Ju~.

Guthrie, J. T., McGough, K., Bennett, L., & Rice, M. E, (1996). Concept-Oriented Reading Instruction: An integrated curriculum to develop motivations and strategies for reading, In L. Baker, P. Afflerbach, & D. Reinking (Eds.), Developing engaged readers in school and home communities (pp. 165-190), Hillsdale, NJ: Erlbaum.

Guthrie, J. T., Van Meter, P., McCann, A., Wigfield, A., Bennett, L., Poundstone, C., Rice, M. E., Fabisch, E, Hunt, B., & Mitchell, A. (1996). Growth of literacy engagement: Changes in motiva- tions and strategies during Concept-Oriented Reading Instruc- tion. Reading Research Quarterly, 31, 306-332.

Guthrie, J. T., Weber, S., & Kimmerly, N. (1993). Searching documents: Cognitive processes and deficits in understanding graphs, tables, and illustrations. Contemporary Educational Psychology, 18, 186-221.

Harris, K. R., & Graham, S. (1992). Self-regulated strategy development: A part of the writing process. In M. Pressley, K. R. Harris, & J. T. Guthrie (Eds.), Promoting academic competence and literacy in school (pp. 277-309). New York: Academic Press.

Harris, K. R., & Graham, S. (Eds.). (1996). Making the writing process work: Strategies for composition and self-regulation. Cambridge, MA: Brookline Books.

Hartman, D. K. (1995). Eight readers reading: The intertextual links of proficient readers reading multiple passages. Reading Research Quarterly, 30, 520--561.

Hiebert, E., & Raphael, T. (1996). Psychological perspectives on literacy and extensions to educational practice. In D. C. Berliner & R. C. Calfee (Eds.), Handbook ofeducationalpsychology (pp. 85-113). New York: Simon & Schuster.

Holliday, W. G., Alvermann, D. E., & Yore, L. (1994). The reading-science learning-writing connection: Breakthroughs, bar- riers, and promises. Journal of Research in Science Teaching, 31, 877-893.

Johnson, D. W., Johnson, R. T., & Stanne, M. B. (1989). Impact of goal and resource interdependence on problem-solving success. Journal of Social Psychology, 129, 621-629.

Linn, M. C., & Mnilenburg, L. (1996). Creating lifelong science learners: What models form a finn foundation? Educational Researcher, 25, 18-24.

Lipson, M. Y., Valencia, S. W., Wixson, K. K., & Peters, C. W. (1993). Integration and thematic teaching: Integration to im- prove teaching and learning. Language Arts, 70, 252-263.

Meece, J. L., Blumenfeld, P. C., & Hoyle, R. H. (1988). Students' goal orientations and cognitive engagement in classroom activi- ties. Journal of Educational Psychology, 80, 514-523.

Meloth, M. S., & Deering, P. D. (1994). Task talk and task awareness under different cooperative learning conditions. Amer/- can Educational Research Journal, 31, 138-165.

Morrow, L. M., Pressley, M. P., Smith, J., & Smith, M. (1997). The effect of a literature-based program integrated into literacy and science instruction with children from diverse backgrounds. Reading Research Quarterly, 32, 54-78.

Mosenthal, P. B. (1996). Understanding the strategies of document literacy and their conditions of use. Journal of Educational Psychology, 88, 314-333.

Oldfather, P., & McLaughlin, J. (1993), Gaining and losing voice: A longitudinal study of students' continuing impulse to learn across elementary and middle school contexts. Research in Middle Level Education, 17, 1-25.

Pate, E. P., McGinnis, K., & Homstead, E. (1995). Creating coherence through curriculum integration. In J. A. Beane (Ed.), Toward a coherent curriculum: 1995 yearbook of the Associa-

tion for Supervision and Curriculum Development (pp. 62-70). Alexandria, VA: Association for Supervision and Curriculum Development.

Peterson, N., Kolen, M., & Hoover, H. D. (1989). Scaling, norming, and equating. In R. L. Lima (Ed.), Educational measurement (pp. 221-263). New York: Macmillan.

Pintrich, P. R., & DeGroot, E. V. (1990). Motivational and self-regulated learning components of classroom academic per- formance. Journal of Educational Psychology, 82, 33--40.

Pintrich, P. R., & Schrauben, B. (1992). Students' motivational beliefs and their cognitive engagement in classroom academic tasks. In D. H. Schunk & J. L. Meese (Eds.), Student perceptions in the classroom (pp. 149-184). Hillsdale, NJ: Erlbaum.

Pressley, M. P., Harris, K. H., & Marks, M. (1992). But good strategy instructors are constructivists. Educational Psychology Review, 4, 3-31.

Purcell-Gates, V., McIntyre, E., & Freppon, P. A. (1995). Learning written storybook language in school: A comparison of Iow-SES children in skills-based and whole language classrooms. Amer/- can Educational Research Journal, 32, 659--685.

Reynolds, R., Sinatra, G., & Jetton, T. (1996). Views of knowledge acquisition: A continuum from experience-centered to mind- centered. Educational Psychologist, 31, 93-105.

Ross, J. A. (1988). Controlling variables: A recta-analysis of training studies. Review of Educational Research, 58, 405--437.

Salomon, G. (1991). Transcending the qualitative-quantitative debate: The analytic and systemic approaches to educational research. Educational Researcher, 20, 10-18.

Schiefele, U. (1996). Topic interest, text representation, and quality of experience. Contemporary Educational Psychology, 21, 3-18.

Schraw, G., Bruning, R., & Svoboda, C. (1995). Source of situational interest. Journal of Reading Behavior, 27, 1-17.

Singer, H., & Donlan, D. (1982). Active comprehension: Problem- solving schema with question generation for comprehension of complex short stories. Reading Research Quarterly, 17, 166- 186.

Skinner, E. A., & Belmont, M. J. (1993). Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of Educational Psychology, 85, 571-581.

Smith, E. L. (1991). A conceptual change model of learning science. In S. M. Glynn, R. H. Yeany, & B. K. Britton (Eds.), The psychology of learning science (pp. 43-63). Hillsdale, NJ: Erlbaum.

Snow, R. E., & Lohman, D. E (1989). Implications of cognitive psychology for educational measurement. In R. L. Linn (Ed.), Educational measurement (pp. 263-333). New York: Macmillan.

Stipek, D. (1996). Motivation and instruction. In D. C. Bediner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 85-113). New York: Simon & Schuster.

Sweet, A., Guthrie, J. T., & Ng, M. (1998). Teachers' perceptions and students reading motivation. Journal of Educational Psychol- ogy, 90, 210-223.

Turner, J. C. (1995). The influence of classroom contexts on young children's motivation for literacy. Reading Research Quarterly, 30, 410-441.

Whitehurst, G. J., Epstein, J. N., Angell, A. L., Payne, A. C., Crone, D. A., & Fischel, J. E. (1994). Outcomes of an emergent literacy intervention in Head Start. Journal of Educational Psychology, 86, 542-555.

Wigfield, A., & Guthrie, J. T. (1997). Relations of children's motivation for reading to the amount and breadth of their reading. Journal of Educational Psychology, 89, 420--432.

CONCEPT-ORIENTED READING INSTRUCTION

A p p e n d i x A

T e a c h e r Q u e s t i o n n a i r e o n C O R I a n d T r a d i t i o n a l T e a c h i n g

277

No. of Correlation with type of Instructional dimension items r instructional program

Real-world interaction (observational vs. abstract) 9 .90 Knowledge (conceptual vs. factual) 12 .89 Direction of control (student vs. teacher) 15 .91 Social (collaborative vs. individual) 13 .66 Strategies taught (explicitly vs. implicitly) 9 .75 Written expression (personal vs. conventional) 9 .47 Integration of language arts and science (integrated vs.

departmentalized) 8 .71

.97***

.97***

.95***

.86***

.61"**

.20

.90***

Note. CORI = Concept-Oriented Reading Instruction. ***p < .001.

A p p e n d i x B

R u b r i c s fo r C o d i n g the P e r f o r m a n c e A s s e s s m e n t

Prior Knowledge

This rubric was the same as the writing rubric.

Wri t ing

Level 1

At this level, the students presented (a) no information; (b) scientifically inaccurate information; (c) one appropriately identi- fied feature or species for each biome; or (d) two features or species for only one of the biomes.

Level 2

Students presented a combination of two to four general or specific features of the biomes or species for at least one biorne. However, no relationships among them were demonstrated. At this level, students also may present scientifically inaccurate information.

Level 3

At this level, the student may have presented accurate and relevant information for a minimum of four specific features of one biome. An alternative was that students could present any relationships or connec- tions between the species and the features of the biome but that the relationships were implicit through comparison or vague. No explana- tions for relationship for features are presented.

Level 4

At this level, a student presented all of the necessary elements for a Level 3, but the relationships or connections between species and the features of the biome were presented explicitly. However, these relations were stated for only one biome and the other remained implicit or vague. Students also may present their answer in the form of a clear relationship in one biome that was not transferrable to the other. Explanations were explicitly included and were distinguishable from particular facts or statements of biome and species characteristics.

Level 5

At this level, students wrote an explicit relationship between species and features for both biomes. The comparison was often

parallel, with one type of animal in one biome being comparable to a type of animal in the other biome showing asymmetry. However, answers at this level lack explanations of the relationships between biomes. An alternative means for obtaining a Level 5 was to present a systemic (Level 6) response for only one biome with limited information on a second biome.

Level 6

At this level, students clearly explained the principles that link the features of biomes in terms of "systems." For example, life cycles or adaptations of certain species and the relationships of these species and features of their respective biomes were stated. The systems were governed by principles centered around the availability of water, temperature, formations of the biome, sur- vival, or food chains.

Drawing

This rubric was the same as the writing rubric.

Search

The rubric for search consisted of six levels. Several aspects of the log were incorporated into the final reading, including the number of relevant packets chosen by the students, the quality of the notes evidenced in the log, and the characteristics of the reasons given for choosing the packets. Students' motivational characteristics, such as elaboration, persistence, and apparent interest in task, were recognized.

Level 1 was assigned to student work in which responses were nonexistent, incoherent, or completely irrelevant to the question. Reasons for choosing the packet were not presented.

Level 2 was assigned to responses in which students reported one to three relevant packets. The notes, however, showed vague information related to one plant or animal. Reasons for choosing the notes were excluded or were self-referenced (e.g., "I liked it").

Level 3 was assigned to responses in which three to five relevant packets were selected and several irrelevant packets (one to three) may have been included. Notes appeared in the form of "captions" or tides and headings. Usually, three to six features or species were listed and a number of these features may have been scientifically

278 GtrrHRIE Ear AL.

trivial. One packet usually showed the notes that reflected a written reason for choosing them. However, the links between notes and packets were otherwise missing, incoherent, or repetitive.

Level 4 consisted of two to five relevant packets. From these, a substantial number of appropriate structures (3-15) for each biome was identified, including plants, animals, weather, and soil of each biome.

Level 5 was assigned to responses in which four to seven relevant packets were selected. The key structures of both biomes of pond and desert were included, and a few simple relations between particular animals or between specific plants and biome features were expressed.

Level 6 consisted of four to seven relevant packets with an explicit reason for them. The notes illustrated the principle of adaptation for both plants and animals. At this level, the life cycles of plants and animals are shown to be related to the features of biomes. The sequence of packets selected during the search showed cumulative development of biome and adaptive information about plants and animals. The notes may have had the quality of argument if information is combined with reasoning to present a cogent viewpoint. The notes consistently contained information that fulfilled the goal stated in the reason.

The reasons for choosing packets were coded separately and combined with the initial levels assigned to packet selection and quality of the notes.

Reasons at Level 1 were not presented or were not distinct from notes.

Reasons at Level 3 were distinct from notes but were not topical or conceptual (e.g., it was interesting). Reasons were not associated or connected to the content of the notes.

Reasons at Level 5 consisted of four to seven reasons stated distinctly from the notes. However, the reasons were too general or too specific, although they were related to the content of the notes for a majority of the packets.

At Level 6, five to nine reasons were presented. These reasons consisted of a goal for reading that was at least as specific as the title for a major heading of the text but was not a verbatim repetition. Reasons were presented in a definable sequence and related to the notes. These levels of supplying reasons for choosing infommfion were related to the initial coding. Having identified the rubric level on the basis of selections and notes, the coder then adjusted the level based on the coding for reasons according to Table B 1.

Table B 1 Adjustments in Search for the Final Code

Selection and notes Reason Final code

7, 6 5 or 7 No change 7, 6 3 or 1 Reduce 1 5, 4 7 or 5 No change 5, 4 3 Reduce 1 5, 4 1 Reduce 2 3, 2 3 or 5 No change 3, 2 1 Reduce 1

1 1, 3, or 5 No change

Transfer

This rubric was the same as the writing rubric.

Informat ion Text Comprehens ion

Each question in this section was scored on a 3-point scale (1-3). The scores ranged from 4 to 12. The general standards for each answer consisted of the following:

1. The answer is inaccurate or incoherent; the answer does not demonstrate an understanding of the question or the text.

2. An accurate response demonstrates an understanding of either of the text or the question.

3. An accurate and elaborated response demonstrates a complete understanding of both the text and the question.

These expectations were applied to both Grade 3 and Grade 5 responses. For Grade 3, Level 1 responses were consistently incoherent or irrelevant to the question. For Grade 5, Level 1 responses were incoherent, inaccurate, or extremely vague and general with one or two remotely relevant pieces of information.

Level 2 responses for Grade 3 included a few important bits of information from the text and one relevant point that addressed the question. For Grade 5, Level 2 responses included a portion but not all of the text-based information needed in the question. Relations between information and explanations were absent or weakly stated.

Level 3 responses for third grade included essential concepts, accurate information, and a completeness of description. For Grade 5, Level 3 responses contained full descriptions with relations among concepts clearly defined. Explanations were provided if requested in the question. The response contained critical details and elaborated statements of their relationships to each other.

Narra t ive Interpretat ion

The three questions in the narrative assessment task were each coded on three levels. This provided a score of 3-9 on the narrative task for each student.

At Level 1, the students provided an irrelevant answer, an incoherent brief statement, no answer, or a self-reference but not a text-reference response.

Level 2 responses for Grade 3 and Grade 5 students included appropriate information that was vague, limited in scope, or marginally appropriate to the question. Level 3 responses for Grade 3 and Grade 5 students included elaborate descriptions that were question relevant and text based. These students supported a general point with specific examples from the text and replied to all parts of the question.

Rece ived February 3, 1997 Revis ion rece ived N o v e m b e r 18, 1997

Accep t ed N o v e m b e r 26, 1997 •