Innovations and Future Directions for Early Numeracy Curriculum-Based Measurement

11
http://aei.sagepub.com/ Intervention Assessment for Effective http://aei.sagepub.com/content/early/2011/07/15/1534508411414154 The online version of this article can be found at: DOI: 10.1177/1534508411414154 published online 18 July 2011 Assessment for Effective Intervention Amanda M. Marcotte Scott A. Methe, Robin Hojnoski, Ben Clarke, Brittany B. Owens, Patricia K. Lilley, Bethany C. Politylo, Kara M. White and the Special Series Innovations and Future Directions for Early Numeracy Curriculum-Based Measurement: Commentary on Published by: Hammill Institute on Disabilities and http://www.sagepublications.com can be found at: Assessment for Effective Intervention Additional services and information for http://aei.sagepub.com/cgi/alerts Email Alerts: http://aei.sagepub.com/subscriptions Subscriptions: http://www.sagepub.com/journalsReprints.nav Reprints: http://www.sagepub.com/journalsPermissions.nav Permissions: at EAST CAROLINA UNIV on July 28, 2011 aei.sagepub.com Downloaded from

Transcript of Innovations and Future Directions for Early Numeracy Curriculum-Based Measurement

http://aei.sagepub.com/Intervention

Assessment for Effective

http://aei.sagepub.com/content/early/2011/07/15/1534508411414154The online version of this article can be found at:

 DOI: 10.1177/1534508411414154

published online 18 July 2011Assessment for Effective InterventionAmanda M. Marcotte

Scott A. Methe, Robin Hojnoski, Ben Clarke, Brittany B. Owens, Patricia K. Lilley, Bethany C. Politylo, Kara M. White andthe Special Series

Innovations and Future Directions for Early Numeracy Curriculum-Based Measurement: Commentary on  

Published by:

  Hammill Institute on Disabilities

and

http://www.sagepublications.com

can be found at:Assessment for Effective InterventionAdditional services and information for     

  http://aei.sagepub.com/cgi/alertsEmail Alerts:

 

http://aei.sagepub.com/subscriptionsSubscriptions:  

http://www.sagepub.com/journalsReprints.navReprints:  

http://www.sagepub.com/journalsPermissions.navPermissions:  

at EAST CAROLINA UNIV on July 28, 2011aei.sagepub.comDownloaded from

Assessment for Effective InterventionXX(X) 1 –10© 2011 Hammill Institute on DisabilitiesReprints and permission: http://www.sagepub.com/journalsPermissions.navDOI: 10.1177/1534508411414154http://aei.sagepub.com

Article

414154 AEI

1East Carolina University, Greenville, NC, USA2Lehigh University, Bethlehem, PA, USA3University of Oregon Eugene, OR, USA4University of Massachusetts at Amherst, Amherst, MA, USA

Corresponding Author:Scott A. Methe, 104 Rawl Building, ECU Main Campus, Greenville, NC 27858, USA Email: [email protected]

Innovations and Future Directions for Early Numeracy Curriculum-Based Measurement: Commentary on the Special Series

Scott A. Methe1, Robin Hojnoski2, Ben Clarke3, Brittany B. Owens1, Patricia K. Lilley1, Bethany C. Politylo4, Kara M. White4, and Amanda M. Marcotte4

Abstract

The purpose of this extended commentary article is to frame the set of studies in the first of two issues and recommend areas of inquiry for future research. This special series issue features studies examining the technical qualities of formative assessment procedures that were developed to inform intervention. This article intends to emphasize issues in the current set of studies that do not appear central to early numeracy curriculum-based measurement (EN-CBM) research. To the extent possible, we expect that this two-volume series will result in scientific and practical advances. Despite this lofty intention, this series can neither represent all issues important to stakeholders nor characterize the full body of applied and basic research in early numeracy assessment. As such, we focus on a set of theoretical frameworks to guide the current and future development of EN-CBM.

Keywords

early mathematics, mathematics assessment, early numeracy

Each study in this series, either explicitly or implicitly, rec-ognizes formative assessment (FA) as a primary component driving its conception. To define the intended reach of the studies in this series, FA serves as the main topic. FA is a special type of assessment used before and during instruc-tion that provides feedback to inform the process of learn-ing (Black & Wiliam, 1998). Each paper in this series uses a type of FA methodology described as curriculum-based assessment (CBA) or curriculum-based measurement (CBM). Broadly, CBA assumes that a scope and sequence of domain-specific content is the standard against which student learn-ing and teaching should be evaluated (Hintze, Christ, & Methe, 2006). CBM is distinguished by efficient, short-duration tests that can be used to examine static and dynamic performance in fundamental content areas. Originally, CBM was conceived as a general outcome measure (GOM), and the logic of a GOM is exemplified by the function of CBM of oral reading fluency.

Oral reading fluency is considered a GOM because the skill itself is a critically important outcome of reading instruction and because oral reading develops gradually over time (as would any skill based on a set of interrelated subskills). The GOM model is frequently contrasted with a

subskill measurement (SM) approach given that the latter examines discrete behaviors that do not grow steadily over larger time spans. A key issue linking GOM and SM mod-els, to an extent, is the nature of an item set within a speci-fied test. Changing item arrangements allows subskills to be sampled with more or less breadth. For example, in early mathematics, tests requiring relative size discrimination between two-dot arrays should be mastered more quickly relative to tests requiring discrimination between two double-digit numerals. In turn, the latter would be mastered quickly relative to tests requiring rank-ordering the relative size among three five-digit numerals. Tests with broad item sets directly address screening and progress monitoring deci-sions, but are limited for instructional diagnosis.

at EAST CAROLINA UNIV on July 28, 2011aei.sagepub.comDownloaded from

2 Assessment for Effective Intervention XX(X)

Emphasizing Data-Based Decision Making and Domain Content

It is doubtful that GOM logic is portable to mathematics and early mathematics assessment primarily because one skill set—computation fluency, for example—fail to fully represent the breadth of the mathematics curriculum and as a result, this skill set demonstrates generally low criterion-related validity (Christ, Scullin, Tolbize, & Jiban, 2008). Fuchs (2004) provides further clarity on the GOM model by describing these as robust indicators—measures that statistically asso-ciate with key mathematics outcomes but are not sampled from curricula. Drawing from Fuchs’s (2004) concept of measurement models in mathematics, Hosp, Hosp, and Howell (2007) propose a link between SM and GOM called skill-based measurement (SBM). SBMs are less discrete than measures developed through an SM approach but not as broadly representative as GOMs. In addition to categoriz-ing tests either nominally or with regard to measurement models, distinctions among measures can also be made in terms of (a) the degree to which they inform important instruc-tionally relevant decisions. With regard to the coverage of the domain, oral counting, identifying numerals, comparing quantities, and filling in numbers from a sequence are all com-mon tasks found in early numeracy CBM systems; how-ever, they differ in terms of how much domain knowledge is required. Making content-based distinctions among mea-sures allow researchers opportunities to examine purported constructs and scrutinize item and stimulus arrangements. Clarke et al. in this issue pay special attention to the item struc-ture of their easyCBM measures.

For the purposes of this article, we avoid distinctions among CBA, CBM, GOM, and SM (although we respect the schol-arly tradition of CBM research and understand that such distinctions are important). Instead, we hope to emphasize the practical decisions that can be facilitated by the tests and diagnostic processes developed in the series. Three instruc-tionally relevant decisions of key importance are those related to screening, progress, and skill diagnosis (Hosp, in press). Clarke et al., Methe, Begeny, and Leary, as well as Seethaler and Fuchs (all in this issue) develop tests with clear utility for screening, diagnosis, and progress monitor-ing, whereas Hampton et al. (this issue) look more closely at progress measures. In a unique application of already estab-lished measures, Burns (this issue) uses CBM measures in a diagnostic process. Although many of the tests developed in this series may differ from the original intent of a CBM technology, we will refer to the collective work as early numer-acy curriculum-based measurement (EN-CBM). Ongoing discussions about measurement models are important, but beyond the scope of this article (see Fuchs & Deno, 1991, for an advanced discussion and Hosp et al., 2007, for a prac-tical discussion).

Reading and early literacy CBMs have tremendous utility for informing response-to-intervention (RtI) models. Drawing

from early literacy research is also useful because EN-CBM research is in an early stage of development (Fuchs, 2004) and because alphabetic and numeric systems are both sym-bol-based cultural imperatives (Deno, 1992). Although advantageous, drawing from early literacy research may lead to imprecision in EN-CBM research. For example, the term fluency is a vestige of early literacy research used commonly throughout EN-CBM research. The construct of fluency emanated from cognitive automaticity theory and applied reading research and, as such, it implies accu-racy, speed, as well as attentiveness to the rhythm of a writ-ten text. Beyond accuracy and speed, what exactly is mathematical fluency? Although we do not answer this question in this commentary or throughout the series, we wish to use this and other issues below as topics that deserve attention in future research. We believe that attending to these issues can assist EN-CBM research in developing indepen-dence from early literacy research. The degree to which EN-CBM research should be independent is unknown, but we are concerned that relying on these comparisons may affect a surging field of inquiry.

What Is Early Numeracy, Exactly?Numeracy

Etymology suggests quite plainly that literacy is knowledge of letters (and alphabetic systems) and numeracy is knowl-edge of numerals (and the numeric system). In reference to young children and schooling, the word numeracy seems to have been used substantially in the past century throughout the United Kingdom, Australia, and New Zealand (Brinkworth, 1985; Scott, 1999). When considering the various descrip-tive terms used in this series and in the broader literature (e.g., mathematics, numeracy, numeric literacy, emergent math), a more careful examination of the construct becomes important, and what may appear a nominal issue can become essential fodder for construct validation. Shadish, Cook, and Campbell (2002) write, “The naming of things is a key problem . . . for names reflect category memberships that . . . have implications about relationships to other concepts, theories, and uses” (p. 66). Indeed, construct inferences are concerned with assessment methodology, and requires “operationalizing each [purported] construct . . . several ways within and across studies; [examining the] match between . . . instances and the target construct; acknowledging legitimate debate . . . given the socially constructed nature of both operations and constructs” (Shadish et al., 2002, p. 68).

EarlyThe term early implies the scope and sequence of learning objectives that can facilitate further learning of (a) what numerals represent (i.e., cardinality) and (b) rules specify-ing how numerals can be combined (i.e., arithmetic) and

at EAST CAROLINA UNIV on July 28, 2011aei.sagepub.comDownloaded from

Methe et al. 3

applied (i.e., problem solving and algebra). In addition to curricular implications, the term early also implies a devel-opmental context. Baroody (2004) suggests that the transi-tion from home to school, occurring at kindergarten, is a major cultural benchmark indicating readiness for learning numerals and procedures. To date, the majority of EN-CBM studies, as well as those featured in this series, features samples of children in kindergarten as well as just prior to (pre-kindergarten) and following this transition (first grade). Broadening this developmental aspect to include ecological inputs such as experience, it is clear that persons of all ages could have deficits in early numeracy but ide-ally, the meaning of early numeracy should emphasize basic skills and knowledge akin to a number sense (Berch, 2005). We hold that early numeracy means something quite similar to number sense (Berch, 2005). Readers of this series are urged to attend to the many unique ways that number sense is operationalized.

Early Numeracy as Number SenseIn their executive summary, members of the National Research Council (2001) assert, “all young Americans must learn to think mathematically, and they must think mathematically to learn” (p. 1). Berch (2005) proffered the idea of reasoning and thinking mathematically as a key definition of number sense. This statement raises the ques-tion of what it means to think mathematically and how mathematical thinking can be made observable to examine the development of competence. Defining mathematical competence remains a considerable challenge. Each author in this series makes note of the obvious lack of consensus about a clear definition for number sense. Nonetheless, the authors find unique ways to operationalize early numeric principles. Clarke et al. use the National Council of Teachers of Mathematics (NCTM) Focal Points (NCTM, 2006) as a policy-based framework for their measures, which cover a range of content yet to be operationalized in the EN-CBM literature. Methe et al. use a variety of models (i.e., the informal math model discussed in the next section) to develop measures of basic counting, cardinality, ordinal-ity, grouping, and decomposing numbers. Seethaler and Fuchs look specifically at the extent to which fundamen-tal computation concepts can be assessed by substituting object arrays for numerals in their screening and progress measures.

A well-defined model of mathematical cognition in young children was proposed by Griffin, Case, and Siegler (1994) and illustrated the concepts that underlie mathemati-cal learning in the area of number and quantity, or number sense. These authors posited the existence of a mental num-ber line consisting of interrelated layers of knowledge and skills. For example, the mental number line helps co-ordi-nate different types of symbolic representation (i.e., written numerals and verbal labels), conceptual representation(i.e.,

quantities and finger patterns), number order, one-to-one correspondence, and procedures of movement along the number line. Griffin et al. (1994) used the term central to convey that the structure is essential to successful perfor-mance on a variety of tasks: central in that future learning is dependent on it and central in that instruction that develops this structure is critical. The central conceptual number line can be tied to a variety of theoretical models of number sense featured in the basic research literature, such as for-mal and informal, conceptual and procedural, triple-code theory (Dehaene, Piazza, Pinel, Cohen, & Campbell, 2005), and number sense access (Wilson, Dehaene, Dubois, & Fayol, 2009).

Early Numeracy in Basic ResearchSince 1958, library databases indicate that approximately 1,000 peer-reviewed studies have been conducted in early numeracy (see Figure 1). Among these are a wide variety of studies examining basic cognitive processes critical to con-sider when developing and implementing assessments to inform intervention. Adams’s (1990) seminal work in early lit-eracy cited substantial cognitive and neuroscience research that subsequently informed the development and use of early literacy CBM. Despite a call to action (Gersten, Jordan, & Flojo, 2005), the majority of applied EN-CBM studies have not clearly drawn from cognitive psychology or neurosci-ence (i.e., basic research) as a theoretical foundation. We hold that one or more of the following theoretical frame-works could be used to guide future EN-CBM research.

Formal and InformalThe distinction between formal and informal mathematical knowledge and skill is important particularly for young children in the early stages of mathematical development who have yet to be exposed to instruction in schools. The notion of formal mathematics was developed to distinguish the mathematics that is learned in school from exposure to mathematical ideas in the home. Formal mathematics involves written symbolic representations (i.e., written numerals and operators), algorithms, and mathematical principles transmitted primarily through instruction (Song & Ginsburg, 1987). In contrast, informal mathematics may be described as “a sense of everyday mathematics” that chil-dren develop through experiences prior to school entry (Song & Ginsburg, 1987). Despite what seems to be a simple idea, informal mathematics involves both concrete and abstract representations underlying mathematical skills and concepts. The mental number line, as a central conceptual structure, involves the integration and coordination of infor-mal and formal knowledge and skill. For example, a child makes an association between the written number “3” and a quantity array (•••), thus developing a more complex rep-resentation of the number and quantity system.

at EAST CAROLINA UNIV on July 28, 2011aei.sagepub.comDownloaded from

4 Assessment for Effective Intervention XX(X)

Figure 1. Studies of early literacy and early numeracyNote. Bars represent number of studies. Databases used were Education Research Complete and PsycINFO. Search terms included early numeracy, literacy; early reading, mathematics; early mathematics assessment; early reading assessment; basic reading, math; reading, math assessment; elementary reading, mathematics.

Methe et al. and Seethaler and Fuchs, in this issue, extend the focus of measurement in EN-CBM to include tasks that focus on informal knowledge. Hampton et al. continue research on the informal skill of oral counting but use numeral-based tests of formal knowledge as prog-ress monitoring measures. Being able to choose among measures operationalizing this dimension is particularly important for young children as there is some evidence to suggest that children perform differently on both task types. Research suggests the performance gap between lower-income and middle-income children is not as signifi-cant on nonverbal calculation, presumably an informal skill, as it is on tasks such as story problems that require more formal knowledge (Jordan, Kaplan, Locuniak, & Ramineni, 2007).

Conceptual and ProceduralIn discussing the strands of mathematical proficiency, the National Research Council (2001) defines procedural flu-ency as “knowledge of procedures, knowledge of when and how to use them appropriately, and skill in performing

them flexibly, accurately, and efficiently” (p. 121). In this report, conceptual and procedural knowledge are presented as separate strands of proficiency, deemphasizing their interdependency. Referring again to the concept of a mental number line, its effectiveness in guiding learning depends on the learner’s ability to access and apply the structure to mathematical learning. That is, in responding to a written word problem such as, “Which is closer to 5: 1 or 6?” a child must see the actual numerals to access the procedure, but must also know their position on a mental number line to answer proficiently. My own child (second author) ini-tially responded to the above question by asking, “Do you mean on the page, or in my head?” Her response (and subsequent correct answer) suggested that she had con-structed a mental number line, differentiated a conceptual structure from the visual presentation of the numerals, and used the concept to access and guide the necessary proce-dure. Although both conceptual and procedural knowledge are thus required, considering them separately as targets for assessment and instruction or intervention may be a useful heuristic. Burns, in this issue, presents a unique preliminary investigation of this heuristic using a nonex-

at EAST CAROLINA UNIV on July 28, 2011aei.sagepub.comDownloaded from

Methe et al. 5

Figure 2. An organizing model depicting growth in number sense accessNote. Dehane et al.’s triple code theory is combined with informal and formal as well as conceptual and procedural knowledge. The box at the bottom of the model reminds readers that the development of number sense access relies on the quality of experience, instruction, assessment, and curriculum.

perimental single case design. This study highlights the importance of a diagnostic process to link assessment to intervention.

Triple Code TheoryTo illustrate number sense, Dehaene et al. (2005) identified three distinct but interrelated cognitive codes for number sense: (a) quantitative (e.g., 12 objects), (b) verbal (e.g., say “twelve”), and (c) visual-abstract (e.g., the numeral 12). Although the research establishing triple code theory is largely based in neuroscience, the application of the model closely follows an informal to formal and conceptual to procedural framework. Like the previous conception of number sense, this model indicates that the development of each code is essentially linear and later development is based on lower-level development. That is, the extent to which a learner can grasp a more abstract mathematical equation depends on the extent to which the quantitative and language-based codes have been engaged. Embedded within the magnitude code is the notion of subitizing (Desoete, Ceulemans, Roeyers, & Huylebroeck, 2009). Subitizing is the automatic recognition of quantities and takes either the more basic form of perceptual recognition or combining perceived objects for conceptual recognition. Perceptual subitizing holds that an array of one to five dots can be easily organized into one unit, but more than five dots would need to be organized into two groups. Integrating subitizing into early childhood assessments is a key policy recommenda-tion (Clements, 2004).

Number Sense Access

Much like a formal and informal distinction, the number sense access view (Wilson et al., 2009) suggests that num-ber sense is nonsymbolic whereas number sense access refers to the conversion of nonsymbolic knowledge into the cultural system of number representation. Figure 2 was constructed to organize the different theories discussed in this section pertaining specifically to the notion of growing access to number. This characterization of development emphasizes key transitions that can be viewed as bench-marks of early math competence (Baroody, 2004). The idea of conversion in level forms the basis for a variety of special education interventions (Riccomini & Witzel, 2010), as a movement from concrete to picture-based to abstract representation is a popular notion in curriculum development (Griffin & Clements, 2007). In terms of the mental number line, the process of linking the various lev-els of representation is a capstone that ultimately creates the central conceptual structure. In the current series, many new EN-CBM tests can be used to potentially examine key benchmarks, such as the transition from a static, one-to-one concept of number to one involving movement and relative size (see Methe et al., this issue). Indeed, the authors in this series pay close attention to the need to address both symbolic and nonsymbolic representations of mathemati-cal knowledge. Attention to this issue allows an examina-tion of number sense access to more precisely determine the source of difficulty for young children who are struggling in early math.

at EAST CAROLINA UNIV on July 28, 2011aei.sagepub.comDownloaded from

6 Assessment for Effective Intervention XX(X)

Figure 3. Broad rationale and context for research cited by authors of EN-CBM studiesNote. Approximately 22 studies were surveyed since 1997 and of these, 12 offered broad-based contextual rationales for the research. Life opportunities included one statement pertaining to general positive function in society and four statements related to workforce entry. Four statements regarding the need to improve conditions for low-SES students were noted and five statements about the need to advance research commensurate with early literacy CBM.

Why Must We Conduct EN-CBM Research?To Emphasize Unaddressed OutcomesDiscussing the ideal research study, Ogden and Goldberg (2002) write, “it is a mistake to place heavy emphasis on the importance of a study that all present would accept as important” (p. 73). Because of the link between assessment and intervention, most would assume that EN-CBM research is important. Although this seems the case, the implications of failing to prevent illiteracy far exceed the classroom and school. With regard to innumeracy, how-ever, Elvis has not left the (school) building. Figure 3 indicates, using a sample of 20 studies conducted in early numeracy CBM since 1997 (see Methe, McCulloch, Betts, Floyd, & Bowler, 2011), that the most common context for the research pertains to later school outcomes. We hold that this obvious rationale no longer bears discussion. Further-more, because EN-CBM enhances prevention, it is neces-sary to know what it is preventing. We hope that future

EN-CBM research can not only cite studies that discuss both risk and resilience related to numeracy but endeavor to trace lines from early to later competence in longitudinal data systems.

Evidence indicates that people with low numeracy are more likely to (a) hold prejudicial views of ethnic and social groups, (b) disengage from civic life, (c) have finan-cial problems, (d) feel less secure in employment, and (e) be at risk for health problems (Ancker & Kaufman, 2007; Lawrence, 2010). Ancker and Kaufman (2007) indi-cate that those with obesity, diabetes, and cancer are more likely to experience complications if they lack numer-acy. In the medical literature, patient numeracy has long been characterized as an “accepted determinant of health” (Donelle, Hoffman-Goetz, & Arocha, 2007). Broadening the context for EN-CBM research can promote a more robust interdisciplinary science where researchers in pediat-ric school psychology and health-oriented special education, for example, can team with assessment and measurement experts.

at EAST CAROLINA UNIV on July 28, 2011aei.sagepub.comDownloaded from

Methe et al. 7

To Catch Up to Early Literacy

In addition to broadening the rationale, early mathematics research simply lags behind early reading research. The results and implications of this substantial gap are unknown, but it is easy to speculate that less attention—both broadly and specifically—is directed toward children who are failing early mathematics. Writing from experience, Methe (2009) indicated that many educators can effectively promote early reading achievement, but are not yet sure about fac-tors leading to early numeracy success. The primary goal of the article was to suggest that the failure of early numeracy assessment research to narrow its content to a clear set of “big ideas” was due to a paucity of research. To date, how-ever, the truth of this premise is unknown. Including only peer-reviewed research conducted since the National Defense Education Act of 1958 (the first law enacting fed-eral support for educational research) gaps in both levels and trends of early literacy (the white bars) and early numer-acy research (the dark bars) are evident in Figure 1. Further-more, although early numeracy research appears to have been competitive with early reading research, the explosion of early literacy research in the middle to late 1990s has resulted in a cumulative difference of 2,827 available stud-ies from 1958 to 2010.

To Reduce Confusion and Enhance RepresentationWhen discussing the profession of school psychology, Fagan (1999) identified two distinct historical stages: (a) “diversity of necessity” and (b) “restriction for iden-tity.” Fagan concluded that a breadth of practices was a fundamental precondition that ultimately led to the selec-tion of a more specified set of functions. Although we struggle to find the exact nomenclature for the phenome-non, any scholar will recognize the value of brainstorming prior to the selection of a specific course of action. Figure 4 summarizes the studies featured in Methe et al. (2011) with regard to how a construct of number sense has been opera-tionally defined in the EN-CBM research. Specific refer-ences to notable components of number sense (Berch, 2005) were extracted, organized, and complied. This figure depicts the number of studies in which each component appears. The most common three EN-CBM tests, appearing more than 30 times across studies, do not appear to directly tap conceptual number sense, and key conceptual topics such as cardinality, ordinality, and measurement concepts are weakly represented in the research.

Remaining Issues and Conclusions

Although a variety of issues were discussed in this article and linked to the current studies in the series, we have left a number of critical issues unaddressed. We hope that the theories proffered as conceptual-to-abstract frameworks for EN-CBM can assist in enhancing a more precise under-standing of number (and perhaps may save practitioners from mistakenly saying “the number nine” when they really mean “the numeral nine”). In addition to precision in operationalizing number, we are also concerned about basic measurement issues such as item analysis and validity. Although the studies in this series used sound scientific procedures, only the Clarke et al. study appeared to use statistical evaluations of item function. Although Methe et al. cited a theory of subitizing to develop their conceptual items, a statistical evaluation of item difficulty, discrimina-tion, and diagnostic accuracy was not conducted. Looking closely at item function helps ensure the precision of screening decisions in an RtI model. Using a test of dot arrays as an example, if all students regardless of ability automatically recognize one to five items, then a great number of false negative classifications for very young students (i.e., students who seem to know the difference between two and four objects) should be evident with items containing less than five objects.

With regard to validity, few researchers have examined the latent structure of EN-CBM measures (see Floyd, Hojnoski, & Key, 2006, and Lago & DiPerna, 2010). In dis-cussing the implications of their research, Hampton et al., Methe et al., and Seethaler and Fuchs (all in this issue) remind the reader that inferences about criterion-related validity are only as good as the criterion tests. As such, the extent to which EN-CBM systems represent the mathemat-ics domain cannot be fully known until relations are estab-lished with well-developed and comprehensive state end-of- grade assessment systems as well as computer-adaptive assessments. In broadening the criterion, EN-CBM can bet-ter represent the domain of mathematics as experienced by students and teachers in schools. Clarke et al. in this issue operationalized the NCTM focal points by including a wider assessment of mathematics including items related to geom-etry. Few studies, however, have examined geometry directly. This article was designed to link studies in this series and to a more robust theoretical literature. In completing this arti-cle and series, our intentions are to positively affect school practice, contextualize a research agenda, guide data-based decision making, and ultimately help students who struggle to learn mathematics.

at EAST CAROLINA UNIV on July 28, 2011aei.sagepub.comDownloaded from

8 Assessment for Effective Intervention XX(X)

Figure 4. Number of EN-CBM studies citing a specific component of number sense

Declaration of Conflicting InterestsThe authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

FundingThe authors received no financial support for the research, author-ship, and/or publication of this article.

References

Adams, M. J. (1990). Beginning to read: Thinking and learning about print. Cambridge, MA: MIT Press.

Ancker, J. S., & Kaufman, D. (2007). Rethinking health numeracy: A multidisciplinary literature review. Journal of the American Medical Information Association, 14, 713–721.

at EAST CAROLINA UNIV on July 28, 2011aei.sagepub.comDownloaded from

Methe et al. 9

Baroody, A. J. (2004). The developmental bases for early child-hood operations and number standards. In D. H. Clements & J. Sarama (Eds.), Engaging young children in mathematics: Standards for early childhood mathematics education (pp. 173–220). Mahwah, NJ: Lawrence Erlbaum.

Berch, D. B. (2005). Making sense of number sense: Implications for children with mathematical disabilities. Journal of Learn-ing Disabilities, 38, 333–339.

Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80, 139–144.

Brinkworth, P. (1985). Numeracy: What, why, how? Australian Mathematics Teacher, 41, 8–10.

Christ, T. J., Scullin, S., Tolbize, A., & Jiban, C. L. (2008). Impli-cations of recent research: Curriculum-based measurement of math computation. Assessment for Effective Intervention, 33, 198–205.

Clements, D. H. (2004). Major themes and recommendations. In D. H. Clements & J. Sarama (Eds.), Engaging young children in mathematics: Standards for early childhood mathematics education (pp. 7–12). Mahwah, NJ: Lawrence Erlbaum.

Dehaene, S., Piazza, M., Pinel, P., Cohen, L., & Campbell, J. I. D. (2005). Three parietal circuits for number processing. In Hand-book of mathematical cognition (pp. 433–453). New York, NY: Psychology Press.

Deno, S. L. (1992). The nature and development of curriculum-based measurement. Preventing School Failure, 36, 5.

Desoete, A., Ceulemans, A., Roeyers, H., & Huylebroeck, A. (2009). Subitizing or counting as possible screening variables for learning disabilities in mathematics education or learning? Educational Research Review, 4, 55–66.

Donelle, L., Hoffman-Goetz, L., & Arocha, J. F. (2007). Assess-ing health numeracy among community-dwelling older adults. Journal of Health Communication, 12, 651–665.

Fagan, T. K. (1999). Training school psychologists before there were school psychologist training programs: A history. In C. R. Reynolds & T. B. Gutkin (Eds.), The handbook of school psychology (pp. 2–33). New York, NY: John Wiley.

Floyd, R. G., Hojnoski, R., & Key, J. (2006). Preliminary evidence of the technical adequacy of the preschool numeracy indica-tors. School Psychology Review, 35, 627–644.

Fuchs, L. S. (2004). The past, present, and future of curriculum based measurement research. School Psychology Review, 33, 188–192.

Fuchs, L. S., & Deno, S. L. (1991). Paradigmatic distinctions between instructionally relevant measurement models. Excep-tional Children, 57, 488–500.

Gersten, R., Jordan, N. C., & Flojo, J. R. (2005). Early identifica-tion and interventions for students with mathematics difficul-ties. Journal of Learning Disabilities, 38, 293–304.

Griffin, S. A., Case, R., & Siegler, R. S. (1994). Rightstart: Pro-viding the central conceptual prerequisites for first formal learning of arithmetic to students at risk for school failure. In K. McGilly (Ed.), Classroom lessons: Integrating cognitive

theory and classroom practice (pp. 25–49). Cambridge, MA: MIT Press.

Griffin, S. A., & Clements, D. (2007). Number worlds: A prevention/intervention curriculum. Columbus, OH: Science Research Associates.

Hintze, J. M., Christ, T. J., & Methe, S. A. (2006). Curriculum-based assessment. Psychology in the Schools, 43, 45–56.

Hosp, J. L. (in press). Using assessment data to make decisions about teaching and learning. In K. R. Harris, S. Graham, & T. Urdan (Eds.), APA educational psychology handbook: Vol. 3. Applica-tion to learning and teaching (pp. 87–109). Washington, DC: American Psychological Association.

Hosp, M. K., Hosp, J. L., & Howell, K. W. (2007). The ABCs of CBM: A practical guide to curriculum-based measurement. New York, NY: Guilford.

Jordan, N. C., Kaplan, D., Locuniak, M. N., & Ramineni, C. (2007). Predicting first-grade math achievement from devel-opmental number sense trajectories. Learning Disabilities Research & Practice, 22, 36–46.

Lago, R. M., & DiPerna, J. C. (2010). Number sense in kindergar-ten: A factor-analytic study of the construct. School Psychol-ogy Review, 39, 164–180.

Lawrence, E. D. (2010). What Americans know and why it matters for politics [Electronic Version]. Retrieved from http://projects .iq.harvard.edu/cces/files/Lawrence-_What_Americans _Know.pdf

Methe, S. A. (2009). Big ideas: Missing pieces in early mathemat-ics assessment. Communiqué, 38, 1, 4–5.

Methe, S. A., McCulloch, L. M., Betts, J., Floyd, R. G., & Bowler, M. B. (2011). Synthesis of early numeracy curriculum-based measurement research. Manuscript submitted for publication.

National Council of Teachers of Mathematics. (2006). Curriculum focal points for prekindergarten through 8th grade mathemat-ics: A quest for coherence. Reston, VA: Author.

National Research Council. (Ed.). (2001). Adding it up: Helping children learn mathematics. Washington, DC: National Acad-emy Press.

Ogden, T. E., & Goldberg, I. A. (2002). Research proposals: A guide to success. San Diego, CA: Academic Press.

Riccomini, P. J., & Witzel, B. S. (2010). Response to intervention in math. Thousand Oaks, CA: Corwin Press.

Scott, D. (1999). Essential ingredients for numeracy. Australian Primary Mathematics Classroom, 4, 4–8.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experi-mental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.

Song, M. J., & Ginsburg, H. P. (1987). The development of infor-mal and formal mathematical thinking in Korean and U.S. children. Child Development, 58, 1286–1296.

Wilson, A. J., Dehaene, S., Dubois, O., & Fayol, M. (2009). Effects of an adaptive game intervention on accessing number sense in low-socioeconomic-status kindergarten children. Mind, Brain, and Education, 3, 224–234.

at EAST CAROLINA UNIV on July 28, 2011aei.sagepub.comDownloaded from

10 Assessment for Effective Intervention XX(X)

About the Authors

Scott A. Methe, PhD, is an assistant professor of school psychol-ogy at East Carolina University. His current research interests are the assessment of mathematical cognition in young children, curriculum-based assessment, and decision science.

Robin Hojnoski, PhD, is assistant professor in the School Psychology Program at Lehigh University. She is interested in the application of school psychology principles and practices to early education. Her research focuses on assessment and intervention to improve educational and social outcomes for preschool children and their families.

Ben Clarke is currently a research associate with Instructional Research Group and Pacific Institutes for Research. He serves as a co-principal investigator on three federally-funded research grants in the area of mathematics instruction and assessment. His current research includes testing the efficacy of a kindergarten mathematics curriculum, evaluating the effectiveness of a first grade mathematics intervention program for at-risk students and developing a first grade mathematics intervention program.

Brittany B. Owens is a recent graduate of the Psychology Department at East Carolina University, and as a research

assistant, Brittany closely examined the research gap between early numeracy and early literacy.

Patricia K. Lilley is a student at East Carolina University pursu-ing a Bachelor of Arts in Psychology. Patricia is a research assis-tant examining the implications of research in early numeracy on later outcomes.

Bethany C. Politylo is a doctoral student at the University of Massachusetts at Amherst, currently studying early numeracy assessment.

Kara M. White is a doctoral student at the University of Massachusetts at Amherst, currently studying early numeracy assessment.

Amanda M. Marcotte is an assistant professor of school psychol-ogy at the University of Massachusetts at Amherst. Her research interests are academic and behavioral interventions, school-based prevention programs, formative assessment and curriculum-based measurement, Specific Learning Disabilities, and preventative reading instruction.

at EAST CAROLINA UNIV on July 28, 2011aei.sagepub.comDownloaded from