Appeared in Journal of Applied School Psychology, 24, 1 1

A cross-cultural validation study of the Application of Cognitive Functions Scales, a

dynamic assessment procedure, with Dutch first grade students from regular primary

schools

G.M. van der Aalsvoort, Leiden University, Department of Special Education, PO Box 9555,

2300 RB Leiden, The Netherlands, aalsvoort@fsw.leidenuniv.nl

Carol S. Lidz, 2206 Lombard Street, Philadelphia, PA, USA, zdilsc@aol.com

A cross-cultural validation study of the Application of Cognitive Functions Scales, a

dynamic assessment procedure, with Dutch first grade students from regular primary

schools

Abstract This article describes a study using the Application of Cognitive Functions Scale

(ACFS), a curriculum-based dynamic assessment for use with young children. The ACFS has

been used primarily with children in the US. This study explores the validity of the ACFS with a

sample of 89 Dutch children aged five to six years from six regular primary schools. The results

raise questions about the validity of the ACFS as a dynamic assessment procedure for use with

Dutch children of primary school age. Issues concerning the use of dynamic assessment with

young children are discussed as well as issues regarding the specific content of the subscales.

Appeared in Journal of Applied School Psychology, 24, 1 2

Introduction

Dynamic assessment (DA) has now been available as a diagnostic procedure for well over

thirty years, with ideas related to its development ranging back to the beginning of the twentieth

century (Lidz, 1987). Applications of this model with very young children are relatively rarer

than those developed for older learners, and development of procedures for this population is a

continuing area of need. Lidz and Jepsen (2000) designed the Application of Cognitive Functions

Scale (ACFS) with the intention of applying the dynamic assessment approach with children of

preschool age. With the exception of a validity study of one of the subscales with children in the

United Kingdom (Brooks, 1997), there has been no published research investigating the

usefulness of this procedure with children other than from the United States. The current study

closely follows a similar exploration of the usefulness of the ACFS with Dutch children in

regular education carried out by the same authors (Lidz & Van der Aalsvoort, 2005). In the

earlier pilot study, evidence of the validity of the procedure with this population was documented

despite the fact that their ages were somewhat older than the range intended by the authors of the

ACFS. The current study investigates aspects of construct validity of the ACFS with a larger

cohort of young Dutch students of primary school age who attend a regular education program.

DA is a generic term for procedures that embed intervention within the ongoing

assessment, and that usually include a pretest-training-posttest format that directly links

assessment to intervention. As the child engages in the assessment task, the assessor can observe

the child’s strengths and weaknesses. By looking at the learning processes during the course of

problem-solving, the examiner can discover how the child learns and how the child can best be

instructed. Thus, both the child’s current levels of functioning as well as responsiveness to

intervention are tapped.

Dynamic assessment (DA) refers to format, and not to content of the testing procedure.

Static, combined with dynamic learning assessments, are better predictors of the abilities of

individual children than either one of these measures alone, reflecting what Vygotsky referred to

as the zones of actual and proximal development that are both required to have a full and

complex understanding of the child’s functioning.

The meaning of DA for diagnosticians is that they need to go beyond what the child

currently knows, to understand how the child learns, as well as to determine obstructions to the

child’s successful learning. Diagnosticians such as school psychologists (e.g., US) or educational

Appeared in Journal of Applied School Psychology, 24, 1 3

psychologists (e.g., The Netherlands, England, Scotland, Wales, Northern-Ireland) need to find

ways to remain close to, yet move beyond and below, the content demands of a specific

curriculum to address issues that result in referral of children who are not successful learners.

DA is most appropriately used when the assessment questions concern responsiveness of

the learner, the nature of processes that characterize the learner, and generation of ideas for

potentially useful interventions. Other assessment approaches fail to consider the nature of the

relationship between the assessor and the child or to use this relationship to generate instruction-

relevant insights or strategies. Insights into the behaviors of the assessor that relate to

responsiveness of the learner are important for providing linkages between the assessment and

the classroom.

Although DA tends to reflect and rely on development of metacognitive processes, the

realization that these processes are emerging and actively developing even at very young ages

makes the use of DA with young children relevant. The use of DA with young children is also

supported by the strong negative criticisms of the use of

standardized approaches with this population by influential groups such as the National

Association for the Education of Young Children. Use of DA with younger populations has been

rapidly developing (e.g., Kahn, 2000; Lidz, 2000, 2003; Tzuriel, 2001). One of the approaches

designed for use with very young children is the Application of Cognitive Functions Scale

(ACFS; Lidz & Jepsen, 2000). The test has been developed and researched primarily in the

United States.

The Application of Cognitive Functions Scale (ACFS)

The ACFS has been designed as a curriculum-based dynamic assessment for use with

children who function between the ages of three through five years. The ACFS includes six

subscales (four core and two supplementary described in the Method section) that represent

learning processes that are typically required for success in most American preschool programs.

The subtests were designed based on insights about neuropsychological functioning derived from

Luria (1973, in Lidz, 2000) who stated that cognitive development should be understood as rapid

development of functions/processes beginning with attention and perception during infancy and

early childhood. Thus, the domains of cognition that would be relevant to tap are attention,

perception, memory and executive functioning that represent the learning processes necessary for

Appeared in Journal of Applied School Psychology, 24, 1 4

school achievement. The ACFS yields scores that indicate the degree to which the child has

mastered each of the tasks, as well as the child’s responsiveness to intervention. The ACFS

includes a Behavior Observation Rating Scale that describes qualitative aspects of the child’s

interaction with the materials and the assessor. The behavior rating descriptors are the same

across the six subscales.

The ACFS pretest and posttest of each subscale are the same or close variations on the

same task, and the instructions for administration of all tasks are standardized. The ACFS

interventions for each subscale are semi-scripted and predetermined as guidelines, in a

compromise effort to impose a degree of standardization on the procedure for purposes of

research, facilitation of interpretation, and ease of administration.

The nature of information yielded by the ACFS is both quantitative and qualitative, but

not normative. The quantitative scores (number of items and per cent of items correct) serve as

indicators of level of mastery of the subscale tasks. The behavior ratings offer qualitative

information, and the assessor is expected to make notations that describe how the child goes

about performing each of the tasks, as well as how the child responds to the assessor and the

assessment process.

Several studies document the reliability and validity of the ACFS. Results from all of the

studies, most of which were carried out as part of a Masters Degree program administered by the

second author, are categorized in Table 1. Table 1 shows the type of validity addressed by each of

the studies in relation to the subscale included in that study. The numbers 1 to 6 under the

heading of subscales in Table 1 refer to Classification (1), Auditory Memory (2), Visual Memory

(3), Pattern Completion (4), Perspective Taking (5), and Verbal Planning (6).

--------------Insert table 1 here---------------

Findings like those in the studies reviewed above provide evidence of the validity of the

ACFS per se, as well as the appropriateness of using a dynamic assessment procedure with

children of preschool age.

Questions remain regarding the validity of the ACFS, as all of the studies involved small

groups of children. Moreover, these studies (other than Lidz’s study with deaf students) did not

investigate whether ceiling effects would occur with students older than five. There is also the

Appeared in Journal of Applied School Psychology, 24, 1 5

question of whether the subscales represent culturally specific abilities, since all but one involved

children from the US, where they often enter preschool at the age of 3, followed by Kindergarten

at the age of 5. This organization of school entrance is not found in all Western countries.

Because it can never be assumed that a procedure designed within one cultural setting can be

usefully applied with children from other cultural backgrounds, research investigating such cross-

cultural applications is necessary.

This last issue was first addressed in a pilot study by Lidz and Van der Aalsvoort (2005)

with a population of 29 Dutch children with typical development between the ages of five and six

years. In this study, only the four core ACFS subscales were administered: The two supplemental

subtests, Verbal Planning and Perspective Taking, were not included in this study, as these tests

rely heavily on the use of language, and therefore may be too sensitive to cultural loading. The

focus was to investigate both construct and predictive validity. Significant pretest-posttest gains

were found for three of the four subscales, with the exception of Visual Memory (p = .07), where

a possible ceiling effect again appeared to be an issue. Relationships with language and

arithmetic tests were investigated. Because the expectations regarding predictive validity in

relation to these domains were not supported, the authors concluded that there was possibly an

underlying common factor of language and reasoning that related to both language and arithmetic

that overrode the specific relationships between subscales and these domains. The

appropriateness of the ACFS with this population appeared supported and similar to results from

studies carried out with children from the US.

The study presented here involves the data from an additional 89 students, and also uses

the alternative Response to Mediation scale included in the previous study.

Three hypotheses were tested. The first concerns the investigation of construct validity of the

ACFS as a dynamic assessment device through significant pretest to posttest gains. The second

hypothesis investigated whether the content of the ACFS subscales coincided with similar

cognitive functions assessed by Dutch norm-referenced tests. A third hypothesis concerned

determination of the relationship between behaviors displayed during the assessment and

response to intervention as reflected in the pretest to posttest gain scores.

Methodology

Participants

Appeared in Journal of Applied School Psychology, 24, 1 6

The participants in this study came from six regular primary schools in The Netherlands.

There were 49 boys and 40 girls, for a total of 89 participants. The mean age range for boys was

67 months (SD = 5.99) and for girls, 66 months (SD = 5.78). All of the children had completed

one year of preschool, and Dutch was their first language. The children resided in mid-sized cities

in The Netherlands. Data on parents’ education and occupation were collected in four schools

from 49 children adding up to information of 27 children received (55 percent of the parents

involved). The socio economic status of each parent was rated using the Standard International

Socio-Economic Index of Occupational Status (Ganzeboom, De Graaf & Treiman, 1992). The

index contains 7 levels of occupation which ranged from 1 (high level, such as scholars) to 7 (low

level, such as factory workers). SES score was computed based on adding the level of occupation

of both parents: The mean SES range was 5,9 (SD = 1, 35) out of a total maximum of 14.

Design

The children were selected from the Grade 2 classes of their regular primary schools. The

order of the class list was used to collect parents’ consent. Testing of the children proceeded

following receipt of parental consent. After the number of participants required from each school

to serve the student’s sample size was reached, assessment sessions were planned. The primary

purpose of this study was to investigate the concurrent validity of the ACFS. Based on the

findings with the pilot study (Lidz & Van der Aalsvoort, 2005), norm-referenced tests were

included to test construct validity of the ACFS subscales Classification, and Auditory Memory.

Moreover, a norm-referenced test was added to test concurrent validity of Visual Memory by

using Digit Span of the WISC-III, using Dutch norm-tables. The assessment sessions of the

ACFS were videotaped to allow optimal scoring and observation of the children’s behavior

during the assessment. Table 2 shows the characteristics of the schools involved, and the tests

used for validation.

------------Insert Table 2 here------------

Instruments

The four core subscales of the ACFS administered are described below. Each subscale is

explained with respect to content per element of the dynamic assessment pursued including the

way the child’s performance is scored.

Classification:

Appeared in Journal of Applied School Psychology, 24, 1 7

Pretest: The child is shown 36 wooden blocks of mixed attributes and, following

introductory comments, they are told to “show me how you can make groups with these blocks.”

If they are successful, they are told: “Show me another way that you can make groups with these

blocks.”

Conversational intervention: Pictures of shapes varying on attributes of color, shape, and

size are placed in front of the child. The child is taught what to notice about the pieces as a basis

for making groups. The assessor provides scaffolding as necessary to help the child make the

groups.

Posttest: Same as pretest. The maximum score of pretest and posttest is 7. The child

receives a minimum of 4 points for correct use of blocks followed by 1 point for each correct

grouping.

Auditory Memory (Story Retelling):

Pretest: The child is read a short story and told to listen carefully and then to tell it back to

the assessor.

Conversational intervention: Abstract pieces are placed on a large felt mat (or,

alternatively, magnetic pieces on a magnet board) and used to create a model for the elements in

the story. The child learns to “read” the symbols to create visual images to facilitate retelling of

the story.

Posttest: The child is invited to tell the same story without the visual support.

Retention: The delayed memory test is assessed following completion of the third

subscale which is the Visual Memory subscale. The child is invited to retell the story without the

visual support. The maximum score of pretest, posttest and retention test is 17. The child receives

one point for each item of the story told to a maximum of 14 points and 1 to 3 points when the

chronological order of the items mentioned mirrors the story line.

Visual Memory:

Pretest: A set of eight pictures of common objects (four animals and four means of

transportation) is placed in random order in front of the child. The child first names the pictures

then is told to try to remember them, and then is asked how they will help themselves remember.

Conversational intervention: With an alternate set of pictures (food and clothes), the child

is taught to use memory strategies of rehearsal, chunking and verbal elaboration.

Appeared in Journal of Applied School Psychology, 24, 1 8

Posttest: Same as pretest. The maximum score of pretest and posttest is 13. This is divided

into 1 point for each remembered picture and another 5 points in case one or more strategies are

displayed.

Sequential Pattern Completion:

Pretest: Sets of six pairs of sequential patterns made with abstract plastic shapes are

placed in front of the child, with the last piece missing. The child must select the missing piece

from a choice of two to three alternatives.

Conversational intervention: The child is led through completion of a series of patterns,

beginning with hand movements to creating drawn patterns with crayons. The child is taught to

feel the "need" for completion through emphasizing the rhythm of the pattern, as well as to say

the pattern that was drawn.

Posttest: Same as pretest, but begins with the item on which the child met his/her last

success. The child must select the missing piece from a choice of two to three alternatives. The

child receives 2 points for each correct answer up to a total of 12, and 1 point for every correct

explanation of the item chosen. The maximum score of pretest and posttest is 18.

Response to Mediation Scale (Van der Aalsvoort & Lidz, 2002)

The assessor rates the students from the videotapes of the dynamic assessment from 1

(low) to 5 (high) on each of the following seven variables: Responsiveness of interaction with

mediator, Self-regulation of attention and impulses, Affective quality of interaction with

mediator, Communication related to shared activity, Comprehension of activity demands, Use of

mediator as a resource, and Reaction to challenge. The ratings were computed to reflect the total

assessment including pretest, instructive conversation/intervention and posttest across all four

subscales. Total scores could range between 7 and 35. Table 3 shows the Product Moment

Correlation Coefficients of the reliability checks carried out with a percentage of the subjects per

school.

--------Insert Table 3 here---------

Table 3 shows that all but one (r = .40 from School 4) of the means of the coefficients are

acceptable. The data of school 4 therefore were excluded from data analysis.

Norm-referenced tests

Digit Span Forward and Backward (Wechsler, 2003)

Appeared in Journal of Applied School Psychology, 24, 1 9

The subscale provides 7 items. The experimenter reads aloud a list of digits at a rate of

one digit per second. At the end of each list the child is asked to repeat the digits in order of

presentation. The task consists of two lists of the same length. The forward version starts with

lists of three digits. The backward version starts with lists of two digits. The test is terminated

when a child fails to perform correctly on two lists of the same length. The test score is the total

number of correct answers before termination. The raw score is compared to a norm table of

Dutch children’s performance. The scores can range from 1 to 18.

Language Test for Children (Van Kuijk, 1996)

The Language Test for Children is a standardized achievement test that includes several

subtests. The ones used in the study are described below. The raw score derived at with each test

is compared to a norm table with scores that range from level E (low) to A (high). Each level

reflects a percentile: E = percentile 0-10; D = percentile 10-25; C = percentile 25-50; B =

percentile 50-75, and A reflects percentile 75 to 100. The scores were coded from 1 (E = low) to

5 (A = high).

Receptive Vocabulary

The subscale includes 96 items of four pictures per page. The child has to point to the

correct picture after the assessor says a word that is related only to one of the pictures. The raw

score is then compared to a norm table.

Story Telling The subscale includes two strips of 8 pictures that represent a story. The first one

is about a boy and a girl who show an old pram to their father. He builds them a go-cart, and the

children push the car up a hill. When they ride down in full speed the cart hits a tree. The boy and

the girl fall out of the cart. The cart is broken down to pieces. They take up the pieces crying.

Their father builds the go-cart once more and the children stop crying. The score is based upon

the number of items answered correctly about the pictures. The second story is about a child

loosing her balloon when she trips over a bag left by a boy who ate fries from the bag. Each strip

is present while the child is invited to tell the story. The number of correct deductions with

respect to the story line is added to the scores up to a maximum of 32. The raw score is then

compared to a norm table.

Story Comprehension

Appeared in Journal of Applied School Psychology, 24, 1 10

The subscale includes four short stories. After each story four questions are asked. There

can be up to a total of 16 correct answers. The raw score is then compared to a norm table.

Ordering test (Van Kuijk, 1997)

The standardized achievement test contains 42 items that require paper and pencil

problem solving. The raw score derived at with each test is compared to a norm table with scores

that range from level E (low) to A (high). Each level reflects a percentile: E = percentile 0-10; D

= percentile 10-25; C = percentile 25-50; B = percentile 50-75, and A reflects percentile 75 to

100. The scores were coded from 1 (E = low) to 5 (A = high).

Procedure

The children from the six schools were tested by six Master‘s thesis students as part of

their Master’s thesis between the spring and the fall of 2005. The students were all primary

school teachers who had asked a colleague of their school for permission to approach the parents

of their class. The student who collected data for school 4 was the classroom teacher of the

children she assessed. Testing of the children proceeded following the order of receipt of parental

consent. The ACFS manual had been translated into Dutch in discussion between both authors to

assure comparability of the items. This manual was used again. The first author trained the

teacher-assessors using direct instruction and discussion of their videotaped trials. The children in

the study were administered the four subscales of the ACFS individually. The assessment was

recorded on videotape. Behavior ratings could thus be made after completion of the assessment to

carry out reliability checks to evaluate scoring of the tasks and ratings of behavior during

assessment. Each child was tested in the morning and required about one hour per child. After the

ACFS assessment, the tests of language, mathematics or memory were administered. The choice

of which additional tests to administer was based on the research questions raised by the specific

Master’s student’s thesis. Then, the videotapes were analyzed with respect to scoring ACFS

scales and Response to Mediation ratings. The data were read into a codebook to allow statistical

analysis.

Results

Appeared in Journal of Applied School Psychology, 24, 1 11

The primary questions for this study concerned the appropriateness of the ACFS for use

with Dutch children, as well as issues of the construct validity of this procedure in relation to this

population. To address these questions, we first compared the means for each of the core tests,

since pretest to posttest gains are relevant to the construct validation of dynamic assessment.

Furthermore, we looked at the means and standard deviations in relation to the maximum scores

for each subscale as a source of information regarding the relevance of this procedure for this

population. This information appears in Table 4

---------Insert Table 4 here----------

Table 4 shows that the means and standard deviations were mediate for Classification and low for

Auditory Memory. In the case of Visual Memory, the mean number of participants remembered

at least six pictures out of the eight provided in the pretest. This finding reveals that students

performed very well with respect to memory, but poor with respect to application of memory

strategies during both the pretest and posttest phases. Similarly, during Pattern Completion, the

mean number of children correctly pointed at the correct sequence of at least five of the six items

during the pretest, but their ability to verbalize the reason for their choice was carried out poorly

for both the pretest and posttest. These two requests (pattern completion and provision of

rationale) make quite different demands, and the scoring for this subscale would not represent a

smooth continuum of task mastery between the two.

Hypothesis 1: Pretest to posttest to describe test construct validity

The mean age differed significantly between the schools, F (5, 83) 2, 318, p, 051.

Therefore, a one-way repeated-measures Analysis of Variance was carried out with age as

covariate and pretest and posttest scores as within-subject variables. Although there were positive

gains on all four subscales, none of these reached statistical significance. This means that we

cannot validate the test for this sample through the construct that was operationalized by pretest-

posttest gains on the four core subscales of the ACFS. Hypothesis 2: Relationship between ACFS

test scores and norm-referenced language ability and arithmetic scores

To address the next validity issue, we looked at the relationships between the ACFS

subscales and tests of language and arithmetic. The means, standard deviations of the tests

involved are presented in Table 5.

---------Insert Table 5 about here-------

Appeared in Journal of Applied School Psychology, 24, 1 12

The mean scores in Table 5 reveal that many children obtained high scores on the Mathematics

and Story Comprehension tests, whereas their performance on Receptive Vocabulary and Story

Telling was mediate. To test the expectation that there would be a relationship between

performance on ACFS subscales and norm-referenced tests Pearson Product-Moment-Correlation

Coefficients were computed between the subscales of Classification and Pattern Completion and

mathematic performance expressed as Ordering, between Auditory Memory and language

performance expressed as Receptive Vocabulary, Story Telling and Story Comprehension, and

between Auditory Memory and Digit Span. The results appear in Table 6.

---------------Insert Table 6 about here ---------------

Table 6 shows that the concurrent validity of the subscale Auditory Memory is strong in relation

to performance on the norm-referenced test Story telling. However, the results show no

meaningful differences in this relationship for Auditory Memory pretest compared with posttest.

The correlations between Visual Memory and Digit Span, or between Classification and

Ordering, and between Auditory Memory and Receptive language were non-existent. These

findings suggest that these three subscales measure skills other than those tapped by the norm-

referenced tests.

Hypothesis 3: Relationship between behavior during assessment and gain scores

Table 7 offers an overview of the Response to Mediation scores obtained per subscale.

------------Insert Table 7 about here-------------

The data in Table 7 suggest that the children felt comfortable during the course of the assessment,

as all of the scores per subscale were at or above 3. Moreover, a considerable range in ratings

with respect to Response to Mediation was found suggesting that the behaviors displayed during

assessment varied considerably among the children of the sample.

The relationship between behaviors displayed during the assessment and response to

intervention as reflected in the gain scores was tested. The correlations appear in Table 8.

-------------Insert Table 8 about here---------------

Table 8 shows a significant relationship between behavior ratings during the assessment of

Auditory Memory and the gain score on the scale. This finding suggests that the more responsive

the students are to scale-related mediation, the better they perform on the scale that measures

ability to use memory related to the auditory modality.

Appeared in Journal of Applied School Psychology, 24, 1 13

Discussion

This study applied the ACFS with 89 regular education kindergarten children from six

Dutch primary schools. The utility of this procedure with this population could not be fully

confirmed with respect to construct validity of the dynamic assessment procedure. The findings

related to the first hypothesis suggest that Classification and Auditory Memory were acceptable

subscales with respect to the age group, whereas Visual Memory and Pattern Completion

produced a ceiling effect with respect to the number of items correctly solved. The scores

obtained by either applying strategies or verbalizing reasons for choosing a specific figure to

complete a pattern, were low. It could be said that the results for these subscales would therefore

express the cognitive functions tapped on the level of task performance only. The concurrent

validity of the subscale Auditory Memory is strong in relation to performance on the norm-

referenced Story telling test. The correlations between Visual Memory and Digit Span, or

between Classification and Ordering, and between Auditory Memory and Receptive language

were non-existent. These findings suggest that these subscales measure skills other than those

tapped by the norm-referenced tests. The findings related to the third hypothesis suggest that

behavior displayed during mediation of Auditory Memory is related to the type of cognitive

function tapped. Children who made gains on the subscale that required ability to visualize

contents and activities of stories read were more responsive in their interactions with the assessor

and were able to self-regulate their attention and impulses more effectively on that scale.

However, this relationship of behavior with gains was not apparent on other subscales. Again,

this may be an issue of ceiling effects, or may reflect aspects of the task per se.

The children’s performance during assessment expressed in the Response to Mediation

Scale suggests that the participants felt comfortable during the assessment so there is no reason to

think that the pedagogy involved prevented them from performing optimally. However, there is a

bi-directionality of social interactions that aims at tapping the child’s ability to learn from

instruction. There is a so-called cooperative principle involved. There should be mutual

dependency in the conversation so that the child feels part of the assessment. When the pretest

format does not allow for responsive explanations, then the conversational instruction during the

intervention portion may required to be more explicitly semi-structured to enable the

achievement of mutual dependency (Wells, 2002).

Appeared in Journal of Applied School Psychology, 24, 1 14

The findings of this study contrast with the results of the previous pilot study with a

smaller group of similar students from Dutch schools and with studies carried out with preschool

children in the U.S. The results of these previous studies generally supported the construct

validity of the ACFS as expressed in significant pretest to posttest gains on the subscales. One

difference between this study and the findings of the validation studies described in Table 1 is the

use of multiple assessors. In the Dutch validation study there were six assessors, each with a

somewhat different thesis topic. Just how this would impinge on the results is not clear, but

maintaining the same level of integrity of assessment implementation across this number of

assessors is clearly a challenge that requires further study. The issue of participants being older

than those for whom the ACFS were designed may be another issue that is related to the scoring

of the subscales which needs further attention to validate the scales. It is likely that this related to

the apparent ceiling effects on at least two of the subscales.

And finally, learners usually are referred because there are diagnostic issues that need to be

addressed. Although it is useful to validate the procedures on non-referred students in regular

education programs, this does not necessarily reflect the ultimate applications of these

procedures. This suggests that there are at least two broad approaches to validity that must be

considered: on the one hand, determination of basic psychometric functioning of the instrument

with a “general” population, and, on the other hand, determination of how the procedure

functions with populations for which it is intended. The current study is an example of the

former, and the data contribute to the ongoing investigation of the usefulness of the specific

procedure involved in this study, but with regard to the use of dynamic assessment procedures

with young children in general.

Acknowledgement: We wish to thank Eveline Bouwman, Matthea Baarda, Kim van Leeuwen,

Joan Hof, Wendy Klaver, and Pascal van Santen for collecting the data of this study.

Appeared in Journal of Applied School Psychology, 24, 1 15

References

Bensoussan, Y. (2002). The effectiveness of mediation on three subscales of the

Application of Cognitive Functions Scale, a dynamic assessment procedure for

young children. New York: Unpublished master’s thesis: Touro College.

Brooks, N.D. (1997). An exploratory study into the cognitive modifiability of preschool

children using dynamic assessment. Newcastle, United Kingdom: Unpublished

master’s thesis: University of Newcastle-Upon-Tyne.

Kahn, R. J. (2000). Dynamic assessment of infants and toddlers. In C.S. Lidz & J.G. Elliott

(Eds.), Dynamic assessment: Prevailing models and applications (pp. 325-

374). Amsterdam: JAI/Elsevier Science.

Levy, C. (1999). The discriminant validity of the Application of Cognitive Functions

Scale (ACFS): A performance comparison between typically developing and

special needs preschool children. New York: Unpublished master’s thesis: Touro

College.

Lidz, C.S. (Ed.) (1987). Dynamic Assessment. An Interactional approach to evaluating

learning potential. New York: The Guilford Press.

Lidz, C.S. (2000). The Application of Cognitive Functions Scale (ACFS): An example of

curriculum-based dynamic assessment. In C.S. Lidz & J.G. Elliott (Eds.), Dynamic

assessment: Prevailing models and applications (pp. 407-439). Amsterdam: JAI/ Elsevier

Science.

Lidz, C.S. (2003). Early childhood assessment. Hoboken, NJ: Wiley.Lidz, C.S. (2004).

Successful application of a dynamic assessment procedure with deaf students

between the ages of four and eight years. Educational and Child Psychology, 21(1), 59-

73.

Lidz, C.S., & Elliott, J.G. (Eds.)(2000). Dynamic assessment: Prevailing models and

applications. Amsterdam: JAI/Elsevier Science.Lidz, C.S., & Jepsen, R.H. (2000).

The Application of Cognitive Functions Scales. Currently available through the

second author: (zdilsc@aol.com).

Lidz, C.S., & Van der Aalsvoort, G.M. (2005). Usefulness of the Application of

Cognitive Functions Scales with young children from the Netherlands.

Transylvanian Journal of Psychology, 5, 83-99.

Appeared in Journal of Applied School Psychology, 24, 1 16

Malowitsky, M. (2001). Investigation of the effectiveness of the mediation portion of two

subscales of the Application of Cognitive Functions Scales, a dynamic assessment

procedure for young children. New York: Unpublished master’s: Touro College,

ED457191.

Shurin, R. (1999). Concurrent and discriminant validity of a dynamic assessment procedure with

special needs and typical preschool children. New York: Unpublished master’s thesis:

Touro College, ED435681

Tatik, T. (2000). A concurrent validity study between the Application of Cognitive

Functions Scale and the Leiter-Revised International Performance Test.

Unpublished master’s thesis, Touro College. ED445033

Tzuriel, D. (2001). Dynamic assessment of young children. New York: Kluwer Academic/

Plenum.

Van der Aalsvoort, G.M., & Lidz, C.S. (2002). Reciprocity in dynamic assessment in

classrooms: Taking contextual influences into account. In G.M. Van der Aalsvoort,

W.C.M. Resing & A.J.J.M. Ruijssenaars (Eds.), Learning potential

assessment and cognitive training: Actual research and perspectives in theory

building and methodology (pp. 111-147). Amsterdam: JAI/Elsevier.

Van Kuijk, J.J. (1997). Ordenen [Ordering]. Arnhem: The Netherlands: Centraal Instituut

voor Toetsontwikkeling.

Van Kuijk, J.J. (1996). Taal voor kleuters [Language of children in Kindergarten].

Arnhem, The Netherlands: Centraal Instituut voor Toetsontwikkeling.

Wechsler, D. (2005). Wechsler Intelligence Scale for Children: Third edition. (Dutch

translation). London: Harcourt.

Wells, G. (2002). Responding in Interviews and Tests: Children learning to participate in

the activity of evaluation. Human Development, 45, 187-193.

Appeared in Journal of Applied School Psychology, 24, 1 17

Table 1

Overview of the results of validation studies on ACFS in the US

Type of

validity

N Design

and

populatio

n**

Subscales

1 2 3 4 5 6

Lidz (1996)*

Construct 30 1 X X X X X

Brooks

(1997)

Construct 22 2 X

Shurin

(1998)

Concurrent,

&

discriminant

26 3 X X X X X X

Levy (1999)

Tatik (2000)

Discriminant

& construct

Concurrent

22

15

3

3

X

X

X

X

X

X

X

X

X

X

X

X

Malowitsky

(2001)

Construct 30

2 X X

Bensoussan

(2002)

Construct 20 2 X X X

Lidz (2004) Concurrent &

construct

13 4 X X X X X X

*

Reported in Lidz (2000): Many subscales differed from subsequent versions.

**

1: Response to intervention with high functioning children

2: Response to intervention versus control group with children with developmental delays

3: Response to intervention and differences between children with/without special needs

4: Response to intervention with deaf children

Appeared in Journal of Applied School Psychology, 24, 1 18

Table 2

Overview of Dutch sample and data gathered

School 1 2 3 4 5 6

Age in months

Sex: b-g

SES

Ordering

Receptive Language

Story Telling

Story

Comprehension

Digit Span (WISC-

R)

65

6-4

x

x

66

5-5

x

x

65

6-8

x

x

68

9-6

x

x

x

x

70

15-11

x

67

8-6

x

x

N 10 10 14 15 26 14

Appeared in Journal of Applied School Psychology, 24, 1 19

Table 3

Product Moment Correlation Coefficient of the reliability check on Response to Mediation Scales

Percentage of videotapes

compared

r

School 1

School 2

School 3

School 4

School 5

School 6

10

10

22

20

12

14

.69

.69

.98

.40

.91

.66

Appeared in Journal of Applied School Psychology, 24, 1 20

Table 4

Means and standard deviations of the ACFS Subscales, and range of scores per subscale obtained

by the sample

Pretest

(n = 88)

Posttest

(n = 88)

Range of

scores per

subscale

obtained by

the sample

M SD M SD

Classification 3,3 1,73 4,0 1,73 0-7

Auditory

Memory

3,4 2,94 6,3 4,28 0-17

AM:

retention

6,2 4,00 0-15

Visual

Memory

6,3 1,61 7,2 1,69 1-11

Pattern

Completion

9,2 5,22 11,0 4,31 0-18

Appeared in Journal of Applied School Psychology, 24, 1 21

Table 5

Means, standard deviations, and range of scores of norm-referenced mathematics and language

ability tests

M SD Range of scores

obtained by the

sample

Mathematics

(n = 21)

4,1 1,18 1-5

Receptive

Vocabulary

(n = 8)

3,0 1,18 1-5

Story Telling

(n = 14)

2,2 1,31 1-5

Story

Comprehension

(n = 14)

3,9 0,54 1-5

Digit Span

(n = 21)

10,8 2,80 6-15

Appeared in Journal of Applied School Psychology, 24, 1 22

Table 6

Pearson Correlations between ACFS Subscales and norm-referenced mathematics and language

ability tests

Mathematics

(n = 21)

Receptive

Vocabulary

(n = 8)

Story

Telling

(n =

14)

Story

Comprehension

(n = 14)

Digit

Span

(n = 21)

Class: pretest ,037

Class: posttest -,075

AM: pretest ,276 ,627* ,200

AM: posttest -,218 ,535* ,092

AM: retention ,023 ,703* -,147

VM: pretest -,040

VM: posttest ,277

* p. 05

Appeared in Journal of Applied School Psychology, 24, 1 23

Table 7

Means, standard deviations, and range of scores obtained by the sample of the ratings on

Response to Mediation per subscale

M SD Range of scores

obtained by the sample

Classification 25,7 3,89 15-35

Auditory Memory 24,7 3,78 14-34

Visual Memory 26,1 4,03 15-34

Pattern Completion 26,0 3,74 20-35

Appeared in Journal of Applied School Psychology, 24, 1 24

Table 8

Pearson Correlations between gain scores on ACFS Subscales and Response to Mediation per

subscale

Response to Mediation during assessment

Class AM VM PC

Classification -,151

Auditory

Memory

.306**

Visual Memory ,052

Pattern

Completion

-,035

** p. 01