Reflecting on Scientific Thinking: Children's Understanding of the Hypothesis-Evidence Relation

Wiley and Society for Research in Child Development are collaborating with JSTOR to digitize, preserve and extend access toChild Development.

http://www.jstor.org

Reflecting on Scientific Thinking: Children's Understanding of the Hypothesis-Evidence Relation Author(s): Ted Ruffman, Josef Perner, David R. Olson and Martin Doherty Source: Child Development, Vol. 64, No. 6 (Dec., 1993), pp. 1617-1636Published by: on behalf of the Wiley Society for Research in Child DevelopmentStable URL: http://www.jstor.org/stable/1131459Accessed: 21-07-2015 22:36 UTC

REFERENCESLinked references are available on JSTOR for this article:

http://www.jstor.org/stable/1131459?seq=1&cid=pdf-reference#references_tab_contents

You may need to log in to JSTOR to access the linked references.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at http://www.jstor.org/page/ info/about/policies/terms.jsp

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].

This content downloaded from 139.80.61.47 on Tue, 21 Jul 2015 22:36:51 UTCAll use subject to JSTOR Terms and Conditions

Reflecting on Scientific Thinking: Children's Understanding of the Hypothesis-Evidence Relation

Ted Ruffman and Josef Perner University of Sussex

David R. Olson Ontario Institute for Studies in Education, University of Toronto

Martin Doherty University of Sussex

RUFFMAN, TED; PERNER, JOSEF; OLSON, DAVID R.; and DOHERTY, MARTIN. Reflecting on Scientific Thinking: Children's Understanding of the Hypothesis-Evidence Relation. CHILD DEVELOPMENT, 1993, 64, 1617-1636. 3 experiments were carried out to examine children's understanding of the role of covariation evidence in hypothesis formation. Previous research suggested that it is not until 8 to 11 years of age that children begin to understand how a given pattern of covariation supports a particular hypothesis about which factor is causally responsible for an observed effect. Experiments 1 to 3 employed a different (fake evidence) technique than previous research and showed that by 6 years of age most children understand how evidence would lead a story character to form a different hypothesis than the subject's own. Experiment 3 showed that most 6- and young 7-year-olds understand how a character's future actions (e.g., choice of an object) and predictions of future outcomes depend on the hypothesis he or she holds.

In scientific practice as well as everyday life we are called upon to evaluate some claim, hypothesis, or theory in light of the evidence. Science involves the creation of hypotheses or theories to describe and explain the observed facts. As researchers we generally recognize the tenuous nature of scientific hypotheses. If we put one forward we suppose from the outset that it might be wrong and consequently examine it for its logical consistency and subject it to empiri- cal scrutiny (i.e., determine its fit with the evidence). In everyday life such claims in- clude everything from a politician's assur- ances about an imminent upswing in the economy to the effect certain pesticide resi- dues in food might have on our health. Furthermore, a full understanding of science requires that children recognize that the hypotheses they encounter in texts are formed on the basis of the available evidence, and which are plausible though not necessarily correct ways of explaining the

data. Clearly, then, the ability to evaluate claims or theories and their relation to the evidence is an important skill. It is the development of children's understanding of this hypothesis-evidence relation that we in- vestigated in the research described herein.

Initial attempts (Kuhn, Amsel, & O'Laughlin, 1988) to investigate children's understanding of hypotheses and evidence suggested that before the age of about 11 to 12 years children have very little insight into how hypotheses are supported or contradicted by evidence, and that even at this age and into adulthood understanding is quite shaky. For instance, in some of Kuhn et al.'s stories, children were told that the type of cake eaten-either chocolate or carrot-had a differential impact on whether persons caught colds. There were two persons, one who thought chocolate cake caused colds and the other who thought carrot cake caused colds. Children were then given ac-

The authors would like to thank the children and staff of Balfour, Queen's Park, and West Blatchington Primary Schools for their participation in this study. The research was supported by a fellowship to Ted Ruffman from the Social Sciences and Humanities Research Council of Canada (award no. 756-91-0189). Requests for reprints should be sent to Ted Ruffman, Depart- ment of Experimental Psychology, University of Sussex at Brighton, Biology Building, Falmer, Brighton BN1 9QG, UK.

[Child Development, 1993, 64, 1617-1636. ? 1993 by the Society for Research in Child Development, Inc. All rights reserved. 0009-3920/93/6406-0001$01.00]


1618 Child Development cess to the evidence (i.e., they were shown which characters ate chocolate or carrot cake and whether each went on to catch a cold). In such tasks, children were either asked to explain how the evidence showed that a particular variable made a difference, or they were asked to say which variable was causal and which story character's hypothesis was correct. Children had difficulty with all three of these requests. When asked to assess the evidence they either ignored the evidence and insisted that it was consistent with their own prior theory, or they used the evidence to construct a new theory but failed to understand that this new theory contradicted a previous theory they had held.

Likewise, children's difficulties have been documented on numerous other sorts of scientific reasoning tasks. Beginning with the work of Inhelder and Piaget (1958), and continuing with that of others (Dunbar & Klahr, 1989; Kuhn & Phelps, 1982; Schau- ble, 1990), children have been shown to make confounded assessments when car- rying out experiments and attempting to assess which of a number of variables is causal, and to make little effort to systematically compare different variables when exploring the reasons for an outcome.

However, Sodian, Zaitchik, and Carey (1991) pointed out that the tasks of Kuhn et al. (1988) may have underestimated children's understanding of the hypothesis- evidence distinction because: (1) they included contexts in which children had strongly held beliefs of their own, and it is very plausible that revising such beliefs is more difficult than forming theories when no prior beliefs exist or when beliefs are not held with any degree of conviction, and (2) because they used complex tasks (i.e., the tasks of Kuhn et al. required children to choose which variable was causal while at the same time ruling out a number of other potentially causal variables). Indeed, more recent studies indicated that children possess an understanding of the distinction between hypotheses and evidence earlier than prior research had indicated.

For instance, Sodian et al. (1991) showed that by 6 or 7 years of age children distinguish between a conclusive and an in- conclusive test for a hypothesis. Children were told that a mouse in a home could be either small or large. They were then shown two boxes, each with a piece of cheese inside, and were told that the mouse would

eat the cheese if it could. But while one box had a large opening which either mouse could fit through, the other box had a small opening which only the small mouse could fit through. Children recognized that to determine the size of the mouse it was best to set out the box with the small opening.

In a second set of experiments, Sodian (1991) set out to investigate children's understanding of covariation evidence. In one task children were told that a story character was investigating whether the size of a tennis racket or the material it was made from affected the quality of the serves that the racket could produce. To do so, the story character carried out an experiment in which she had friends make serves with tennis rackets that either varied according to size (with type of material held constant) or material (with size held constant), and raters who rated the quality of the serves. The end product of these individual ratings was an overall score for each of the rackets. Chil- dren were required to understand how the evidence affected the hypothesis that it was size alone that was responsible for serve quality. Thus, they had to pay attention to the way each of the potential causal factors (size and material) varied with the outcome (quality of serve). Children were asked to justify why a particular pattern of evidence was in favour of one hypothesis but contrary to another. By 8 years of age more than half of the children were good at giving such ex- planations, and by 11 years almost all children could do so.

Likewise, Bullock (1991) obtained results consistent with this pattern. She asked children which lanterns a story character should construct to determine whether having a roof on the lantern makes a difference in how well it will stay lit in windy conditions. To help them answer the question, children were shown cards depicting lanterns that varied along three dimensions: whether they had roofs, whether they had small or large holes in the side, and whether the candle inside was short or tall. Most 9- year-olds could choose the correct cards, indicating that the story character should test the hypothesis by building a lantern with a roof and one without, while at the same time keeping the other variables (hole and candle size) constant. Further, children were asked to interpret covariation evidence given hypothetical situations (e.g., they were asked to imagine that the story character made several test objects that varied along each of the three dimensions, tried them out, and then


Ruffman et al. 1619

obtained a given pattern of results). Over half of the 8-9-year-old group were able to justify why the results showed that the focal dimension (e.g., roof presence) did or did not make a difference to outcome, by referring to the covariation of the focal variable with good and bad outcomes.

Yet, it may be that even these results underestimate children's understanding of covariation evidence. First, to a great extent previous research has relied on justifications, and there is a widespread finding (Fla- vell, 1985) that children's ability to explain a phenomenon often lags behind their understanding of the phenomenon itself (as indicated by correct responses on tasks requiring minimal verbal demands). Second, in the research of Bullock (1991), Kuhn et al. (1988), and Sodian (1991), children were required to consider two or more possible causes-for example, in Sodian's study, size and material-and their relation to the outcome. It may be that children could show their understanding of the hypothesis- evidence relation in simplified tasks with only one potential cause. Third, it seems plausible that some understanding of the hypothesis-evidence relation is present before children are able to suggest ways of testing a hypothesis (as Bullock required children to do). In sum, it is possible that a more sen- sitive test of children's abilities would re- veal that they come to understand how a pattern of evidence relates to a hypothesis sometime before 8 years.

There is one other reason for thinking that previous research might have underestimated children's abilities, this one tied to what related research has revealed about children's abilities rather than potential pro- cedural shortcomings. Kuhn (1989) pointed out that differentiating between a claim and evidence requires metacognitive abilities. For instance, children must understand how a pattern of evidence leads one to form a particular idea as to what is causal. In addition, Kuhn argues that the "key" insight involves an "awareness that things could be otherwise" and that a theory or claim is recognized as such when "its possible false- hood and the existence of alternative theories are recognized" (Kuhn, 1989, p. 684). Because it was not until adolescence or even adulthood that subjects solved her tasks, Kuhn argued that children must possess a metacognitive deficit. Yet, given the new in- sights that "theory of mind" research has provided into children's metacognitive abilities, such a claim seems curious (e.g., see

Perner, 1991). For instance, recent research (Ruffman, Olson, Ash, & Keenan, 1993) indicates that as early as 4 or 5 years of age children are able to infer how misleading evidence will lead an onlooker to form a false belief with regard to who took an item. Be- liefs and hypotheses are mental states, and in each case an understanding of their origin requires metacognitive or metarepresenta- tional abilities. If children possess the metacognitive abilities to judge how evidence will affect an onlooker's beliefs by this early age, then it is odd that they cannot begin to assess how evidence will affect a hypothesis until age 8 or 9 (Bullock, 1991; Sodian, 1991), or until age 11 or even later (Kuhn et al., 1988).

In the first experiment, we introduced the Faked Evidence task primarily in an effort to simplify task demands relative to previous research. In this task, we showed children drawings of boys who had eaten either green food or red food. Some of the boys had lost teeth and others had not. The evidence covaried perfectly with outcome, so that all boys who ate one type of food (e.g., green food) lost teeth while all boys who ate the other type maintained all their teeth. We asked subjects which type of food caused tooth loss. Then, in the presence of the subject but not a story character, we tampered with the evidence so that it would appear to anyone not exposed to the initial evidence pattern (e.g., the story character) that the opposite food (e.g., red food) caused tooth loss. The story character then arrived on the scene and we asked subjects which type of food he would say caused tooth loss. In this way the story character, but not the subject, was led to a mistaken hypothesis with regard to which food caused tooth loss.

This Faked Evidence task had three main advantages, the first two of which were aimed at reducing task complexity. First, unlike the studies of Bullock (1991), Kuhn et al. (1988), and Sodian (1991), it tapped children's understanding of the hypothesis- evidence distinction without requiring ver- bally demanding justifications. Second, the number of potential causes of an outcome was reduced from two or more (as in previous research) to the bare minimum of one. Third, this task included a safeguard to ensure that children's responses reflected a genuine understanding of the hypothesis- evidence distinction. That is, it ensured that children assess a given hypothesis regarding which variable was causal independent of their own such hypothesis. In our task, chil-


https://www.researchgate.net/publication/20356898_Children_and_Adults_as_Intuitive_Scientists?el=1_x_8&enrichId=rgreq-e3e1ef43a5973afaebd11128bd67b213-XXX&enrichSource=Y292ZXJQYWdlOzE1MDg3MzYyO0FTOjI1ODExOTU0MDQwODMyMEAxNDM4NTUxOTAyMjE0

1620 Child Development dren had to understand that the story character would form a different hypothesis to their own. Because this hypothesis was different to their own, children could not rely on their own knowledge about what was correct when assessing the character's hypothesis. Instead, their recognition that the story character would form a different hypothesis must have been based on the story character's access to different evidence because this was the only way in which the character's experience differed from that of the child. As a result, our task had a distinct advantage over some of the tasks of Kuhn et al. (i.e., those in which children were asked to assess which variable was causal and assess the impact of the evidence on story characters' hypotheses).

In such tasks, Kuhn et al. (1988) (a) asked children if a given pattern of evidence showed that a particular variable (e.g., type of cake) made a difference as to whether an outcome (e.g., colds) would occur, and (b) told children of two people who held different theories and asked them to assess whether one person was right in light of a given pattern of evidence. To answer these questions children could simply use the available evidence to form a hypothesis about which agent was causal. They could then use this hypothesis (their own hypothesis) to assess whether a variable such as "type of cake" did make a difference as to whether persons caught colds, and they could use a simple matching procedure to compare this hypothesis with that of another person and say whether that person's hypothesis was correct. Yet forming the correct hypothesis and being able to employ a matching strategy do not address whether children possess the metacognitive abilities to reflect on the evidence to understand how it could lead the other person to form her hypothesis in the first place. This can only be determined (a) when children explicitly refer to the available evidence in their justifications for why a particular hypothesis is supported-what Bullock (1991), Sodian (1991), and in other experiments, Kuhn et al. required children to do-or (b) through the use of our method. Once again, in our method we required not just that children form the correct hypotheses themselves, but that they understand how the evidence would lead the story character to form a different hypothesis.

Thus, while Kuhn (1989, p. 679) claimed that children's difficulty on the tasks of Kuhn et al. (1988) was likely due to a metacogni-

tive deficit, that is, an inability to reflect on a theory "as an object of cognition" and understand how evidence bears upon it, it was actually possible for children to succeed on such tasks without possessing metacognitive abilities.

Experiment 1 METHOD

Subjects In the first experiment we tested 16 4-

year-olds (mean: 4-7, range 4-1 to 4-11, 7 girls and 9 boys) and 16 5-year-olds (mean: 5-4, range 5-0 to 5-10, 6 girls and 10 boys). Subjects were drawn from a largely white, lower-middle-class, elementary school in a mid-sized city in England. The first language of all subjects was English. Materials

Materials for the Hypothesis-Evidence task included a doll (20 cm tall) who represented another person and whose hypothesis children were asked to assess, and a sheet of white paper (21 x 34 cm) with 10 boys' heads drawn on it. Five of the boys had healthy teeth, while five had missing teeth. Directly in front of the boys' mouths was a piece of "food" (a small shape cut out of either red or green construction paper). Mate- rials for the False Belief task included a pencil, two small plastic dolls (different from the one used in the Hypothesis-Evidence task), and two hiding places (two differently col- ored envelopes). Procedure

Hypothesis-Evidence task.-First, children were introduced to the story character, a doll named "Sally." Sally then left to go to the "playground" (underneath the experimental table), where it was said she could no longer see or hear what happened. Children were shown a drawing of a boy eating one type of food-for instance, green food-and who had several teeth missing: "Look, this boy ate some green food. And look, he has some teeth missing." They were then shown nine more drawings in sequence, four more of boys eating green food who had also lost teeth, and five of boys eating red food who possessed the full complement of strong and healthy teeth. For half the children green food was associated with tooth loss and for the other half red food was associated with tooth loss. Next, children were asked to assess the covariation evidence. They were asked, "Which type of food makes kids' teeth fall out?" All children associated the correct


Ruffman et al. 1621

food with tooth loss, showing that they had no difficulty interpreting the covariation evidence.

Then the experimenter faked the evidence by rearranging the 10 pieces of "food" so that it now suggested the opposite food to be the source of tooth loss. Thus, if children had observed green food as the cause of tooth loss, the five pieces of green food were moved to the drawings of boys who had healthy teeth and the five pieces of red food were moved to the drawings of boys who had teeth missing. Prior to faking the evidence, and in order to prevent confusion about what the real state of evidence actually was, children were reminded of which type of food each group of boys had actually eaten: "Now remember, really and truly, these boys ate green food didn't they? And these ones ate red food. Now you and I know that's true don't we? But let's do something before Sally gets back. Let's move the food around. So we'll move this piece here and this one here, and this piece here and this one here,..." Once the evidence had been faked, children were again asked which type of food each group of boys had actually eaten: "Now remember, really and truly, which color of food did these boys eat? And which color did these boys eat?" If children erred on either of these questions they were corrected.

Next, Sally returned and observed the evidence in the misleading state, and children were asked to say what kind of food she would say causes kids' teeth to fall out: "Sally comes back and sees things the way they are now. She didn't see which food the boys ate, did she? When Sally sees things the way they are now, which food will she say makes kids' teeth fall out?" To assist children in answering this question they were shown two pieces of "food": one green and one red. Then, they were asked to justify their response: "Why will Sally say the [red/ green] food makes kids' teeth fall out?" Fi- nally, children were asked a control question to ensure that they had not revised their own hypothesis regarding which kind of food was the culprit. That is, children who understood that the doll would hold a mistaken hypothesis regarding which food caused tooth loss but who had revised their own hypothesis, basing it on the misleading evidence rather than the real evidence, would not have assessed the doll's hypothesis independent of their own hypothesis. If so, children's responses to these questions would not tell us anything over and above

the data of Kuhn et al. (1988). Children could obtain the correct response simply by using the data to form the correct hypothesis. They would not have to understand how the evidence resulted in a particular hypothesis (an insight that requires metacognitive abilities). Thus, the experimenter said, "And what about you? What food do you say makes kids' teeth fall out?"

False Belief task.-First, children were introduced to the two dolls ("Katy" and "John"). John put his pencil in one envelope and then went to the "playground." While he was in the playground Katy moved the pencil to the second envelope. At this point children were asked a control question to ensure they remembered the actual location of the pencil ("Now remember, where is the pencil now?"). John then returned and children were asked the Belief question ("Where will John say the pencil is?") and two more control questions to make sure they had kept track of important events ("Did John see Katy move the pencil?" and "Where did John put the pencil in the beginning?").

Half the children received the Hypothe- sis-Evidence task first and half received the False Belief task first.

RESULTS AND DISCUSSION

Recall that in the Hypothesis-Evidence task children were asked to assess both their own and the story character's hypothesis. Because there were two questions and two possible answers for each question, the probability of being correct on both simply by chance was 0.25. Consequently, one would expect one-quarter of the children in each age group-four 4-year-olds and four 5-year-olds-to succeed simply through chance. Furthermore, if 12 or more children did succeed, then it would be clear that these children, as a group, were performing above chance. Row 1 of Table 1 shows that, in fact, six 4-year-olds and 13 5-year-olds answered both questions correctly. Thus, on the basis of the answers to the two hypothesis-evidence questions alone, it is clear that the 5-year-old group was performing above chance and understood the hypothesis- evidence relation (binomial test: N = 16, k = 13, p < .01).

Another two children (one 4- and one 5-year-old) attributed a hypothesis to the doll that was consistent with the faked evidence (e.g., green food makes teeth fall out) but then failed to maintain their original


1622 Child Development TABLE 1

CHILDREN'S RESPONSES TO THE HYPOTHESIS QUESTION IN EXPERIMENT 1

Doll Self 4-Year-Olds 5-Year-Olds

Correct Correct 6 (38%) 13 (81%) Correct Incorrect 1 (6%) 1 (6%) Incorrect Correct 6 (38%) ... (0%) Incorrect Incorrect 3 (19%) 2 (13%)

and contradictory hypothesis (e.g., red food makes teeth fall out) and claimed that they too thought green food was causal (row 2 of Table 1). These children may have been confused by the manipulation of evidence and believed the faked evidence constituted the real evidence. Unlike children who maintained a contradictory hypothesis, it is not clear whether such children identified the doll's hypothesis independent of their own hypothesis. The remaining children (nine 4-year-olds and two 5-year-olds) failed even to assess the story character's hypothesis correctly (rows 3 and 4 of Table 1).

Children's justifications of why the story character would form a particular hypothesis ("Why will John say the [red/green] food makes kids' teeth fall out?") are also of interest. Of the children who answered the two hypothesis-evidence questions correctly, 1 of 6 4-year-olds and 2 of 13 5-year-olds provided justifications in which they clearly contrasted the evidence in favor of the doll's hypothesis and the evidence in favor of the alternative hypothesis (e.g., "Because that's got two teeth and that's got more"; "The red does and that one [green] doesn't"). A large number of other children (three 4-year-olds and nine 5-year-olds) provided justifications that suggested that they may have understood how the evidence led the doll to form her hypothesis, but that were not "convincing" because they did not involve a direct contrast of the two sets of evidence (e.g., "They've only got two teeth"; "Because that [red food] was supposed to be over there and you changed it"; "'Cuz we switch it around").

A third result concerns children's performance on the Hypothesis-Evidence task relative to the False Belief task. As expected, children did very well on the False Belief task. All children recognized that the doll who had not witnessed the transfer of the pencil would come to hold a false belief regarding its whereabouts. At the same time,

these children also passed the three control questions (including the question about where they thought the pencil was). This indicates, among other things, that these children assessed the story character's belief independent of their own belief. Thus, children's performance on our Hypothesis- Evidence task lagged behind that on the False Belief task. At first glance this result is somewhat surprising because of the paral- lels between the two tasks: each requires a "metacognitive" insight into how a person's mental state arose on the basis of information in the world.

It may be, however, that the difference in children's performance on the two tasks was due to (a) the fact that the Hypoth- esis-Evidence task involved 10 pieces of evidence (what each of the 10 boys ate), whereas the False Belief task involved only one piece (the pencil), or (b) confusion regarding which information was veridical in the Hypothesis-Evidence but not the False Belief task.

As for the latter of these possibilities, in the False Belief task there could be little doubt as to which information (location) was veridical. First, it was simply not possible that both pieces of information (pencil in envelope A, pencil in envelope B) were cur- rently true. Second, the veridical state was specified, not by the experimenter's words as in the Hypothesis-Evidence task, but by a much less confusing and more reliable means: what children could see with their own eyes. So the pencil could only be in one envelope, and when children were asked to say which state was veridical all children were correct in doing so. That is, when they were asked to name the pencil's current location (the question asked immediately prior to John's return), all children named the second envelope. In contrast, there could be confusion in the Hypothesis-Evidence task. First, it was possible for children to believe both sets of evidence were veridical, in


Ruffman et al. 1623

other words, to disregard or misunderstand what the experimenter said and think that both kinds of food harmed teeth. Second, it was possible that children did believe only one type of food harmed teeth, yet were confused by the faked evidence manipulation and the experimenter's words, and believed that the faked evidence was veridical.

Experiment 2 Experiment 2 had two purposes. First,

it was designed to rule out an alternative in- terpretation for the results of Experiment 1. In Experiment 1, the evidence was faked in order to lead the story character to a mistaken hypothesis. We did this so that children's prediction of the character's hypothesis was based on an understanding of the hypothesis-evidence relation (i.e., how hypotheses are formed on the basis of evidence). However, children may have correctly predicted the character's hypothesis, not on account of the pattern of (faked) evidence, but simply because the evidence had been manipulated. If so, children might believe that any manipulation would lead the character to a mistaken hypothesis irrespec- tive of the pattern of evidence the manipulation resulted in. To examine this possibility, we manipulated the evidence in one condition of Experiment 2, but in such a way that it did not affect the pattern of evidence or theoretical conclusion (i.e., the story character would come to the same conclusion as the subject who had access to veridical information).

A second purpose of Experiment 2 was to examine children's understanding of imperfect evidence. In the first experiment, all pieces of evidence were consistent with one hypothesis and inconsistent with the other. However, it is possible that patterns of evidence can be ignored when evidence covaries perfectly with outcome. Instead, individual cases can be singled out and used as evidence for a given hypothesis, particularly if such cases are consistent with a child's own hypothesis (as Kuhn et al. [1988] found children so prone to do). However, in Exper- iment 2 we employed imperfect covariation: most but not all of the evidence was in favor of a particular hypothesis and contradicted a second hypothesis. In this case, individual pieces of evidence cannot be singled out in the same way because not all evidence associated with a particular outcome is consistent. Thus, it is paramount to consider the pattern the evidence presents. And the ability to understand the significance of patterns

of evidence is of great value given that in many areas of science (e.g., psychology) evidence that varies imperfectly with outcome is the norm. For this reason, the imperfect evidence employed in Experiment 2 provides important information.

METHOD

Subjects Subjects included 27 5-year-olds (mean:

5-4, range: 4-9 to 5-11, 17 girls and 10 boys), 17 6-year-olds (mean: 6-7, range: 6-3 to 6-9, 9 girls and 8 boys), and 10 7-year-olds (mean: 7-7, range: 7-1 to 7-11, 3 girls and 7 boys). Subjects were drawn from a largely white, middle-class, elementary school in a mid- sized city in England. The first language of all subjects was English. Materials

A doll named "John" (20 cm tall) served as the character whose hypothesis children were asked to assess. Materials included six cardboard figures (4 cm tall), each of whom had a small blue hat on her head. There were two lots of "food": six green and six red pieces. The food consisted of a green or red star (1 cm) stuck to a small, white cardboard backing. Procedure

Children were shown the two types of food, which sat in separate piles on the table, and the six girls and their hats, which sat a small distance away. Before the stories be- gan, children were asked to pretend the story characters were real people, and that the stars were really two types of food. Then the experimenter said, "The girls will go and eat some food and then I'll ask you which type of food they like to eat. John is going to go away so he won't see which food the girls eat but when he comes back I'll ask you which type of food he'll say the girls like to eat. Now we've got some pictures to help you answer the questions." At this point the experimenter pointed to three drawings that sat on the center of the table. All three drawings depicted a girl smiling broadly but one showed her thinking of red food, one showed her thinking of green food, and one showed her thinking of both types of food. It was explained to children that this one girl represented all the girls and that her thought and smile combined to indicate which type(s) of food she liked. Then, to ensure that children understood what the pictures meant, the experimenter said, "Show me the picture which shows the girls liking [red/green/both types of] food." All children answered these questions correctly.


1624 Child Development There were four stories in total. In all

stories children (but not the story character, John) watched while, in succession, each of the girls went over to one of the two food piles, ate a piece of food, and then returned to where she had been near the center of the table. When each girl bent down to eat some food, her hat fell off her head and remained behind as a clue to which food she had eaten. There were two Likes Both stories ("[3,3]--+ [1,5]" and "[3,3]-- [3,3]") and two Likes One stories ("[5,1]--+[1,5]" and "[5,1]-- [5,1]"). The veridical evidence in the two Likes Both stories indicated that the girls liked both kinds of food; three girls ate red food and three ate green food. The veridical evidence in the two Likes One stories indicated that the girls liked only one type of food; five girls ate one type of food (e.g., green) and only one ate the other (e.g., red).

Then, before John returned, the experimenter manipulated the evidence. In the

"(3,3)---(1,5)" story, the experimenter moved two hats from one pile and placed them beside the second pile, leaving only one hat beside the first pile and five beside the second pile. Thus, the faked evidence would suggest an incorrect hypothesis to John upon his return: that the girls like to eat a particular type of food rather than both types. In the

"(3,3)---(3,3)" story, the experimenter moved two hats from one pile and placed them beside the second pile, and then moved two other hats back to the first pile, resulting in the same number of hats as were originally beside each pile. In this case the manipulation would not lead John to the wrong hypothesis.

In the "(5,1)---(1,5)" story, the experi-

menter moved four hats from the first pile and placed them beside the second pile so that John would form an incorrect hypothesis upon his return (as in the "[3,3]--[1,5]" story). In the "(5,1)--* (5,1)" story, the experimenter moved four hats from the first pile and placed them beside the second pile, and then moved four hats to the first pile, leaving the original number of hats beside each pile. As in the "(3,3)--(3,3)" condition, this manipulation would not lead John to the wrong hypothesis. These stories are described in greater detail below.

In all conditions, before the evidence was manipulated, the experimenter asked children to state their own hypothesis about which food the girls liked. Thus, after children had viewed which food the girls had

eaten, but before the hats were manipulated, the experimenter said, "Which picture best shows which type of food the girls like to eat? Do they like to eat the red food, or the green food, or do they like to eat both types of food?" The experimenter counterbalanced the order in which each of the three pictures was named and pointed to each upon mention.

Then the experimenter said, "Now you and I know which food the girls ate. We know that three girls ate red food and three ate green food. But let's do something before John gets back. We'll move some hats. We'll move these hats over here [and these hats over here-in the "(3,3)---(3,3)" and "(5,1)-*-(5,1)" stories]. Now, how many hats are by the green pile now? And how many are by the red pile?" Then John returned and children were asked the Hypothesis questions: "John comes back and sees things this way. Which picture best shows which type of food John will say the girls like to eat? Will John say they like to eat the red food, or the green food, or will he say they like to eat both types of food? And what about you? Do you say they like to eat the red food, or the green food, or do they like to eat both types of food?" Although children had been asked previously about their own hypothesis, it was their answer to this second question that we considered because it required them to maintain their original hypothesis having just ascribed a different hypothesis to John. Finally, children were asked the Justification question: "Why will John say that?"

Children received (in a counterbalanced order) two stories each: either the two Likes Both stories or the two Likes One stories. In each story, children were asked two Hypothesis questions (one about themselves and one about John), and one Justification question (about John). Children were ran- domly assigned to either of the condition pairs in the following proportions: Likes Both stories-14 5-year-olds, 9 6-year-olds, and 8 7-year-olds; Likes One stories-13 5- year-olds, 8 6-year-olds, and 2 7-year-olds. More 7-year-olds were not tested on the Likes One stories because testing revealed that performance had reached near ceiling level by 6 years of age. Thus the two who were tested were included essentially as the result of experimenter error. RESULTS AND DISCUSSION

Recall that children were given three choices-red, green, or both-when asked


Ruffman et al. 1625

TABLE 2

NUMBER AND PERCENTAGE OF CHILDREN PASSING THE HYPOTHESIS AND/OR JUSTIFICATION QUESTIONS OF THE LIKES BOTH AND LIKES ONE

STORIES IN EXPERIMENT 2

5-YEAR-OLDS 6-YEAR-OLDS 7-YEAR-OLDS

N % N % N %

Passing all 4 Hypothesis questions:

Likes Both .............. 3 21 5 56 6 75 Likes One .............. 3 23 7 88 2 100 Subtotal ................ 6 22 12 71 8 80

Failing a Hypothesis question but offering a convincing justification 9 33 4 24 Total .................. 15 56 16 94 8 80 NoTE.-5-year-olds: N = 27; 6-year-olds: N = 17; 7-year-olds: N = 10.

which food the story character would think the girls liked, and which food they themselves thought the girls liked. Further, each subject, whether given the Likes Both or Likes One stories, was asked four such Hy- pothesis questions (two for self, two for the story character). The probability of obtaining the correct answer on any one of these questions simply by chance was 0.33, and the probability of obtaining a correct answer on all four questions by chance was .012. Thus, children who passed all four Hypothesis questions would have been at a level significantly above chance. It would be clear that such children understood the hypothesis-evidence relation because they would have (a) assessed the story character's hypothesis independent of their own and understood how the evidence could lead the character to a mistaken hypothesis, and (b) not attributed a different hypothesis to the doll simply because the evidence had been manipulated. With regard to this latter point, recall that it could not have been the mere manipulation of evidence because in both the

"(3,3)---(3,3)" and

"(5,1)---(5,1)" stories

the evidence was also manipulated but the pattern remained the same. Children who recognized that the doll's hypothesis would have been identical to their own in these two stories would have demonstrated that their assessments were based on the pattern of evidence. Over the two Likes Both stories, three 5-year-olds (21%), five 6-year-olds (56%), and six 7-year-olds (75%) obtained the pattern described above, while over the two Likes One stories, three 5-year-olds (23%), seven 6-year-olds (88%), and both 7- year-olds (100%) passed all four Hypothesis questions.

One can also determine when it can be said that children as a group (and not just as individuals) understand the hypothesis- evidence relation. Table 2 includes the number and percentage of children passing all four Hypothesis questions in each of the story pairs. Because performance on the Likes One and Likes Both stories was not significantly different, we have collapsed across the two story pairs for this test. The proportions of 6-year-olds (binomial test: N = 17, k = 12, p < .04) and 7-year-olds (binomial test: N = 10, k = 8, p < .02) passing the four Hypothesis questions were significantly above chance.

Justifications.-After children had assessed the story character's hypothesis, they were then asked why he would say the girls liked green, red, or both kinds of food. As in Experiment 1, children had to contrast the evidence in favor of each hypothesis to be counted as offering a "convincing" justification. In the

"(3,3)---(3,3)" condition, when the correct response to the question about the character's hypothesis was that he would think the girls liked both kinds of food, examples of justifications counted as indicating an understanding of the hypothesis- evidence distinction were of the following form: "Because they've both got the same amount [of hats]. They've eaten the same"; or "Because there's three of that [pointing to the hats by one food pile] and three of that [pointing to the hats by the other food pile]." In the remaining stories-"(3,3)--+ (1,5)", "(5,1)--+(1,5)," and "(5,1)--(5,1)"- when the general or simplifying hypothesis was that the character would think the girls liked only one kind of food, examples of jus-


1626 Child Development tifications counted as indicating an understanding of the hypothesis-evidence distinction were of the following form: "'Cuz there's the most hats there," or "'Cuz there's more hats there than the green." Justifica- tions not counted as convincing included: "Because there's lots of hats"; "Because there's five there"; or "'Cuz they got three hats by it." These justifications were not deemed "convincing" because children had not explicitly contrasted the two sets of evidence.

The "convincing" justifications provide converging evidence for children's understanding of the hypothesis-evidence relation. First, consider the children who passed all four Hypothesis questions of the Likes Both stories. Two of three 5-year-olds, four of five 6-year-olds, and six of six 7-year-olds also provided at least one (of two possible) convincing justifications. Next, consider the children who passed all four Hypothesis questions of the Likes One stories. Two of three 5-year-olds, six of seven 6-year-olds, and two of two 7-year-olds provided at least one (of two possible) convincing justifications. Children's performance on the Justi- fication questions of the Likes Both and Likes One stories was (like the Faked Evi- dence questions) not significantly different so that we have collapsed across the Likes Both and Likes One stories for all remaining tests.

The justifications also suggest that con- sideration only of children's answers to the Hypothesis questions might lead to an un- derestimation of their abilities. That is, there were some children who failed a Hypothesis question yet offered a convincing justification. For this reason, we have computed a liberal estimate that indicates the number of children whose performance suggests that they have at least some understanding of the hypothesis-evidence relation.

For instance, as in Experiment 1, a number of children apparently became confused as to which state of evidence was veridical. These children assessed the story character's hypothesis correctly in both stories. However, in the story in which the manipulation of evidence would suggest a mistaken hypothesis to the story character

("[3,3]-- [1,5]" or "[5,1]-- [1,5]"), such children also

based their own hypothesis on the faked evidence. On the basis of their responses to the Hypothesis questions alone, it was not clear whether such children had assessed the doll's hypothesis independent of their own

hypothesis (i.e., understood the hypothesis- evidence relation). Seven 5-year-olds, three 6-year-olds, and one 7-year-old responded in this manner over the Likes Both or Likes One stories. Yet six of these children (four 5-year-olds and two 6-year-olds) provided a convincing explanation when asked why the story character would form a mistaken hypothesis.

Another group of children assessed the story character's hypothesis correctly in one story of a given story pair, but not in the other. Although such children did not answer all the Hypothesis questions correctly, they often provided a convincing justification in one story for why the story character would form a particular hypothesis. There were 13 5-year-olds and two 6-year-olds who assessed the story character's hypothesis correctly one out of two times. Of this group, five 5-year-olds and both 6-year-olds offered a convincing justification.

In total, then, there were nine 5-year- olds and four 6-year-olds who did not pass all four Hypothesis questions, yet whose justifications suggest they have some understanding of the hypothesis-evidence relation. Whether such children's failure on one or more of the Hypothesis questions stems from a genuine difficulty or from other factors (e.g., wandering attention during testing) is not clear. Nevertheless, Table 2 includes the number of children either passing the conservative measure (all four Hypothe- sis questions) or the liberal measure (failing a Hypothesis question but offering a convincing justification). Importantly, including children who passed the liberal measure does not change our claim as to when children (as a group) can be said to understand the hypothesis-evidence relation. It simply provides added support to our claim that by 6 years of age children have grasped this insight, and suggests that a good many 5-year- olds may also understand the relation.

Children's ability to justify their answers to the questions about the story character's hypothesis is somewhat surprising because we thought that the reliance on explicit justifications was what made the tasks of Bullock (1991), Kuhn et al. (1988), and Sodian (1991) so difficult. It shows that young children's failures in previous studies could not have resulted simply from a general inability to provide "explicit justifications." This topic is addressed below in the General Discussion, where we suggest a reason why children may have difficulty offering certain types of justifications.


Ruffman et al. 1627

Patterns of evidence.-In Experiment 2 we employed imperfect covariation so that children had to understand that the story character's hypothesis would be based on the pattern of evidence, even if all evidence was not consistent with that pattern. Our results show that by 6 years of age children understand that evidence need not be perfect for one to arrive at a particular hypothesis. This result is important in that (a) a great deal of evidence in everyday life is imperfect, and (b) Kuhn et al.'s (1988) results suggested that this understanding was not present before 11 or 12 years of age, and Sodian's (1991, Experiment 1) results suggested that it was not present before about 9 years of age.

Experiment 3 The first purpose of Experiment 3 was

to rule out an alternative explanation for the results of Experiments 1 and 2, and the results obtained in previous research (Bullock, 1991; Kuhn et al., 1988; Sodian, 1991). Ex- periments 1 and 2 suggested that by 5 or 6 years of age children understand something about the way in which evidence leads to hypothesis formation. Yet it could be argued that when children attributed a particular view to the story character their answer was based on a mere description of the state of affairs to which the story character had access (e.g., "the evidence indicates that the boys experienced tooth loss when they ate red food") without any notion that things were likely to turn out that way again (e.g., "whoever eats red food will experience tooth loss"). If so, children would not have been attributing a "hypothesis" to the story character because one of the characteristic features of theories and hypotheses is that they allow prediction of future outcomes.

To test their understanding of the pre- dictive properties of hypotheses, children were told a story based on that of Sodian (1991, Experiment 2). Two story characters wanted to know if a given feature (e.g., size) made a difference in how hard tennis players could hit a ball. One character was shown veridical evidence (large rackets give hard shots, small rackets give soft shots) and the other was shown faked evidence (large rackets give soft shots, small rackets give hard shots). Children were asked two questions about each character. First, they were told that a player was going to take another shot with the large racket and asked whether each character would say the shot would be hard or soft. Second, the two players went to

a store where large and small tennis rackets (similar to the ones used in the experiment) were sold. Children were asked which racket each player would buy.

A second purpose of Experiment 3 concerns the relation between false belief tasks and our faked evidence task. There are some important similarities between our task (which we have claimed investigates children's understanding of the hypothesis- evidence relation), and tasks that investigate children's understanding of false belief. In one (Misleading Container) version of the false belief task (e.g., Perner, Leekam, & Wimmer, 1987, Experiment 2) children are shown that a familiar candy box contains a pencil rather than the expected candy (Smarties). They are then asked what another person who has not seen inside the container will think is in there. In another (Translocation) version of the false belief task, a story character hides an object at location A and the object is moved to location B in the character's absence (e.g., Perner et al., 1987, Experiment 1). Children are asked where the character will think the object is.

In these tasks children must apply general rules (e.g., Smarties boxes contain Smarties; a person thinks an object is where they have last seen it) and arrive at specific predictions (the story character will think this box contains Smarties; the story character will think the object is at A). There is, then, a similarity between these false belief tasks and the hypothesis-evidence task we employ in Experiment 3. In the hypothesis- evidence task children must also apply some general rules (e.g., people form beliefs about efficacy on the basis of evidence) to arrive at a specific prediction (the story character shown veridical evidence will think the large tennis racket which was used in the experiment gave harder shots than the small one which was used in the experiment). Yet there is an important difference because children must go beyond this specific prediction and recognize that the character will also make more general predictions (all large tennis rackets will give harder shots). These generalizations are tacitly based on the causal properties of size ("bigness" vs. "smallness"). In sum, our hypothesis- evidence task taps an understanding of something that is "hypothesis-" or "theory"-like in that children must understand that the evidence will lead the story character to form generalizations. Success on some false belief tasks requires children to have made generalizations (e.g., Smarties


1628 Child Development boxes contain Smarties) but does not require them to understand that another person would use evidence to form generalizations.

A third purpose of Experiment 3 was to directly compare the differences between the way children's understanding of the hypothesis-evidence relation was assessed in Experiments 1 and 2, on the one hand, and in previous research on the other. Bullock (1991), Kuhn et al. (1988), and Sodian (1991) found that children begin to understand the hypothesis-evidence relation sometime between 8 and 11 or 12 years of age, whereas we found understanding by age 5 or 6. Two major differences between these two sets of studies were (a) that we used the faked evidence paradigm, whereas previous researchers (at least in some studies) relied on children giving justifications, and (b) there was only one potential causal factor in our studies (e.g., color of food), whereas there were two such variables in the study of Sodian and three or more in the studies of Bullock and Kuhn et al. In Experiment 3, we asked children a number of different questions. Some were similar to those employed by Bullock, Kuhn et al., and Sodian, whereas others were an extension of the questions included in the Faked Evidence conditions of Experiment 1 and 2. In this way we could determine whether the justifications required by previous researchers were the reason why they reported later understanding than our results indicated. In addition, we manipulated the number of potential causal variables to determine whether this factor affected children's performance.

The fourth purpose of Experiment 3 was to ensure that subjects in Experiments 1 and 2 really did understand the way in which exposure to a particular state of evidence would lead to the formation of a particular hypothesis (and only that hypothesis). In Ex- periments 1 and 2, children observed different evidence than a story character and so formed a different hypothesis. For this reason we claimed that children could not rely on their own knowledge of what was right when assessing the story character's hypothesis, and had to understand how the evidence led the story character to form that hypothesis. However, an alternative explanation is possible. Children could have rea- soned that the story character saw a different state of evidence and would therefore form a different hypothesis (one that was opposite to the child's own). Yet they might not have understood that a given state of evidence specifies a particular hypothesis, not just a

different hypothesis. In Experiment 3 we assessed whether young children are limited in this respect by presenting one state of evidence to the child, another state to one doll, and a third state to the other doll. This way the child had to understand, not just that the dolls would be led to different hypotheses than the child herself, but that each doll would be led to a specific hypothesis corre- sponding to the particular evidence each was shown.

We only tested children no younger than 6 because 6-year-olds formed the youn- gest age group in both Experiments 1 and 2 for which understanding was very solid. If such children passed the additional controls of Experiment 3 then we could feel very confident that they understood the hypothesis-evidence relation.

METHOD

Subjects Subjects included 18 6- and 7-year-olds

(mean: 6-11, range 6-7 to 7-1, 11 girls and 7 boys) and were drawn from a largely white, middle-class, elementary school in a mid- sized city in England. The first language of all subjects was English.

PROCEDURE

There were four material sets: Racket- Size, Strings-Number, Ball-Size, and Head- Shape. All children were told four stories, each story employing a different material set. In each story children were told that two dolls, "Katy" and "John," wanted to know whether a given factor (i.e., size of racket, number of strings, material of ball, shape of head) mattered in how hard a player could hit the ball. These tasks are described in greater detail below. For ease of exposition, all descriptions will concern the Racket-Size material set. The assignment of material sets to ordinal positions and story types was counterbalanced across children. Four lists were created, and a child received one of those lists. The lists were designed so that a given material set was used once in the first story a child was told, once in the second story, once in the third story, and once in the fourth story. Further, a given material set directly preceded another given set only once in the four orders. The counterbalanc- ing also was such that, among the four lists, a given material set was combined with a particular story type only once.

There were four different story types: One Variable-Two Hypotheses, Two Vari-


Ruffman et al. 1629

FIG. 1.-Materials of Experiment 3: top row, Two Variables-Two Hypotheses story; middle row, One Variable-Three Hypotheses story; bot- tom row, One Variable-Two Hypotheses story and One Variable-One Hypothesis story. (Note: for each subject the sets of materials used differed from story to story.)

ables-Two Hypotheses, One Variable- One Hypothesis and One Variable-Three Hypotheses.

One Variable-Two Hypotheses story.-- In this story, the subject and two dolls were shown two tennis rackets that differed only in size. Figure 1 illustrates the nature of the materials used in this and the three other stories. While different materials were used in each story for a given child, only the Ten- nis-Size materials are included in Figure 1. The experimenter always pointed out how the materials were different, and subjects were told that the dolls would perform an

experiment to determine if size matters in how hard a player can hit the ball. One (Un- informed) doll then left and the dolls' experiment was described to the subject and the remaining (Informed) doll in detail to dis- suade children from claiming the results were invalid due to improper experimental practices.

For instance, the experimenter said, "Do you know what Katy and John did? They got six of their friends and each friend took five shots with each racket. So they each took five shots with this racket, and five shots with this racket. Then, after each shot, they got two people who said whether or not the players were able to hit the ball hard. Then they came up with an overall score for each iacket. That tells us how hard the players could hit the ball with each of the rackets." The experimenter then asked the child, "Do you want to know how the results turned out?" The experimenter placed a star next to one tennis racket and a black circle next to the other. Children were told that the star meant players were able to hit the ball hard and the black circle meant they were only able to hit soft shots. The experimenter then manipulated the evidence: "Now you, and I, and Katy know that this is the way the results turned out. Does John know this? Well let's do something before he gets back. We'll move the star over here and the circle over here." Then the experimenter asked the child to recall the important story information: "Now really and truly, where does the star go? And where does the circle go? Does John know that?"

At this point in each story (after the evidence had been manipulated and subjects had been asked to recall the important story information), John returned and observed the faked evidence, and children were asked the Faked Evidence questions. In addition, children were asked a second set of questions: the Hypothesis-Evidence questions. These questions are included in Table 3 for the Racket-Size material set. For half the children the Faked Evidence questions preceded the Hypothesis-Evidence questions and for the other half the order was reversed. When the Hypothesis-Evidence questions came first, they followed the presentation of the veridical evidence and preceded the manipulation of the evidence. When the Hy- pothesis-Evidence questions came second, they immediately followed the Faked Evi- dence questions, and the evidence was returned to its original form before the Hy- pothesis-Evidence questions were asked.


1630 Child Development TABLE 3

EXPERIMENTAL QUESTIONS OF EXPERIMENT 3

Hypothesis-Evidence (based on the questions of previous researchers, e.g., Sodian [19911):

Evidence Assessment: Remember, Katy and Johnny wanted to know whether the size of a tennis racket matters in how hard a player can hit the ball. Do these results show that size is what matters or don't they show that? Evidence Justification: How do they show that size matters/doesn't matter?

Faked Evidence: Next Serve Prediction: Suppose one of the players were to serve again with the big racket. Before the player serves, Katy/Johnny is going to say whether s/he thinks it will be a hard serve or a soft serve. What will s/he say? Racket Buying Prediction: Suppose Katy/Johnny went to a store where they sold rackets just like the ones which were used in the experiment. They're not the rackets that were used in the experiment [experimenter points to rackets used in the experiment and shows that two other rackets were used], but they're like the ones used in the experiment. There are big ones and little ones. Which racket will Katy/Johnny buy if s/he wants a racket which will help her/him hit the ball hard?

The two Hypothesis-Evidence questions were comparable to those employed by Bullock (1991), Kuhn et al. (1988), and So- dian (1991). The Next Serve Prediction and Racket Buying Prediction questions were extensions of the Faked Evidence questions of Experiments 1 and 2 and provided us with an indication of whether children understood how a hypothesis would affect one's future predictions and behavior. Correct responding on the Next Serve Prediction and Racket Buying Prediction questions required that children ascribe opposite predictions to Katy and John (given the dolls' different access to the evidence).

Two Variables-Two Hypotheses story.-The procedure for this story was identical except that there were four tennis rackets rather than two, and the rackets could vary in two respects (size and material) rather than one (size). Regardless of the material set that was used in the Two Vari- ables-Two Hypotheses story, the second variable (way in which the items could differ) was always material. Further, in all cases, although there were two kinds of material, material was irrelevant to outcome (e.g., only size affected outcome). In this way our task was identical to that of Sodian (1991), who also asked children to assess the impact of evidence on hypotheses when there were two variables, only one of which was causal. Consequently, it allowed us to determine whether it was the number of potential causal variables that accounted for children's difficulty in the studies of Bullock (1991), Kuhn et al. (1988), and Sodian.

One Variable-One Hypothesis story.- The procedure for this story was identical to that of the One Variable-Two Hypotheses story except that once the evidence had been manipulated (the star and circle had been moved), it was returned to its original form. As in Experiment 2, this story was included to ensure that it was not the mere manipulation of evidence that led the subject to attribute a different hypothesis to the Uninformed doll. In this task, the evidence had been manipulated but the Uninformed doll would not be led to a mistaken hypothesis.

One Variable-Three Hypotheses story.-This story was included to ensure that children understood that a particular state of evidence specifies a particular hypothesis, not just a different hypothesis. The procedure for the One Variable-Three Hypotheses story was similar to that for the One Variable-Two Hypotheses story except that there were three levels of the causal variable rather than two, and the subject and two dolls were all led to unique hypotheses. Thus, in the One Variable-Three Hypothe- ses story both dolls were uninformed.

The subject and two dolls were shown three different-sized tennis rackets. Whereas in the other stories, one (Uninformed) doll then went to the playground while the subject and other (Informed) doll viewed the veridical evidence, in this story both dolls went to the playground so that the subject alone was shown the veridical evidence (e.g., only the big tennis racket gave hard


Ruffman et al. 1631

shots). The experimenter then manipulated the star and black circles (e.g., so it appeared that only the medium-sized tennis racket gave hard shots), and one of the dolls returned from the playground and observed this set of evidence. This doll then went back to the playground and the experimenter manipulated the evidence again (e.g., so that it appeared only the small tennis racket gave hard shots). The second doll then returned from the playground and viewed this set of evidence.

As in the other three stories, children were asked the Hypothesis-Evidence and Faked Evidence questions after the evidence had been manipulated to ensure that they had kept track of important information. The experimenter said, "Now really and truly, which racket had the hard shots? And which ones had the soft shots? Do John and Katy know that?" Furthermore, because there were two separate manipulations of the evidence in this story rather than just one (as in the other stories), the experimenter reminded subjects of the story events: "So really and truly, you and I know this racket had the hard shots. Then we did this [moving star and black circles about] and John came back. Then we did this [again moving star and circles] and Katy came back." The subject was then asked the Faked Evidence questions.

Correct performance on the One Vari- able-Three Hypotheses task required children to recognize, for example, that while they themselves would form the hypothesis that large tennis rackets produce hard shots (and make predictions accordingly), one character would form the hypothesis that medium-sized rackets produce hard shots, and the other character would form the hypothesis that small rackets produce hard shots. A child who answered in this way would have demonstrated an understanding of the precise relation between evidence and hypotheses, that is, that a particular state of evidence leads to a particular hypothesis, one that is specified directly by the evidence.

Design For a given child, one doll always acted

as the Informed doll, although the doll des- ignated as Informed was counterbalanced across children. Subjects were asked the two Hypothesis-Evidence questions in all stories except the One Variable-Three Hypotheses story. They were not included with this story because it was not clear what

the correct answer to the Evidence Assess- ment question should be. In one sense, the results showed that size did matter because, for instance, the largest racket produced harder shots than the other two. Yet in another sense the results showed that size did not matter because there was no difference in performance between the medium-sized and smallest racket; they both produced soft shots. Thus, children were asked a total of three Evidence Assessment questions and three Evidence Justification questions.

As for the Faked Evidence questions, children were asked what each of the story characters (Katy and John) would predict (Next Serve and Racket Buying Prediction questions) in all four stories. Further, in the One Variable-Three Hypotheses story children were also asked a Next Serve Predic- tion and Racket Buying Prediction question for themselves. This meant children were asked four Faked Evidence questions in each of the One Variable-Two Hypotheses, Two Variables-Two Hypotheses, and One Variable-One Hypothesis stories (for a total of 12 such questions), and six Faked Evi- dence questions in the One Variable-Three Hypotheses story.

Within the Faked Evidence question block, the order in which children were asked the two Faked Evidence questions was counterbalanced, as was the order in which children were asked about Katy, John, or (in the One Variable-Three Hypotheses story) themselves.

RESULTS Two 2 (sex) x 2 (order of Hypothesis-

Evidence vs. Faked Evidence question block) x 4 (question order within Faked Ev- idence block) x 4 (story order) analyses of variance were carried out. All variables were between-subjects variables, and the dependent variable was either (a) responses on the 12 Faked Evidence questions of the One Variable-Two Hypotheses, Two Variables- Two Hypotheses, and One Variable-One Hypothesis stories, or (b) responses on the three Evidence Justification questions of the One Variable-Two Hypotheses, Two Vari- ables-Two Hypotheses, and One Variable- One Hypothesis stories. A third analysis consisted of a 2 (sex) x 4 (question order within Faked Evidence block) x 4 (story order) x 4 (story type) analysis of variance, in which the dependent variable was responses on the 18 Faked Evidence questions of all four stories. The first three vari-


1632 Child Development ables were between-subjects variables and story type was a within-subjects variable. The data for each analysis were subjected to an arcsine transformation so that the distribution on each dependent variable approxi- mated the normal distribution. No interac- tions between between-subjects variables and within-subjects variables were permit- ted because of the large number of empty cells when the full model was specified. No effects were significant for any of the analyses (all F's < 1.60).

Hypothesis-Evidence question.-To convincingly demonstrate an understanding of the hypothesis-evidence relation on the Hypothesis-Evidence questions, a child would have to correctly answer all three Evi- dence Assessment questions (e.g., by claiming that the results did show that the size of a tennis racket mattered in how hard a player could hit the ball), and then on at least one occasion justify this answer by referring to the evidence in an unambiguous way. In fact, the Evidence Justification questions were of utmost importance because correct responses to the Evidence-Assessment questions required only that children form the correct hypothesis, but not that they reflect on the reasons for why that hypothesis was warranted. Five of 18 children (28%) correctly answered all three Evidence As- sessment questions and then, on at least one occasion, justified their answer by making a contrast between the evidence in support of the potential causal variables (i.e., "Because that's got a star and it hits better 'cuz it's got a stronger material"; "Because the big one hits harder than the small one"; "Because that one gives a soft shot and that one gives a hard shot"; "Because these two are wooden and wooden gives a harder shot"; "The small one doesn't have the same size and the racket will have to be bigger"). In addition, two children (making 39% in total) answered one of the three Evidence Assess- ment questions correctly and then justified their answer in this manner ("By the star and the circle. They show this one can hit hard and this one can hit a soft one"; "Because if you use the yellow ones it won't hit properly and if they use the red ones it would hit properly"). These children answered the other two Evidence Assessment questions incorrectly, however, suggesting that their understanding was somewhat shaky.

Somewhat surprisingly, there was no difference between children's performance on the Evidence Justification question over the three stories in which this question was

included. Two children correctly justified their answer to the Hypothesis-Evidence question only in the Two Variables-Two Hypotheses story (and not in the other two stories), one child did so only in the One Variable-Two Hypotheses story, and one did so only in the One Variable-One Hy- pothesis story. Another two children gave a correct justification in all three stories. Thus, children's performance on the Evidence Jus- tification question was not affected by the number of potential causal variables.

Faked Evidence questions.-Children's performance on the Faked Evidence questions of the One Variable-Two Hypotheses story, the Two Variables-Two Hypotheses story, and the One Variable-One Hypothe- sis story is included in Table 4. Their performance on the Faked Evidence questions of the One Variable-Three Hypotheses story also is included in Table 4. Table 4 shows that children did about equally well when answering questions (a) about either of the two dolls, (b) about Racket Buying behavior versus Next Serve predictions, and (c) whether there were two potential causal variables or just one. (This lack of differences was confirmed by the analyses of variance discussed earlier.)

Because children performed compara- bly on the Faked Evidence questions of the four stories, their performance on these questions was collapsed (see Table 5). Table 5 shows that most children passed all, or nearly all, of these Faked Evidence questions. In fact, 13 of 18 (72%) children answered at least 15 of 18 questions correctly. These children were, as individuals, above chance performance (binomial test: N = 18, k = 15, p < .01). Furthermore, as a group, children were also above chance performance (binomial test: N = 18, k = 13, p < .03).

Thus, most children showed a clear understanding of how the exposure to different evidence would lead the Informed and Un- informed story characters to different predictions and behavior, and how exposure to the same evidence (in the One Variable-One Hypothesis story) would lead them to the same predictions. Furthermore, 6- and 7- year-olds' understanding of the hypothesis- evidence relation is not restricted to a basic understanding that different evidence results in different hypotheses (e.g., opposite evidence = opposite hypothesis). Instead, their performance on the One Variable- Three Hypotheses story shows that they rec-


Ruffman et al. 1633

TABLE 4 NUMBER OF CHILDREN CORRECT ON THE FAKED EVIDENCE QUESTIONS OF EXPERIMENT 3

One Variable- Two Variables- One Variable- One Variable- Two Hypotheses Two Hypotheses One Hypothesis Three Hypotheses

Racket buying prediction: (Informed Doll) .......... 15 17 16 (Uninformed Doll) ........ 14 17 14

Next serve prediction: (Informed Doll) .......... 15 13 16 (Uninformed Doll) ........ 14 14 16

Racket buying prediction: (1st Uninformed Doll) .... ..... ... 12 (2d Uninformed Doll) ..... ... ...... 14 (Self) ............. ...... ......... 16

Next serve prediction: (1st Uninformed Doll) ..... ... ...... 14 (2d Uninformed Doll) ..... ... ...... 13 (Self) .............. .... ......... 16 NOTE.-N = 18.

ognize that one forms a very particular hypothesis that is directly related to the state of evidence one has access to.

One final point of interest is how children did on the Evidence Justification questions (modeled after Bullock [1991], Kuhn et al. [1988], and Sodian [1991]) in comparison to the Faked Evidence questions. Because the Evidence Justification question was only asked in the One Variable-Two Hypothe- ses, Two Variables-Two Hypotheses, and One Variable-One Hypothesis stories, we made our comparison only on these stories. Children were significantly better on the 12 Faked Evidence questions than they were on the three Evidence Justification questions: seven children were above chance performance on the Faked Evidence questions (at least 10 of 12 correct) yet did not offer a convincing justification on any of the three Evidence Justification questions, while no children showed the opposite pattern (binomial test: N = 7, k = 7, p < .001, one-tailed). One could argue that this result is misleading because there were consider- ably fewer Evidence Justification questions and hence opportunities for children to dem-

onstrate an understanding of the hypothesis- evidence relation here. However, even if one carries out a stricter test the result is the same. For instance, one can compare the number of children who correctly answered all 12 Faked Evidence questions to the number who offered convincing justifications on just one of three Evidence Justification questions. There were five children who passed all 12 Faked Evidence questions but none of the three Evidence Justification questions, while one child passed an Evi- dence Justification question while failing one or more Faked Evidence questions (binomial test: N = 6, k = 5, p < .02, one- tailed). These results are consistent with those of Experiments 1 and 2 but provide a better comparison of the two sorts of mea- sures (Evidence Justification vs. Faked Evi- dence) because they were obtained with the same group of subjects using the same materials and stories.

DIscUSSION

The results of Experiment 3 suggest that a critical factor in how well children do on tests of their understanding of the hypoth-

TABLE 5 NUMBER OF CHILDREN CORRECT ON THE 18 FAKED EVIDENCE QUESTIONS OF

THE FOUR STORIES IN EXPERIMENT 3

Number Questions Correct

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

2 1 1 - - 1 - - 2 - 2 9


1634 Child Development esis-evidence relation is how that understanding is assessed. The faked evidence paradigm reveals substantially earlier understanding than asking about why the evidence supports a particular hypothesis. In Experiment 3 we found no evidence that it was the number of potential causal variables that caused children greater difficulty in the study of Sodian (1991) in comparison to the present study (recall that there were two potential causal variables in Sodian's study). Furthermore, our results showed that children aged 6 to 7 understand a central feature of a hypothesis-that evidence leads one to make generalizations about events (e.g., it is not just the rackets that were used in the experiment that give hard shots, but all large tennis rackets) and to make predictions (e.g., the next shot with the large tennis racket will result in a hard shot).

General Discussion Summary of results.-The results of

Kuhn et al. (1988) suggested that it is not until 11 or 12 years that children begin to understand how a hypothesis relates to covariation evidence, and the results of Bul- lock (1991) and Sodian (1991) suggested that by as early as 8 or 9 years of age children have some insight into this relation. Yet, our results suggest that this understanding de- velops even earlier. By 6 years of age most children understand how simple covariation evidence forms the basis for a hypothesis. It should be noted, however, that our results do not speak to when children begin to understand more complicated forms of covariation evidence, for instance, of the form studied by Inhelder and Piaget (1958). In addition, Experiment 2 showed that 6-year- olds understand that hypotheses are formed on the basis of patterns of evidence. Such children showed no signs of basing their answers on individual cases which contradicted the patterns as Kuhn et al. found children were prone to do. This provides an important test of children's abilities, given that patterns of evidence are what one must assess when evidence covaries imperfectly with outcome, as it often does. Furthermore, the results of Experiment 3 provide evidence that children's understanding of the hypothesis-evidence relation extends beyond their simply recognizing the role of evidence in hypothesis formation. They also have some insight into two of the fundamen- tal features of a hypothesis or theory: the way in which it shapes future actions and predictions.

Coordinating hypotheses and evidence.-Kuhn (1989) claimed that young children do not differentiate or coordinate hypotheses (or what Kuhn calls a "theory") and evidence. Kuhn (p. 679) lists three as- pects of what it means to coordinate theories and evidence: ". .. first that the evidence be encoded and represented separately from a representation of the theory. If new evidence is merely assimilated to a theory, as an instance of it, the possibility of con- structing relations between the two as separate entities is lost. Second, the subject must represent the theory itself as an object of cognition.... Third, and paradoxically, co- ordination of theories and evidence requires temporary bracketing, that is, disregarding or setting aside one's acceptance of the theory."

Children in Kuhn et al.'s (1988) tasks were said either to have disregarded the evidence in favor of their own theories, or to have used the evidence to construct a new theory but failed to understand that this new theory contradicted a pervious theory they had held.

Yet coordinating hypotheses and evidence is precisely what our tasks required children to do, and it is clear that most 6- year-olds could do it. First, children's performance in Experiment 2 (when the evidence was imperfect) shows that they did not treat the evidence as a mere instance of a hypothesis. Here, some of the instances (pieces of evidence) were actually inconsistent with the simplifying hypothesis, yet most 6-year- olds recognized that the story character would nevertheless form that hypothesis. Second, children recognized that the story character would form a hypothesis that was actually incorrect. In doing so, children had to reflect on (treat as an object of cognition) the character's mistaken hypothesis (recognizing how the faked evidence would result in this hypothesis), and in the process had to set aside their own hypothesis about what was true. The question, then, is what made the tasks of previous researchers more difficult than our tasks?

Why a difference in results?-We pointed out one reason earlier. Our results show that the Evidence Justification question employed in Experiment 3 and in the research of Bullock (1991), Kuhn et al. (1988), and Sodian (1991) (e.g., Sodian: "Do the results show that size is what matters or don't they show that?") is quite difficult for children. One could imagine that the diffi-


Ruffman et al. 1635

culty with this question is that to a young child the results do not "show" anything. Showing is something that people do to other people; it is not something a pattern of results can do.

Although a direct within-subject comparison was not carried out, children's difficulty in explaining what the results showed contrasted with their relative ease at explaining why a character would form a particular hypothesis (the Faked Evidence Jus- tification questions of Experiment 2). The reason for this may be that hypotheses are mental states, and research has shown that these are a natural topic of interest for children. Thus, by as early as 2 or 3 years of age children ask a great many questions about why people act as they do (Dunn, 1988), and by 3 to 5 years of age they understand the sources of a variety of different mental states, for instance, visual experiences (Ya- niv & Shatz, 1988), knowledge (Wimmer, Hogrefe, & Perner, 1988), and beliefs (Per- ner et al., 1987; Wimmer & Perner, 1983). Indeed, it is easy to see the significance of such information for children's ability to make sense of the world and, consequently, to understand why children would do well on the Faked Evidence Justification questions. Whether an ability to answer the Hy- pothesis-Evidence Justification question is indicative of a "deeper" understanding of the hypothesis-evidence distinction or whether this question is unnecessarily complex is not clear.

Another possible reason for the difference in results is that in the tasks of Bullock (1991) and Kuhn et al. (1988), there were three or more potential causal variables that children had to evaluate. Although the results we obtained in Experiment 3 suggest that children do not have difficulty when two potential causal variables are present, it stands to reason that they might have difficulty as more variables are introduced.

Sodian (1991) and Sodian et al. (1991) point out other possibilities, for instance, that children in Kuhn et al.'s (1988) studies may have misunderstood the request to evaluate the evidence as a request to defend or elaborate their own views, or they may have failed because the evidence in Kuhn et al.'s studies often supported implausible hypotheses that children may have rejected out of hand. With respect to this latter suggestion, children might retain their prior hypothesis as counter-evidence was initially presented, regarding the first instances as spurious and

misleading until eventually, when a large amount of counter-evidence had accumu- lated, they would relinquish their initial hypothesis. Indeed, Kuhn reports that most children do eventually relinquish their hypothesis as counter-evidence grows. This suggests, not that children are unable to coordinate hypotheses and evidence, as Kuhn's claims imply, but that this ability will not be manifest in all circumstances, for instance, when the evidence suggests implausible hypotheses. To avoid this potential problem, we deliberately chose tasks in which the evidence suggested plausible hypotheses about what factors were causal, and this may have made it easier for children to coordinate hypotheses and evidence right from the outset of our tasks. That is, because the evidence suggested a hypothesis which was plausible, children may have had no re- luctance to attribute it to another person even if they themselves opted for a different hypothesis.

Conclusions.-In sum, our results show that by as early as 6 years of age children possess some understanding of a very basic prerequisite that is needed to properly understand much of science education. Fur- ther, our results are compatible with recent theory of mind research. By 6 years of age children's metacognitive abilities are suffi- ciently developed to allow them some form of insight into hypotheses and the way they are constructed from patterns of evidence.

References Bullock, M. (1991, April). Scientific reasoning in

elementary school: Developmental and individual differences. Paper presented at the bi- ennial meeting of the Society for Research in Child Development, Seattle.

Dunbar, K., & Klahr, D. (1988). Developmental differences in scientific discovery processes. In D. Klahr & K. Kotovsky (Eds.), Complex information processing: The impact of Her- bert A. Simon (pp. 109-143). Hillsdale, NJ: Erlbaum.

Dunn, J. (1988). The beginnings of social understanding. Cambridge, MA: Harvard Univer- sity Press.

Flavell, J. H. (1985). Cognitive development (rev. ed.). Englewood Cliffs, NJ: Prentice-Hall.

Inhelder, B., & Piaget, J. (1958). The growth of logical thinking from childhood to adolescence. London: Routledge & Kegan Paul.

Kuhn, D. (1989). Children and adults as cognitive scientists. Psychological Review, 96, 674- 689.

Kuhn, D., Amsel, E., & O'Laughlin, M. (1988).


1636 Child Development The development of scientific thinking skills. New York: Academic Press.

Kuhn, D., & Phelps, E. (1982). The development of problem-solving strategies. In H. Reese (Ed.), Advances in child development and behavior (Vol 17, pp. 1-44). New York: Aca- demic Press.

Perner, J. (1991). Understanding the representa- tional mind. Cambridge, MA: MIT Press.

Perner, J., Leekam, S. R., & Wimmer, H. (1987). Three-year-olds' difficulty with false belief: The case for a conceptual deficit. BritishJour- nal of Developmental Psychology, 5, 125- 137.

Ruffman, T., Olson, D. R., Ash, T., & Keenan, T. (1993). The ABCs of deception: Do young children understand deception in the same way as adults? Developmental Psychology, 29, 74-87.

Schauble, L. (1990). Belief revision in children: The role of prior knowledge and strategies for generating evidence. Journal of Experimen- tal Child Psychology, 49, 31-57.

Sodian, B. (1991). Evaluating causal hypotheses: A metaconceptual deficit in children? Un- published manuscript, University of Munich and MIT.

Sodian, B., Zaitchik, D., & Carey, S. (1991). Young children's differentiation of hypothetical beliefs from evidence. Child Development, 62, 753-766.

Wimmer, H., Hogrefe, J., & Perner, J. (1988). Chil- dren's understanding of informational access as a source of knowledge. Child Develop- ment, 59, 386-396.

Wimmer, H., & Perner, J. (1983). Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children's understanding of deception. Cognition, 13, 103-128.

Yaniv, I., & Shatz, M. (1988). Children's understanding of perceptibility. In J. W. Astington, P. L. Harris, & D. R. Olson (Eds.), Developing theories of mind. Cambridge: Cambridge University Press.


Reflecting on Scientific Thinking: Children's Understanding of the Hypothesis-Evidence Relation

Documents

Transcript of Reflecting on Scientific Thinking: Children's Understanding of the Hypothesis-Evidence Relation