ORIGINAL ARTICLE

Doris M. Dehn á Edgar Erdfelder

What kind of bias is hindsight bias?

Received: 10 December 1995 /Accepted: 29 October 1997

Abstract When people are asked to recollect a formerresponse after having received feedback information,their recollection tends to approach the feedback an-swer. This phenomenon is referred to as hindsight bias.Recently, Erdfelder and Buchner proposed a multi-nomial model designed to independently measure thecontributions of reconstruction and recollection pro-cesses. On its basis, they only found ®rm evidence for thecontribution of reconstruction biases to hindsight ef-fects. In the present study, we tried to experimentallyenhance the probability of recollection biases by (a) re-ducing the depth of processing of the original estimate,(b) minimizing the distinctness of the original estimateand feedback, and (c) combining both treatments. Theempirical data were analyzed using a variant of theErdfelder and Buchner model which allows for the ex-perimental manipulation of the feedback answer. Thismodel is shown to adequately describe the data of ourexperiment. Moreover, although both treatments weree�ective in that the ®rst one reduced correct recollec-tions in general and the second one diminished di�er-ences between original estimates and recalled estimates,the parameter estimates indicate that the probability ofrecollecting the original estimate is not hampered byfeedback information, even under conditions which arethought to enhance memory impairments.

Introduction

In 1967, Walster described the phenomenon that ``afteran event occurs individuals tend to overestimate their

likelihood of correctly anticipating such an outcome''(p. 239). Eight years later, initiated by the work ofFischho� (1975), the systematic investigation of thisphenomenon ± now labelled ``hindsight bias'' or the``I-knew-it-all-along e�ect'' ± began. In the followingyears, hindsight e�ects were investigated in a variety ofdi�erent arti®cial and natural settings. The contextschosen for investigation comprised historical and socialevents (e.g., Fischho�, 1975; Pennington, 1981a; Was-serman, Lempert, & Hastie, 1991), general knowledgequestions (Fischho�, 1977; Hell, Gigerenzer, Gauggel,Mall, & MuÈ ller, 1988; Kohnert, 1996; Pohl, in press;Pohl & Hell, 1996; Schmidt, 1993; Wood, 1978), psy-chological experiments (Davies, 1987; Slovic & Fisch-ho�, 1977), political elections (Leary, 1982; Pennington,1981b; Powell, 1988; Synodinos, 1986), and object at-tributes (Pohl & Gawlik, 1995). Hindsight e�ects werealso examined in expert judgements such as in medicaldiagnoses (e.g., Arkes, Faust, Guilmette, & Hart, 1988;Detmer, Fryback, & Gassner, 1978), in psychologists'expert knowledge (Pohl, 1992), in jurors' decisions(Caspar, Benedict, & Perry, 1989), and in the judgementsof school career advisers (Mullet, 1987). Further, out-come information was shown to in¯uence not only in-dividual decisions, but also decisions made in groups(Bukszar & Connolly, 1988; Stahlberg, Eller, Maass, &Frey, 1995).

Regardless of the domain and the individuals chosenfor investigation, the presentation of outcome informa-tion in all of the above studies resulted in a mean in-crease in the perceived predictability of this outcome.Findings of no or reversed hindsight e�ects, on the otherhand, were very rare and are probably due to the con-sciously perceived unpredictability or implausibility ofthe outcome (e.g., Kohnert, 1996, Chap. 7; Mazursky &O®r, 1990; Pohl, in press; Verplanken & Pieters, 1988).In addition to the stability across domains and partici-pant groups, the observed bias towards the actual out-come turned out to be very resistant against attempts ofelimination. For instance, neither the instructions ``totry hard'' or ``to adopt the perspective of a person

Psychol Res (1998) 61: 135 ± 146 Ó Springer-Verlag 1998

D. M. Dehn (&)Department of Psychology, Building 1.1,University of the Saarland, PO Box 151150,66041 SaarbruÈ cken, Germanye-mail: doris@cops.uni-sb.de

E. ErdfelderUniversity of Bonn,Bonn, Germany

without outcome knowledge'' nor information about thenature of the hindsight bias succeeded in signi®cantlyreducing hindsight e�ects (Fischho�, 1977; Pohl, inpress; Pohl & Hell, 1996).

The crucial component of every hindsight bias ex-periment is some sort of numerical estimate, for exam-ple, a probability judgement or an estimate of a physicalquantity. Estimates made without outcome knowledge,that is, neither knowledge of whether a certain eventoccurred nor knowledge about the exact physicalquantity (i.e., foresight answers), are then comparedwith estimates made with outcome knowledge (i.e.,hindsight answers). Experiments on hindsight bias di�erin what kind of comparison is made: in a memory design(as introduced by Fischho� & Beyth, 1975), foresightand hindsight judgements stem from the same individ-uals. Here, individuals ®rst give a numerical estimate(re¯ecting again a probability or a physical quantity).This estimate is referred to as the ``original answer.''After being informed about the correct answer (the ac-tual outcome or physical quantity), participants are in-structed to recollect their original answer. In ahypothetical design (as introduced by Fischho�, 1975),foresight and hindsight judgements stem from di�erentparticipants. Here, one group of participants isprompted to give a numerical answer without receivingoutcome information (foresight group); the other groupof participants is ®rst informed about the actual out-come and then asked to give a numerical estimate as ifthey had not been informed about the outcome (hind-sight group).1

In order to explain the average shift of the hindsightresponse towards the feedback answer ± which is nor-mally observed in both designs ± di�erent sets of ex-planations have been proposed (for detailed overviewssee Hawkins & Hastie, 1990; Kohnert, 1996; Schmidt,1993). The empirical evidence so far supports two pos-sible sources of hindsight bias: recollection processes andreconstruction processes. Recollection processes may bea�ected by outcome knowledge because the latter couldeither distort the memory trace for the original know-ledge or reduce its accessibility (e.g., Fischho�, 1975;Hell et al., 1988). In addition, if the original knowledgeis not recollectable, then the reconstruction of the origi-nal knowledge may be a�ected because the outcomeinformation serves as a biasing ``anchor'' in the recon-struction process (e.g., Hell et al., 1988; Schmidt, 1993;Stahlberg & Eller, 1993) or in¯uences the ``rejudgement''in some other way (cf. Hawkins & Hastie, 1990). Obvi-ously, a clear separation of recollection and recon-struction processes in hindsight judgement generationcan only be made if participants in fact generated an

original answer. For this reason, the present study isrestricted to the memory design in hindsight research.

Following a suggestion by Erdfelder (1992), Erdfelderand Buchner (in press) employed the methodology ofmultinomial processing tree modelling (Batchelder &Riefer, 1990; Hu & Batchelder, 1994; Riefer & Batch-elder, 1988) to develop a model of the cognitive pro-cesses involved in hindsight judgements. Their13-parameter hindsight-bias model ± labelled HB13 ±applies to the memory design and aims primarily atseparating recollection and reconstruction biases inhindsight. Erdfelder and Buchner showed that the HB13model adequately ®ts the data of four hindsight experi-ments using almanac-type general knowledge questions.Moreover, the model's parameters (in particular, theparameters representing reconstruction and recollectionprocesses) displayed their experimental manipulations inthe predicted way. This lends some support to the psy-chological validity of the model and its parameters. Withregard to the basis of hindsight e�ects, the experimentsconducted by Erdfelder and Buchner showed either noor only very small recollection biases in hindsightjudgements. Instead, the major ± if not the single ± causefor hindsight e�ects appeared to be a systematic recon-struction bias. Speci®cally, Erdfelder and Buchner ob-served only one signi®cant recollection bias (p < .05) innine independent data sets obtained in their four ex-periments. By contrast, signi®cant and sizeable recon-struction biases were observed in all nine data sets.2

Similar results were recently reported by Kohnert(1996), who analyzed some of his data within a nine-parameter submodel of the full HB13 model. He ob-served three signi®cant recollection biases (p < .05) ineight data sets collected in ®ve experiments. In addition,he re-analyzed Exp. 3 of Schmidt (1993) and also foundevidence for a small, but signi®cant recollection bias. Asin Erdfelder and Buchner's (in press) experiments, sig-ni®cant reconstruction biases were found in all of thesedata sets. The slightly larger proportion of signi®cantrecollection biases in Kohnert's results is probably dueto the larger sample sizes employed in his experiments(cf. p. 107). This may have led to signi®cant results evenin case of very small e�ects.3 In fact, Erdfelder andBuchner and also Kohnert agree that if there are reliable

1In addition to the distinction between memory design and hypo-thetical design, Hertwig, Gigerenzer, and Ho�rage (1997) proposeda theoretical distinction between (1) hindsight bias as a shift ofprobability ratings due to the repetition of an assertion and (2)hindsight bias as a shift of numerical estimates due to the provisionof the correct physical quantity. In the present article, we areconcerned only with the latter phenomenon.

2 In the HB13 model, recollection biases are tantamount to positivedi�erences between the model parameters rc and re (probability ofrecollecting the original answer to a control item and to a feedbackitem, respectively). Reconstruction biases are indicated by themodel parameter b (probability of a biased reconstruction, givenrecollection failure). If b > 0, then reconstruction biases contributeto the hindsight bias e�ect (see Erdfelder & Buchner, in press, formore details).3Unfortunately, the proper statistical evaluation of Kohnert's(1996) results is di�cult because he analyzed his data within theframework of a submodel of the HB13 model, which obviouslyfailed to ®t at least some of his data sets. As is well known, hy-pothesis tests may be misleading if the underlying statistical modelis violated. Nevertheless, the descriptive evidence presented byKohnert appears strong enough to support his interpretation of theresults.

136

recollection biases in their experiments, then they mustbe small. Using a completely di�erent methodology,Stahlberg and Eller (1993) and Schmidt arrived at thesame conclusion.

Nevertheless, the available empirical evidence ap-pears too weak to conclude that recollection biases inhindsight are negligible in general. Firstly, despite beingsmall, recollection biases seem to be reliable in the sensethat the recollection of original answers is consistentlyslightly worse with outcome knowledge as compared toa control condition without outcome knowledge. Sec-ondly, no speci®c e�ort was undertaken so far to max-imize the chances of recollection biases to occur.Therefore, it is not particularly surprising that strongrecollection biases did not show up in the experimentsreported above.

Goals of the present research

The aim of the present study is to identify and to carryout experimental manipulations under which memorybiases are more likely to arise than in previous research.It seems plausible to assume that the provision of feed-back information is more likely to provoke misrecol-lections if the memory trace for the original answer isdi�cult to discriminate from the memory trace for thefeedback information (cf. Hell et al., 1988). Two vari-ables which can be expected to in¯uence the discrimin-ability of the original answer's memory trace are, amongothers, the depth of processing during encoding and thesimilarity of the original answer and the feedback an-swer.

Depth of processing. In hindsight studies, the shift of therecalled answer towards the correct answer is typicallygreater if the original response was processed in a shal-low way as compared to deep processing at encoding(Arkes et al., 1988; Davies, 1987; Hell et al., 1988; Ko-riat, Lichtenstein, & Fischho�, 1980; Slovic & Fischho�,1977). A shallower way of processing than in the stan-dard memory paradigm can be achieved, for instance, byproviding the participants with a set of answers amongwhich they choose their original response instead ofgenerating it. Based on the levels-of-processing expla-nation of the so-called generation e�ect (e.g., Graf, 1982;Slamecka & Graf, 1978), we predicted that an answerprovided by the experimenter and selected by the par-ticipant would produce a less elaborate memory traceand hence give rise to a lower probability of recollectionthan an answer that was generated by the individual her-or himself. According to the memory-trace strengthhypothesis put forward by Hell et al. (pp. 536±537), aweaker or less elaborate memory trace for the originalanswer, given a ®xed trace strength for the feedbackanswer, should enhance the probability of recollectionbiases.

Di�erence between the original answer and the feedbackanswer. Another way of enhancing the probability ofrecollection biases is to increase the similarity betweentarget information (original answer) and interferinginformation (feedback answer). This hypothesis derivesfrom both research on retroactive inhibition (McGeoch& Irion, 1952; Slamecka & Ceraso, 1960) and severaltheories of the hindsight bias e�ect. It has been hy-pothesized, for instance, that the less distinct twomemory traces representing original and feedback an-swer are, the higher is the probability that the twotraces merge (cf. Fischho�, 1975; Pohl & Gawlik,1995), that one superimposes the other (cf. Hasher,Attig, & Alba, 1981; Schkade & Kilbourne, 1991), orthat both become indistinguishable during recall (Koh-nert, 1996). In the context of two numerical estimates,similarity might, for instance, mean that two estimatesare close to each other. Thus, a possible way of fos-tering memory biases consists in minimizing the dif-ference between the original and the feedback answers.This assumption is indirectly supported by ®ndings ofno or reversed hindsight e�ects for those outcomesthat strongly diverged from the original belief (Ko-hnert, 1996; Mazursky & O®r, 1990; Pohl, in press;Verplanken & Pieters, 1988).

Assessing the e�ect of the di�erence between originaland feedback answers, however, is hampered by amethodological problem: If general knowledge questionsare used (like in the experiments by Erdfelder & Buch-ner, in press), the deviation of the initial estimate fromthe correct answer at least partially re¯ects participants'knowledge on a speci®c topic. That is, on average, itemsthat draw on a superior knowledge base give rise tosmaller di�erences between original answer and feed-back answer than items that draw on an inferiorknowledge base. Kohnert's (1996, Exps. 1±3) analyses ofthe e�ect of the di�erence between original answer andfeedback answer su�er from exactly this problem. Be-cause he was aware of the fact that the di�erence scoreswere confounded with the participants' knowledge state(see Kohnert, p. 124), Kohnert decided to experimen-tally manipulate the (supposedly) ``correct'' answers toalmanac-type questions in his Exp. 4. However, in orderto reduce the risk that feedback manipulations would bedetected by some participants, he had to adjust thefeedback answers to a participant's individual know-ledge state. As a consequence, treatments were realizeddi�erently for participants with di�erent knowledgestates. This precaution might have diminished the e�ectof his manipulation.

In order to disentangle the e�ect of the di�erencebetween the original answer and the feedback answer onthe one hand and the e�ect of the knowledge state on theother, we decided to use material for which the know-ledge state can safely be assumed to be weak for allparticipants. In this way, the deviation of the originalanswer from the feedback answer could be determinedirrespective of the knowledge state associated with thisitem. Material that meets the above requirement was

137

proposed by Eller, Stahlberg, Maass, and Frey (1991). Itconsists of a set of questions on the results of a ®ctitiouspublic opinion inquiry for which almost all ``outcomes''appear, in principle, possible.

The use of material lacking a unique correct answer,though, confronts us with a further problem: The modelproposed by Erdfelder and Buchner (in press) cannot beapplied to the material by Eller et al. (1991) because itrequires the existence of a unique ``correct'' answer. Forthis reason, we developed a variant of the HB13 modelwhich allows for the experimental manipulation of thefeedback answer. This model is outlined in the nextsection.

A multinomial model for hindsight studieswithout unique feedback answers

The model proposed here is a joint multinomial modelwith separate processing trees for di�erent combinationsof the original answer and the feedback answer. Like theoriginal HB13 model, it applies to the memory design inhindsight experiments. Generally, three experimentalconditions are distinguished: (1) No feedback answer isgiven (Fig. 1a), (2) the feedback value is smaller than theoriginal estimate (Fig. 1b), and (3) the feedback value isgreater than the original estimate (Fig. 1c). In the fol-lowing, the original answer is abbreviated to OA, the

Fig. 1 a Processing tree forcontrol items without feed-back information. b Process-ing tree for experimental itemswith FA<OA. c Processingtree for experimental itemswith FA>OA

138

feedback answer to FA, and the recalled original answerto ROA. If we refer to the numbers associated with theOA, the FA, and the ROA, they are typeset in italics(i.e., OA, FA, and ROA, respectively).

As the model is especially designed for the analysis ofdi�erences between the recalled original answer and theoriginal answer, ROA ) OA, we label it as the ``hind-sight bias di�erence score'' (HBDS) model.

Response categories are represented by rectangles onthe right-hand side of Fig. 1. The model refers to ®veresponse categories, one for a hindsight bias (HB) of 0,and four for non-zero hindsight biases. Because in thepresent study hindsight bias is de®ned as the di�erencebetween ROA and OA, a hindsight bias index HB � 0indicates perfect recollection, whereas a non-zero hind-sight bias indicates a more or less strong deviation of theROA from the OA. In order to distinguish betweendi�erent sizes of the deviation, a (®xed) cut-o� point c isintroduced. Apart from preserving information on thesize of the deviation between OA and ROA, c also servesthe purpose of increasing the number of response cate-gories. A su�cient number of response categories is aprerequisite for the testability of the model.4 Ideally, thecut-o� point is chosen in such a way that the number ofcases in each of the categories HB < )c, )c � HB < 0,0 < HB � c, and HB > c is approximately the same.This ensures that each category contains su�cient ob-servations to estimate parameters and to conductgoodness-of-®t tests in a statistically feasible way.

The cognitive processes which are assumed to lead toa particular response are represented by ovals in Fig. 1.Two types of cognitive processes are distinguished:recollection and reconstruction processes. Recollectionprocesses result in the cognitive events ``OA retrievable''or ``OA not retrievable.'' In case the OA is not retriev-able, reconstruction processes are initiated. Recon-struction processes follow the strategies ``unbiasedreconstruction of the OA'' or ``biased reconstruction ofthe OA.'' If the OA is reconstructed unbiased by the FA,then the participant probably generates her or hishindsight estimate by drawing an answer from the samedistribution of possible answers as was used in thegeneration of the OA. A biased reconstruction ofthe OA, on the other hand, might consist in anchoringat the FA and adjusting towards the OA (cf. Tversky &Kahneman, 1974). Other strategies of reconstructing theOA based on the FA are also conceivable (see Hawkins& Hastie, 1990). Clearly, a participant can only be bi-ased by the FA if she or he received one. This is why theprocessing tree for the control items (Fig. 1a) lacks thelower branch denoting the cognitive state ``biased re-construction of the OA.''

Conditional probabilities of cognitive events and re-sponses are displayed on the branches connecting dif-ferent cognitive events or response categories. Themodel contains three sorts of parameters ± namely,recollection parameters, reconstruction parameters, andguessing parameters. The probability of retrieving theOA given a control item (i.e., no feedback) is denoted byrc. The probability of retrieving the OA given an ex-perimental item is given by rk, where the index k indi-cates the size of the feedback deviation from the OA(i.e., feedback category k). Two sorts of reconstructionparameters are suggested. The parameter bk denotes theprobability of an OA-reconstruction process in feedbackcategory k that is biased by the outcome information.Accordingly, 1 ) bk denotes the probability of an un-biased reconstruction in feedback category k. The pa-rameter ak indicates the probability that a participant infeedback category k employing a biased reconstructionproduces an ``extreme'' hindsight bias (classi®ed on thebasis of the cut-o� point c). In case the feedbackquantity is smaller than the original quantity (as depic-ted in Fig. 1b), ak denotes the probability that anROA ) OA di�erence is produced which is less than )c.In case the feedback quantity is bigger than the originalquantity (as depicted in Fig. 1c), ak denotes the proba-bility that an ROA ) OA di�erence is produced which isbigger than c. The guessing parameters g1, g2 and g3re¯ect the probability that a response reconstructedunbiased by the FA falls into a particular answer cate-gory. In contrast to the other parameters, g1, g2 and g3are assumed to be stable across feedback conditions. Ifthe guessing processes can in fact be thought of asdrawing a random ROA from the same distribution theOA was drawn from, then it follows thatg1 � 1 ) g1 � 0.5 and g2 � g3 should hold. To illus-trate, let us assume that the random variable OA isnormally distributed with mean l and variance r2. Inmost applications, this assumption should provide agood approximation to the data. Then, by assumption,the distribution of ROAs resulting from unbiased re-constructions must also be a normal distribution withmean l and variance r2. As a consequence, the linearcombination HB � ROA ) OA follows a normal dis-tribution, too, albeit with mean l ) l � 0 and variance2(r2 ) Cov(ROA, OA)), where Cov(ROA, OA) denotesthe covariance between ROAs and OAs. Because anynormal distribution with mean 0 satis®es the constraintsp(HB < 0) � p(HB ³ 0) � 0.5 and p(HB < )c) �p(HB > c), the restrictions g1 � 1 ) g1 � 0.5 andg2 � g3 must hold.

The HBDS model incorporates the main theoreticalassumptions made in the HB13 model, a decision whichappears sensible in light of the successful demonstrationsof HB13's validity. One major simpli®cation of ourmodel arises, though, as a consequence of the experi-mental manipulation of the feedback answer: We do notneed to distinguish between original over- and under-estimations of the feedback answer because the FA isnot included in the de®nition of the response categories.

4 The hindsight bias di�erence score ROA ) OA does not distin-guish between ROAs which are shifted towards the FA and ROAswhich are shifted beyond the FA (with regard to the position of theOA). For our material (in which the FA is an experimental vari-able), alternative hindsight bias indices that take into account theFA would be unde®ned in the control condition. Therefore, theycannot be employed in the present study.

139

Another simpli®cation ± dropping the possibility ofsource confusion between the original answer and thefeedback answer ± is due to our providing the feedbackanswer at test, which minimizes the chances of sourceconfusions (see below). Further, we do not take intoaccount chance hits of the original answer, because theprobability for a chance hit given a freely generated re-called answer appears to be negligible (cf. Erdfelder &Buchner, in press).

Hypotheses of the present research

In the following, we report an experiment that was de-signed to enhance the probability of memory biases tooccur. For the reasons described above, the variableschosen for manipulation were (1) the type of answer tobe given, that is, whether the original answer was gen-erated freely or had to be chosen out of ®ve alternatives,and (2) the di�erence between the original answer andthe feedback answer, with di�erences being either+/)10 or +/)20. The e�ects of these independentvariables were analyzed by means of (a) traditionalANOVA methods and (b) the HBDS multinomial modeldescribed in the previous section.

Our hypotheses were as follows: First, if the hindsightbias is at least partially due to recollection biases, thenthe probability of remembering the original answershould be lower for experimental than for control items;in terms of the HBDS model parameters: rc > rk.Generally, this e�ect should be found for both answerformats (generated vs chosen) and for all levels of dif-ferences between original and feedback answers ()20,)10, +10, +20). However, according to the above as-sumptions, memory biases are particularly likely toemerge if the memory trace for the OA is relatively weak(as in the case of experimenter-provided answers) or ifthe original answer and the feedback answer are easilyconfused (as in the case of small di�erences like +10and )10). If both treatments are combined, then thedi�erence rc ) rk should be most pronounced. If, incontrast, recollection biases do not contribute to hind-sight e�ects, then the probability of recollecting theoriginal answer should be the same for control and ex-perimental items, regardless of answer format and dis-tance from the feedback answer.

Second, on the basis of the empirical evidence citedabove, we hypothesize that reconstruction biases con-tribute to the hindsight bias. Hence, it is expected that theprobability of the feedback answer in¯uencing the re-construction of the original answer, bk, is signi®cantlylarger than 0. We do not expect the experimental ma-nipulations to a�ect the probabilities of biased recon-structions, that is, all bk parameters should be of the samesize. Whether or not the reconstruction of a non-re-trievable OA is biased by the FA should not depend onthe initial depth of encoding of the OA, and it should alsobe una�ected by the di�erence FA ) OA, simply becausethe OA is not available in the reconstruction process.

Third, we expect the FA ) OA di�erence manipula-tion to a�ect the deviation of the ROA from the OA incase of biased reconstructions. That is, the parameter a20should substantially exceed the parameter a10.

Method

Participants

Forty-four ®rst-year students of psychology at the University ofBonn participated in this experiment in order to gain a subjectcredit.

Materials

The material consisted of questions on the results on a ®ctitiouspublic opinion inquiry as proposed by Eller et al. (1991). Thismaterial does not have unique feedback answers, thus enabling theexperimenter to control the deviation between original answer andfeedback answer. For example, one question could read ``Whatpercentage of German adults support banning cars from city cen-tres?'' Because all questions refer to percentages, the answers(original answer as well as feedback answer) are restricted to arange from 0 to 100. The questions were chosen in such a way thatoriginal answers at the extremes should be unlikely. The materialoriginally used by Eller et al. comprised a set of 18 questions. In thepresent study, we used a total of 30 questions, of which 17 werepart of Eller et al.'s original set. The remaining 13 questions wereconstructed by the present authors.

Two questionnaires were created, both containing the above 30questions. In the ®rst questionnaire (i.e., the foresight question-naire), each question was followed by a line prompting the par-ticipant to predict the percentage of adults making a particularstatement (``Your prediction to this question is...''). In the secondquestionnaire (i.e., the hindsight questionnaire), questions werefollowed by either one or two lines, depending on their status asexperimental or control items. If a question served as an experi-mental item, the participant was ®rst informed about the ``actualoutcome'' of the public opinion inquiry (e.g., ``The correct answeris 86 percent.'') before being prompted to recollect her or hisoriginal response (``Your prediction was...''). If a question served asa control item, it was simply followed by the line prompting theparticipant to recollect the original response.

Procedure

The experiment consisted of two phases, one in which the originalanswer (i.e., the foresight answer) was collected and another inwhich feedback information was provided and the recalled originalanswer (i.e., the hindsight answer) was collected. Between bothphases, there was a delay of three days. Feedback information ±that is, information about the allegedly ``correct'' answer ± wasgiven in the second phase, immediately before participants had torecollect their original answer to every question. Participants weretested as a group.

In the ®rst phase of the experiment, participants were providedwith the ®rst questionnaire. This questionnaire was identical for allparticipants. In the instructions covering the questionnaire, par-ticipants were told that the present study investigated humans'ability to make quantitative judgements under uncertainty. For thisreason, they should predict some results of a recent public opinioninquiry whose results were not published yet. In the instructions,participants were also informed about the second phase of theexperiment. In this phase, they would receive some more infor-mation on the public opinion inquiry, on the basis of which theywould be asked to make new estimates. This procedure shouldserve to investigate how changes in the information basis are re-¯ected in the judgements. Finally, participants were informed that

140

results of public opinion inquiries are always given as wholenumbers. Therefore, their estimates should be given as wholenumbers, as well.

In the second phase of the experiment, participants were pro-vided with the second questionnaire. This questionnaire was indi-vidually prepared for each participant. In contrast to priorannouncements ± and hence unexpectedly for the participants ±they were instructed to recollect their predictions made three daysbefore. Participants were told that the actual outcome was listed formost of the questions in order to examine whether the proper resultcould facilitate recollection. Those items served as experimentalitems. Questions for which no outcome was provided served ascontrol items.

Design

Independent variables. We manipulated two independent variables:the format of the original answer and the feedback information.The format of the original answer was manipulated by eitherproviding (multiple choice format) or not providing (free format) aset of possible answers in the ®rst questionnaire. The feedbackinformation was manipulated by varying the di�erence between theoriginal answer and the feedback answer communicated to theparticipant. Both variables were manipulated within subjects.

Answer format. For each participant, half of the phase 1 questionsrequired a free generation of the prediction; the other half weremultiple choice questions where the appropriate estimate had to bechosen out of ®ve possible answers. For example, the possibleanswers for the question. ``How large is the percentage of adultsfavouring prohibition of smoking at work?'' were ``7 percent,'' ``21percent,'' ``44 percent,'' ``61 percent,'' and ``85 percent.'' The an-swer alternatives were generated in such a way that they were ap-proximately evenly distributed over the range of possible answers,with at least three of the answers falling into the interval from 20 to80. In contrast to the original answers required in the ®rst ques-tionnaire, recalled original answers required in the second ques-tionnaire were always given in a free answer format.

Feedback information. The di�erence between the original answerand the feedback answer was varied on four levels di�ering in theabsolute value and the direction of the deviation. The values chosenas di�erences were 10 and 20. This was done in order to ensure thatfor almost all original answers (i.e., answers in the interval [20, 80]),the feedback information fell into the interval of possible answers(i.e., [0, 100]). Combining these two FA ) OA deviations 10 and 20with the two possible directions (+ and )), we obtained four ex-perimental conditions. In addition to these experimental condi-tions, a control condition without feedback information wasrealized, thus yielding the following ®ve FA ) OA feedback con-ditions: )20, )10, no feedback, +10, +20. A random noise be-tween +2 and )2 was added to the feedback answers in each of thefour experimental conditions in order to avoid obvious regularitiesin the pattern of feedback answers. For example, if a participantoriginally responded ``60'' to a question, she could receive asfeedback ``42''()20 condition), ``49'' ()10 condition), ``70'' (+10condition), ``82'' (+20 condition), or no feedback on the ``correct''answer at all (control condition).

The items were assigned to the answer formats and feedbackconditions as follows: Of the 30 questions on the ®ctitious publicopinion inquiry, 15 were randomly determined to be multiplechoice questions, and the other 15 were determined to be freegeneration questions. The answer type for a particular question didnot vary across participants because the assignment of feedbackconditions to items had to proceed di�erently for the two types ofquestions. For the free generation questions, feedback conditionswere assigned randomly to items after the participants had pro-vided their original answers. For multiple choice questions, how-ever, there were some restrictions as to the assignment of items tofeedback conditions. Percentages could only be plausibly presented

as ``correct'' outcomes if they were included in the set of answeralternatives. Otherwise, a participant who was able to recollect notonly her predictions but also the other answer alternatives woulddoubt the credibility of the feedback information. Hence, formultiple choice questions it was determined in advance which itemswould serve as control items and, in case an item served as anexperimental item, what the absolute value of the deviation be-tween the original answer and the feedback answer was. The di-rection of the deviation (i.e., the particular experimental condition)was then assigned randomly.

Dependent variable. The di�erence between the recalled originalanswer and the original answer (i.e., ROA ) OA) served as an in-dicator of the size of the hindsight bias.

Results

The mean original answer across all items was 47.4, witha standard deviation of 20.6. Of the original answers,83.9% (corresponding to a total of 1106) fell into arange from 21 to 79. In the following, only items withinthis range are analyzed. This restriction is due to the factthat a random choice of feedback conditions (subject tothe restrictions discussed above) was only feasible foritems with original answers between 21 and 79.

In order to statistically analyze the e�ects of thewithin-subject factors ``answer format'' and ``feedbacktype'' on the participant's mean ROA ) OA deviations,we chose the MANOVA approach to repeated measuresdesigns (see, e.g., O'Brien & Kaiser, 1985). Since werestricted this analysis to ROAs with OAs larger than 20and less than 80, almost half of the 44 participants hadat least one missing value in some of the 10 treatmentcombinations (2 answer formats ´ 5 feedback condi-tions). To avoid loss of data, we replaced missing valuesby the mean of valid values generated by other partici-pants. This procedure is defensible because missingvalues were generally rare and the procedure did notchange the sample means. In fact, this replacement ofmissing values had only minor e�ects on the results ofthe signi®cance tests.

MANOVA analysis

After replacing the missing values with the means of thevalid values, a 2 (answer format) by 5 (type of feedback)repeated measures MANOVA of the ROA ) OA devi-ations revealed a signi®cant main e�ect of the feedbacktype (Pillai's V� 0.84, approximate F(4, 40)� 53.45,p<.01). Neither the main e�ect of the answer formatnor the interaction of answer format and feedback typeproved to be signi®cant (all Fs<1). Figure 2 shows thatthe mean ROA ) OA di�erence increases with increas-ing FA ) OA di�erence in an approximately linearfashion. If the feedback answer is smaller than theoriginal answer, the recalled answer tends to be smallerthan the original answer; if the feedback answer is largerthan the original answer, the recalled answer tends to belarger than the original answer.

141

Multinomial analysis

Data analyses based on multinomial models involvethree components: parameter estimation, goodness-of-®ttests, and tests of parameter restrictions. Parameter es-timation and goodness-of-®t tests were carried out usinga computer program by Hu (1991). The cut-o� point cfor di�erentiating between weak and strong hindsightbiases was chosen to be 15. This choice was guided bythe aim to obtain response categories with relativelyhomogenous response frequencies. The raw frequenciesfor the 2 ´ 5 ´ 5 response categories obtained across allexperimental conditions are displayed in Table 1.

The maximum likelihood estimates for the HBDSparameters and their 95% con®dence intervals are dis-played in Table 2. The estimates and con®dence inter-vals were computed separately for generated and formultiple choice answers. As predicted, the estimates ofthe recollection parameters rc, r10, and r20 are higher forgenerated than for multiple choice answers.

The goodness-of-®t tests were carried out using thelikelihood-ratio v2 statistic G2. The level of signi®cance

chosen was a � .01. This choice was guided by thefollowing considerations. After elimination of extremeoriginal answers, the number of observations wasNg � 510 for generated answers and Nm � 596 formultiple choice answers. Given these sample sizes anddf � 10 (for the test of the general model), df � 1 ordf � 2 (for the tests of the subsequent restrictions), thepower of the goodness-of-®t test for detecting ``medium''deviations from the model (i.e., w� .3, cf. Cohen, 1977)already approaches 1 for a � .01. That is, the b-errorprobability is less than .01 for substantial model viola-tions.5

For generated answers, the HBDS model ®ts the dataperfectly, G2(10)� 8.9, n.s. For multiple choice answers,the ®t is su�cient as well, G2(10)� 20.8, n.s., although itis not as good as for generated answers. Since the power

Fig. 2 Size of hindsight biasby answer type and feedbackcondition

Table 1 Frequencies in the response categories

Answer format Feedback type Hindsight bias category

HB<)15 )15 £ HB<0 HB=0 0<HB £ 15 HB >15

Generated )20 24 52 13 18 4)10 20 38 15 17 5control 11 30 14 21 1510 7 21 27 40 1820 8 17 14 35 26

Multiple choice )20 32 42 4 13 3)10 19 56 7 34 5control 19 36 1 37 1710 7 51 5 66 1820 7 36 3 42 36

5 These power values were computed using the GPOWER program(Faul & Erdfelder, 1992). Erdfelder and Bredenkamp (in press,Footnote 3) presented a formula showing how the overall e�ect sizew depends on the H0 and H1 cell probabilities in case of jointmultinomial models.

142

of the G2 test is still acceptable when a� .01, the factthat the model only just ®tted the data for multiplechoice answers does not seem problematic.

Once it has been established that the basic substantivemodel adequately ®ts the data, hypotheses concerningspeci®c parameters can be tested. This is done by suc-cessively imposing constraints on the parameters, eitherby setting parameters to be equal to each other or bysetting them to speci®c values. After the parameters forthe restricted models have been re-estimated, the modelsare again subjected to a likelihood-ratio test. In this test,the dfs correspond to the number of restricted parame-ters. The test statistic DG2 then re¯ects the decrease inthe goodness of ®t due to the parameter restriction. Inthis way, the adequacy of each restriction introduced canbe evaluated.

The ®rst hypothesis to be tested concerns the role ofrecollection processes. That is, we tested the hypothesisthat memory performance in the experimental condi-tions is worse than in the control condition. Further, incase feedback information is provided, the di�erencebetween the original answer and the feedback answerwas expected to in¯uence memory performance. Thesehypotheses were tested by restricting recollection pa-rameters to be identical for experimental and controlitems as well as within di�erent experimental conditions(i.e., rc� r10� r20). The resulting model did not describethe data signi®cantly worse than the model without thisrestriction: for generated answers, DG2(2)� 4.3, n.s.; formultiple choice answers, DG2(2)� 3.8, n.s. Thus, hind-sight bias does not seem to be due to memory impair-ment after the provision of outcome information, bothfor generated and for selected answers. Further, there isno e�ect of the di�erence between original answer andfeedback answer on the probability of recollecting theoriginal answer. Note also that the descriptive pattern ofparameter estimates contradicts the predictions madeabove: Rather than being worse than r̂c, the estimates of

r̂10 are better for both generated and multiple choiceanswers (see Table 2).

The next hypotheses to be tested involved the pa-rameters bk and ak representing reconstruction process-es. The fact that b̂k is not 0 already indicates theexistence of reconstruction biases. In order to test oursecond hypothesis, we tested the conjecture that theprobability of a reconstruction bias, bk, does not dependon the di�erence between the original answer and thefeedback answer. This assumption was tested by settingthe reconstruction parameters b10 and b20 equal to eachother. This constraint also turned out to be tenable,given an a of .01: for generated answers, DG2(1)� .7,n.s.; for multiple choice answers, DG2(1)� 4.9, n.s.

To investigate our third hypothesis stating that theprobability of producing a large hindsight bias in case ofbiased reconstructions depends on the di�erence be-tween the original answer and the feedback answer, weset a10� a20. As predicted, for multiple choice answers,the restricted model describes the data signi®cantlyworse than the unrestricted model, DG2(1)� 16.2,p<.01. For generated answers, however, the restrictedmodel does not have to be rejected, DG2(1)� .4, n.s.

In addition to hypotheses directly connected to thee�ects of the feedback manipulations, hypotheses con-cerning the guessing parameters g1, g2, and g3 were ex-amined. As outlined above, an unin¯uencedreconstruction of the original response should result inhindsight bias values which are distributed symmetri-cally around a mean of 0. Thus, we expected g1� 0.5 andg2� g3. In fact, ®xing the guessing parameters in such away does not result in a signi®cant increase of G2: forgenerated answers, DG2(2)� 2.3, n.s.; for multiple choiceanswers, DG2(2)� 4.9, n.s.

Finally, we tested the e�ect of the depth of processingmanipulation (i.e., the OA format) on the HBDS modelparameters. As expected, generating the OA instead ofselecting it increased each of the three recollection pa-

Table 2 Parameter estimates and 95% con®dence intervals for the HBDS model

Parameters Answer format of original estimates

Generated Multiple choice

ML estimate 95% Con®denceinterval

ML estimate 95% Con®denceinterval

Recollectionrc .15 (.09±.22) .01 (.00±.02)r10 .20 (.16±.25) .04 (.02±.07)r20 .13 (.09±.17) .03 (.01±.05)

Reconstructionb10 .40 (.28±.52) .25 (.26±.35)b20 .48 (.38±.59) .45 (.34±.55)a10 .36 (.22±.49) .25 (.20±.40)a20 .41 (.30±.52) .59 (.46±.71)

Guessingg1 .55 (.49±.60) .56 (.51±.60)g2 .27 (.20±.34) .22 (.17±.27)g3 .30 (.22±.39) .22 (.16±.28)

143

rameters rc [DG2(1)� 17.2, p<.01], r10 [DG2(1)� 14.3,p<.01], and r20[DG2(1)� 29.4, p<.01] signi®cantly. Incontrast, the other seven model parameters were nota�ected by the OA format, DG2(7)� 10.3, n.s.

Discussion

In the present experiment, we replicated the well-known®nding that providing an individual with outcome in-formation results, on average, in a shift of the recalledanswer towards the feedback answer. A mean-baseddata analysis of the hindsight bias index ROA ) OAshowed that this shift increases with the di�erence be-tween the original answer and the feedback answer,whereas the answer format of the original estimate doesnot a�ect the ROA ) OA di�erence. Besides these gen-eral ®ndings, the present study mainly illustrated twopoints: First, the HB13 model can be modi®ed in such away that it accommodates material without a uniquefeedback answer, and second, there is again no evidencefor the memory impairment hypothesis in hindsight re-search. Both aspects are elaborated below.

A variant of the HB13 model: The HBDS model

The HBDS model is a multinomial processing-treemodel for hindsight studies which ± in contrast to theHB13 model suggested by Erdfelder and Buchner (inpress) ± allows for the experimental manipulation offeedback answers. This variant of HB13 was shown todescribe the data of the present experiment adequately.Minor reservations concern the model's ®t to multiplechoice responses. For this kind of questions, the modeljust ®tted the data. Nevertheless, it is the experimentalvariation of the answer format carried out to reducememory performance, rather than HBDS itself, whichappears to be problematic. In a memory design, theoriginal answer is usually generated freely. For freelygenerated answers, the HBDS model ®tted the datasatisfactorily. Thus, it can be concluded that the HBDSmodel adequately describes the cognitive processes in-volved in generating a hindsight judgement, given astandard memory design.

Moreover, the model's parameters behaved in apsychologically plausible way. For instance, the gener-ation e�ect predicts better memory performance forgenerated as compared with experimenter-provided an-swers. Consistent with the generation e�ect, the recol-lection parameters rc and rk, k� 10, 20, were larger forgenerated answers than for multiple choice answers.However, this ®nding has to be regarded with somecaution. In the present study, answer type and itemmaterial were confounded, that is, it was always thesame set of questions which served as multiple choiceitems or as items with freely generated answers. Asoutlined above, this confounding was unavoidable be-cause the feedback manipulation had to proceed di�er-

ently for the two answer formats. However, because thetwo item sets underlying the two answer formats werecreated randomly, it is hardly conceivable that the itemmaterial alone accounts for di�erences in the rc, r10, andr20 estimates ranging between .10 and .20.

If the original answer cannot be recollected but has tobe reconstructed, then it is plausible to assume that thedi�erence between the original answer and the feedbackanswer in¯uences the magnitude of hindsight bias in thebiased reconstructions (as re¯ected in ak). In fact, thedi�erence between the original answer and the feedbackanswer was shown to a�ect ak for multiple choice an-swers. Unexpectedly, however, it did not in¯uence ak forgenerated answers. Perhaps the variation of FA ) OAdi�erences realized in the present study was not greatenough to reach signi®cance for generated answers aswell. This consideration is partially supported by the factthat the e�ect in the parameter estimates, although notsigni®cant, is in the right direction.

The role of recollection and reconstruction processesin hindsight bias

The main goal of the present study was to subject the®ndings of Erdfelder and Buchner (in press) to astronger test. In a series of experiments, Erdfelder andBuchner found only weak evidence for the contributionof recollection biases to hindsight bias. However, inthose experiments the authors did not manipulate vari-ables that maximize the probability of memory biases.

The present study was based on the rationale thatmemory impairments should be more likely to occur if(a) the memory trace for the original answer is weak andless elaborate or (b) the memory traces for the originalanswer and the feedback answer are less distinct. How-ever, even under these conditions we failed to ®nd anyevidence for memory impairment hypotheses of thehindsight bias. Neither a shallow encoding of the origi-nal answer (like in multiple choice answers), nor a smalldi�erence between the original answer and the feedbackanswer (like in the FA ) OA�+/)10 condition), northe combination of both factors gave rise to impairedmemory performance due to the provision of feedbackinformation. This is implied by the empirical adequacyof the restriction rc� r10� r20 for both generated andmultiple choice answers. That is, in none of our experi-mental conditions does the probability of recollectingthe original answer depend on whether feedback infor-mation is provided or not. Rather, there was a weaktrend in the parameter estimates that even opposed thepredictions.

Interestingly, these results match those reported byKohnert (1996, Exp. 4). Although he did not analyze thee�ect of the FA ) OA di�erence within a multinom-ial model, he reported the percentage of perfectrecollections which can be directly interpreted in termsof our recollection parameters (Kohnert, 1996, p. 127,Fig. 5.7). These perfect recollections, also, did not di�er

144

signi®cantly for small and large FA ) OA di�erences.Analogous to our results, the recollection estimatestended to be larger for small FA ) OA di�erences, incontrast to what was predicted.

Like in several previous studies, the hindsight biasobserved in our experiment seems to be due to a re-construction process which is biased towards the feed-back answer (as demonstrated by bk>0). Although this®nding is in line with the results of Erdfelder andBuchner (in press), Kohnert (1996), Schmidt (1993),Stahlberg and Eller (1993), and others, the evidenceagainst memory impairment hypotheses still appears in-conclusive. There might be other ways of revealingsubstantial memory impairments in the hindsight biasparadigm which we have not pursued. For instance,BroÈ der and Erdfelder (1997) recently reported prelimi-nary results of a series of experiments designed to ex-plore the e�ects of treatments known to enhanceinterference in retroactive inhibition experiments. In-terestingly, the design feature of within-subject versusbetween-subject manipulations of the feedback versuscontrol condition turned out to be a major source ofvariance in the results: Weaker or absent recollectionbiases were found in within-subject designs like the oneemployed in our study, whereas reliable and sizeablerecollection biases were found if experimental and con-trol items were presented to di�erent groups of partici-pants. Regardless of whether this ®nding turns out to betenable, additional studies are needed in order to de-termine whether the role of recollection biases to hind-sight e�ects is in fact negligible or not.

References

Arkes, H.R., Faust, D., Guilmette, T.J., & Hart, K. (1988). Elim-inating the hindsight bias. Journal of Applied Psychology, 73,305±307.

Batchelder, W.H., & Riefer, D.M. (1990). Multinomial processingtree models of source monitoring. Psychological Review, 97,548±564.

BroÈ der, A., & Erdfelder, E. (1997). Hindsight Bias: BeeintraÈ chtigteErinnerung, verzerrte Rekonstruktion oder beides? [Hindsightbias: Impaired memory, distorted reconstruction, or both?] InH. Mandl (Ed.), Bericht uÈber den 40. Kongreû der DeutschenGesellschaft fuÈr Psychologie in MuÈnchen 1996 (pp. 31±36).GoÈ ttingen, Germany: Hogrefe.

Bukszar, E., & Connolly, T. (1988). Hindsight bias and strategicchoice: Some problems in learning from experience. Academy ofManagement Journal, 31, 628±641.

Casper, J.D., Benedict, K., & Perry, J.L. (1989). Juror decisionmaking, attitudes, and the hindsight bias. Law and HumanBehavior, 13, 291±310.

Cohen, J. (1977). Statistical power analysis for the behavioral sci-ences (Rev. ed.). New York: Academic Press.

Davies, M.F. (1987). Reduction of hindsight bias by restoration offoresight perspective: E�ectiveness of foresight-encoding andhindsight-retrieval strategies. Organizational Behavior and Hu-man Decision Processes, 40, 50±68.

Detmer, D.E., Fryback, D.G., & Gassner, K. (1978). Heuristicsand biases in medical decision-making. Journal of Medical Ed-ucation, 53, 682±683.

Eller, F., Stahlberg, D., Maass, A., & Frey, D. (1991, March). DerKnew-it-all-along-E�ekt in Einzel- und Gruppensituationen. [The

knew-it-all-along e�ect in individual and group situations].Paper presented at the 33. Tagung experimentell arbeitenderPsychologen, Gieûen, Germany.

Erdfelder, E. (1992, April). Multinomiale Modelle zum Hindsight-Bias-Paradigma [Multinomial models for the hindsight biasparadigm]. Paper presented at the 34. Tagung experimentellarbeitender Psychologen, OsnabruÈ ck, Germany.

Erdfelder, E., & Bredenkamp, J. (in press). Recognition of script-typical and script-atypical information: E�ects of cognitiveelaboration. Memory & Cognition.

Erdfelder, E., & Buchner, A. (in press). Decomposing the hindsightbias: A multinomial processing tree model for separatingrecollection and reconstruction in hindsight. Journal of Exper-imental Psychology: Learning, Memory, and Cognition.

Faul, F., & Erdfelder, E. (1992). GPOWER: A priori, post-hoc, andcompromise power analyses for MS-DOS [Computer program].Bonn, Germany: Bonn University, Department of Psychology.

Fischho�, B. (1975). Hindsight ¹ foresight: The e�ect of outcomeknowledge on judgment under uncertainty. Journal of Experi-mental Psychology: Human Perception and Performance, 1, 288±299.

Fischho�, B. (1977). Perceived informativeness of facts. Journal ofExperimental Psychology: Human Perception and Performance,3, 349±358.

Fischho�, B., & Beyth, R. (1975). ``I knew it would happen'':Remembered probabilities of once-future things. OrganizationalBehavior and Human Performance, 13, 1±16.

Graf, P. (1982). The memorial consequences of generation andtransformation. Journal of Verbal Learning and Verbal Behav-ior, 21, 539±548.

Hasher, L., Attig, M.S., Alba, J.W. (1981). I knew it all along.Or, did I? Journal of Verbal Learning and Verbal Behavior, 20,86±96.

Hawkins, S. A., & Hastie, R. (1990). Hindsight: Biased judgmentsof past events after the outcomes are known. PsychologicalBulletin, 107, 311±327.

Hell, W., Gigerenzer, G., Gauggel, S., Mall, M., & MuÈ ller, M.(1988). Hindsight bias: An interaction of automatic and moti-vational factors? Memory and Cognition, 16, 533±538.

Hertwig, R., Gigerenzer, G., & Ho�rage, U. (1997). The reiteratione�ect in hindsight bias. Psychological Review, 104, 194±202.

Hu, X. (1991). General program for processing tree models [Com-puter program]. Irvine, CA: University of California, Irvine.

Hu, X., & Batchelder, W.H. (1994). The statistical analysis ofgeneral processing tree models with the EM algorithm. Psy-chometrika, 61, 199±240.

Kohnert, A. (1996). Grenzen des RuÈckschaufehlers: Die Verzerrungvon Erinnerungen an fruÈheres Wissen durch neue Informationen.[Limits of the hindsight bias: The distortion of recollections offormer knowledge by new information]. Bonn, Germany: Holos.

Koriat, A., Lichtenstein, S., & Fischho�, B. (1980). Reasons forovercon®dence. Journal of Experimental Psychology: HumanLearning and Memory, 6, 107±118.

Leary, M. R. (1982). Hindsight distortion and the 1980 presidentialelection. Personality and Social Psychology Bulletin, 8, 257±263.

Mazursky, D., & O®r, C. (1990). ``I could never have expected it tohappen'': The reversal of the hindsight bias. OrganizationalBehavior and Human Decision Processes, 46, 20±33.

McGeoch, J. A., & Irion, A. L. (1952). The psychology of humanlearning. New York: Longmans & Green.

Mullet, E. (1987). Information integration, hindsight e�ect andexpert judgements. Cahier de Psychologie Cognitive/EuropeanBulletin of Cognitive Psychology, 7, 295±308.

O'Brien, R. G., & Kaiser, M. K. (1985). MANOVA method foranalyzing repeated measures designs: An extensive primer.Psychological Bulletin, 97, 316±333.

Pennington, D. C. (1981a). The British ®remen's strike of 1977/1978: An investigation of judgements in foresight and hindsight.British Journal of Social Psychology, 20, 89±96.

Pennington, D. C. (1981b). Being wise after the event: An investi-gation of hindsight bias. Current Psychological Research, 1,271±282.

145

Pohl, R. F. (1992). Der RuÈ ckschaufehler: Systematische VerfaÈ l-schung der Erinnerung bei Experten und Novizen. [Hindsightbias: Systematic distortion of recollections in experts and nov-ices]. Kognitionswissenschaft, 3, 38±44.

Pohl, R. F. (in press). The e�ects of feedback source and plausi-bility on hindsight bias. European Journal of Cognitive Psy-chology.

Pohl, R. F., & Gawlik, B. (1995). Hindsight bias and the misin-formation e�ect: Separating blended recollections from otherrecollection types. Memory, 3, 21±55.

Pohl, R. F., & Hell, W. (1996). No reduction in hindsight bias aftercomplete information and repeated testing. Organizational Be-havior and Human Decision Processes, 67, 49±58.

Powell, J. L. (1988). A test of the ``knew-it-all-along'' e�ect in the1984 presidential and statewide elections. Journal of AppliedSocial Psychology, 18, 760±773.

Riefer, D. M., & Batchelder, W. H. (1988). Multinomial modelingand the measurement of cognitive processes. PsychologicalReview, 95, 318±339.

Schkade, D. A., & Kilbourne, L. M. (1991). Expectation-outcomeconsistency and hindsight bias. Organizational Behavior andHuman Decision Processes, 49, 105±123.

Schmidt, C. (1993). Verzerrte Vorstellungen von Vergangenem:Vorsatz oder Versehen? [Distorted memories of the past: In-tention or mistake?]. Bonn, Germany: Holos.

Slamecka, N. J., & Ceraso, J. (1960). Retroactive inhibition ofverbal learning. Psychological Bulletin, 57, 449±475.

Slamecka, N. J., & Graf, P. (1978). The generation e�ect: Delin-eation of a phenomenon. Journal of Experimental Psychology:Human Learning and Memory, 4, 592±604.

Slovic, P., & Fischho�, B. (1977). On the psychology of experi-mental surprises. Journal of Experimental Psychology: HumanPerception and Performance, 3, 544±551.

Stahlberg, D., & Eller, F. (1993). Hindsight-E�ekte: Eine ur-teilstheoretische ErklaÈ rung. [Hindsight e�ects: The role ofjudgmental processes]. In L. Montada (Ed.), Bericht uÈber den38. Kongreû der Deutschen Gesellschaft fuÈr Psychologie in Trier1992, 2 (pp. 735±741). GoÈ ttingen, Germany: Hogrefe.

Stahlberg, D., Eller, F., Maass, A., & Frey, D. (1995). We knew itall along: Hindsight bias in groups. Organizational Behavior andHuman Decision Processes, 63, 46±58.

Synodinos, N. E. (1986). Hindsight distortion: ``I knew it all alongand I was sure about it.'' Journal of Applied Social Psychology,16, 107±117.

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty:Heuristics and biases. Science, 185, 1124±1131.

Verplanken, B., & Pieters, R. G. M. (1988). Individual di�erencesin reverse hindsight bias: I never thought something likeChernobyl would happen. Did I? Journal of Behavioral DecisionMaking, 1, 131±147.

Walster, E. (1967). ``Second guessing'' important events. HumanRelations, 20, 239±249.

Wasserman, D., Lempert, R.O., & Hastie, R. (1991). Hindsight andcausality. Personality and Social Psychology Bulletin, 17, 30±35.

Wood, G. (1978). The ``knew-it-all-along'' e�ect. Journal of Ex-perimental Psychology: Human Perception and Performance, 4,345±353.

146