A Hierarchical Process-Dissociation Model

Jeffrey N. RouderUniversity of Missouri—Columbia

Jun LuAmerican University

Richard D. Morey, Dongchu Sun, and Paul L. SpeckmanUniversity of Missouri—Columbia

In fitting the process-dissociation model (L. L. Jacoby, 1991) to observed data, researchers aggregateoutcomes across participant, items, or both. T. Curran and D. L. Hintzman (1995) demonstrated howbiases from aggregation may lead to artifactual support for the model. The authors develop a hierarchicalprocess-dissociation model that does not require aggregation for analysis. Most importantly, the Curranand Hintzman critique does not hold for this model. Model analysis provides for support of processdissociation—selective influence holds, and there is a dissociation in correlation patterns among partic-ipants and items. Items that are better recollected also elicit higher automatic activation. There is nocorrelation, however, across participants; that is, participants with higher recollection have no increasedtendency toward automatic activation. The critique of aggregation is not limited to process dissociation.Aggregation distorts analysis in many nonlinear models, including signal detection, multinomial pro-cessing tree models, and strength models. Hierarchical modeling serves as a general solution foraccurately fitting these psychological-processing models to data.

Keywords: process dissociation, human memory, Bayesian hierarchical models, aggregation bias

In many mid-level cognitive domains, such as memory andattention, researchers rely on relatively simple measurement mod-els. Examples include the theory of signal detection (Green &Swets, 1966), the model of conjoint recognition (Brainerd, Reyna,& Mojardin, 1999), process dissociation (Jacoby, 1991), the powerlaw of cognitive learning (Newell & Rosenbloom, 1981), and thesimilarity-choice model (Luce, 1963). These measurement modelsare nonlinear—that is, data are assumed to come from a morecomplex process than the addition of true scores and noise. Theadvantage of using nonlinearity is that the researcher can faithfullymodel psychological process and measure constructs inaccessibleto analyses of variance and regression models.

In most analyses, researchers need to aggregate outcomes overpeople, items, or both to produce summary measures. This aggre-gation is not overly problematic for analysis with linear models(Rouder & Lu, 2005). Unfortunately, this sanguine state does notapply for nonlinear models. Over the past 50 years, several authorshave critiqued applications of nonlinear measurement models that

require aggregation of outcomes across participants, items, or both(e.g., Ashby, Maddox, & Lee, 1994; Curran & Hintzman, 1995;Estes, 1956; Estes & Maddox, 2005; Heathcote, Brown, & Me-whort, 2000; Luce, 1959; Rouder & Lu, 2005). Although thesecritiques appear in different domains and sometimes without ref-erence to one another, they are part of a more general critique:Aggregation in nonlinear models often distorts the measurement ofpsychological process.

These critiques have not been well heeded in practice, and wesuspect it is because of a lack of alternatives to aggregation.Aggregation seems necessary, especially in fields such as memoryand psycholinguistics. In most experiments, a participant is testedonce on an item; each participant–item combination is unrepli-cated. The resulting performance for any participant–item combi-nation is often dichotomous. For example, in stem completion, thestem is either completed with a studied word or not; in recognitionmemory, the item is judged either new or old. For these paradigms,the estimation of parameters in memory models relies on estimat-ing probabilities of events. For example, process-dissociation pa-rameters are functions of stem-completion probabilities; signaldetection parameters are functions of old-response probabilities.To generate proportions that estimate these probabilities, research-ers aggregate over dichotomous outcomes; they aggregate acrossitems, participants, or both. The critique that aggregation distortsanalysis is problematic in many paradigms because aggregationseems necessary.

We have recently advocated nonlinear hierarchical models asan alternative to aggregation in experimental designs (Rouder &Lu, 2005; Rouder, Lu, Speckman, Sun, & Jiang, 2005; Rouder, Lu,et al., 2007; Rouder, Morey, Speckman, & Pratte, 2007). Thesehierarchical models account for participant and item variability.Unlike conventional hierarchical linear models (e.g., Raudenbush

Jeffrey N. Rouder and Richard D. Morey, Department of PsychologicalSciences, University of Missouri—Columbia; Jun Lu, Department of Sta-tistics, American University; Dongchu Sun and Paul L. Speckman, De-partment of Statistics, University of Missouri—Columbia.

We thank Mike Pratte, Courtney Gilbert, and Andrew Kent for help inrunning the reported experiments. We thank Timothy Curran and JeffreyToth for providing us with stimuli. We thank Larry Jacoby and Jeff Tothfor supportive comments. This research was supported by National ScienceFoundation Grant SES-0351523 and National Institute of Mental HealthGrant R01-MH071418.

Correspondence concerning this article should be addressed to Jeffrey N.Rouder, Department of Psychological Sciences, 210 McAlester Hall, Univer-sity of Missouri, Columbia, MO 65211. E-mail: rouderj@missouri.edu

Journal of Experimental Psychology: General Copyright 2008 by the American Psychological Association2008, Vol. 137, No. 2, 370–389 0096-3445/08/$12.00 DOI: 10.1037/0096-3445.137.2.370

370

& Bryk, 2002), ours are nonlinear and are tailored for specificpsychological-processing models. In this article, we present ahierarchical version of Jacoby’s (1991) process-dissociationmodel. To account for item and participant variability, there areseparate recollection and automatic activation parameters for everyparticipant–item combination. By imposing a hierarchical structureon these participant–item parameters, the model provides theneeded constraint for accurate estimation without recourse to ag-gregation.

We divide the article as follows. In the next section, we provideexamples of how aggregation compromises analysis with the Ja-coby process-dissociation model. Then, we introduce the hierar-chical process-dissociation model and show how it does indeedprovide for accurate measurement. We analyze two experimentswith the hierarchical model. Although the interpretation requires acaveat, the results provide support for the basic tenets of theJacoby process-dissociation model.

The Jacoby Process-Dissociation Model

In the past 25 years, research in memory has been greatlyinfluenced by multiple-systems approaches. These theories aresupported, in part, by studies on anterograde amnesiacs. Amnesi-acs exhibit greatly degraded performance on tasks that directlyprobe memory, such as cued recall. Yet they show somewhat intactperformance on indirect tasks, such as priming (Cave & Squire,1992) and stem completion (Graf & Schacter, 1985). These resultsprovide evidence that there may be an implicit form of memorypreserved in amnesiacs that is separate from conscious recollection(e.g., Schacter, 1992; Tulving & Schacter, 1990). Although thereare several different descriptions of this implicit form (Schacter &Tulving, 1994), it is common to consider it automatic and based onfamiliarity.

The goal of Jacoby’s process-dissociation model is to separatelymeasure conscious recollection and automatic activation. We de-scribe its application to a stem-completion task. In the standardversion of the task, participants study a list of words and are thentested on them. At test, items appear as to-be-completed stems(e.g., the stem br__ may be completed to form the word bread). Inthe process-dissociation procedure, there are two testing condi-tions: an include and an exclude condition. In the include condi-tion, participants are instructed to complete the stem with a pre-viously studied word. In the exclude condition, participants areinstructed to complete the stem with a word not studied. In theinclude condition, either successful conscious recollection or suc-cessful automatic activation is sufficient for successful stem com-pletion. In the exclude condition, successful automatic activationand a failure of conscious recollection are sufficient and necessaryfor successful stem completion. By contrasting these conditions,an estimate of the probability of successful recollection and suc-cessful automatic activation may be obtained. The process-dissociation model is formally given as follows: Let I and E be theprobabilities of completing a stem with a studied word in theinclude and exclude conditions, respectively. For simplicity, rec-ollection and automatic activation are assumed to be all-or-none,dichotomous processes. Let R and A be the probability of con-scious recollection and automatic activation, respectively. TheJacoby process-dissociation model is given by

I � R � (1 � R)A (1)

and

E � (1 � R)A. (2)

Estimators may be obtained by solving the above equations for Rand A:

R � I � Ei, (3)

and

A �E

1 � �I � E�, (4)

where I and E are the proportion of stems completed with studiedwords in the include and exclude conditions, respectively.

One subtle aspect of the process-dissociation equations is thatthey are compatible with many psychological models (Buchner,Erdfelder, & Vaterrodt-Plunneck, 1995). Figure 1 shows a generalmodel in which the probability of automatic activation depends onwhether recollection occurred (A* denotes the probability of auto-matic activation given retrieval success; A denotes the probabilityof automatic activation given retrieval failure). The probabilities ofstem completion in the include and exclude conditions are R �(1 � R)A and (1 � R)A, respectively, which are the same as inEquations 1 and 2. The parameter A* does not enter into theequations. Consequently, only automatic activation conditioned onrecollection failure is measured in the process-dissociation frame-work (see Buchner et al., 1995).

The measurement of automatic activation conditioned on re-trieval failure is undesirable. Most psychologists conceptualizeautomatic processes as quicker than deliberative ones (Kahneman,1973), implying that automatic activation could not be dependenton conscious recollection. Moreover, early memory-system theo-rists explicitly championed the independence of implicit and ex-plicit systems (e.g., Schacter, 1992). To make the measurement ofA interpretable and psychologically plausible, it is widely assumedthat automatic activation and recollection are independent (Jacoby,1998). This independence assumption implies that A* � A and thatmeasurement of A applies to cases in which recollection succeedsas well as fails. Importantly, independence, in this case, means that

Include Trial

R

0

0+

0

R

+

++

0

A*

1−A

A

1−A

*

* *

1−A1−A

AA

Exclude Trial

1−R 1−R

Figure 1. A process-dissociation model for stem completion in whichautomatic activation depends on whether recollection succeeds or fails. Theparameter R denotes the probability of successful conscious recollection.The parameters A* and A denote the probability of successful automaticactivation contingent on recollection success and failure, respectively. � �stem completed with studied item; 0 � stem completed with item notstudied.

371HIERARCHICAL PROCESS DISSOCIATION

the model does not imply any specific time course of processing.Recollection could come after automatic activation or be per-formed simultaneously with it. All that is implied is that theprobability of successful automatic activation does not depend onthe state of recollection and vice versa.

Aggregation in the Process-Dissociation Model

Our main concern is about the effects of aggregation in applyingthe process-dissociation model. Curran and Hintzman (1995) firstexplored this concern, and our development follows theirs. Thefirst step is expanding the notation of the model such that it canaccount for participant and item effects. Let Iij and Eij be theprobability that the ith participant completes the jth stem with thecorresponding studied item in the include and exclude conditions,respectively. The expanded process-dissociation model is

Iij � Rij � (1 � Rij)Aij (5)

and

Eij � (1 � Rij)Aij, (6)

where Rij and Aij are the probabilities that the ith participantconsciously recollects and elicits automatic activation from the jthitem, respectively. To allow the measurement of Aij to be inter-pretable and psychologically plausible, we assume that recollec-tion and automatic activation are independent but only at the levelof participant–item combinations. That is, for the ith participantobserving the jth item, the success of recollection is independent ofthe success of automatic activation. This form of independencerefers to the independence of the underlying psychological pro-cesses. Consequently, we term it process independence.

Because each participant–item combination is unreplicated, re-searchers do not estimate Iij and Eij. Instead, they aggregate. Weconsider here aggregation of items to compute participant-specificproportions and denote these by Ii and Ei. The goal, then, is torecover participant-specific averaged parameters R� i� � ��jRij�/Jand A� i�� (�jAij)/J, where J is the number of items. The question athand is whether the estimation procedure of Equations 3 and 4yield good estimates of R� i. and A� i�.

Curran and Hintzman (1995) argued that participants who areparticularly good at conscious recollection may also be particularlygood at eliciting automatic activation. Likewise, items that areparticularly easy to consciously recollect may elicit a greaterautomatic activation. Additionally, they argued that even interac-tions may be correlated: Greater conscious recollection of partic-ular participant–item combinations may be accompanied bygreater automatic activation of those combinations. None of thesecorrelations violate process independence. Process independencedescribes what happens conditioned on a particular participant anditem. These correlations are at a secondary level and describe thedistribution of item and participant effects rather than the under-lying psychological processes.

Curran and Hintzman (1995) showed that these secondary corre-lations lead to systematic distortions in automatic activation estimatesfrom aggregation. In our notation, estimates of automatic activationderived from Ii. and Ei. do not converge in the large-sample limit to A� i.Curran and Hintzman provided a cogent informal explanation of howsecondary correlations distort estimates of automatic activation (see

also Curran & Hintzman, 1997; Jacoby, Begg, & Toth, 1997). Weprovide a simple example.

Consider the case of a single participant, and suppose that halfof the studied items are easy and the other half are hard. Assumeeasy items are consciously recollected with probability R � .5 andautomatically activated with probability A � .5. Hard items havecorresponding probabilities R � .1 and A � .1. Process indepen-dence may be assumed; that is, recollection for easy items isindependent of automatic activation for easy items and the samefor hard items. Following Equations 1 and 2, the true values ofstem completion for easy items are I � .75 and E � .25 for theinclude and exclude conditions, respectively. The true values forhard items are I � .19 and E � .09, respectively. Aggregatingthese probabilities across items yields stem-completion probabili-ties of I� � .47 and E� � .17. The goal is to estimate item-averagedconscious recollection and automatic activation. True values ofthese item averages are R� �.3 and A� � .3. Estimates are obtainedby substituting probabilities I� � .47 and E� � .17 into Equations 3and 4. The resulting estimates are R � .3 and A � .243.Although the estimate of conscious recollection is accurate, theestimate of automatic activation is too low. The underestimationreflects the fact that automatic activation is measurable only whenrecollection fails. If the two processes are correlated, instances ofrecollection failure will be greatly associated with instances ofautomatic activation failure. Importantly, this bias is asymptotic aswe used true probabilities as data.

Perhaps the most powerful element of the Curran and Hintzman(1995) critique is that they assumed correlations that do not violateprocess independence. These correlations violate independencewhere it is secondary, that is, across the distributions of items andparticipants. Therefore, the Curran and Hintzman critique showshow secondary sources of correlation, which are often not ofgeneral interest, play havoc in analysis.

Curran and Hintzman (1995) showed how this bias threatens thegold-standard evidence for the veracity of Jacoby’s process-dissociation model: selective influence. In a selective influencetest, the researcher manipulates a select variable. In the model,there is a prespecified set of parameters that should be affected bythis manipulation and a complementary set that should not (esti-mates should be the same across different levels of the manipula-tion within sampling error). If selective influence holds, the resultsare interpreted as support for the model (see Rouder, 2004).

One popular selective influence test is to assess whether studyduration affects recollection but not automatic activation. Thegeneral finding is that study duration does indeed affect recollec-tion but not automatic activation (Jacoby et al., 1997; Jacoby,Toth, & Yonelinas, 1993; cf. Curran & Hintzman, 1995). Thisselective influence finding has been crucial: It provides evidencefor the validity of the model with the independence assumption(see Cowan & Stadler, 1996, for extended discussion).

Curran and Hintzman (1995) argued that the systematic biasundermines tests of the invariance of A. Here, we provide a simpleexample: Consider the case in which both easy and hard items arepresented for short and long study times. Suppose that the truevalues in the short study-time condition are R � .5, A � .5, foreasy items and R � .1, A � .1, for hard items. As discussed before,the true averages are R� � .3, A� � .3, while the estimates are

372 ROUDER, LU, MOREY, SUN, AND SPECKMAN

R � .3, A � .243. Suppose that true values in the longstudy-time condition are R � .9, A � .9, for the easy items andR � .15, A � .15, for the hard items. The true averages for the longstudy-time condition are R� �.525, A � .525. The effect of studytime, on average, is to raise both R and A to .525 from .3. Theinvariance of A should be rejected, for A varies as much as R does.The estimates in this long study-time condition, obtained via

Equations 3 and 4, are R � .525, A � .229. Unfortunately, theestimates of A� are nearly equal in the two study-time conditions(.243 vs. .229), even though the true difference is large. In thiscase, this estimated invariance is an artifact of aggregation. AsCurran and Hintzman explained, the underestimation of A is di-rectly related to the size of R. When R is increased, the underes-timation increases as well. Even though the automatic activation isgreater for longer study times in the example, the underestimationbias is more extreme. These opposite trends nearly cancel, pro-ducing the appearance of invariance.

In this article, we present a hierarchical model that provides foraccurate estimation without recourse to aggregation. Before pre-senting the model, we address whether there are empirical meansof mitigating distortions from aggregation, as has been claimed atleast implicitly. Curran and Hintzman (1995), who first demon-strated distortions from aggregation, ran a set of experiments todemonstrate that aggregation is problematic. They showed thataggregated estimates of automatic activation decrease with studytime. This decrease, they argued, is evidence of downward bias inthe estimation of automatic activation. The situation, however, isnot so clear. The result may be interpreted as indicating thatautomatic activation simply decreases with study time. The Curranand Hintzman interpretation only holds if one assumes that trueautomatic activation is invariant to study time. The modelingapproach presented here tests the invariance rather than assuming it.

Jacoby et al. (1997) made an analogous argument, though in anopposite direction. They argued that Curran and Hintzman’s(1995) data set is an anomaly. More typically, automatic activationis nearly constant across study time and other benchmark manip-ulations (e.g., divided attention). This constancy is taken as evi-dence for a lack of downward bias in process-dissociation analysis.Yet this statement also assumes that true automatic activation isinvariant to study time. It could be that the observed invariancereflects both an increase in automatic activation with study timecounteracted by an increased downward bias (as in the aboveexample). The relevant point is that there is no empirical argumentin this framework that would be decisive.

Currently, although selective influence has been established, itmay be an artifact of aggregation. We addressed this problem bydeveloping a model for which aggregation is not necessary. Af-terward, we applied the model to the results of a selective influenceexperiment (Experiment 1) to decide whether selective influencetruly holds.

A Hierarchical Process-Dissociation Model

Model Specification

Our approach is to extend Jacoby’s process-dissociation modelwith parameters that account for participant and item variability.We follow the development in Lu, Speckman, Sun, and Rouder

(2007), who provided the following three-level hierarchical model.The first level is the process-dissociation model:

Iijk � Rijk � (1 � Rijk)Aijk, (7)

and

Eijk � (1 � Rijk)Aijk, (8)

where as before i and j index participants and items, respectively,and k indexes study-duration condition. For each participant i, eachitem j appears in only one of the study conditions. Therefore, wecannot obtain all combinations of participants, items, and studydurations, necessitating further constraint.

The second level describes how Rijk and Aijk are constrained toreflect only main effects of participants, items, and conditions.Parameters Rijk and Aijk are probabilities that range between zeroand one. It is convenient and common to place linear models onparameters that span the reals, and one popular method is to modelprobit-transformed probabilities. This transform and the closelyrelated logit transform are the main staple in psychometrics (Nun-nally & Bernstein, 1994). Let aijk and rijk denote the transformedparameters, that is, Aijk � �(aijk) and Rijk � �(rijk), where � is thestandard normal cumulative distribution function. The curve inFigure 2 shows the probit transform.

The second level is a main effects model on transformed pa-rameters a and r:

rijk � �i(r) � j

(r) � k(r), (9)

and

aijk � �i(a) � j

(a) � k(a). (10)

Parameters �i(r) and �i

(a) denote the ith participant’s consciousrecollection and automatic activation abilities, respectively; pa-rameters j

(r) and j(a) denote the jth item’s propensity to elicit

conscious recollection and automatic activation, respectively; andparameters k

(r) and k(a) denote the overall level of conscious

recollection and automatic activation in the kth condition, respec-tively. Figure 2 shows an example of the model on consciousrecollection for values of a condition effect (k

(r) � .5), a partici-pant effect (�i

(r) � �1) and an item effect (j(r) � 1.5). After

adding these main effects and transforming, the correspondingvalue of parameter Rijk is .84. To maintain identifiability, partici-pant and item effects are treated as zero-centered while conditioneffects are unconstrained. This approach of using additive modelson appropriately transformed variables is analogous to the Raschmodel in item response theory (Lord & Novick, 1968).

The models in Equations 9 and 10 are not identifiable withoutadditional constraint. For example, a constant C may be added to�i

(r) and subtracted from j(r) with no effect on rijk. If people and

items were fixed effects, then the constraint would be �i�i(r) � 0,

�i�i(a) � 0, �jj

(r) � 0, and �jj(a) � 0. In memory experiments,

both participants and items are sampled from larger populations towhich researchers wish to generalize findings. In this case, it isappropriate to consider participant and item effects as random(Clark, 1973). A natural choice in this application is to modelparticipant and item effects as draws from bivariate normals witharbitrary covariance matrices:

373HIERARCHICAL PROCESS DISSOCIATION

� �i�r�

�i�a� � � N2�0,���, i � 1, . . ., I, (11)

and

� j�r�

j�a� � � N2�0,��, j � 1, . . ., J, (12)

where I and J reference the number of participants and items,respectively. The covariance matrices describe how the populationof random effects are distributed. The covariance matrix �� hasthree unique elements: the variance of participant effects on con-scious recollection, the variance of participant effects on automaticactivation, and the covariance among these participant effects. Ifthe covariance is positive, then participants who have a higherdegree of conscious recollection tend to have a higher degree ofautomatic activation. The model is flexible and may account for anydegree of correlation: positive, negative, or none. These covariancesare free parameters that are estimated rather than assumed.

We term the model defined by Equations 7–12 as the hierar-chical process-dissociation model. To make the estimates of au-tomatic activation interpretable, we assume process independence.That is, for a specific participant observing a specific item in aspecific condition, recollection and automatic activation are as-sumed to be independent. The model, however, does not assumeindependence across people and items. People who are particularlygood at recollection may be particularly good at automatic activa-tion, and the same holds for items. Therefore, the hierarchicalmodel captures process independence yet may be analyzed inreal-world conditions in which secondary sources of correlationare present.

Analysis

The approach we take to analyze the model is Bayesian. Al-though many researchers prefer the Bayesian approach on philo-sophical grounds (e.g., Berger & Wolport, 1988; Edwards, Lind-

man, & Savage, 1963; Wagenmakers, in press), we have far morepragmatic reasons for this choice. Simply put, while Bayesianimplementation is relatively straightforward, a more classical (fre-quentist) implementation is not. Moreover, Bayesian modeling isin the mainstream in statistics, economics, epidemiology, andpsychometrics. To help bring Bayesian analysis to the mainstreamof experimental psychology, we have provided a tutorial on Bayes-ian hierarchical analysis for experimental psychologists (seeRouder & Lu, 2005).

In Bayesian analysis, the researcher provides a prior distributionfor the parameters. In many applications, the prior may be madenoninformative or nearly noninformative, as they are in this ap-plication. In this model, priors are needed for grand means (k

(r)

and k(a)) and covariance matrices (�� and �). The priors for

these parameters, as well as a general outline of model analysis,are provided in the Appendix. Software in the R language isprovided at http://web.missouri.edu/�umcaspsychpcl/code. Lu etal. (2007) provided more complete coverage of the model, includ-ing derivations of conditional posterior distributions. Lu et al. alsostudied the behavior of the model under a number of alternativepriors.

Model Assumptions

There are three new assumptions in the hierarchical process-dissociation model: the probit link; the bivariate normal distribu-tion on random effects; and the additivity between participants,items, and conditions. The first two of these, the probit link and thebivariate normal, are common choices in models of this type. Thethird assumption about additivity deserves more scrutiny. Trueunderlying additivity implies that there are no interactions. Aninteraction in this context would imply that a particularparticipant–item combination is different from the sum of maineffects. Curran and Hintzman (1995) argued that these interactionsare not only plausible but may be positively correlated acrossrecollection and automatic activation. Such unmodeled correla-

−4 −2 0 2 4

Rijk

Rijk

ijkr

ijkr

Model on Recollection:

Pro

babi

lity

0.0

0.2

0.4

0.6

1.0

Probit Value

=.84

=

=1=.5 −1 +1.5

µk +α +βi j

=1

Figure 2. Additive probit model on recollection: Rijk � �(k � �i � j). Figure depicts the case in which k �.5, �i � �1, and j � 1.5. Resulting value of recollection is Rijk � .84.

374 ROUDER, LU, MOREY, SUN, AND SPECKMAN

tions would exert downward bias on the estimation of automaticactivation. The argument is analogous to that used to show down-ward bias with easy and hard items in the previous example.Therefore, it is necessary to evaluate the practical consequences ofthe additivity assumption. The results of Experiments 1 and 2 serveas a guide for evaluation; therefore, we defer discussion of theconsequences of additivity until after presenting the experiments.To foreshadow, these consequences are not large and do notchange our conclusions.

Experiment 1

Although selective influence has been previously established, itmay be an artifact of aggregation. Experiment 1 was designed toassess whether selective influence holds even when the data areanalyzed without recourse to aggregation via the hierarchicalprocess-dissociation model.

Experiment 1 is modeled after Curran and Hintzman’s (1995)Experiments 4 and 5. Participants were asked to study a list ofitems and then asked to stem-complete these items in accordancewith include and exclude instructions. Following Curran andHintzman, we manipulated study duration through three levels:items not studied, items studied for 1 s, and items studied for 10 s.The predictions about the effect of study time on recollection arestraightforward: More study time promotes greater recollection.More to the point, recollection should be better for items studiedfor 10 s than for items studied for 1 s. The predictions aboutautomatic activation are more subtle. Automatic activation mayrise with the duration of study until a point, after which additionalstudy does not raise automatic activation further. On the basis ofthe relationship between priming and study duration, Neill, Beck,Bottalico, and Molloy (1990) concluded that additional study past1 s does not increase automatic activation. Following previousprocess-dissociation researchers, we predicted that the automaticactivation for the 1-s and 10-s study conditions would be aboutequal and greater than the no-study condition.

Method

The method for Experiment 1 closely follows that of Jacoby etal. (1993) and Curran and Hintzman (1995).

Participants. Sixty-six undergraduate students at the Univer-sity of Missouri—Columbia received partial class credit for par-ticipation.

Materials. The items consisted of 116 five-letter words fromJacoby et al. (1993). The stems were the first three letters of theitems followed by two blanks. Each stem yielded only two validcompletions, one of which was the item. For example, item lapelyielded a stem lap__ which has valid completions of lapel andlapse. Twenty of these items were used as fillers; the remaining 96comprised the critical set used in analysis. The 96 critical itemswere those used by Curran and Hintzman (1995, Experiments 4and 5). All items were presented and tested on a PC. The size ofeach letter on the display was 5 mm � 4 mm. All items werepresented in an uppercase type.

Design. The main independent variables were study durationand instructions at test. Study duration was manipulated throughthree levels (no study, 1-s presentation, 10-s presentation). Studyduration was blocked—all items of the same study duration were

presented together to prevent selective rehearsal of short-durationitems during the longer study period of the long-duration items.The order of study condition blocks was counterbalanced acrossparticipants. Testing instructions were either the include or theexclude instructions. These were randomly intermixed throughoutthe test list. Altogether, there were six conditions obtained bycrossing study duration and item instructions. These six conditionswere crossed with participants in a balanced Latin square design.

Procedure. The experiment was divided into instructions,study, and test phases. In the instructions phase, participants wereshown a sample item and a sample stem and were taught theinclude and exclude instructions. In the study phase, items werepresented sequentially and in the center of the display. The orderof items within a block was randomized, with the exception thateach block began and ended with five filler items.

The test phase began on completion of the study phase. Partic-ipants were cued with the simple phrases “Use a word from thelist” or “Do NOT use a word from the list” to indicate whether astem was to be completed under the include or exclude conditions,respectively. To make these cues more salient, the former waspresented in green, and the latter was presented in red. Participantstyped in the two letters that completed the stem. They wereallowed to complete the stem with “XX” if they could not think ofa valid completion. The order of stems was randomized. Partici-pants were self-paced at test. Total time to complete a session wasapproximately 45 min.

Results

The data were submitted to conventional (aggregation-based)analysis and to the hierarchical model analysis.

Conventional analysis. Stem-completion rates for include andexclude conditions were generated for each participant by aggre-gating over items. The resulting rates were submitted to Equations3 and 4 to compute estimates of Ri�k and A� i�k for each participant ineach condition. In some cases, especially those in which itemswere not studied, the proportion of stem completion was greater inthe exclude than in the include condition. When data are orderedthis way, the resulting estimates from Equation 3 for Ri�k arenegatively valued. Whereas negative values are not interpretable inthe process-dissociation model, we used maximum likelihood sub-ject to the constraint that parameter estimates be positive. Con-strained maximization was performed with the simplex algorithm(Nelder & Mead, 1965). Means of Ri�k and Ai�k are shown in the firstrow of Table 1. We refer to these estimates as the item-aggregationestimates.

Table 1Estimates of Overall Recollection and Automatic Activation as aFunction of Study Condition for Experiment 1

Recollection Automatic activation

Nostudy 1 s 10 s

Nostudy 1 s 10 s

Item aggregation .100 .233 .336 .270 .348 .315Double aggregation .065 .216 .328 .279 .356 .329Bayesian equivalent .042 .220 .320 .287 .378 .358

375HIERARCHICAL PROCESS DISSOCIATION

In addition to item aggregation, we also computed estimatesfrom aggregating stem-completion rates for each condition overboth participants and items. These double-aggregated stem-completion rates were submitted to Equations 3 and 4 to computeestimates of R��k and A� ��k for each condition. These are shown inthe second row of Table 1. The third row is from the hierarchicalBayesian model, which is discussed subsequently.

The aggregation-based analyses tell a consistent and plausiblestory. Greater study durations yielded increasing levels of recol-lection. The story for automatic activation is only slightly morecomplex. Studied items elicited greater automatic activation thanunstudied items. The effect of duration was modest, with some-what more automatic activation in the 1-s study condition than inthe 10-s study condition. (Inferential statements are made with thehierarchical Bayesian model discussed subsequently.)

There are some differences between the two aggregation meth-ods, especially for the no-study condition. The difference reflectsa small-sample bias in estimation. As the true value of R becomesvanishingly small, half of all stem completions are better for theexclude than for the include condition. For this half, the estimateof R is forced to be positive by the constrained maximum-likelihood procedure. For the other half, the half in which stemcompletion is better for the include than the exclude condition, theestimate of R from Equation 3 is small but positive. The averageof these two cases yields a small upward bias in estimating R anda corresponding small downward bias in estimating A. Fortunately,these biases decrease with increasing sample size. Not surpris-ingly, then, the bias is more noticeable for item aggregation thanfor double aggregation as the sample sizes are smaller for theformer than for the latter. Because item-aggregation estimates havesome nonasymptotic bias for conditions with low recollection, theydo not serve well in this application. Consequently, we use thedouble-aggregation estimates for comparison with the hierarchicalBayesian model.

Many previous researchers in process dissociation have notdiscussed the problem of negative recollection estimates whenoverall recollection estimates are very low. We strongly suspectresearchers have admitted negative values of recollection withitem aggregation; all reports simply stated that their estimates of Rfollow R � I � E without qualification. In several studies, therewere very low averaged values of R (e.g., R .05), which indicatethat negative individual estimates were admitted. Fortunately, thehierarchical model allows researchers to avoid this issue as esti-mates are constrained to be positive by the probit transform.

There is an inconsistency in the literature with regard to whetherautomatic activation decreases with increasing study time. Jacobyand colleagues (see Jacoby et al., 1997, Table 1) reported that theaverage decrease in A with study time across eight experiments is.003. In contrast, Curran and Hintzman (1995) found more sizabledeclines. They found an average of .047 across three experiments,with a .10 decline in their Experiment 5, which they considered tobe the most valid of their experiments. The size of the decline inour data (double aggregation) is .03, which is intermediate be-tween those found by the two groups.

Hirarchical model analysis. We follow Lu et al. (2007) inanalyzing the data of Experiment 1.1 In Bayesian analysis, the endproduct of estimation is a posterior distribution, that is, a distri-bution of the parameters given the data. The left column of Figure3 provides posterior distributions of parameters k

(r) and k(a), the

overall grand mean effect on recollection and automatic activation,for different study conditions. In frequentist analysis, such as theabove aggregation analyses, the end product is a point estimate,such as those in Table 1. In Bayesian analysis, the posteriorsthemselves provide all information needed for estimation andinference. The mean of the posterior serves as a point estimate;an interval that contains 95% of the mass is called a 95%credible interval and is analogous to a frequentist confidenceinterval.

Posterior distributions of recollection (see Figure 3, top panels)vary considerably as a function of duration, whereas posteriors ofautomatic activation (see Figure 3, bottom panels) vary much lessso. This graph serves as the main evidence for selective influencein the experiment. Parameters k

(r) and k(a) are on the probit scale

and are not directly comparable to double-aggregation estimates ofR��k and A� ��k. To provide for a direct comparison, we first com-puted Bayesian posterior mean point estimates of all Rijk and Aijk,which are on the probability scale rather than the probit scale.These point estimates were then averaged over participants anditems for each condition. The resulting values are displayed inTable 1 and are labeled Bayesian equivalent.

Selective influence. Selective influence is obtained if studyduration affects only recollection. The Curran and Hintzman(1995) critique centered on the possibility that automatic activationincreases with study duration, but this effect is masked by anincreasing downward bias from aggregation. The hierarchicalmodel provides for an accurate test of selective influence; thecritical question is whether the hierarchical model estimators re-veal an increase in automatic activation with study duration.

The left-hand column of Figure 3 reveals a strong effect of studyon recollection and a weak one on automatic activation. We focuson the contrast between the 1-s and 10-s study conditions as thisprovides for the strongest test of selective influence as discussedpreviously. Let k � 1, 2 denote the 1-s and 10-s study conditions,respectively. The contrasts of interest are 2

(r) � 1(r), for recollec-

tions, and 2(a) � 1

(a), for automatic activation. Posterior distri-butions for these contrasts are shown in the right-hand column ofFigure 3. The dotted vertical lines indicate the 95% credibleinterval between the 2.5 and 97.5 percentiles, the Bayesian ana-logue to the 95% confidence interval. In the top panel (for recol-lection), the value of zero difference is outside the 95% credibleinterval, indicating that the difference in recollection between the1-s and 10-s conditions is statistically significant. The bottom-rightpanel shows the same plot for automatic activation. Automaticactivation trends lower in the 10-s than in the 1-s study condition,indicating that there is a lack of evidence for the claim thatautomatic activation increases with study duration. Note that thevalue of zero is well within the 95% credible interval, indicating alack of evidence for a reliable difference across the 1-s and 10-sconditions. The corresponding 95% confidence interval on theprobability scale is (�.06, .02). Therefore, from a Bayesian point

1 The current experiment is a Latin square design rather than a completefactorial design. Each participant observed each item in only one of thethree study conditions. Fortunately, the additive structure of the modelmakes estimation possible in incomplete designs. The technical constraintsfor design matrices for linear models are provided in Christensen (1996),and the design of Experiment 1 meets these constraints.

376 ROUDER, LU, MOREY, SUN, AND SPECKMAN

of view, there is less than a 2.5% chance that the automaticactivation in the 10-s condition is more than .02 greater than thatin the 1-s condition.

One of the statistical problems in testing selective influence isthat invariances are assessed by failing to reject the null hypoth-esis. Such practices may be abused if the power of the test is notsufficient. When the null hypothesis is not rejected, it is useful tosupplement the analysis with a relative comparison, and we com-pared the effects of study time on automatic activation with thoseon recollection. We constructed an effects-ratio statistic by divid-ing the difference in A (see Figure 3, bottom-right panel) by thedifference in R (see Figure 3, top-right panel) and denote it by �.Because these differences are random in the Bayesian framework,

the effects-ratio statistic has a distribution. The posterior mean of� is �� � �.169, indicating that the magnitude of the effect ofstudy duration on automatic activation is less than a fifth of that onrecollection and in the opposite direction. The Curran and Hintz-man (1995) concern is that automatic activation increases withstudy duration. This concern may be addressed by computing theposterior probability that � is positive and large. We decided toplace a one-fifth criterion on the effect-size ratio to address Curranand Hintzman’s concern. The appropriate test statistic is the pos-terior probability that � � .2 given the data. This probability is lessthan .016, which indicates that Curran and Hintzman’s concernsabout aggregation masking an increase in automatic activation areunwarranted for this data set.

−3.0 −2.5 −2.0 −1.5 −1.0 −0.5 0.0

0

1

2

3

4

Probit Units

Den

sity

No Study1 Second10 Seconds

−3.0 −2.5 −2.0 −1.5 −1.0 −0.5 0.0

0

1

2

3

4

5

6

Probit Units

Den

sity

Probit Units

Den

sity

−0.2 0.0 0.2 0.4 0.6

0

1

2

3

4

Probit Units

Den

sity

−0.2 0.0 0.2 0.4 0.6

0

2

4

6

Posterior of Grand−Mean of Recollection

Posterior of Grand−Mean of Automatic Activation

Posterior of Difference in Recollection

Posterior of Difference in Automatic Activation

Figure 3. The top-left and bottom-left panels show posterior distributions of overall recollection (k(r)) and

automatic activation (k(a)), respectively. The top-right panel shows the contrast between 10-s and 1-s study

conditions on recollection. The zero point is outside the 95% credible interval indicating a significant effect ofstudy duration. The bottom-right panel shows the same for automatic activation. The zero point is well withinthe 95% credible interval, indicating a lack of evidence for an effect of study duration. Densities in left panelsare obtained by Gaussian kernel smoothing of model-based outputs.

377HIERARCHICAL PROCESS DISSOCIATION

Participant and item random effects. The hierarchical analysisallows for the study of individual and item differences. The toprow of Figure 4 shows the relevant effects as scatterplots. Thetop-left plot is for participants and shows participants’ automaticactivation estimates as a function of their recollection estimates.The top-right plot shows the same for items. The scatterplot formatis convenient as it shows the variability and correlation acrossrandom effects. The range of variation is modest. Standard devi-ations, measured in probit units, are on the order of .4. Thecorrelations between recollection and automatic activation areclearly different for participants than for items, with there beingonly a small correlation for the former (r � .11) and a large one forthe latter (r � .89).

Parameters �� and � account for correlations among the ran-dom effects for the population of participants and items. Thebottom row of Figure 4 shows posterior distributions of correlation

coefficients as calculated from the posteriors on �� and �. Thelack of evidence for a correlation for participant effects can be seenin the posterior distribution of the correlation coefficient derivedfrom the covariance matrix �� (see Figure 4, bottom-left panel;posterior M � .03). The substantial correlation across items isevident in the posterior distribution of the correlation coefficientderived from the population-level covariance matrix � (see Fig-ure 4, bottom-right panel; posterior M � .67). The correlationsfrom the covariance matrices are more conservative than thosefrom the individual random-effect estimates. This conservatism isexpected. The correlations from the random effects are akin to samplecorrelations in that they describe properties of these particular itemsand participants. The correlations from parameters �� and � arepopulation-level estimates. More evidence is needed to produceextreme estimates of these population-level parameters than toproduce extreme estimates for a particular set of random effects.

−1.0 −0.5 0.0 0.5 1.0

−1.

0−

0.5

0.0

0.5

1.0

Participant Effects

Effect on Recollection

Effe

ct o

n A

utom

atic

Act

ivat

ion

−1.0 −0.5 0.0 0.5 1.0

−1.

0−

0.5

0.0

0.5

1.0

Item Effects

Effect on Recollection

Effe

ct o

n A

utom

atic

Act

ivat

ion

Posterior of Participant Correlation

Correlation

Den

sity

−1.0 −0.5 0.0 0.5 1.0

0.0

0.5

1.0

1.5

2.0

2.5

Posterior of Item Correlation

Correlation

Den

sity

−1.0 −0.5 0.0 0.5 1.0

01

23

4

Figure 4. Random effects for Experiment 1. Top left: Participant random effects on automatic activation as afunction of those on recollection. Top right: Same plot for item random effects. Bottom: Plots of the posteriordistributions of the correlation coefficients as derived from �� and �, respectively.

378 ROUDER, LU, MOREY, SUN, AND SPECKMAN

Discussion

The process-dissociation model passes the selective influencetest: Study duration affected recollection to a far greater extentthan automatic activation. This result cannot be due to aggregationartifacts as the hierarchical model avoids aggregation. In the re-mainder of this discussion, we use the hierarchical model to assesshow badly aggregation distorts estimation in conventional process-dissociation analysis. The results are fairly encouraging, indicatingthat many of the previous demonstrations of selective influence arereasonable.

Hierarchical estimates of overall automatic activation aregreater in value than their double-aggregation counterpart (seeTable 1). The difference between estimation methods is .022 in the1-s condition and .029 in the 10-s condition, which suggests asmall underestimation bias. The more important consideration ishow this asymptotic bias varies with study duration. Curran andHintzman (1995) predicted that aggregation estimates for auto-matic activation would be greater for the 1-s study condition thanfor the 10-s study condition. Their rationale is as follows. Asdiscussed in the prior example with easy and hard items andselective influence, the amount of downward bias in automaticactivation increases with greater true values of recollection. Be-cause recollection is greater in the 10-s study condition than in the1-s study condition, there is greater downward bias on the auto-matic activation estimate in the 10-s than in the 1-s study condi-tion. Indeed, the double-aggregation estimate of automatic activa-tion is .027 greater in the 1-s than in the 10-s condition. Yet asimilar pattern was found for Bayesian estimates, which are notasymptotically biased. Automatic activation is .020 greater in the1-s than in the 10-s study condition. The difference, .007, is anestimate of the amount of difference in downward bias acrossconditions. The significance of this amount is a function of the sizeof the effects, and in many applications, this degree of bias is fairlyinconsequential.

We performed a small simulation study to assess the effects ofaggregation more generally. The hierarchical model provides es-timates of condition, participant, and item effects for recollectionand automatic activation, with the item effects being highly cor-related. We used these estimates as true values and then simulateddata from the hierarchical model. Parameters were then estimatedwith the double-aggregation method. This process of simulatingdata from fixed true values and estimating parameters was re-peated 500 times to provide a reasonable estimate of the expecteddegree of bias in double aggregation. This simulation provided theexpected downward biases for the 1-s and 10-s conditions, whichwere .022 and .029, respectively.

The amount of downward bias increases with the degree ofcorrelation and the amount of recollection. We used simulations tostudy the effect of different values of recollection on downwardbias. Once again, estimates from Experiment 1 served as truevalues for simulation, with the exception of the overall recollectionparameter (r). This parameter was manipulated through severalvalues to simulate the effect of different study-duration conditions.For each value of (r), we repeated the simulation 500 times.Figure 5 shows the expected bias in the double-aggregation esti-mate of automatic activation as a function of overall recollection(plotted on a probability scale). Although the amount of biasincreased with recollection, the rate of increase was not too great

for smaller values of recollection. Aggregation becomes suspecteither when recollection varies greatly across conditions or whenrecollection is of a high value such that small differences result inlarge differences in bias. The two vertical lines in Figure 5 showthe recollection values for the 1-s and 10-s study conditions. Thedifference in bias across these values is not substantial.

Preexperimental Automatic Activation

The striking pattern of correlations across items and its absenceacross participants is a strong result. Curran and Hintzman (1995)found similar patterns, though not to the same degree. Theiraverage sample correlation across items is about .5, whereas oursis about .8. Even though the results are concordant, ours is fareasier to interpret, especially with regard to statistical significance.Their result is based on aggregation, and it is unknown what effectaggregation has on correlation estimates. Moreover, it is wellknown that their aggregation across participants increases the TypeI error in subsequent statistical tests (Clark, 1973). Our estimatesare not only greater in magnitude, they are made without recourseto aggregation, and statistical significance is easily assessed.

Curran and Hintzman (1995) provided a straightforward con-jecture for the correlation across items. Stems themselves vary inhow conducive they are to completion by the studied item. Forexample hum__ is a better stem for humor than hou__ is for hound.This variability induces a correlation in performance across in-clude and exclude conditions. In this section, we describe a test ofthis and other explanations of the correlation. Unfortunately, wefind little evidence for these explanations in our data.

In Experiment 1, there was a no-study baseline condition. In-clusion of this condition allows for the assessment of how muchautomatic activation is preexperimental and how much results

0.0 0.2 0.4 0.6 0.8 1.0

−0.

12−

0.08

−0.

040.

00

Overall Recollectability

Est

imat

ed B

ias

in A

utom

atic

Act

ivat

ion

Figure 5. Expected degree of downward bias in automatic activationestimates from double aggregation. Points denote results of the simula-tions; the fitted line is a smoothed in a nonparametric manner (Cleveland,1981). The two vertical lines show the recollection values for the 1-s and10-s study conditions in Experiment 1, respectively.

379HIERARCHICAL PROCESS DISSOCIATION

from the study list. We adapted an extension from Buchner et. al(1995) to separate these two contributions. On include trials wherethe item has been studied, either recollection, study-driven auto-matic activation, or preexperimental automatic activation is suffi-cient for stem completion. On exclude trials where the item hasbeen studied, stem completion comes about when recollection failsand either preexperimental or stimulus-driven automatic activationsucceeds. When an item is not studied, stem completion occursonly from preexperimental automatic activation. These statementsyield the following model equations for a particular participanttested with a particular stem when items have been studied:

Iijk � Rijk � (1 � Rijk)(Aijk � (1 � Aijk)Bj),

and

Eijk � (1 � Rijk)(Aijk � (1 � Aijk)Bj).

The corresponding equation in the no-study condition (denoted byk � 0) is

Iij0 � Eij0 � Bj.

Equations 9 and 10 were used to constrain Rij and Aij. Parameter Bj

was modeled as

bj � �b) � j(b),

where bj � ��1(Bj). There are no participant effects in bj. Anyparticipant effects would occur as an interaction, that is, givenstems are completed at different rates by different participants. Itis impossible to model such interactions without replicatingparticipant–item pairings. We discuss the consequences of notmodeling interactions after presenting all of the experimental data.

Participant random effects are modeled by extending Equation11; a trivariate normal is used to model random item effects:

�j

�r�

j�a�

j�b� � � N3�0,��, j � 1, . . ., J,

where � is a 3 � 3 covariance matrix of six free parameters. Weassume process independence in the model as previously dis-cussed. We term this extended model with preexperimental auto-matic activation as the extended hierarchical process-dissociationmodel and used it to reanalyze Experiment 1.

With this extended hierarchical model, the pattern of correlationamong items observed in the previous hierarchical model-basedanalysis may be decomposed into correlations among three pro-cesses: recollection, study-driven automatic activation, and preex-perimental automatic activation. Parameter j

(b) serves as a relativeindex of how well a stem primes its completion in the absence ofany study. Curran and Hintzman’s (1995) speculation about thecorrelation between recollection and automatic activation moti-vated two questions: (a) Is there a correlation between study-driven automatic activation and recollection, and (b) if so, is thiscorrelation solely due to preexperimental automatic activation?Question (b) is answered by assessing whether the correlationbetween study-driven automatic activation and recollection is at-tenuated to zero when preexperimental automatic activation iscontrolled (partialed out).

Analysis with the extended hierarchical model reveals thatmuch, though not all, of the automatic activation is preexperimen-tal. Preexperimental automatic activation is .266 on average,whereas study-driven automatic activation is .051 on average.Almost all of the previous findings held: Figure 6A shows thatstudy-driven automatic activation did not differ substantiallyacross the 10-s and 1-s study conditions. Figure 6B shows a lackof covariation in study-driven automatic activation and recollec-tion across participants.

The relationship between the three item-based processes (recol-lection, study-driven automatic activation, and preexperimentalautomatic activation) is shown as pairwise scatterplots in Figures6D–6F. All three processes covary together to some degree. Thescatterplots show that there is a moderate correlation betweenstudy-driven automatic activation and recollection (r � .68 forsample values and r � .42 for generalization to the population). Toassess whether this correlation was driven by preexperimentalautomatic activation, we estimated a partial correlation. There aretwo approaches to doing so: one based on individual randomeffects and another using the population-level covariance matrix�. The latter is more appropriate for our purposes. If we had usedthe former, it could be claimed that a lack of attenuation in partialcorrelation is due to noisy individual random-effect estimates (thisphenomenon is referred to as an errors-in-variables problem in thestatistics literature). The use of the population-level covariancematrix corrects for sampling noise in the individual item estimates.The posterior distribution of the population-level partial correla-tion of stimulus-driven automatic activation and recollection, whenconditioned on preexperimental automatic activation, is shown inFigure 6C. The posterior mean partial correlation is .35, which isnot that different from the population marginal correlation of .42.Moreover, there is little evidence that this population-level corre-lation is zero, a value that would indicate that all of the observedcorrelation is from variation in preexperimental automatic activa-tion. In fact, the probability that the population-level partial cor-relation is greater than zero is .93.

Stem-completion baselines are not the only property that variesacross items. We assessed whether commonly considered wordproperties, age of acquisition (Gilhooly & Logie, 1980), wordfrequency (Kucera & Francis, 1967), meaningfulness (Toglia &Battig, 1978), familiarity, and imagability (Coltheart, 1981) couldaccount for the correlation between study-driven automatic acti-vation and recollection. Consequently, we computed the partialcorrelation between study-driven and automatic activation andrecollection conditioned on each of these properties. We make theassumption that these properties are measured without error.Hence, there is no errors-in-measures problem, and the samplepartial correlation coefficient is not subject to distortion in atten-uation. There was only a small degree of attenuation in partialcorrelation, the largest of which was from imagability. The partialcorrelation is r � .66, which is only slightly smaller than themarginal sample partial correlation of r � .68. The correlationof study-driven familiarity and recollection is not readily drivenby common dependencies on word properties. Given that theseword properties typically account for only a small fraction ofoverall variance in memory experiments, the result is not toosurprising.

380 ROUDER, LU, MOREY, SUN, AND SPECKMAN

Difference in Stimulus−DrivenAutomatic Activation

Probit Units

Den

sity

−0.6 −0.4 −0.2 0.0 0.2

0.0

0.5

1.0

1.5

2.0

2.5

A

−1.0 −0.5 0.0 0.5 1.0

−1.0

−0.5

0.0

0.5

1.0

Participant Effects

Effect on Recollection

Effe

ct o

n S

timul

us−

Driv

enA

utom

atic

Act

ivat

ion

B

Sample corr.: −0.25 Pop. corr.: −0.17

Correlation of Recollectionand Study−driven Automatic Activation

given Pre−experimental Automatic Activation

Partial Correlation

Den

sity

−1.0 −0.5 0.0 0.5 1.0

0.0

0.5

1.0

1.5

C

−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

−1.5

−1.0

−0.5

0.0

0.5

1.0

1.5

Effect on Recollection

Effe

ct o

n S

tudy

−D

riven

Aut

omat

ic A

ctiv

atio

n

Sample corr.: 0.68 Pop. corr.: 0.42

D

−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

−1.5

−1.0

−0.5

0.0

0.5

1.0

1.5

Effect on Recollection

Effe

ct o

n P

re−

Exp

erim

enta

l A

utom

atic

Act

ivat

ion

Sample corr.: 0.85 Pop. corr.: 0.66

E

−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

−1.5

−1.0

−0.5

0.0

0.5

1.0

1.5

Effect on Study−Driven Automatic Activation

Effe

ct o

n P

re−

Exp

erim

enta

l A

utom

atic

Act

ivat

ion

Sample corr: 0.45 Pop. corr.: 0.27

F

Figure 6. Reanalysis of Experiment 1 with the extended hierarchical process-dissociation model. A: Posterior of thedifference in study-driven automatic activation between the 1-s and 10-s study conditions shows no evidence of aneffect. B: The scatterplot of participant specific recollection and study-driven automatic activation effects shows littlecorrelation. C: Posterior distribution of the partial correlation of recollection and study-driven activation for controlledpreexperimental automatic activation. The vertical dashed line is the one-tailed 95% credible bound. D–F: Pairwisescatterplots of the item effects on recollection, study-driven automatic activation, and preexperimental automaticactivation. Sample corr. � sample-level correlation; Pop. corr. � population-level correlation.

381HIERARCHICAL PROCESS DISSOCIATION

Experiment 2

Analysis of Experiment 1 provided support for the hierarchicalprocess-dissociation model. The goal in Experiment 2 was toprovide the following tests of the validity and usefulness of themodel: (a) The pattern of correlations in Experiment 1, with nocorrelation in participant effects and sizable correlation in itemeffects, is noteworthy. We assessed its replicability under differentstudy conditions. (b) If the model is accurately capturing system-atic item-based effects, then these should be largely stable. Exper-iment 2 provided an opportunity to investigate the stability of theseitem-based estimates. (c) The usefulness of the model is its abilityto provide accurate estimates in conditions where aggregationfails. In Experiment 2, we explored whether differences in recol-lection could be made sufficiently large between conditions suchthat aggregation led to consequential distortions.

In Experiment 2, we implemented a levels-of-processing ma-nipulation, as this is known to dramatically affect performance(Craik & Lockhart, 1972). In previous process-dissociation anal-yses of stem-completion tasks, processing level has been shown todramatically affect estimates of recollection (Bergerbest &Goshen-Gottstein, 2002; McBride & Dosher, 1999; Toth, Rein-gold, & Jacoby, 1994). The effect on automatic processing, on theother hand, has been inconsistent. Bergerbest and Goshen-Gottstein (2002) found more automatic activation for rating thepleasantness of items than for counting vowels or syllables. Tothet al. (1994) and McBride and Dosher (1999), in contrast, found noincrease in automatic activation across similar deep and shallowstudy conditions. Whereas there is some disagreement in theliterature on this point, our study provides needed evidence. In theshallow-processing condition, participants were asked to countvowels in items at study; in the deep-processing condition, partic-ipants were encouraged to use whatever strategies they believedbest facilitated recall for a subsequent memory test.

Method

Participants. Forty-four undergraduate students at the Univer-sity of Missouri—Columbia received partial class credit for par-ticipation.

Materials. Thirty-two words from Experiment 1 served ascritical items. Fewer critical items were used in Experiment 2 toguarantee a high rate of recollectability for the deep-processingcondition. The items were chosen to maximize variance of itemeffects. There were 16 additional filler items.

Design. The main independent manipulations were the pro-cessing level at study and the instructions at test. For the shallowlevel, participants were asked to count vowels, and the items weredisplayed for 1 s. For the deep level, participants were asked tostudy items for a later memory test, and the items were displayedfor 12 s. Processing level was blocked to prevent selective re-hearsal of shallow-condition items during the study period fordeep-condition items. The order of study conditions was counter-balanced across participants. Testing instructions were either theinclude or the exclude instructions. These were randomly inter-mixed throughout the list. Altogether, there were four conditionsobtained by crossing the two levels-of-processing conditions withthe item instructions. These four conditions were crossed withparticipants in a balanced Latin square design.

Procedure. Procedure was identical to that of Experiment 1.Total time to complete Experiment 2 was approximately 15 min.

Results

Conventional analyses and the hierarchical model analyses wereperformed. We used the hierarchical model with a single source ofautomatic activation (Equations 7–12) instead of the extendedhierarchical model with preexperimental and stimulus-driven au-tomatic activation because the later needs a no-study controlcondition, which was not included in the design. Overall recollec-tion and automatic activation estimates from both aggregationmethods and from the hierarchical process-dissociation model areshown in Table 2.

There is a sizable effect of processing level on recollection anda more modest one on automatic activation for all methods ofanalysis. Item-aggregation estimates seem compromised in thiscase as evidenced by the substantially higher estimate of recollec-tion in the shallow condition than in the other methods. As statedearlier, this high estimate is likely from nonasymptotic bias. Ac-cording to the double-aggregation analysis, the difference in au-tomatic activation across the different processing levels is .037.Unfortunately, there is no way of assessing the statistical signifi-cance of this difference from double aggregation unless one wishesto assume that there are neither item nor participant effects in stemcompletion.

To construct a test of this difference with aggregation, wereanalyzed the data with single aggregation in a dubious manner.Negative values of recollection for individuals were admittedwhere indicated by Equation 3. We do not recommend this practiceand use it simply to compare analyses. The resulting item-aggregation automatic activation averages were .219 and .255 forthe shallow and deep conditions, respectively. A paired t test ofthese estimates revealed no significant difference of study condi-tion on automatic activation, t(42) � .91, p � .1. Hence, if wefollowed the standard practice of doing estimation solely throughEquations 3 and 4, we would have concluded an invariance inautomatic activation.

The hierarchical model analysis yields a higher estimate ofautomatic activation for the deep condition than did aggregation.Moreover, the difference across study conditions is .059, which isabout 50% greater than that from aggregation (.037). Figure 7shows the posterior distributions for overall recollection and au-tomatic activation. The plots in the top row show a substantiallevels-of-processing effect on recollection. The effect on automaticactivation is more subtle. The bottom-left panel shows the poste-rior of the appropriate contrast. The 95% credible interval for this

Table 2Estimates of Overall Recollection and Automatic Activation as aFunction of Study Condition for Experiment 2

Recollection Automatic actvation

Shallow Deep Shallow Deep

Item aggregation .128 .412 .211 .271Double aggregation .068 .398 .241 .278Bayesian equivalent .063 .393 .253 .312

382 ROUDER, LU, MOREY, SUN, AND SPECKMAN

contrast excludes zero, although not by much. There is, therefore,some evidence that automatic activation is greater in the deepcondition than in the shallow condition. This evidence would nothave been available from aggregation analysis; distortions fromaggregation resulted in a downward bias that differentially affectedestimates of automatic activation in the deep condition. This biasmasked the increase in automatic activation with study conditionin exactly the manner described by Curran and Hintzman (1995).

Figure 8 shows participant and item random effects. There islittle evidence for correlation of recollection and automatic acti-vation across participants. Correlation across the individual par-ticipant effects is .081; the estimated population correlation from�� is .007. There is evidence for a sizable correlation, however,across items. Correlation from the item effects is .841; populationcorrelation from � is .590. This is the same pattern seen inExperiment 1.

The items used in Experiment 2 are a subset of those used inExperiment 1. If the model is well specified, then item effects

should be stable across different participants and study conditions.Figure 9 is a scatterplot of item effects for those items that wereused in both experiments. There is a high degree of correlation (.69and .79 for recollection and automatic activation, respectively).Item-effect estimates are surprisingly similar across the two ex-periments even though these experiments entailed different partic-ipants and study instructions. This stability provides convergingevidence that the hierarchical Bayesian model is well specified.

Discussion

Experiment 2 provides for three noteworthy conclusions: First,the pattern of correlations replicated. Items that elicited higherlevels of recollection tended to elicit higher levels of automaticactivation. Across participants, however, recollection and auto-matic activation appeared uncorrelated. Second, there is stabilityamong item estimates. Items that led to good performance inExperiment 1 tended to do so in Experiment 2. Third, the model

−3 −2 −1 0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

Posterior of Grand−Mean of Recollection

Probit Units

Den

sity

ShallowDeep

−3 −2 −1 0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

Probit Units

Den

sity

Posterior of Difference in Recollection

Probit Units

Den

sity

0.0 0.5 1.0 1.5 2.0 2.5 3.0

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Probit Units

Den

sity

0.0 0.5 1.0 1.5 2.0 2.5 3.0

01

23

4

Posterior of Difference in Automatic ActivationPosterior of Grand−Mean of Automatic Activation

Figure 7. The top-left and bottom-left panels show posterior distributions of overall recollection (k(r)) and

automatic activation (k(a)), respectively. The top-right panel shows the contrast between deep and shallow

conditions on recollection. The zero point is outside the 95% credible interval, indicating a significant effect ofstudy duration. The bottom-right panel shows the same for automatic activation. The zero point is just outsidethe 95% credible interval, indicating some evidence of an effect. Densities in left panels are obtained by Gaussiankernel smoothing of model-based outputs.

383HIERARCHICAL PROCESS DISSOCIATION

proved exceptionally useful when compared with aggregationmethods. The hierarchical model shows that there is a levels-of-processing effect on automatic activation; conventional aggrega-tion methods miss this effect due to increases in downward biaswith recollection. Here is a case in which bias from aggregationhas a practical consequence. Therefore, the need to consider biasfrom aggregation depends on the particular application at hand.

The levels-of-processing effect is a failure of selective influ-ence. Bergerbest and Goshen-Gottstein (2002) proposed that thisfailure may result from exceedingly shallow encoding. The vowel-counting task may preclude even automatic encoding of the or-thography of the word. Bergerbest and Goshen-Gottstein ran anadditional experiment in which participants counted syllables in-stead of vowels in the shallow conditions. Although this manipu-lation was hypothesized to increase the likelihood of encoding theorthography of the word, these researchers still found greaterautomatic activation for deep study than for shallow study. It isunclear if the model fails or if the shallow-level manipulationfocuses attention away from encoding the orthography of thewhole word.

Caveat: Correlated Interactions

The hierarchical model is a vast improvement over conventionalmethods. Even so, one important issue remains unresolved. Likeconventional methods, it does not account for interactions betweenparticipants and items. A particular participant may be able torecollect a particular word well because of recent experience.Curran and Hintzman (1995) gave the following hypothetical: Thestem mus__ may be more often completed with music than muscleby most participants. Participants who are physical-therapy ma-jors, however, may reverse the ordering. Interactions of this typeare assumed not to exist in the hierarchical model.

Concern arises if these interactions exist and are correlatedacross recollection and automatic activation. For example, thephysical-therapy major may both better recollect and elicit moreautomatic activation from studying MUSCLE than other partici-pants do relative to other items. As pointed out by Curran andHintzman (1995), ignoring the interactions will lead to asymptoticdownward bias. In the hierarchical model, the lack of an interac-tion term means that correlated interactions will result in a down-

−1.0 −0.5 0.0 0.5 1.0

−1.

0−

0.5

0.0

0.5

1.0

Participant Effects

Effect on Recollection

Effe

ct o

n A

utom

atic

Act

ivat

ion

−1.0 −0.5 0.0 0.5 1.0

−1.

0−

0.5

0.0

0.5

1.0

Item Effects

Effect on Recollection

Effe

ct o

n A

utom

atic

Act

ivat

ion

Posterior of Participant Correlation

Correlation

Den

sity

−1.0 −0.5 0.0 0.5 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Posterior of Item Correlation

Correlation

Den

sity

−1.0 −0.5 0.0 0.5 1.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

Figure 8. Random effects for Experiment 2. Top left: Participant random effects on automatic activation as afunction of those on recollection. Top right: Same plot for item random effects. Bottom: Plot of posterior of thecorrelation coefficient as derived from �� and �.

384 ROUDER, LU, MOREY, SUN, AND SPECKMAN

ward bias in automatic activation parameter (a). Unfortunately,this downward bias will be greater for conditions with higheramounts of recollection, making it difficult to assess selectiveinfluence. The argument for this case is analogous to the previ-ously provided simple example of the effects of selective examplewith easy and hard items.

It would be advantageous to allow arbitrary interaction terms inthe model. Unfortunately, it is simply impossible to include theseterms for most memory designs as these designs yield no repli-cates. Whereas interactions cannot be modeled, the best approachis to assess the consequences of these possible interactions withsimulations, as we do below. These simulations reveal that thepresence of interactions is of no practical consequence for thepreceding analyses.

To assess the impact of correlated interactions, we performed aseries of simulations with interactions perfectly correlated acrossrecollection and automatic activation, which represents a worstcase scenario. In these cases, the additivity assumption in thehierarchical model is misspecified. To simulate data, we used thefollowing equations to generate true recollection and automaticactivation probability parameters:

Rijk � �(�i(r) � j

(r) � k(r) � �ij), (13)

and

Aijk � �(�i(a) � j

(a) � k(a) � �ij). (14)

The equations are the second level of the hierarchical model, withthe addition of an interaction term, �ij. Note that the same value ofinteraction, �ij, enters both equations. Hence, interactions areperfectly positively correlated. We sampled each �ij from a zero-centered normal with variance ��

2 . The size of this variance, ��2 ,

determines the amount of covariance of the interaction and thedegree of downward bias in estimating automatic activation. Wepicked a value for ��

2 � .2, which is 1.25 times as large as theaverage variance of the random effects across both experiments.

The simulations are modeled after the 1-s and 10-s study con-ditions in Experiment 1. True values were the estimates obtained

from the hierarchical model, with the exception that perfectlycorrelated interactions were added and grand means of automaticactivation were set equal across both study conditions (1

(a) �2

(a) � �.394; this value is the average of the estimates fromExperiment 1). For each trial in Experiment 1, values of Rijk andAijk were computed according to Equations 13 and 14. If the trialwas an include trial, the stem-completion probability was p �Rijk � (1 � Rijk)Aijk; if the trial was an exclude trial, p � (1 �Rijk)Aijk. From these probabilities, dichotomous stem-completionevents (successes or failures) were sampled. The resulting data setwas then submitted for hierarchical model estimation with theassumption of additivity. This process of generating data withlarge correlated interactions and analyzing them with the hierar-chical model with additivity was repeated 1,000 times. The criticalquestion is the degree of asymptotic bias in automatic activationestimates.

As expected, there is some downward bias. For the 1-s condi-tion, the bias is �.032 in probit units (�.012 in probability units).For the 10-s condition, the bias increases to �.052 in probit units(�.020 in probability units). The greater bias in the 10-s conditionis expected as downward bias increases with increasing recollec-tion. Comparisons of automatic activation across conditions entailsubtraction, and the biases cancel to a large degree. The amount ofbias in the difference is about �.022 in probit units and �.0078 inprobability units.

Automatic activation estimates in Experiment 1 (see Figure 3,bottom-right panel) may be reexamined in light of this bias. Thebias is negative, indicating that a small positive adjustment may beadded to the posterior. When added, the posterior mean shifts to�.036 from �.058, which strengthens the claim of invariance. Theeffect-size ratio changes to �.100 from �.168. The probabilitythat �, the effect-size ratio, is greater than .2 changes to .036 from.016. Therefore, the possibility of correlated interactions poses nopractical threat to the previous conclusions.

Although interactions pose no practical threat to our data anal-ysis, there are designs in which interactions could potentially be aproblem, especially when there are large differences in recollection

−1.0 −0.5 0.0 0.5 1.0

−1.

0−

0.5

0.0

0.5

1.0

Recollection of Items

Estimates from Experiment 1

Est

imat

es fr

om E

xper

imen

t 2

Correlation: 0.69

−1.0 −0.5 0.0 0.5 1.0

−1.

0−

0.5

0.0

0.5

1.0

Automatic Activation of Items

Estimates from Experiment 1

Est

imat

es fr

om E

xper

imen

t 2

Correlation: 0.79

Figure 9. Stability of item random effects across two experiments. The diagonal denotes the equality ofestimates.

385HIERARCHICAL PROCESS DISSOCIATION

across conditions. The presence of interactions does not threatenthe validity of the rejection of selective influence in Experiment 2;additional downward bias would make the effect larger. Perhapsthe most important point of the exercise is that diligent researchersmay assess the potential effects of correlated interactions throughsimulation.

Conclusions

Process Dissociation

The hierarchical Bayesian model of experimental data yieldsthree lines of support for the process-dissociation model. First, themodel passes a study-duration selective influence test. Increasingstudy duration from 1 s to 10 s affects recollection but not auto-matic activation. Second, the model reveals a striking pattern ofcorrelations. Participants’ recollection and automatic activationabilities are uncorrelated; that is, participants with high recollec-tion abilities tend to be no better at automatic activation. A clearlydifferent pattern emerges for items. Items that are better recol-lected tend to elicit a greater degree of automatic activation. Thefact that the model can dissociate these two opposing patternsprovides informal evidence of support. Third, the model showsstability in item estimates across different participants and encod-ing instructions.

One contribution of the model is that it provides a means ofassessing the detrimental effects of aggregating across participantsand items in process-dissociation experiments. For Experiment 1,where the difference in recollection across conditions was moder-ate, there was no practical harm in aggregation. In Experiment 2,in which there was a large difference in recollection across con-ditions, the effects of aggregation were consequential. It may turnout that in many experiments, there are no practical disadvantagesto aggregation. We emphasize that such knowledge is only acces-sible after hierarchical model analysis and not before it.

Modeling Mnemonic Processes

The dangers of aggregation are well known in several contexts.Even so, the problem has not been articulated in a sufficientlygeneral manner. We cannot understate the following point: Allnonlinear models are subject to distortion when data are aggre-gated. This problem holds not only for process dissociation but formost memory models in the field, including most multinomialprocessing tree models (Batchelder & Riefer, 1999), most strengthmodels (e.g., similarity-choice model; Luce, 1963), and cognitive-skill-acquisition models (Heathcote et al., 2000; Rickard, 2004).Ignoring the problem is tantamount to assuming that there is novariation across items or participants.

We have recently proposed that aggregation may be avoidedentirely by implementing hierarchical models (Rouder & Lu, 2005;Rouder, Lu, et al., 2007). In Rouder, Lu, et al. (2007), for example,we developed a hierarchical version of the theory of signal detec-tion that avoids problems with aggregation. Fortunately, the skillsand techniques encompassed in the modeling generalize wellacross many domains. For example, much of the modeling dis-cussed here is similar to that for our signal detection models. Thehierarchical elements of the models presented here will generalizeto other multinomial and random utility type models. In this vein,

while the focus has been on process dissociation, the modelingendeavor is far more general.

On a final note, we realize that the message that aggregation iscause for concern is not good news. The advocated modelingsolution places burdens on analysts. These hierarchical models arebased on a level of statistics that is beyond the training afforded inmost psychology departments in the United States; however, webelieve the time spent learning the requisite statistics is well worththe investment. There are now several texts on Bayesian modelingwith coverage of hierarchical models (e.g., Congdon, 2006;Gelman, Carlin, Stern, & Rubin, 2004; Lee, 2004). We haveprovided a primer of Bayesian hierarchical modeling for experi-mental psychologists (Rouder & Lu, 2005). Bayesian hierarchicalmodels of mnemonic processes are not implemented in commer-cial software. Nonetheless, they may be programmed in high-levellanguages such as R, Winbugs, and MATLAB.

References

Ashby, F. G., Maddox, W. T., & Lee, W. W. (1994). On the dangers ofaveraging across subjects when using multidimensional scaling or thesimilarity-choice model. Psychological Science, 5, 144–151.

Batchelder, W. H., & Riefer, D. M. (1999). Theoretical and empiricalreview of multinomial process tree modeling. Psychonomic Bulletin &Review, 6, 57–86.

Berger, J. O., & Wolport, R. L. (1988). The likelihood principle (2nd ed.).Hayward, CA: Institute of Mathematical Statistics.

Bergerbest, D., & Goshen-Gottstein, Y. (2002). The origins of levels-of-processing effects in a conceptual test: Evidence for automatic influ-ences of memory from the process-dissociation procedure. Memory &Cognition, 20, 1252–1262.

Brainerd, C. J., Reyna, V. F., & Mojardin, A. H. (1999). Conjoint recog-nition. Psychological Review, 106, 160–179.

Buchner, A., Erdfelder, E., & Vaterodt-Plunneck, B. (1995). Towardunbiased measurement of conscious and unconscious memory processeswithin the process dissociation framework. Journal of ExperimentalPsychology: General, 124, 137–160.

Cave, C., & Squire, L. (1992). Intact and long-lasting repetition priming inamnesia. Journal of Experimental Psychology: Learning, Memory, andCognition, 18, 509–520.

Christensen, R. (1996). Plane answers to complex questions: The theory oflinear models (2nd ed.). Berlin, Germany: Springer-Verlag.

Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique oflanguage statistics in psychological research. Journal of Verbal Learningand Verbal Behavior, 12, 335–359.

Cleveland, W. S. (1981). LOWESS: A program for smoothing scatterplotsby robust locally weighted regression. American Statistician, 35, 54.

Coltheart, M. (1981). The MRC psycholinguistic database. Quarterly Jour-nal of Experimental Psychology: Human Experimental Psychology,33(A), 497–505.

Congdon, P. (2006). Bayesian statistical modelling (2nd ed.). New York:Wiley.

Cowan, N., & Stadler, M. A. (1996). Estimating unconscious processes:Implications of a general class of models. Journal of ExperimentalPsychology: General, 125, 195–200.

Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: Aframework for memory research. Journal of Experimental Psychology:General, 104, 671–684.

Curran, T., & Hintzman, D. L. (1995). Violations of the independenceassumption in process dissociation. Journal of Experimental Psychol-ogy: Learning, Memory, and Cognition, 21, 531–547.

Curran, T., & Hintzman, D. L. (1997). Consequences and causes of

386 ROUDER, LU, MOREY, SUN, AND SPECKMAN

correlations in process dissociation. Journal of Experimental Psychol-ogy: Learning, Memory, and Cognition, 23, 496–504.

Edwards, W., Lindman, H., & Savage, L. J. (1963). Bayesian statisticalinference for psychological research. Psychological Review, 70, 193–242.

Estes, W. K. (1956). The problem of inference from curves based ongrouped data. Psychological Bulletin, 53, 134–140.

Estes, W. K., & Maddox, W. T. (2005). Risks of drawing inferences aboutcognitive processes from model fits to individual versus average perfor-mance. Psychonomic Bulletin & Review, 12, 403–408.

Gelfand, A., & Smith, A. F. M. (1990). Sampling-based approaches tocalculating marginal densities. Journal of the American Statistical As-sociation, 85, 398–409.

Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2004). Bayesiandata analysis (2nd ed.). London: Chapman & Hall.

Gilhooly, K. J., & Logie, R. H. (1980). Age of acquisition, imagery,concreteness, familiarity and ambiguity measures for 1,944 words. Be-havior Research Methods & Instrumentation, 12, 395–427.

Graf, P., & Schacter, D. L. (1985). Implicit and explicit memory for newassociations in normal and amnesic subjects. Journal of ExperimentalPsychology: Learning, Memory, and Cognition, 11, 501–518.

Green, D. M., & Swets, J. A. (1966). Signal detection theory and psycho-physics. New York: Wiley.

Heathcote, A., Brown, S., & Mewhort, D. J. K. (2000). The power lawrepealed: The case for an exponential law of practice. PsychonomicBulletin & Review, 7, 185–207.

Jacoby, L. L. (1991). A process dissociation framework: Separating auto-matic from intentional uses of memory. Journal of Memory and Lan-guage, 30, 513–541.

Jacoby, L. L. (1998). Invariance in automatic influences of memory:Toward a user’s guide for the process-dissociation procedure. Journal ofExperimental Psychology: Learning, Memory, and Cognition, 24, 3–26.

Jacoby, L. L., Begg, I. M., & Toth, J. P. (1997). In defense of functionalindependence: Violations of assumptions underlying the process-dissociation procedure? Journal of Experimental Psychology: Learning,Memory, and Cognition, 23, 484–495.

Jacoby, L. L., Toth, J. P., & Yonelinas, A. P. (1993). Separating consciousand unconscious influences of memory: Measuring recollection. Journalof Experimental Psychology: General, 122, 139–154.

Kahneman, D. (1973). Attention and effort. Englewood Cliffs, NJ:Prentice-Hall.

Kucera, H., & Francis, W. N. (1967). Computational analysis of present-day American English. Providence, RI: Brown University Press.

Lee, P. M. (2004). Bayesian statistics: An introduction (3rd ed.). NewYork: Wiley.

Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental testscores. Reading, MA: Addison-Wesley.

Lu, J., Sun, D., Speckman, P. L., Rouder, J. N., & Morey, R. D. (2007). ABayesian approach for assessing conscious and unconscious processesin human memory. Manuscript submitted for publication.

Luce, R. D. (1959). Individual choice behavior. New York: Wiley.Luce, R. D. (1963). Detection and recognition. In R. D. Luce, R. R. Bush,

& E. Galanter (Eds.), Handbook of mathematical psychology (Vol. 1, pp.103–189). New York: Wiley.

McBride, D. M., & Dosher, B. A. (1999). Forgetting rates are similar in

conscious and automatic memory: A process-dissociation study. Journalof Experimental Psychology: Learning, Memory, and Cognition, 25,583–607.

Neill, W. T., Beck, J., Bottalico, K. S., & Molloy, R. D. (1990). Effects ofintentional versus incidental learning on explicit and implicit memorytests. Journal of Experimental Psychology: Learning, Memory, andCognition, 16, 457–463.

Nelder, J. A., & Mead, R. (1965). A simplex method for function mini-mization. Computer Journal, 7, 308–313.

Newell, A., & Rosenbloom, P. S. (1981). Mechanisms of skill acquisitionand the law of practice. In J. R. Anderson (Ed.), Cognitive skills andtheir acquisition (pp. 1–55). Hillsdale, NJ: Erlbaum.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. New York:McGraw-Hill.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models:Applications and data analysis methods (2nd ed.). Thousand Oaks, CA:Sage.

Rickard, T. C. (2004). Strategy execution in cognitive skill learning: Anitem-level test of candidate models. Journal of Experimental Psychol-ogy: Learning, Memory, and Cognition, 30, 65–82.

Rouder, J. N. (2004). Modeling the effects of choice-set size on theprocessing of letters and words. Psychological Review, 111, 80–93.

Rouder, J. N., & Lu, J. (2005). An introduction to Bayesian hierarchicalmodels with an application in the theory of signal detection. Psy-chonomic Bulletin & Review, 12, 573–604.

Rouder, J. N., Lu, J., Speckman, P. L., Sun, D., & Jiang, Y. (2005). Ahierarchical model for estimating response time distributions. Psy-chonomic Bulletin & Review, 12, 195–223.

Rouder, J. N., Lu, J., Sun, D., Speckman, P. L., Morey, R. D., & Naveh-Benjamin, M. (2007). Signal detection models with random participantand item effects. Psychometrika, 72(4). Advance online publication.doi:10.1007/s11336-005-1350-6

Rouder, J. N., Morey, R. D., Speckman, P. L., & Pratte, M. S. (2007).Detecting chance: A solution to the null sensitivity problem in sublim-inal priming. Psychonomic Bulletin & Review, 14, 597–605.

Schacter, D. L. (1992). Understanding implicit memory: A cognitiveneuroscience approach. American Psychologist, 47, 559–569.

Schacter, D. L., & Tulving, E. (1994). What are the memory systems of1994? In D. L. Schacter & E. Tulving (Eds.), Memory systems 1994 (pp.1–38). Cambridge, MA: MIT Press.

Tanner, M. A. (1998). Tools for statistical inference: Methods for theexploration of posterior distributions and likelihood functions. NewYork: Springer.

Toglia, M. P., & Battig, W. R. (1978). Handbook of semantic word norms.New York: Erlbaum.

Toth, J., Reingold, E. M., & Jacoby, L. L. (1994). Toward a redefinition ofimplicit memory: Process dissociations following elaborative processingand self-generation. Journal of Experimental Psychology: Learning,Memory, and Cognition, 20, 290–303.

Tulving, E., & Schacter, D. L. (1990, January 19). Priming and humanmemory systems. Science, 247, 301–306.

Venables, W. N., & Ripley, B. D. (2003). Modern applied statistics with S.New York: Springer.

Wagenmakers, E. M. (in press). A practical solution to the pervasiveproblem of p-values. Psychonomic Bulletin & Review.

(Appendix follows)

387HIERARCHICAL PROCESS DISSOCIATION

Appendix

Analysis of the Hierarchical Process-Dissociation Model

We provide a complete specification of the priors and an over-view of the method of analysis. A more detailed analysis of andjustification for these priors may be found in Lu et al. (2007). Luet al. also provided alternative analyses of process dissociationwith other priors.

Specification of the Prior

Priors are needed for k(r), k

(r), ��, and �. Priors on grandmeans are

k(r) � Normal(0, �

2 ), k � 1, . . . , K, (A1)

and

k(a) � Normal(0, �

2 ), k � 1, . . . , K. (A2)

The value of �2 must be chosen beforehand. Any large values will

lead to an appropriately diffuse prior. We use �2 � 1,000 in

application.Priors on covariance matrices are

�� � Inverse Wishart�m, �) (A3)

and

� � Inverse Wishart�m, �). (A4)

The density of the inverse Wishart distribution is

f�S�m,�� ����m/ 2�S���m�3�/ 2

2m�2�m/ 2�exp�� 1

2tr��S�1�� , (A5)

where S is a positive definite matrix and �p is the multivariategamma function given by �p(a) � �p( p � 1)/4�p

j � 1�(a � 1⁄2(j �1)) (see Gelman et al., 2004, p. 574).

The Wishart prior is perhaps the most common choice forcovariance matrices. It is conjugate for the covariance matrix of amultivariate normal distribution, and this simplifies analysis. De-tails of the inverse Wishart and its use as a prior for covariancematrices can be found in standard texts on statistics (e.g., Tanner,1998). The inverse Wishart has two parameters, m and �, which

0 1000 3000 5000

−1.

0−

0.8

−0.

6−

0.4

−0.

20.

0

Grand Mean

Iteration

Est

imat

e

0 5 10 20 30

0.0

0.2

0.4

0.6

0.8

1.0

Lag

Aut

ocor

rela

tion

0 1000 3000 5000

−2.

0−

1.0

0.0

1.0

Item Effect

Iteration

Est

imat

e

0 5 10 20 30

0.0

0.2

0.4

0.6

0.8

1.0

Lag

Aut

ocor

rela

tion

0 1000 3000 5000

−6

−4

−2

02

46

Participant Precision

Iteration

Est

imat

e

0 5 10 20 30

0.0

0.2

0.4

0.6

0.8

1.0

Lag

Aut

ocor

rela

tion

Figure A1. Convergence in Markov chain Monte Carlo sampling. Top row: Values on each iteration forselected parameters (from left to right: grand mean (a) for the deep condition, item effect 11

(r), and theoff-diagonal term of the participant precision matrix ��

�1). Bottom row: Autocorrelation of the chains for theselected parameters. The modest degree of autocorrelation is offset by the long length of the chains.

388 ROUDER, LU, MOREY, SUN, AND SPECKMAN

must be set beforehand. The choice m � 2 leads to the leastinformative prior. Parameter � is a 2 � 2 matrix. An appropriatechoice in this context is to set � to the identity matrix. The inverseWishart prior is informative; that is, the prior does provide infor-mation about the scale of random effects (Lu et al., 2007; Rouder,Lu, et al., 2007). To ensure the choice of � did not overly biasestimates, we repeated the analyses with the alternatives � � .2Iand � � 5I. Results for these choices were highly similar to thereported ones for � � I.

Analysis

The target of analysis in Bayesian statistics is the derivation ofthe marginal posterior distribution for each parameter. Often, thesemarginal posteriors cannot be derived as closed-form expressions.We follow the common approach of deriving closed-form expres-sions for full-conditional posterior distributions and samplingthese with Markov chain Monte Carlo (MCMC) techniques (Gel-fand & Smith, 1990). The derivation of full-conditional posteriordistributions for this application may be found in Lu et al. (2007).These full-conditional posteriors are inverse Wishart and multi-

variate normal, which may be sampled directly without recourse tothe more computationally intensive Metropolis-Hastings algo-rithm. Details are found in Lu et al.

We sampled chains of 6,000 iterations, with the first 1,000iterations serving as a burn-in period. Convergence was rapid, andchains exhibited only modest autocorrelation. The top panels ofFigure A1 show chains for select parameters for Experiment 2. Thebottom panels show the degree of autocorrelation in the chains forthese parameters. As can be seen, mixing is sufficient.

All analyses were done in the R statistical language (http://cran.r-project.org/). Sampling from the inverse Wishart andmultivariate normal was accomplished with the MCMCpack(MCMCpack.wustl.edu) and MASS package (Venables & Ripley,2003), respectively. A full computer implementation of the anal-ysis may be found at http://web.missouri.edu/�umcaspsychpcl/code.

Received March 16, 2007Revision received August 21, 2007

Accepted August 23, 2007 �

389HIERARCHICAL PROCESS DISSOCIATION