Multilevel Modeling in Psychosomatic Medicine Research

Multilevel Modeling in Psychosomatic Medicine ResearchNICHOLAS D. MYERS, PHD, AHNALEE M. BRINCKS, PHD, ALLISON J. AMES, MS, GUILLERMO J. PRADO, PHD, FRANK J. PENEDO, PHD,AND CATHERINE BENEDICT, MA

The primary purpose of this study is to provide an overview of multilevel modeling for Psychosomatic Medicine readers andcontributors. The article begins with a general introduction to multilevel modeling. Multilevel regression modeling at two levels isemphasized because of its prevalence in psychosomatic medicine research. Simulated data sets based on some core ideas from theFamilias Unidas effectiveness study are used to illustrate key concepts including communication of model specification, parameterinterpretation, sample size and power, and missing data. Input and key output files from Mplus and SAS are provided. A clusterrandomized trial with repeated measures (i.e., three-level regression model) is then briefly presented with simulated data based on somecore ideas from a cognitive-behavioral stress management intervention in prostate cancer. Key words: mixed regression models,random coefficient models, hierarchical linear models, centering, power, missing data.

MLM(s) = multilevel model(s); MLRM(s) = multilevel regressionmodel(s); MLSMs = multilevel structural equation model(s); RAS =raw score; GMC = grand mean centering; CWC = centering withincluster; RSB = risky sexual behavior; PA = perceived peer abstinence;SF = adolescent functioning in school; GAP = gap between parentaland adolescents Americanism; FAM = family functioning; ICC =intraclass correlation coefficient; ML = maximum likelihood; CBSM= cognitive-behavioral stress management; A = time; W = treatmentcondition; MCAR = missing completely at random; MAR = missingat random.

INTRODUCTION

Multilevel data, sometimes referred to as hierarchical data,are common in psychosomatic research. A common hier-

archical structure has repeated observations nested within par-ticipants (1,2). Time-varying data from the repeated observationsconstitute level 1 variables, whereas time-invariant data fromthe individual participants constitute the level 2 variables. Amultilevel model (MLM) can be defined as a statistical modelthat has two or more distinct hierarchical levels, with variablesat each of these levels, and a theoretical interest in relation-ships at one or more of, or across, these levels (3).

Cross-sectional data can also be viewed as multilevel. For ex-ample, data may be collected from patients who are receivingtreatment for the same illness within different hospitals, or withrespect to behavioral interventions, patients may receive apsychological treatment from different therapists. Suppose thatthe relevant population of hospitals (or therapists) was viewedas large and that some of the randomly sampled hospitals wereassigned to an experimental treatment condition, whereas other

hospitals were assigned to the control condition (e.g., treatmentas usual). From a conceptual perspective, variability in patientoutcomes may be attributable to both patient-level characteris-tics (e.g., severity of condition) and hospital-level characteristics(e.g., experimental treatment versus control condition). An MLMenables researchers to partition variability in the outcome(s) bylevel (e.g., patient and hospital levels) and allows for modelingwithin and across these levels simultaneously.

From a statistical perspective, a key concern is to whatdegree the assumption of independence of observations is vi-olated. If the outcome measure for a patient who receivestreatment at hospital A is more similar to the outcome measuresfor other patients at hospital A than to the outcomes for a pa-tient at hospital B, the independence assumption is violated.When this nonindependence is nontrivial and is attributable toat least one cluster variable (e.g., the hospital; the therapist),imposing a fixed-effects model that ignores the nonindepen-dence is often problematic from both an empirical perspective(e.g., increased type I error) and a theoretical perspective (e.g.,inability to model relationships at and across different levels).

A review of Psychosomatic Medicine from 2006 to 2010revealed that 33 studies used an MLM (see Supplemental Dig-ital Content 1, http://links.lww.com/PSYMED/A52). There is aconsiderable variability within this set of studies and acrossapplications of MLMs in other disciplines (4), with regardto some core methodological issues: communication of modelspecification, parameter interpretation, sample size and power,and missing data. Modern approaches to address these coremethodological issues are illustrated with example data sets inthis study (i.e., example 1: Familias Unidas; example 2: stressmanagement). Before these illustrations, we provide a briefgeneral introduction to MLMs.

MLMS: A BRIEF INTRODUCTIONResearchers may recognize MLMs from multiple labels in-

cluding mixed models, random coefficient models, covariancecomponents models, and hierarchical linear and nonlinear mo-dels. We adopt the nomenclature of Hox and Roberts (3) andclassify MLMs into two groups: multilevel regression models(MLRMs) and multilevel structural equation models (MLSMs).MLRMs have an observed outcome or outcomes at level 1, re-gression coefficients as outcomes at level 2 and above, andgenerally observed independent variables. MLSMs have latent

Psychosomatic Medicine 74:925Y936 (2012) 9250033-3174/12/7409Y0925Copyright * 2012 by the American Psychosomatic Society

S T A T I S T I C S A N D M E T H O D S

From the Departments of Educational and Psychological Studies (N.D.M.,A.J.A.), Epidemiology and Public Health (A.M.B., G.J.P.), Psychology (F.J.P.,C.B.), University of Miami, Coral Gables, Florida.

Address correspondence and reprint requests to Nicholas D. Myers, PhD, De-partment of Educational and Psychological Studies, University of Miami, MerrickBldg 311E, Coral Gables, FL 33124-2040. E-mail: [email protected]

The Familias Unidas effectiveness study is supported by National Instituteon Drug Abuse Grant No. 1R01DA025192-01A1 andNational Institute onAlcoholAbuse and Alcoholism Grant No. 3R01DA025192-01A1S1 (awarded to GuillermoJ. Prado). The Cognitive Behavioral Stress Management in Prostate Cancer studywas supported by National Cancer Institute Grant No. 1P50CA84944 (awarded toMike Antoni).

Supplemental digital content is available for this article. Direct URL citationsappear in the printed text and are provided in the HTML and PDF versions ofthis article on the journal’s Web site (www.psychosomaticmedicine.org).

Received for publication September 28, 2011; revision received July 18, 2012.DOI: 10.1097/PSY.0b013e3182736971

Copyright © 2012 by the American Psychosomatic Society. Unauthorized reproduction of this article is prohibited.

http://links.lww.com/PSYMED/A52

or observed outcomes at each level, latent or observed indepen-dent variables, and greater flexibility thanMLRMswith regard tothe path models that may be imposed. There are types of MLMs,however, that do not fit neatly into either category (e.g., multilevelsurvival analysis).

Multilevel substantive theories predated the ability to esti-mate what are currently considered MLMs (5Y7). MLMs wereconceptualized only a few decades ago and initially based onMLRMs (8,9). Initial algorithms that were implemented forthe estimation of these models followed (10Y13). In the late1980s and early 1990s, several MLM texts were presented(14Y16). In each of these texts, a primary focus was given tocross-sectional and/or longitudinal two-level linear regressionmodels. Application of this type of MLM is most commonwithin Psychosomatic Medicine (see Supplemental Digital Con-tent 1, http://links.lww.com/PSYMED/A52). Therefore, this studyprimarily focuses on this particular implementation of an MLRMunless otherwise noted.

It is worthwhile to provide a conceptual summary of an earlyforerunner of the MLRM that was alluded to in the previousparagraph: regression coefficients as outcomes. Under such anapproach, a linear regression model was fitted to level 1 (e.g.,patient) data separately within each level 2 unit (e.g., hospital).The regression coefficients (i.e., intercept and slope(s)) werethen saved in a ‘‘level 2 data set’’ where level 2 predictors werethen specified (i.e., addressing a ‘‘unit of analysis’’ problem).Problems with such an approach included suboptimal estima-tion of regression coefficients within individual level 2 units,forcing variability in the regression coefficients into a restric-tive and often unrealistic covariance structure, and the approachbeing inefficient (conceptually and empirically) and generallyimprecise (particularly in regard to accurate estimation of stan-dard errors). The ideas, however, that some level 1 predictiverelationships should be free to randomly vary across level 2 unitsand that some of this variance should be explained by level 2characteristics both remain central to the present day MLRM.

Deciding how to specify level 1 coefficients at level 2 is akey consideration in MLRMs.1 Specifying a level 1 coefficientas random at level 2 implies that the magnitude of the coeffi-cient truly varies (beyond sampling error and that which can beaccounted for in the structural model) across the relevant pop-ulation of level 2 units (e.g., mean risky sexual behavior [RSB]of adolescents truly varies across a specified population ofschools).2 Specifying a level 1 coefficient as nonrandomly vary-ing at level 2 implies that the magnitude of the coefficient hasa finite number of true values that can be adequately handledin the structural part of the model, thereby precluding the needfor an accompanying random effect for that coefficient (e.g., themagnitude of the true within-school effect of adolescent func-tioning in school (SF) on RSB of adolescents depends on meanfamily functioning [FAM] in each school). Specifying a level 1

coefficient as fixed at level 2 implies that the magnitude of thecoefficient has only one true value. Treating all level 1 coeffi-cients initially as random and then working backward based ononly statistical information from a particular nonlarge samplewhile conceptually alluring has known problems with regard toaccurately recovering a true model (19). Therefore, deciding howto specify level 1 coefficients at level 2 is a key decision thatshould be guided by theoretical (e.g., primary versus secondarypredictors) and empirical information. The relevant decisionsmade within the subsequent examples of the current study weremade only to illustrate modeling possibilities.

MLM Notation and SoftwareAs with any quantitative methodology with more than

one set of contributors, notation, terminology, and software forMLM vary. Notation and terminology used in this study aremost consistent with Raudenbush and Bryk (19), which isconsistent with the studies listed in Supplemental DigitalContent 1, http://links.lww.com/PSYMED/A52. The softwareused for demonstration is Mplus (20) and SAS (21), but manyother software options are available (22).

Consider the notation for independent variables in MLMs.Independent variables at level 1 are denoted Xqij, where q iden-tifies the variable (e.g., X1ij, X2ij,I, Xkij). The subscript idenotes a particular observation at level 1. The subscript j de-notes a particular cluster at level 2. In the repeated-measuresexample, X123 identifies variable X1 measured at time 2 forperson 3. In a cross-sectional example, X123 identifies variableX1 measured for person 2 within hospital 3. Independent vari-ables at level 2 are often denoted in one of two ways. Whena variable is measured at level 2, it is denoted Wqsj, where qcommunicates which level 1 coefficient, Aqj, a given variable,was specified to predict and s uniquely identifies the variable(e.g., Wq1j, Wq2j, etc). In a cross-sectional example, W03j (orW21j as another example) identifies A0j (or A2j) as the relevantoutcome, the third predictor of A0j (or the first predictor of A2j),for the jth hospital. Note that W03j and W21j could actuallydenote the same hospital-level predictor variable. When a var-iable is measured at level 1 and then aggregated at level 2, it isdenoted asX

;qsqj to make clear that observations of the variable

were aggregated across i individuals within each j group (oracross i time points within each j person). The potential im-portance of including the means of level 1 predictors at level 2within relevant medical statistical models (e.g., modeling ofecological momentary assessment data) has been discussed forsome time (23). Fixed effects at level 2, Fqs, adopt the previ-ously defined subscript notation. Subscripts are dropped in thisstudy when their function is redundant.

Centering Predictors in MLMSCentering predictors is an important aspect of MLMs that

has long been misunderstood (24). Manifestations of such

1It is possible to specify all effects as fixed effects and then use boot-strapping for clustered data to get accurate standard error estimates. Estimatesof the variance components of the random effects could also be found withbootstrapping (17).

2Another view of random effects could be framed within the Neyman-Scottproblem (18), where the unit effects may be construed as a nuisance to beaddressed via the inclusion of random effects to increase the likelihood ofobserving consistent fixed effects.

N. D. MYERS et al.

926 Psychosomatic Medicine 74:925Y936 (2012)





misunderstanding include omission of centering decisions inpublished studies (rendering regression coefficients impossible tointerpret) and basing centering decisions on only empirical infor-mation (e.g., trying various centering options in search of statisticalsignificance). Centering level 1 predictors should be informed bysubstantive multilevel theory, well-defined research questions, andempirical information. Although a full review of centering level 1predictors is beyond the scope of this study, thorough reviews areavailable elsewhere (25,26).

Parallel in time to the publication of initial texts on MLMs,a literature emerged on the consequences of centering level 1predictors within MLMs (27Y31). Primary options debated werethe use of raw scores (RASs; no centering), grand mean cen-tering (GMC; subtracting the mean across all level 1 observationsfrom each observation), and centering within cluster (CWC;subtracting the group mean within each cluster from each ob-servation within the relevant cluster). Within this study, GMCwill be communicated as follows: Xqij �X

;q:: at level 1 and

X;qsqjjX

;;

qs:: at level 2, where a double bar indicates the meanof the group means. CWC will be denoted XqijjX

;q:j; CWC

does not apply at level 2 because there are no upper-level unitsfor these observations to be nested within (i.e., level 3).

Kreft et al. (32) mathematically defined conditions underwhich imposing different centering options (RAS, GMC, andCWC) for level 1 predictors in two common specifications ofMLMs resulted in theoretically equivalent models. The twomodels of primary focus were a) random coefficients regressionand b) intercepts and slopes as outcomesVboth of these modelsare introduced in the subsequent section. Within both models,RAS and GMC are equivalent to each other and nonequivalentto CWC. That two parameterizations of an MLM, which differonly by centering decision(s), are statistically equivalent doesnot necessarily mean that both models do an equally good jobof representing specific research questions and/or theory-drivenhypothesesVa point that will be made clear in the FamiliasUnidas example. Before proceeding to the Familias Unidas ex-ample, it is worth noting that centering (as described earlier)within an MLM can also be conceptualized within a broaderframework as one way to adjust the estimated within-clustereffect in the presence of confounding by cluster (33Y35).3

EXAMPLE 1: FAMILIAS UNIDASThe HIV epidemic constitutes a major public health con-

cern in the United States and around the globe. Although HIV

has affected persons of all ages, from 2005 to 2008, the ratesof HIV infection for those 15 to 19 years of age have increased(36). The prevalence of unprotected sexual intercourse amongadolescents is one of the factors responsible for the high inci-dence of new HIV cases. Results from The CDC’s Youth RiskBehavior Surveillance Survey indicate that 46% of high schoolstudents reported having had sexual intercourse and 13.8%reported four or more lifetime sexual partners (37). Althoughthese national statistics are alarming, what may be even moredisconcerting is that Hispanics, the largest and fastest growingminority group in the United States (37), seem to be at height-ened risk for engaging in HIV risk behaviors including unpro-tected sexual behavior and, consequently, at disproportionaterisk of HIV infection (38).

Ecodevelopmental theory is a conceptual framework that hasbeen put forth to explain risk and protective processes linked toHIV risk behaviors in Hispanic youth (39Y41). RSB is a pri-mary health outcome within an ecodevelopmental model pre-dicting HIV risk behaviors. Theory-based predictors of RSBinclude perceived peer abstinence (PA), SF, parental and ado-lescent Americanism gap (GAP), and FAM.

Ecodevelopmental theory data are often collected withinmiddle schools and/or high schools, where only one adolescentper family is sampled. Sampling through schools provides re-searchers with access to a large number of adolescents but alsointroduces the possibility that the data collected may be non-independent. From a statistical perspective, RSB is measured atlevel 1 (i.e., adolescent), but these data may be nonindependentbased on the school (i.e., level 2) that an adolescent is nestedwithin. From a conceptual perspective, RSB may be partiallyexplained by independent variables at level 1 (e.g., FAM), level2 (e.g., average GAP Americanism at the school level), andcross-level interactions (e.g., FAM * average GAP American-ism at the school level).

The first example data set was simulated based on some coreideas from an ongoing randomized trial evaluating the relativeeffectiveness of Familias Unidas (42), where RSB (mean [M][standard deviation {SD}] 0.04 [1.06] minimum = j4.16, max-imum = 5.13) was a composite variable that was continuous andnormally distributed. Familias Unidas is a Hispanic-specific,family-based preventive intervention designed to reduce risksand increase protection for substance use and unprotected sex-ual behavior in Hispanic adolescents. Familias Unidas is guidedby ecodevelopmental theory and aims to prevent substance useand unprotected sexual behavior by increasing positive parent-ing, family support of the adolescent, and parental involvementand by improving general parent-adolescent communication andparent-adolescent communication specific to substance use, un-safe sexual behavior, and HIV. In the Familias Unidas effective-ness study, a total of 744 Hispanic youth and their parents arebeing recruited from 24 middle schools in Miami-Dade County.Participants are assessed at baseline, randomized to either Fam-ilias Unidas or a community practice control condition, and re-assessed at 6, 18, and 30 months postbaseline. In a preliminarybaseline data set, adolescents (N = 460) were nested withinmiddle schools (J = 16) and within-cluster size was unbalanced

3The cited studies (33Y35) provide a thorough account of when the estimatedwithin-cluster effect is likely to be accurately adjusted in the presence of con-founding by cluster under various models (e.g., conditional linear/logistic re-gression, general(ized) linear mixed models) and link functions (e.g., identity,logit, and log-log). The approach taken in the current study (e.g., Eq. 6) has beenshown to produce consistent and efficient estimates of the within-cluster effectwhen the fitted model is correctly specified (33,34). Although the approach takenin the current study is both practical and useful (particularly for the type ofexamples reviewed in this study), it also has been shown to be a special case of amore general approach (35).

MULTILEVEL MODELING

Psychosomatic Medicine 74:925Y936 (2012) 927



ðnj; ¼ 28:75Þ, with the smallest school having 22 adolescentsand the largest school having 32 adolescents.

A larger data set using some characteristics from the Fam-ilias Unidas data set was created because multilevel modelinggenerally is a large sample technique (43), a point that will beelaborated upon in the section on sample size and power. In thesimulated data set (i.e., data set_1), adolescents (N = 2869)were nested within schools (J = 100) and within-cluster sizewas unbalanced ðnj; ¼ 28:69Þ, with the smallest cluster having22 observations and the largest cluster having 32 observations.Because data set_1 through data set_3 were simulated, the resultsthat follow should be viewed as only a means for providing arelevant context for understanding multilevel modeling.

MLRM AND FAMILIAS UNIDASThe Familias UnidasYsimulated data set is used to illustrate

key concepts in an MLRM including the following: communi-cation of model specification, parameter interpretation, samplesize and power, and missing data. Model specification followsthe general strategy outlined in Raudenbush and Bryk (19)where MLRMs are introduced by way of six ‘‘submodels.’’ Inpractice, a single MLM may be fitted when sufficient a prioritheory exists, but proceeding in this incremental way is con-

sistent with the purpose of this study. For spatial reasons,however, most of these submodels (i.e., model 2: means as out-comes regression model; model 3: one-way analysis of covari-ance with random effects; model 4: nonrandomly varying slopes;model 5: random coefficients regression) are elaborated upon inSupplemental Digital Content 2 only (along with Eqs. 2Y5),http://links.lww.com/PSYMED/A53. The first four models arerandom intercept models. The final two models are ran-dom intercepts and slopes models. The > for all hypothesis testswas set equal to .05.

Communication of model specification is provided in an equa-tion form and also via a path diagram. Each equation is writtenin an uncombined form with variable acronyms (e.g., RSBij) andthen in a combined form with general notation (e.g., Yij) in Sup-plemental Digital Content 2, http://links.lww.com/PSYMED/A53.The uncombined equation is a way to emphasize the meaningof the parameters by writing equations at each level. The com-bined equation is an equivalent, compact expression that col-lapses the uncombined equation across levels. An interpretationof only key terms (to reduce redundancy) follows each equation.The causal language used (e.g., effect) communicates the di-rection of the relationship assumed by the model specification.In practice, any causal inference would also need to account fordesign considerations and substantive theory in addition to the

TABLE 1. Familias Unidas: Parameter Estimates (Standard Errors), Observed Power, and Model-Data Fit

Parameter Submodels Fitted to Data set_1 Power

Fixed effects Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 J = 25 J = 100

F00 0.040 (0.033) 0.039 (0.031) 0.039 (0.032) 0.030 (0.036) 0.040 (0.033) 0.039 (0.031) V V

F01 0.020 (0.031) 0.023 (0.032) 0.016 (0.034) 0.025 (0.031) V V

F02 0.030 (0.042) 0.037 (0.044) 0.025 (0.046) 0.037 (0.041) V V

F03 0.036* (0.014) 0.035* (0.014) 0.036* (0.015) 0.037* (0.014) 33.4% 79.7%

F04 j0.007 (0.047) j0.010 (0.047) j0.005 (0.048) j0.009 (0.048) V V

F10 j0.017*** (0.003) j0.016*** (0.003) j0.015*** (0.003) j0.015*** (0.003) 80.8% 100.0%

F20 j0.009** (0.003) j0.008* (0.003) j0.008* (0.003) j0.007* (0.003) V V

F24 j0.011*** (0.002) j0.009*** (0.002) 97.5% 100.0%

F30 0.011*** (0.002) 0.010*** (0.002) 0.010*** (0.002) 0.010*** (0.002) 95.1% 100.0%

F40 j0.034* (0.016) j0.033* (0.016) j0.026 (0.014) j0.027* (0.013) V V

F43 0.017*** (0.004) 40.6% 90.4%

Variance components

R2 1.058*** (0.053)1.058*** (0.053) 1.012*** (0.048) 0.961*** (0.047) 0.747*** (0.019) 0.711*** (0.019) 100.0% 100.0%

T00 0.072*** (0.013)0.061*** (0.011) 0.065*** (0.012) 0.070*** (0.013) 0.083*** (0.013) 0.073*** (0.011) 53.6% 100.0%

T44 0.018*** (0.003) 0.015*** (0.002) 96.9% 100.0%

T04 0.001 (0.005) j0.005 (0.004) V V

Model-data fit

Deviance 8412 8401 8280 8185 7679 7567

# par 3 7 11 12 9 15

AIC 8418 8415 8302 8209 7697 7597

BIC 8436 8457 8368 8281 7751 7686

# par = number of parameters estimated; AIC = Akaike information criterion; BIC = Bayesian information criterion.*p G .05.** p G .01.*** p G .001.

N. D. MYERS et al.






model specification. A path diagram of each model specifica-tion (44) is also provided in Supplemental Digital Content 2,http://links.lww.com/PSYMED/A53 (see Figs. 1Y6). Providingpath diagrams in addition to equations is done to provide futureresearchers with options for how to succinctly and clearlycommunicate model specification.

Model 1: One-Way Analysis of Variance With RandomEffects4

A primary purpose ofmodel 1 is to partition the unconditionalvariance in the outcome(s) by level to assess the degree of non-independence of observations. The degree of nonindependence istypically assessed by the intraclass correlation coefficient (ICC):ICC = T00/(T00 + R

2). Simulation research has shown potentialconsequences (e.g., inflated type I) of ignoring the nonindepen-dence (e.g., imposing a fixed-effects model) when the ICC is assmall as 0.05 (45).

This model is fully unconditional in that there are no pre-dictors at either level.5 This model can be written as follows:

RSBij ¼ A0j þ rijA0j ¼ F00 þ u0j

ð1Þ

RSBij observed RSB for the ith adolescent in the jth schoolA0j = mean RSB in the jth schoolF00 = grand mean RSB across schoolsrij = unique effect of the ith adolescent on RSB in the jth schoolu0j = unique effect of the jth school on RSBR2 = Var(rij) = unconditional adolescent-level variance in RSBT00 = Var(u0j) = unconditional school-level variance in RSB

Table 1 provides parameter estimates, standard errors, andmodel-data fit information for this model and all subsequentmodels. Supplemental Digital Content 3 provides inputand key output for this model and all subsequent models,http://links.lww.com/PSYMED/A54.6 The ICC was equal to0.064, which implied that 6.4% of the variance in RSB was at-tributable to school-level differences. The estimated grand meanRSB across schools, F00, was equal to 0.040. The unconditionalschool-level variance estimate, T00, around this average was equalto 0.072 and was statistically significant, p G .001.7 Table 2 pro-vides descriptive statistics at level 1 (i.e., ‘‘within’’ or adolescentlevel) and level 2 (i.e., ‘‘between’’ or school level) due to evidenceof nonindependence by school. This information can be used todescribe the bivariate associations that exist at each level.

Model 6: Intercepts and Slopes as OutcomesSuppose that the interest is in modeling adolescent- and

school-level effects on RSB while specifying random interceptsand a mix of fixed, nonrandomly varying, and random slopes.Level 1 predictors are specified to address research questions based

on ecodevelopmental theory (e.g., what is the effect of perceivedPA on RSB at the adolescent level?). Level 2 predictors of A0j andA2j are specified to address research questions based on ecodeve-lopmental theory (e.g., what is the effect of mean perceived PA onmean RSB at the school level?). For a fuller explanation of thewider array of research questions, see models 2 to 5 in Supple-mental Digital Content 2, http://links.lww.com/PSYMED/A53.Suppose further that there is reason to believe (1) that the magni-tude of the within-school effect of FAM on RSB depends, in part,on the level of mean GAP Americanism (i.e., a cross-levelstatistical interaction) and (2) that there will be substantialresidual variance in the set of A4j unaccounted for. This modelcan be written as follows:

RSBij ¼ A0j þ A1jðPAij j PA:;

jÞ þ A2jðSFij j SF:;

jÞþA3jðGAPij j GAP:

;

jÞ þ A4jðFAMij j FAM :;

jÞ þ rij

A0j ¼ F00 þ F01ðPA:;

j j PA::;;Þ þ F02ðSF:

;

j � SF::;;Þ

þ F03ðGAP:;

j �GAP::;;

Þ þ F04ðFAM :;

j � FAM ::;;

Þ þ u0jA1j ¼ F10

A2j ¼ F20 þ F24ðFAM :;

j � FAM ::;;

ÞA3j ¼ F30

A4j ¼ F40 þ F43ðGAP:;

j � GAP::;;

Þ þ u4j ð6Þ

u4j = residual within-school effect of RSB regressed onFAM (after controlling for the effect of PA, SF, and GAPAmericanism at level 1) after controlling for the effect of meanGAPAmericanism8

t44 = variance of the U4j

t04 = school-level covariance between the u0j and the u4j

As can be readily viewed in the combined form ofEq.6 (see Fig. 6 in Supplemental Digital Content 2,http://links.lww.com/PSYMED/A53), F24 represents a cross-

level interaction: F24

�ðX2ijjX

;2::Þ*ðX

;4:jjX

;;

4::Þ�. One way

to interpret this interaction is that the magnitude of the effect ofSF on RSB depends on the level of mean FAM. The statisticalsignificance of this cross-level interaction, F24 = j0.009,p G .001 (see Table 1), supported rejection of the null hy-pothesis that the magnitude of the effect of SF on RSB didnot depend on level of mean FAM: H0: F24 = 0.

A question unanswered by testing H0: F24 = 0 was as fol-lows: at what values of mean FAM (M [SD] = 0.00 [1.13],minimum = j2.13, maximum = 3.45) was the effect of SF(minimum = j42.19, maximum = 38.11) on RSB statisticallysignificant? More formally, what were the regions of significance

6Results reported in the text of this manuscript are from Mplus output. Re-gardless of statistical program used to fit a particular MLM, individual Level-1coefficients are not model parameters per se. Estimating reasonably accurateLevel-1 coefficients can be a complex task (often accomplished with a Bayesianframework) and is an area of research (51).

7Testing the statistical significance of the variance of a level 2 random effect,uqj, is a somewhat controversial issue in MLM. The problem is occurs at theboundary of the parameter space because the true value of a variance cannot benegative. Corrections to instances where such a hypothesis test is imposed havebeen offered (e.g., 52).

4This model also can also be referred to as a nested 1-way analysis of vari-ance with the level s units treated as a random factor.

5Non-multilevel modeling options for such data include treating the clustervariable as a fixed factor and thereby limiting inferences to the given sample ofcluster units; and, modeling only at Level-1 while applying a statistical cor-rection for the non-independence by cluster (46Y48).

MULTILEVEL MODELING







http://http://links.lww.com/PSYMED/A53

of mean FAM for the simple slope of RSB on the effect of SF(53,54)? Figure 1 plots the simple slopes of RSB on the effect ofSF on a few values of mean FAM:jSD, M, +SD. The slope waspositive when mean FAM was low, jSD, but the slope becameincreasingly negative as mean FAM increased.

Figure 2 identifies the regions of significance of mean FAMwhere the effect of SF on RSB was statistically significant andnonsignificant. When mean FAMwas ‘‘low’’ (i.e., ej1.37), theeffect of SF on RSB was statistically significant and positive.When mean FAMwas ‘‘low-average’’ (i.e.,j1.37 Gmean FAMG j0.17), the effect of SF on RSB was not statistically signifi-cant. When mean FAM was ‘‘average or greater’’ (i.e., Qj0.17),the effect of SF on RSB was statistically significant and negative.

The interpretation of the cross-level interaction, F43, fol-lowed the same form as that of F24. As can be viewed in Table 1,mean GAPAmericanism was a statistically significant predictorof the within-school effect of FAM on RSB: F43 = 0.017, p G .001.The statistical significance of T00= 0.015, p G.001, suggested thatthere was a substantial residual variance in the A4j slopes unac-counted for by mean GAPAmericanism.

Determining the explained variance in MLMs, particularlywhen at least one level 1 slope coefficient is not fixed, is a complextask and an active area of research. As can be observed in Sup-plemental Digital Content 3, http://links.lww.com/PSYMED/A54,in the current model, information regarding variance accountedfor was not provided. When a slope is not fixed (i.e., TYPE =TWOLEVEL RANDOM), Mplus does not provide a varianceaccounted for in the output. An ad hoc procedure for determiningproportion of variance accounted for was detailed in Raudenbushand Bryk (19).9

At level 1, the relevant equation can be written as follows:

R21 ¼

R 2baseline � R2

full

R2baseline

ð7Þ

A key consideration is which model is designated the baselinemodel and which model is designated the full model. Using

model 1 as the baseline model and model 6 as the full model,32.8% of the level 1 variance was accounted for by the set oflevel 1 predictors.

At level 2, relevant equations can be written as follows:

R2A0j¼ T00 baseline � T00 full

T00 baseline

ð8Þ

R2A4j¼ T44 baseline � T44 full

T44 baseline

ð9Þ

Given that level 2 predictors were specified for both A0j

and A4j, both Eqs. 8 and 9 were used to determine the varianceaccounted for at level 2. Model 5 (see Supplemental DigitalContent 2, http://links.lww.com/PSYMED/A53) is used as thebaseline model and model 6 as the full model.10 Mean PA, meanSF,meanGAPAmericanism, andmean FAM jointly accounted for12%of the variance in the unadjustedRSB schoolmeans (i.e.,A0j).Mean GAPAmericanism accounted for 3.75% of the variance inthe within-school slopes of RSB regressed on FAM (i.e., A4j).

SAMPLE SIZE AND POWER IN MLRMThree practical questions often arise when designing and/or

analyzing data from an experiment: ‘‘what N is needed in orderto use a particular statistical method?’’ ‘‘how much power (P)do I have with a fixed N?’’ and ‘‘what N is needed in order toachieve a fixed level of P?’’ A definitive answer to each of thesesomewhat underdefined questions is not possible across themyriad of model-data conditions observed in practice. Each ofthese questions is generally more difficult to answer underan MLRM as compared with a fixed-effects linear regression

8An alternative interpretation, offered by an anonymous reviewer, is that u4jis that piece of the effect of the within-school effect of family functioning onrisky sexual behavior (after controlling for the effects of peer abstinence,adolescent functioning in school, and GAPAmericanism at level 1), b4j, that isnot accounted for by the mean GAPAmericanism.

TABLE 2. Descriptive Statistics at Levels 1 and 2

Covariance/Correlation Matrix

Level 1

Risky sexual behavior 1.06 j0.06 0.02 0.15 j0.12

Peer abstinence j0.70 135.26 j0.83 0.06 j0.31

Adolescent functioning in school 0.20 j121.68 159.28 j0.05 0.24

Parental and adolescent Americanism gap 1.74 8.03 j6.92 131.77 j0.26

Family functioning j0.45 j13.00 10.99 j10.68 13.31

Level 2

Risky sexual behavior 0.07 j0.05 0.02 0.37 j0.16

Peer abstinence j0.07 21.23 j0.95 j0.20 j0.63

Adolescent functioning in school 0.02 j13.84 9.91 0.05 0.67

Parental and adolescent Americanism gap 0.32 j2.87 0.44 9.77 j0.33

Family functioning j0.05 j3.29 2.37 j1.17 1.27

Means 0.04 0.04 j0.02 0.04 j0.01

Correlation values are above the diagonal, variance values are on the diagonal, and covariance values are below the diagonal within each covariance/correlation matrix table.

N. D. MYERS et al.






model because there aremultiple sample sizes to consider (i.e., nj,J, andN) and the random part of the model is more complex (e.g.,u0j + rij in Eq. 1 versus ri in Fig. 2 of Supplemental DigitalContent 2, http://links.lww.com/PSYMED/A53).

Sample Sizes in GeneralOur response to the question, ‘‘what N is needed in order

to use an MLRM?’’ is informed by summarizing the relevantsimulation research (56). MLRM generally is a large-sampletechnique that often uses maximum likelihood (ML) estima-tion. Results from full ML are summarized later. Estimation ofthe fixed effects, F, is generally unbiased. Estimation of thestandard errors for F has been shown to be negatively biased,increasing type I error rate, when J G 50, although this biasgenerally is small when J , 30 (57Y59). Accurate estimation ofthe random component at level 1, R2, tends to be robust. Ac-curate estimation of the random component(s) at level 2, T(where T is the level 2 variance’ tqq, covariance; tqq¶, matrix),tends to be the most difficult, particularly when J G 100, al-though estimates may be adequate (if somewhat negativelybiased) when J , 50 (57,60). Using restricted ML may mark-edly improve the estimation of Twhen J is not large (57,60).

Increasing J tends to be more important than increasing njfor the purpose of maximizing the possibility of producingprecise and powerful estimates. This trend along with the morespecific findings summarized in the previous paragraph has in-formed a set of general guidelines (56). The 30/30 (i.e., (J = 30)/(nj = 30)) guideline is offered for when the research questions arefocused on F without cross-level interactions. When the researchquestions are focused on cross-level interactions within F, theguideline is modified: 50/20. When the research questions arefocused on T, the guideline is 100/10. Like all sample sizeYrelated

guidelines, a limitation of the above guidelines is that they donot take fully into account study-specific characteristics.

Sample Sizes and Power in a Specific StudyOur response to the questions, ‘‘how much power (P)

do I have with fixed values of nj and J?’’ and ‘‘what nj and/or Jis/are needed in order to achieve a fixed level of P?’’ isdemonstrated by using a flexible two-step Monte Carloapproach in Mplus (see Supplemental Digital Content 3,http://links.lww.com/PSYMED/A54). A full accounting ofthis approach, input and output, is available elsewhere(20,61). The first step is referred to as a real-data analysis. Thesecond step is referred to as a Monte Carlo simulation. Thisapproach fits within the more general framework of using MonteCarlo methods in data analysis (62). The contribution of usingMonte Carlomethods (e.g., simulating data) for a clinical trialwithreal data is toward improved application of a particular statisticalmodel (e.g., MLRM) for a particular clinical trial and, hence, re-sultant conclusions and not to statistical theory per se (63).

Returning to the Familias Unidas study, another data set(i.e., data set_2) was generated following the same protocolused to generate data set_1, with the exception that N = 633,J = 25, and n�j ¼ 25:32, which probably more closely resemblessample sizes inmany trials due to budgetary constraints. Supposethat model 6 is of interest (see Eq. 6) and that answering thequestion ‘‘how much P do I have with fixed values of nj and J?’’for each parameter of interest is desired. Parameters of interest,5i, were all statistically significant parameter estimates, exceptthe main effects of the variables involved in the cross-levelinteractions, from model 6 in Table 1. In step 1, model 6 wasfitted to data set_2 and parameter estimates were saved. In step 2,these parameter estimates were then treated as population valuesfor a Monte Carlo simulation where 10,000 data sets, withn�j ¼ 25:32 and J ¼ 25; were simultaneously generated andanalyzed. Observed power for 5i, Pi, was defined as the per-centage of replications that rejected H0 : 5i = 0. A commonlyused desired of 80% was adopted (64).

As displayed in Table 1, Pi for F ranged from 33.4% to97.5%. Two fixed effects, F03 andF43; had low power, and in bothcases, the simulation suggested that there was less than 50/50chance that the relevant null hypothesis would be rejected. Threefixed effects, F10; F24 and F30; were sufficiently powered, and inall cases, the simulation suggested that there was a very goodchance that the relevant null hypothesis would be rejected. Asimilar range of Pi was observed for the variance components:from 53.6% to 100%. Clearly, in this example andmost examplesin practice, an answer to the question ‘‘howmuchP do I havewithfixed values of nj and J’’will be complex (e.g., sufficiently poweredfor primary research questions while insufficiently powered fornonprimary research questions). Providing an answer to this

9There are limitations (e.g., possibility of negative variance accounted for) tothis approach to variance explained (52). Other approaches, including a singleindex of explained variance for the entire model, are beginning to emerge (55).

10Both Equation 8 and Equation 9 assume that the specification of the Level-1model is constant in the baseline model and the full model.

Figure 1. Simple slopes showing the effect of adolescent school functioning onrisky sexual behavior for a few values of mean family functioning. SD = standarddeviation; M = mean; HLM = hierarchical linear modeling.

MULTILEVEL MODELING






question, however, may provide a useful context for evaluatingresults from a trial (e.g., chance of committing a type II error).

Given the results summarized in the previous paragraph andassuming that nj is fixed, suppose the following question is asked:‘‘what J is needed in order to achieve Pi Q 80% for each 5i?’’.Through relatively minor manipulation of input for step 2, it isdetermined that the answer to the question is J , 115. Becausesampling more level 2 units often requires more resources thansampling more level 1 units, however, derivation of an optimal de-sign in practice frequently considers tradeoffs (in addition to Pi) dueto budgetary constraints and other study-specific characteristics(65Y67). Software has been developed for this purpose (e.g., 68).

MISSING DATA IN MLRMA key to understanding the missing data literature is to com-

prehend Rubin’s taxonomy of missing data (69). Missing com-pletely at random (MCAR) is when the propensity for missingdata on a variable, G, is unrelated to values of other variables inthe model and is also unrelated to values of G itself. Missing atrandom (MAR) is when the propensity for missing data on G isrelated to values of one or more other variables in the model butunrelated to values ofG itself. Missing not at random is when thepropensity for missing data on G is related to values of G itself.Which mechanism the missing data actually follow is typicallyimpossible to identify with relative certainty because MCAR isthe only missing datamechanism for which a statistical test exists(70). Modern approaches that address missing data generallyassumeMAR (71), although approaches that assumemissing notat random are beginning to be usedwithmore frequency (72Y74).

Incomplete data have long been a difficult issue in MLRMwhen a nontrivial proportion (e.g., Q5%; 71) of data are missingfor both the outcome(s) and at least one predictor (75,76).11

Common default approaches in many statistical packages are

to delete level 1 observations with any incomplete level 1 data(especially predictors) and to delete all observations within acluster if any level 2 data (especially predictors) are missingwithin that cluster. These defaults may inefficiently use theobserved data, assume MCAR, and may yield untrustworthyresults (77,78). The use of a few simple strategies that directlyprovide a single set of observations for missing data, such aslast observation carried forward and class mean imputation, hasgenerally been discouraged (79Y81).

Likelihood-based methods for handling missing data inMLRM address the missing data and fit the MLRM in a singlestep. Likelihood-based methods are preferred when the missingdata are in the outcome(s) only, the specification of the MLRMis correct, and MAR can reasonably be assumed (76). Missingdata in the predictors generally cannot be handled in likelihood-based methods (with typical model commands), although someprograms (e.g., Mplus) do allow missing data on auxiliaryvariables (82). Likelihood-based methods can, however, handlemissing data in the predictors if the predictors are brought intothe model by specifying additional model commands that makedistributional assumptions regarding the predictors (see Sup-plemental Digital Content 3, http://links.lww.com/PSYMED/A54).A current and thorough reviewof likelihood-basedmethods forhandling missing data in medical research is available else-where (81).

Multiple-imputation methods for handling missing data inMLRM address the missing data and then fit the MLRM tocomplete data sets in separate steps. The first step addresses themissing data problem by imputing multiple data sets while ac-knowledging the multilevel structure of the data.12 The secondstep fits the complete data sets to the MLRM and combines theresults to provide a single set of parameter estimates and standarderrors (83,84). Multiple-imputation approach of multilevel datais generally the preferred approach in MLRM (under the MARassumption) because of its greater flexibility than likelihood-based approaches (76). It should be noted, however, that muchmethodological research remains to be done with regard to ex-ploring the full range of possible missing data scenarios (andthe relative performance of likelihood-based versus multiple-imputation approaches) in MLRM.Multiple imputations of mul-tilevel data can be performed in Mplus under Bayesian estimation(85). Both unrestricted and restricted models can be used forimputation, but for multilevel data, specifying a restricted modelmay increase the likelihood of convergence under model-dataconditions frequently observed in practice (e.g., a large number ofvariables, continuous and categorical variables, and a nonlargenumber of level 2 units). A review of the unrestricted modeloptions available in Mplus is available elsewhere (85).Returningto the Familias Unidas study, data set_2 was altered (i.e., dataset_2_md) such that approximately 10% of cases were missing

Figure 2. Regions of significance of mean family functioning a mean familyfunctioning of j1.37 or less, or a mean family functioning of 0.17 or greaterwhere the effect of adolescent functioning in school on risky sexual behaviorwas statistically significant.

11The problem generally is considerably easier to address if incomplete dataare confined to the outcome(s) and the missing data can reasonably be viewedas MAR.

12Addressing the missing data problem by imputing multiple data setswhile ignoring the multilevel structure of the data can be described as multipleimputation-flat file. There is evidence that multiple imputations of multileveldata outperform multiple-imputation flat file for handling missing data thatfollow a multilevel structure (76).

N. D. MYERS et al.





data on both RSB (i.e., the outcome) and FAM (i.e., a predictor).Using the default setting of most statistical packages, 323 caseswould have been ignored when fitting model 6 to data set_2_md.Instead, before fitting model 6 to the incomplete data set, fiveimputed data sets were generated in Mplus (i.e., step 1) under arestricted approach (i.e., model 6Vsee Supplemental DigitalContent 3, http://links.lww.com/PSYMED/A54). Generating fiveimputed data sets was consistent with the small amountof missing data (71). In step 2, model 6 was fitted to the fiveimputed data sets in Mplus (see Supplemental Digital Content3, http://links.lww.com/PSYMED/A54), and the results wereconsistent with those previously listed in Table 1.

EXAMPLE 2: STRESS MANAGEMENTThree-level MLRMs occasionally appear in the literature (see

Supplemental Digital Content 1, http://links.lww.com/PSYMED/A52).When three-level MLRMs are used, the data structure often isrepeated measures nested within people who are nestedwithin level 3 units (e.g., treatment provider, dyads, etc).Thus, for example 2, we briefly present a cluster randomized trialwith repeated measures based on some core ideas from a realintervention: cognitive-behavioral stress management in prostatecancer (86). Fuller treatments of three-levelMLRMs are availableelsewhere (19).13

Prostate cancer is the most common cancer among men inthe United States, excluding skin cancer. There are more than2.2 million current survivors of prostate cancer, with more than240,000 new diagnoses estimated each year (87). Radical pros-tatectomy and radiation therapy are the standard treatment optionsfor localized disease. Although both treatments are highly effectivein disease management and extending survival, men often expe-rience chronic and debilitating treatment-related adverse effectssuch as urinary and/or bowel incontinence and erectile dysfunc-tion that challenge quality of life (88,89). Sexual dysfunction, forexample, is associated with increased distress, depression andanxiety, and lowered self-esteem, quality of marital/partnerrelationships, and general QOL (90Y93). Psychosocial inter-ventions tailored specifically for posttreatment prostate cancersurvivors have been shown to have positive effects on survi-vors’ ability to adjust to and cope with disease- and treatment-related changes.

The cognitive-behavioral stress management (CBSM) in-tervention is a 10-week, group-based intervention designed toimprove quality of life through improvements in CBSM skills.It was modified original protocol designed by Antoni et al. (94)to address the specific needs and concerns of men coping withlocalized prostate cancer. A review of the CBSM interventionhas been reported elsewhere (95). Briefly, intervention groupsmet once per week (over the course of 10 weeks) for approxi-mately 2 hours, which consisted of a 90-minute didactic com-ponent and 30-minute relaxation training. Participants wereinstructed in a number of stress management skills including

identifying distorted or irrational thoughts, effective coping strat-egies, anger management, assertiveness training, and eliciting so-cial support. The control condition consisted of a group-based,1-day seminar that lasted for approximately 4 hours duringwhich participantswere instructed in the same stressmanagementskills that were taught in the CBSM intervention sessions.

Suppose that the CBSM intervention example summarizesa relatively controlled and large trial. Treatment is randomlyassigned at the cluster-level, where 50 therapists are to ad-minister the control condition and 50 therapists are to admin-ister the experimental condition (J = 100). Participants arerandomly assigned to a therapist, with 10 participants assignedto each therapist (I = 1000). A quality of life measure is takenrepeatedly from each participant at four equally spaced mea-surement occasions (M = 4) that are common across partici-pants: immediately before the trial (i.e., baseline), 1 week intothe trial, 2 weeks into the trial, and 3 weeks into the trial. Thenesting structure can be conceptualized as repeated quality oflife measures (i.e., level 1) within participants (i.e., level 2)within therapists (i.e., level 3). The hypothesis is that the av-erage linear change in quality of life will be greater for patientsin the treatment condition compared with those in the controlcondition, after accounting for the cluster (‘‘therapist’’) effect.This model can be written as follows:

quality of lifemij ¼ P0ij þ P1ij Amij þ emijP0ij ¼ A00j þ r0ijP1ij ¼ A10j þ r1ijA00j ¼ F000 þ F001 Wj þ u00jA10j ¼ F100 þ F101 Wj þ u10j

ð10Þ

where A represents time (and is coded 0 at baseline,1 at week 1,2 at week 2, and 3 at week 3) and W is a dummy variable fortreatment condition (0 = 1-day educational intervention, 1 =CBSM intervention). A data set was generated (i.e., data set_3)consistent with this example. Equation 10 was fitted to data set_3(see Supplemental Digital Content 3 for input and key output,http://links.lww.com/PSYMED/A54).14 Interpretation of a fewkey results follows, but a fuller set of interpretations and therelevant path model are both available in Supplemental DigitalContent 2, http://links.lww.com/PSYMED/A53.

The fixed effects are typically emphasized in three-levelMLRMs. In this example, the estimate of F001 was not staticallysignificant, F001 = 1.437, p = .207. This result provided evidencethat the difference in expected quality of life at baseline for par-ticipants in the experimental condition as compared with thecontrol condition was not greater than what we would expect bychance, which was consistent with the random assignment ofparticipants to therapists. The estimate of F100 was staticallysignificant, F100 = 1.206, p G .001. This result provided evidencethat the expected rate of weekly linear change in quality of life forparticipants in the control condition was greater than what wewould expect by chance, which was consistent with the 1-day

13The Monte Carlo approach in Mplus for sample size and power determi-nation described previously in the Familias Unidas example could be extendedto this example (20,61).

14Mplus takes a multivariate (not univariate as in some dedicated MLRMsoftware) approach to changemodeling. Thus, the program considers a three-levelMLRM to be a two-level model.

MULTILEVEL MODELING









educational seminar that this group received (i.e., the controlgroup received a standard treatment). Perhaps most importantly,the estimate of F101 was statically significant, F101 = 0.843,p = .004. This result provided evidence that the difference in theexpected rate of weekly linear change in quality of life for par-ticipants in the experimental condition as compared with thecontrol condition was greater than what we would expect bychance, which was consistent with the CBSM intervention thatthis group received (i.e., in addition to the standard treatmentreceived by the control group).15

The random part of three-level MLRMs also is important. Inthis example, the variance of the level 1 error term, emij; wasstatically significant R2 = 46.823, p G .001). This result providedevidence for residual within-participant heterogeneity and thusthe possibility of adding time-varying covariates (e.g., urinaryincontinence, erectile dysfunction, and pain) to the model iftheoretically defensible. The variance of each level 2 error term,r0ij and r1ij; was statically significant (TP00

= 125.026 [ p G .001]and TP11

= 4.960 [ p G .001]). This result provided support forallowing each participant to have her/his own change trajectoryrelative to other participants within the same therapist. The ob-served level of between-participant heterogeneity would supportadding participant-level covariates (e.g., age, comorbid medicalconditions, and pretreatment physical functioning) to the model iftheoretically defensible. The variance of each level 3 error term,u00j andu10j; andu10j;was statically significant (TA00

= 19.168 [pG.001] and TA11

= 0.531 [p = .043]). This result provided support forallowing each therapist to have her/his own average participanttrajectory relative to the other therapistsVeven after controllingfor the effect of the treatment condition. The observed level ofbetween-therapist heterogeneity would support adding therapist-level covariates to the model (e.g., sex and years of experience)if theoretically defensible.

It may have been ideal if evidence had been observed tosupport treating A00j as fixed (i.e., droppingWj and u00jÞ and A10j

as nonrandomly varying based on treatment condition (i.e., drop-ping u10jÞ. Treating A00j as fixed, A00j ¼ F000; would have sug-gested thatmean participant quality of life at baselinewas relativelyconstant across therapists. Conceptually, such a result could havebeen interpreted as the result of successful random assignment ofparticipants to therapistsVgiving each therapist a similar patientprofile, with regard to mean quality of life, at baseline. TreatingA10j as nonrandomly varying, A10j ¼ F100 þ F101Wj, wouldhave suggested that the mean participant rate of weekly linearchange in quality of life was relatively constant across thera-pists after controlling for the effect of treatment condition.Conceptually, such a result could have been interpreted as theaverage rate of linear change in quality of life being indepen-dent of which therapist a participant was assigned to, afteradjusting for the effect of treatment condition. What may be

theoretically ideal may not be observed in practice and thecomplex error structure of three-level MLRMs allow for flexiblemodeling of complex phenomenon often observed in clusterrandomized trials with repeated measures.

SUMMARYMultilevel data, sometimes referred to as hierarchical data,

are common in psychosomatic research. A common hierar-chical structure has repeated observations nested within partici-pants (e.g., example 2: stress management), but cross-sectionaldata can also be viewed as multilevel (e.g., example 1: FamiliasUnidas). When nonindependence in the data is nontrivial andis attributable to at least one cluster variable (e.g., adolescentsnested within schools; repeated measures nested within partici-pants who are nested within therapists), imposing a model thatignores the nonindependence is often problematic from both anempirical perspective (e.g., increased type I error) and a theo-retical perspective (e.g., inability to model relationships at, andacross, different levels).

MLMs, particularly MLRMs, are beginning to appear withsome frequency in psychosomatic research (see SupplementalDigital Content 1, http://links.lww.com/PSYMED/A52). Somecore methodological issues within applications of MLMs thatrequire careful attention include the following: communicationof model specification, parameter interpretation, sample sizeand power, and missing data. Examples of how to communicatethe model specified and how to interpret the model parameterswere provided in example 1, example 2, and Supplemental DigitalContent 1, http://links.lww.com/PSYMED/A52. Special sectionswere dedicated to both sample size and power in MLRM and tomissing data in MLRM. Supplemental Digital Content 3 pro-vides annotated input and key output from Mplus and SAS,http://links.lww.com/PSYMED/A54. Readers are referred toextant resources for excellent didactic treatments of longitu-dinal data from a mixed model (2) and/or latent growth curvemodel (96) perspective. The current study, in combination withthese existing statistical corner studies (2,96), combine to providereaders and contributors to Psychosomatic Medicine with a co-herent and reasonably broad set of resources for potentially in-teresting possibilities with regard to modeling nonindependenceof observations.

We note that up to this point, we have focused much of ourdiscussion on MLRMs with a continuous and normally dis-tributed outcome (i.e., what is sometimes referred to as thehierarchical linear model) and where the level 1 link function isthe identity. MLRMs, as well as relevant software programs(e.g., PROC GLIMMIX), can model other types of outcomes(e.g., binary, ordinal, and count), as well (i.e., what are sometimesreferred to as hierarchical generalized linear models). For a fulldiscussion of how MLRMs accommodate (e.g., level 1 samplingmodel, level 1 link function), these other types of outcomes readersare referred to some relevant resources (97Y99). Finally, althoughMLRMswere emphasized in this study, we note that MLMs havealso been implemented within a more general latent variableframework that may be used with considerable frequency infuture research (e.g., MLSMs (100Y102)).

15Note that b101 is a cross-level interaction. The regions of significanceapproach described in the Familias Unidas example could be extended to thisexample (53,54), including in a way that could identify at which time points thedifference between groups is statistically significant.

N. D. MYERS et al.







REFERENCES1. Sullivan LM, Dukes KA, Losina E. Tutorial in biostatistics: An intro-

duction to hierarchical linear modelling. Stat Med 1999;18:855Y88.2. Blackwell E, Mendes de Leon CF, Miller GE. Applying mixed regression

models to the analysis of repeated-measures data in psychosomaticmedicine. Psychosom Med 2006;68:870Y8.

3. Hox JJ, Roberts JK. Multilevel analysis: where we were and where we are.In: Hox JJ, Roberts JK, editors. Handbook of Advanced MultilevelAnalysis. New York: Routledge; 2010:3Y11.

4. McCoach BD. Hierarchical linear modeling. In Hancock GR, Mueller RO,editors. Reviewer’s Guide to Quantitative Methods in the Social Sciences.New York: Routledge; 2010:123Y40.

5. Robinson WS. Ecological correlations and the behavior of individuals.Sociol Rev 1950;15:351Y7.

6. Lazarsfeld PF, Menzel H. On the relation between individual and col-lective properties. In: Etzioni A, editor. A Sociological Reader on Com-plex Organizations. New York: Holt, Rinehart &Winston; 1969:499Y516.

7. Cronbach LJ, Webb N. Between- and within-class effects in a reportedaptitude-by-treatment interaction: reanalysis of a study by G. L. Anderson.J Educ Psychol 1975;67:717Y24.

8. Lindley DV, Smith AFM. Bayes estimates for the linear model. J R StatSoc Series B Stat Methodol 1972;34:1Y41.

9. Smith AFM. A general Bayesian linear model. J R Stat Soc Series B StatMethodol 1973;35:61Y75.

10. Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incom-plete data via the EM algorithm. J R Stat Soc Series B Stat Methodol1977;39:1Y8.

11. Dempster AP, Rubin DB, Tsutakawa RK. Estimation in covariancecomponents models. J Am Stat Assoc 1981;76:341Y53.

12. Goldstein H. Multilevel mixed linear model analysis using iterativegeneralized least squares. Biometrika 1986;73:43Y56.

13. Longford N. A fast scoring algorithm for maximum likelihood estimationin unbalanced models with nested random effects. Biometrika 1987;74:817Y27.

14. Goldstein H. MultilevelModels in Education and Social Research. London:Griffin; 1987.

15. Bock RD. Multilevel Analysis of Educational Data. New York: AcademicPress; 1989.

16. Bryk AS, Raudenbush SW. Hierarchical Linear Models. Newbury Park,CA: Sage; 1992.

17. Goldstein H. Bootstrapping in multilevel models. In: Hox JJ, Roberts JK,editors. Handbook of AdvancedMultilevel Analysis. NewYork: Routledge;2010:163Y72.

18. Neyman J, Scott EL. Consistent estimates based on partially consistentobservations. Econometrica 1948;16:1Y32.

19. Raudenbush SW, Bryk AS. Hierarchical Linear Models. 2nd ed. NewburyPark, CA: Sage; 2002.

20. Muthen LK, Muthen BO. Mplus User’s Guide. 6th ed. Los Angeles, CA:Muthen & Muthen; 1998Y2010.

21. SAS Institute Inc, SAS 9.2 (English) Help and Documentation, Cary, NC:SAS Institute Inc, 2002Y2008.

22. Roberts JK, McLeod P. Software options for multilevel models. In:O’Connell AA, McCoach BD, editors. Multilevel Modeling of Educa-tional Data. Charlotte, NC: Information Age Publishing; 2008:427Y67.

23. Schwartz JE, Stone SA. Strategies for analyzing ecological momentaryassessment data. Health Psychol 1998;17:6Y16.

24. Enders CK, Tofighi D. Centering predictor variables in cross-sectionalmultilevel models: a new look at an old issue. Psychol Methods 2007;12:121Y38.

25. Algina J, Swaminathan H. Centering in two-level nested designs. In: Hox JJ,Roberts JK, editors. Handbook of Advanced Multilevel Analysis. NewYork: Routledge; 2010:285Y312.

26. Myers ND, Brincks AM, Beauchamp MR. A tutorial on centering incross-sectional two-level models. Meas Phys Educ Exerc Sci 2010;14:275Y94.

27. Longford N. Contextual effects and group means. Multilevel Model Newsl1989;1:5,11.

28. Longford N. To center or not to center. Multilevel Model Newsl 1989;1:7,11.

29. Plewis I. Comment on ‘‘centering’’ predictors in multilevel analysis.Multilevel Model Newsl 1989;1:6,11.

30. Raudenbush SW. ‘‘Centering’’ predictors in multilevel analysis: choicesand consequences. Multilevel Model Newsl 1989;1:10Y2.

31. Raudenbush SW. A response to Longford and Plewis. Multilevel ModelNewsl 1989;1:8Y11.

32. Kreft IGG, de Leeuw J, Aiken LS. The effect of different forms of cen-tering in hierarchical linear models. Multivariate Behav Res 1995;30:1Y21.

33. Brumback BA, Dailey AB, Brumback LC, LivingstonMD, He Z. Adjustingfor confounding by cluster using generalized linear mixed models. StatProbab Lett 2010;80:1650Y4.

34. Goetgeluk S, Vansteelandt S. Conditional generalized estimating equa-tions for the analysis of clustered and longitudinal data. Biometrics2008;64:772Y80.

35. Neuhaus JM, McCulloch CE. Separating between- and within-clustercovariate effects by using conditional and partitioning methods. J R StatSoc Series B Stat Methodol 2006;68:859Y72. Accessed August 7, 2011.

36. Centers for Disease Control and Prevention. HIV Surveillance Report2008;2010;20. Available at: http://www.cdc.gov/hiv/topics/surveillance/resources/reports/. Accessed August 7, 2011.

37. Ramirez RR, de la Cruz GP. The Hispanic Population in the United States:March 2002 Population Report P20-545). Washington, DC: US CensusBureau; 2003.

38. Prejean J, Song R, Hernandez A, Ziebell R, Green T,Walker F, Lin LS, An Q,Mermin J, Lansky A, Hall I. Estimated HIV incidence in the United States,2006Y2009. PLoS ONE 2011;6:e17502. doi:10.1371/journal.pone.0017502.

39. Prado GJ, Schwartz SJ, Maldonado-Molina M, Huang S, Pantin HM,Lopez B, Szapocznik J. Ecodevelopmental � intrapersonal risk: sub-stance use and sexual behavior in Hispanic adolescents. Health EducBehav 2009;36:45Y51.

40. Prado GJ, Huang S, Maldonado-Molina M, Bandiera F, Schwartz SJ, de laVega P, Brown CG, Pantin H. An empirical test of Ecodevelopmentaltheory in predicting HIV risk behaviors among Hispanic youth. HealthEduc Behav 2010;37:97Y114.

41. Pantin HM, Schwartz SJ, Sullivan S, Prado GJ, Szapocznik J. Ecodeve-lopmental HIV prevention programs for Hispanic adolescents. Am JOrthopsychiatry 2004;74:545Y58.

42. Prado G, Pantin HM. Reducing substance use and HIV health disparitiesamong Hispanic youth in the U.S.A.: the Familias Unidas program ofresearch. Interv Psicosoc 2011;20:63Y73.

43. Moerbeek M, Teerenstra S. Optimal design in multilevel experiments. In:Hox JJ, Roberts JK, editors. Handbook of Advanced Multilevel Analysis.New York: Routledge; 2010:257Y81.

44. Curran PJ, Bauer DJ. Building path diagrams for multilevel models.Psychol Methods 2007;12:283Y97.

45. Julian MW. The consequences of ignoring multilevel data structures innonhierarchical covariance modeling. Struct Equ Model 2001;8:325Y52.

46. Muthen BO.Multilevel covariance structure analysis. Sociol Methods Res1994;22:376Y98.

47. Satorra A, Bentler PM. Corrections to test statistics and standard errorson covariance structure analysis. In: von Eye A, Clogg CC, editors. LatentVariable Analysis. Thousand Oaks, CA: Sage; 1994:399Y419.

48. Stapleton LM. Using multilevel structural equation modeling techniqueswith complex sample data. In: Hancock GR,Mueller RO, editors. StructuralEquation Modeling: A Second Course. Greenwich, CT: Information AgePublishing; 2006:345Y83.

49. Ludtke O,Marsh HW, RobitzschA, Trautwein U, Asparouhov T,Muthen B.The multilevel latent covariate model: a new, more reliable approachto group-level effects in contextual studies. Psychol Methods 2008;13:203Y29.

50. MarshHW, Ludtke O, RobitzschA, TrautweinU,Asparouhov T,MuthenB,Nagengast B. Doubly-latent models of school contextual effects: Integratingmultilevel and structural equation approaches to control measurement andsampling errors. Multivariate Behav Res 2009;44:764Y802.

51. Raudenbush SW, Willms J. The estimation of school effects. J EducBehav Stat 1995;20:307Y35.

52. Snijders T, Bosker R. Multilevel Analysis. Thousand Oaks, CA: Sage;1999.

53. Bauer DJ, Curran PJ. Probing interactions in fixed and multilevel re-gression: inferential and graphical techniques. Multivariate Behav Res2005;40:373Y400.

54. Preacher KJ, Curran PJ, Bauer DJ. Computational tools for probing in-teraction effects in multiple linear regression, multilevel modeling, andlatent curve analysis. J Educ Behav Stat 2006;31:437Y48.

55. Roberts JK, Monaco JP, Stovall H, Foster V. Explained variance inmultilevel models. In: Hox JJ, Roberts JK, editors. Handbook ofAdvanced Multilevel Analysis. New York: Routledge; 2010:219Y30.

MULTILEVEL MODELING




http://www.cdc.gov/hiv/topics/surveillance/resources/reports/

http://www.cdc.gov/hiv/topics/surveillance/resources/reports/

56. Hox J. Multilevel Analysis: Techniques and Applications. 2nd ed. Mahwah,NJ: Erlbaum; 2010.

57. Maas CJM, Hox JJ. Robustness issues in multilevel regression analysis.Stat Neerl 2004;58:127Y37.

58. Maas CJM, Hox JJ. The influence of violation of assumptions on mul-tilevel parameter estimates and their standard errors. Comput Stat DataAnal 2004;46:427Y40.

59. Van der Leeden R, Meijer E, Busing F. Resampling multilevel models. In:de Leuw J, Meijer E, editors. Handbook of Multilevel Analysis. NewYork: Springer; 2008:401Y33.

60. Browne WJ, Draper D. Implementation and performance issues in theBayesian and likelihood fitting of multilevel models. Comput Stat 2000;15:391Y420.

61. Muthen LK, Muthen BO. How to use a Monte Carlo study to decide onsample size and determine power. Struct Equ Model 2002;9:599Y620.

62. Gentle JE. Random Number Generation and Monte Carlo Methods. 2nded. New York: Springer; 2003.

63. Myers ND, Ahn S, Ying J. Sample size and power estimates for a con-firmatory factor analytic model in exercise and sport: a Monte Carloapproach. Res Q Exerc Sport 2011;82:412Y23.

64. Cohen J. Statistical Power for the Behavioral Sciences. 2nd ed. Hillsdale, NJ:Lawrence Erlbaum Associates; 1988.

65. Snijders TAB, Bosker R. Standard errors and sample sizes for two-levelresearch. J Educ Behav Stat 1993;18:237Y59.

66. Raudenbush SW, Liu X. Statistical power and optimal design for multi-site randomized trials. Psychol Methods 2000;5:199Y213.

67. Moerbeek M, van Breukelen GJP, Berger M. Design issues for experi-ments in multilevel populations. J Educ Behav Stat 2000;25:271Y84.

68. Optimal Design. Version 2.01 [Computer Software]. New York, NY:William T. Grant Foundation; 2011.

69. Rubin DB. Inference and missing data. Biometrika 1976;63:581Y90.70. Little RJA. A test of missing completely at random for multivariate data

with missing values. J Am Stat Assoc 1988;83:1198Y202.71. Schafer JL, Graham JW. Missing data: our view of the state of the art.

Psychol Methods 2002;7:147Y77.72. Enders CK. Missing nota at random models for latent growth curve

analyses. Psychol Methods 2011;16:1Y16.73. Muthen BO. Asparouhov T, Hunter AM, Leuchter AF. Growth modeling

with nonignorable dropout: alternative analyses of the STAR*D antide-pressant trial. Psychol Methods 2011;16:17Y33.

74. Enders CK. A primer on the use of modern missing-data methods inpsychosomatic medicine research. Psychosom Med 2006;68:427Y36.

75. Longford NT. Missing data. In: de Leuw J, Meijer E, editors. Handbookof Multilevel Analysis. New York: Springer; 2008:377Y99.

76. Van Buuren S. Multiple imputation of multilevel data. In: Hox JJ, Roberts JK,editors. Handbook of Advanced Multilevel Analysis. New York: Routledge;2010:173Y96.

77. Little RJA. Regression with missing X’s: a review. J Am Stat Assoc1992;87:1227Y37.

78. Little RJA, Rubin DB. Statistical Analysis With Missing Data. 2nd ed.New York: Wiley; 2002.

79. Little RJA, Yau L. Intent-to-treat analysis for longitudinal studies withdrop-outs. Biometrics 1996;52:1324Y33.

80. Schafer JL, Scheneker N. Inference with imputed conditional means.J Am Stat Assoc 2000;95:144Y54.

81. Beunckens C, Molenberghs G, Thijs H, Verbeke G. Incomplete hierar-chical data. Stat Methods Med Res 2007;16:457Y92.

82. Collins LM, Schafer JL, Kam CH. A comparison of inclusive and re-strictive strategies in modern missing data procedures. Psychol Methods2001;6:330Y51.

83. Rubin DB. Multiple Imputation for Nonresponse In Surveys. New York:Wiley; 1987.

84. Schafer JL. Analysis of Incomplete Multivariate Data. London: Chapman& Hall; 1997.

85. Asparouhov T, Muthen B.Multiple imputation with Mplus (Version 2). 2010.Available at: http://www.statmodel.com/download/Imputations7.pdf. Acces-sed April 14, 2011.

86. Penedo FJ, Molton I, Dahn JR, Shen BJ, Kinsinger D, Traeger L, Siegel S,Schneiderman N, Antoni M. A randomized clinical trial of group-basedcognitive-behavioral stress management in localized prostate cancer: de-velopment of stress management skills improves quality of life and benefitfinding. Ann Behav Med 2006;31:261Y70.

87. American Cancer Society. Cancer Facts & Figures 2011. Atlanta, GA:American Cancer Society; 2011.

88. Hoffman RM, Gilliland FD, Penson DF, Stone SN, Hunt WC, Potosky AL.Cross-sectional and longitudinal comparisons of health-related quality oflife between patients with prostate carcinoma and marched controls. Cancer2004;101:2011Y9.

89. Miller DC, Sanda MG, Dunn RL, Montie JE, Pimentel H, Sandler HM,McLaughlin WP, Wei JT. Long-term outcomes among localized prostatecancer survivors: health-related quality-of-life changes after radical prosta-tectomy, external radiation, and brachytherapy. J Clin Oncol 2005;23:2772Y80.

90. Beck AM, Robinson JW, Carlson LE. Sexual intimacy in heterosexualcouples after prostate cancer treatment: what we know and what we needto learn. Urol Oncol 2009;27:137Y43.

91. Chambers SK, Schover L, Halford K, Cluttons S, Ferguson M, Gordon L,Gardiner RA, Occhipinti S, Dunn J. ProsCan for couples: randomisedcontrolled trial of a couples-based sexuality intervention for men withlocalised prostate cancer who receive radical prostatectomy. BMC Cancer2008;8:226Y32.

92. Helgason AR, Adolfsson J, Dickman P, Fredrikson M, Arver S, Steineck G.Waning sexual functionVthe most important disease-specific distress forpatients with prostate cancer. Br J Cancer 1996;73:1417Y21.

93. Penson DF, Feng Z, Kuniyuki A, McClerran D, Albertsen PC, Deapen D,Gilliland F, Hoffman R, Stephenson RA, Potosky AL, Stanford JL.General quality of life 2 years following treatment for prostate cancer: whatinfluences outcomes? Results from the prostate cancer outcomes study. JClin Oncol 2003;21:1147Y54.

94. Antoni M, Baggett L, Ironson G, LaPerriere A, August S, Klimas N,Schneiderman N, Fletcher MA. Cognitive-behavioral stress managementbuffers distress responses and immunologic changes following notifica-tion of HIV-1 seropositivity. J Consult Clin Psychol 1991;59:906Y15.

95. Penedo FJ, Antoni MH, Schneiderman N. Cognitive-Behavioral StressManagement for Prostate Cancer Recovery. New York: Oxford UniversityPress, Inc; 2008.

96. LlabreMM, Spitzer S, Siegel S, Saab PG, SchneidermanN. Applying latentgrowth curve modeling to the investigation of individual differences incardiovascular recovery from stress. Psychosom Med 2004;66:29Y41.

97. Agresti A. An Introduction to Categorical Data Analysis. New York: JohnWiley; 1996.

98. Hedeker D, Gibbons R. MIXOR: a computer program for mixed-effectsordinal probit and logistic regression analysis. Comput Methods Pro-grams Biomed 1996;49:157Y76.

99. Schall R. Estimation in generalized linear models with random effects.Biometrika 1991;40:719Y27.

100. Muthen B. Latent variable modeling in heterogeneous populations. Psy-chometrika 1989;54;557Y85.

101. Muthen B. Multilevel covariance structure analysis. Sociol Methods Res1994;22:376Y98.

102. Muthen B, Asparouhov T. Beyond multilevel regression modeling: multi-level analysis in a general latent variable framework. In: Hox JJ, Roberts JK,editors. Handbook of AdvancedMultilevel Analysis. NewYork: Routledge;2010:15Y40.

N. D. MYERS et al.




http://www.statmodel.com/download/Imputations7.pdf

Multilevel Modeling in Psychosomatic Medicine Research

Documents

Transcript of Multilevel Modeling in Psychosomatic Medicine Research