This article was downloaded by:[Brigham Young University]On: 20 May 2008Access Details: [subscription number 776115749]Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Personality AssessmentPublication details, including instructions for authors and subscription information:http://www.informaworld.com/smpp/title~content=t775653663

Construct Validity of the Outcome Questionnaire: AConfirmatory Factor AnalysisReed M. Mueller; Michael J. Lambert; Gary M. Burlingame

Online Publication Date: 01 April 1998

To cite this Article: Mueller, Reed M., Lambert, Michael J. and Burlingame, GaryM. (1998) 'Construct Validity of the Outcome Questionnaire: A Confirmatory FactorAnalysis', Journal of Personality Assessment, 70:2, 248 — 262

To link to this article: DOI: 10.1207/s15327752jpa7002_5URL: http://dx.doi.org/10.1207/s15327752jpa7002_5

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf

This article maybe used for research, teaching and private study purposes. Any substantial or systematic reproduction,re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expresslyforbidden.

The publisher does not give any warranty express or implied or make any representation that the contents will becomplete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should beindependently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings,demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with orarising out of the use of this material.

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

JOUR? 4 L .3P PERSOUALITj ASSESSMENT 1998, ?fl!i) 2-8-26: Cop)i;ght & 1998. Law :mce Erlbaum 4ssociate5, :lac

Construct Validity of the Out come Questionnaire :

A Confirmatory Factor Andy sis

Michael J. Larnbert and Gary M. Burlingame Dapartrnerir ng'Psyk:~!op Brigham Young C:niwrsit,t

Three content domains appear to be esserrtizI in the evaluation of treatment of psycho- logical disnrders: symptoma:ic disrress (e.g.. depressson. anxiety:, interpersonal pro5ierns, and socoi :ale dysfa;lctioc. The Outcome Questionnaire (OQ: was de- higned to measure func:nonin~ in ail three of hese domans. Constmct vaiidation of the muiti?ac:or structure of the OQ was anemptcd using confirmatory factor amiysls. This s t ~ ~ d y failed to support thc multjdimensiond construct system cspmsed by the developers of :he OQ. This f~i lure was due tcr high correialions betweex the :hree con- m a t groupings and poor a\,erall model fi:. Ir. general clinical appi~ca:ions. the three suhscaies cieveioped for :he QQ may still be of usc to che in~crcsted clinician, bur cau- tion needs to be nsed ix interpreti~g thcsc stlbscrle scores; however. the use of rhe OQ when using a giobat scare ccmposcd of (he sum of :kree suSsc;les was suppocted.

Whereas the pressure to study psychetherap! outcome has trad~tionallq come from mternal challenges i e g . Eysenck, 1352j, present-day Empetus for examlnataons of psychorherapeura: oatcome 1s coming from external sollrces such a s the health mansgenerat ~ndustry (P~gof t , 1995: Increasngly, psychdoglsts and mental health care prnfess!onds in genera! are comrng under rhe pwvren of managed beal:h care systems I Krupnlck B Pmcus, 1992 L n d e n & Wen, 1990, Pax!. 1992: Yoken & Berman, 1987) Realrstlcally. thl5 means that therapst5 are berng e w h - ated as to the efficacy and co\t effectrvenw of the service they provsde (Krupnnch & P I ~ C U S , 1932).

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

There IS little agreement as to wh~ch measures are most suitable for assessing outcome (Froyd, Ixnher t , & Froyd. 199h.i Tile mmt common method t>f measar- ing psychotherapq outcome is ttaro~:.ph the use of self-report measures rhx are zom- pleted by patients iLarnbert B Hiil, !994). This nmhod of msnjtonng chanpe r:p- pears to fit weii with h e needs of third-pa-[\; payers zs other means ~ f ' ourc::me assessment are not as cost- or time-effective (e,g.. observer r a m p . ph~s~i~logicriI studles, etc.) whereas self-report measures can be.

Of the measures availahje. most focus on one of three domains: symptomitic distress. interpersonal relationship problems. or socia1 role dysfuncrmn {Lambert & Hill, 1994). :% simiiar cclnoeptualization of &mains has been reporred ky Krupnick and Piracus i 14921. This breakdown of areas that are t,piczl;y messed is similar :c the definitior: of a mental Cisorder as found in the Dfcgriostic Srciris:i- c:n/ Manmi c$M~rziai Di.co~.der.yr (4th ed.'i:

Even though these areas are generally i.lewed as important to assess, there is a pcn- era; lack of researcher e~aiuat i~ln of the Interpersonal and social role fuzctioning I:) favor of cvsluatinp nr.Iy subjective d~s:ress Larnheri. Ogles. k Masters. 1992j. Ai- though each isi' these areas Cali he assessed indxlduail:\: rhrough the use of d~fferent measures. i t u.ould be rime- and cost-effective ro utilize a measure :ha[ atrernpti tc;

e\rolua:e aIi three of these dormins conc.urrently. In addition. a s:ngle measure char zssesses these areas at one time could be utilized across studies and vwulil assist re- searchers in assessing a11 three of'the importan: content domaiw (Lamberr et ai ... 1992).

Another ;mportant consideration rn outcome research i5 how often ic measure change occurring in a patient while she or he is undergo~np psychotherapy. It rnzpht be an asset :o the micone yesearcher to be able rc. measure chanpe at mar::; ciffrenr occasions, inclading drnnng ihe course of therapy. Doing so n,irh a measure that is sensitive to chanpe wouid a'iZow :he researcher to study :he effecrs ~ ? f therap! or! a dose-response basis as well as evaluate the qualit! of ongoing treatment forqual:r?- assurance reasons !Buriingame. Larnbert. Reisinger. Neff. & Mosier. i 995: Lam- bert & Hili, i994j. With regard to dose-response studies, some of the muiriscak measures preseniiy available are far too long to adminisrer frequentty ae.g.. L!I~

Mir,neso:a Multiphasic Personality hventory). and tbme that are short enough to admi~ister on a per-session basis cio !lor assess ail three cruciat dornalzs .;upgesreci previously <e.g . the Beck Depression Envenmry; Larnber? 6r Hil!. i994 !.

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

The O u ~ c u n ~ e Questionnaire (OQ] was developed to he used as a brief measure of patie,at functioning 11 was designed to be sensitjve to patient change over rime, to be utilized with a wide rmge of mentai disorders, and tc? be acost-effective measure of outcome (Lan~ber~, Lunnen. Umphressl Hansen. B Buringame. 1994). Specifi- cally, the OQ is an instrument purported te? assess (a) subjective distress, (b) inter- personal relationshps, and (c) socis! raHe performance: "These areas of functioning suggest a continuum from how the person feeis inside, how they are getting along with significant others; an3 how they are doing ir, important life tasks such as work and schooly' (Lsrnberr eta:.. 1994, p. 1). Subjective distress, interpersonal relation- ships, and social role performance are assessed through their own subscale in the insuument and a summation of these slrbscales @yes nu overali score of a@us:rnent from which clinically significant change can be assessed.

A critical aspect in the devebopinenf of a ncw c!inical measure is demonstration of its abiiity to measure the theorized latent constructs thar it purports to assess: this is ehe crnx of !he nssue cf cmstruci validity iAliec Sr Yec, I9?Yj. Even though the research already completed enhances confidence in t;he OQ !e.g.. it has adequate in- ternal consistency and test--retest reliability estimxes and it has appropriate contect and concurrent validity), the theoretical factor structure underlying its interpreta- tjon has ye: to he empirically tested rn a direct manner. In fact, review of previous research on the OQ has suggested thar the subscales n:ay nor ho;d ~p lrnder factor analysis beca~ase of the high internal consistency and subscaie correladcans (Lam- bert et aE.: 1394): the OQ may be a unidirnennsionai measure.

This study emiilates the construct validity of the OQ utiiizing confirmatory fac- tor analysis (CF&. Specifically. three models were assessed rasing a split-sample. cross-valiciating desigr:. The first mociel tested was theoretically derived by t:he de- vejopers of the 8Q and has three oblique factors: Suhjecti\;e Distress, Interpersonal Relations. and Social Role Perfomance.

The second mode;, a iwo-factor soiution, was developed in rexkw of previwis research coniiucted on the OQ, which observed hEg5 correlations between the OQ s~bscales and high inrernai consistency estimates (Lanbert et a!., 1994). A possible explanation for this is that the items on both the lnterpersonai Reladons and Sociai RoBe Performance scales irmv~lve the evaluation of s s p e ~ s of a person's external situations. Another way of stating :his is that the individuals is evaluating his or her iife situatiori 011 these twc, scales whereas questions from the Subjective D~stress scale prompt answers regarding the person7s internal state. Thus, therc appears to be a bifurcation in the questions as they selectively focus or, either external: (life situatior,) or internal jsub.jective distress) events. Thus. the second rnodei has two oblique factors: Subjectwe Distress and Observed Life Situation.

The rbird model; s singie-factor b m a t of the OQ, was developed for twa :ea- sons. First, the OQ may be used to assess cIinicaHy significant change only after ail three subscale scores are coIIapsed into a single scare: this use implies a single fac- tor that is being assessed. Second, review of the research on widely utilized self--

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

repnrz measures suggests that marry measures, even though the!, are purported robe rnuitidimens~onal, xi lapse into one large. general distress fx ta r (Bi!n>- i~g~. 1993: Piersnsa? Boes. ti: Reaume. 1994,. One prommesat ex:qAe :s ihat of the Symptcm Checkl~st-90 ISCL-90: Detagatis, 19-7) 2nd its variants such as :he Symptom Checklist-90-Revised (Xi-9 l i -R; Derogatis, l983! and ihe Brief Symptom In- \.entor)- (MI: Derogatis & hklisaratos. 19831. These measures have been widely urilized as a self-report measure of psychnpathcdogq- m nutcome studies. The> have been presented by the developers as measures that haw nine d~fferen! subscales that assess unique factorsp and rndude three cornposire ixhces of distress that are based on the ~ i n e sabscaies @erngaris. 1 9 7 : Schwxzu~ild. Weisenberg. & Solomoz. 19'3 1 i. re vie^ of s e m A contempo:ary studres of these measu-es iI4eutier 6: I-lam- Mm, 1366: Bonynge. IW?: Piersmaet a:.. 1994; Schwarzwalde': a).. ; 991 ': have sug- gesred rhat cmiy one factor accc~ilnred for :t sngnificant amount if the variance at bo:h the item and dimension score )eve!. Pjersmaer al. (1995). ~ ( h i ) studled the BSL. in&- cateci rhar ir i s r r m i iikel!. a measure of a unidimensionsl zunsnuct of se~eriry of geri- era1 distress underlying psychnpnrholap:\. and that littie is ganed from uciliring the dimension scores. Bonynge ! ! 993L who sfudled rhe SCL-9Q-R. reported rhan it is most iikeiy a unldimensionai measure of se~erity of genera1 distress.

METHOD

The total sample :A'= I .ibti5: consisred of Empioyee Assistance Progra:~:. patienth

t%n = 504; 45.43%). comxunip m e m l heallh cenx; patients it; = 16%; l i.;S%.;, corn!n.iln~ry norrnak ( n - 104: 9.59%). andcdkge studencs (!: = 309: 28.48% ). Ir in- cluded 63.2% fernaIe and 36.8% male participants with zi mean age far ali parucj- pants of 32.41 years (SD = i ! .89). The range of age was 6.3 years. with the youngest particiganr be3ng 12 years oici a d the oldest being '75 years old. Of the- participants. X7.9Yr were Whiie. 7.7% were Afrjcari 11rneric:ai:. 3.3% were As:ar; or Facrific is- Bander, and I .8% were Hispanic.

The item p o d of the OQ consists a9 45 questions designed to assess different as- pects of the patiem's life inciuding s);rnpto:r~a~oiog;v 2nd functinriai status. These terns are grouped into EiPree subscaies, The Sub~ec:i\:e D~stress subscaie is a 2.5- ifem scale rkat ewluates zynap~orns such as depression 2nd acxiety The Interper- sonai Relationships subscale consists nf i I items Lhar attempt fo assess func:ioning in interpersona1 reiationships by nerisuring "'friction! conflict. isoiatiiin. inade- quacy. and withdrawal" (Lamherr et a!.. ; 994. i;. S 1. The fi'ina! subscale. Social Rok .

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

252 MUELLER, LAMBERT, BURLINGAME

consists of 9 items and attempts to measure dysfunction in tasks or roles such as work and leisure Iife acf vities.

The OQ has been evaluated regarding Its test-retest reliability. internal consis- tency, and the concurrent validity of its specific. subscaies as well as the composite OQ score. Studies thus far have supported the reliability of the OQ with %week test-retest reilability being equal to .83 and internal consistency at 9 3 . Concurrent validity appears LC be high with coefficients ranging from 51 to .88 depending on the measure with which i: was compared je.g., SCIA0-R IDerogatis, ! 9771; Beck Depression inventory [Beck, Ward, Mendelson, Mock, & E:baugh, 19611; State-Trait .4nxiery Invenrory f Spielberger. 1983:). Additional studies of concur- rent validity were conducted by Umphress: Lambert, Smart, Barlow, and Clause ( 1 997) and Umphress ( 1 $95). These studies indicate that the OQ has high to mode:- ate concunerx validity with a variety of measures that are intended ro measwe con- structs similar to those purported to be measured by the OQ subscales. These seud- ies were conducted across cniiege counseiing center, conrnlrnity outpatient clinic. and i~patjent psychiatric groups. They re-eponed SCL-90-R (Derogatis, 1947) cor- relations with the Subjective Distress subsczie of .82 to .92> Inventory of Interpe,r- smal Probiems (Horowitz, Rose~berg, Bayer, Ureno, Br ViIlasenor, 1988'; cozela- :ions w ~ t h the Interperso~al Relations subscaie e f .49 to .60, and Social .k@ustmenr Rating Scale (Weissman &r Bothwel!. 1976) conelat:ons of .54 to "73 with the So- cial Koie subscale ofthe OQ. Howej~er, the highest corre1a:ions were observed be- tween each of these measures and the OQ rota1 score or the Subjective Distress subscale score regardless of the putative construct meosared for comparison.

Procedure

The CFA prxedure uti!rhed wlthm this sttasq was based on the work. of Joreskog an3 Sorbom (l9W i \Then analyzmg dam wnth LHSREL. theresearcher 1s presented w ~ t h many optrons regardrng wh~ch procedures hear she may irse FO den\ e n modei solubon. Martshorne i!993\r reported that the methods used ta produce results dc not require auitr \ ma te normal samples but h e nonnermal simple\ may rrnpact the results obtzined Some methods that may be used to der~ve a solut~on are more robust than others when the \ample is not multrva~rate normal. Accordrng tc Wrne ( 1991 ) and Loehlrn (1992). one nf the more r o h s t estrnatlon methods IS general- : ~ e b least scjuares :GLS); therefore, GLS was used to derzke model solutnons In thls study.

After the appropnatenes5 of the solutmr: wai re\ l e w d . results of CFAs were andyzed urtln~mp \e\leral mdjces Altogether, e ~ g k t mezsures of fxt were ntl lmd lo thls study Root Mean Square Resldnal (RMSR!; Goodness-sf-Rt Index (GFI,; Adjusted Goodness-of-Frt Index (AGFI): C o m p a r a t ~ ~ e Frt Index ICFIj, Kormed FI: Index jKFI); chi -sq~are ($:: ch-square dweded by degrees of freedom rn :he

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

rnodei (X"!'df); and Critical N (CNI. General guidelines regarding cutoff w r h the use of indices are as foliows iIereskag & Sorbom. 1989: Kline. 199 i ;: RMSR less thar: . l9 : GFE, CFI, and NFI greater than .99: AGFI grearer thar: .80: f tha t 1s n m - significant: X':c$;';ess than md CN greater than 200.

Kline (1991:, writing abour w h a ~ he termed "goodness-of-5: tunnel u5ion." slated thal "researchers must understand that globai gnociness-of-fit indices pro- vide limited info;rna:ic>s a b o ~ t the adequacy of path modeis . . :I-s possibk to oh- taln respectable \ dues of fit ~ndices even a h o y h s o m parameter estimates ere nonsznsrcal" Cp. 3%). Others have suggested that assessment of fit should mvclive assessmen! of the enrire model, not simply the goi?dness-of-f3 criteria IBreckier, i990: Muiaik et a:.. j989). Reise. Widaman. and Pugh 11993, silggesied tha? i r is useful to utilix more than one fit index and that "EIC CFA model shouid be accepted or rejected on statistical grcunds alone: theor>,,jucigmen:. arid persr;asi\ s argumenr sioald play a key role in defending the adequacy cf any estimated CFA model" ip. 554).

The resear& reported here utilized the entire data pool to create two random groups from which tc conduct inilia! and s e c o n d a ~ CF.4 studies. Partmpanrs were ran-

dcsnly assigned ro one uf rwo groups. Demographic informatio~ for b x h groups is presented in Table 1 . A two-tailed t test of age differences between p r o u p !.ieltisd insignificant results ( r = -1.05. p = .39,. In a d d ~ t ~ o n . chi-square anal!ses i x f s t * .

TABLE 1 Split Sample Dernograph~cs

Age M 32.835 (SD = 12.391; 33.9gs !,..a = i1.335; Range 63 53 Lowes: i 2 :- A - Iiighes; 7 ; 7;

Sex Mtik 36.7?& 37 .(l?,o Fernaic 63.5% t.?.!!"i,

Race '.%"bite 87.4?:, XS.5"; -4frican American 7.d?.rt 7 , , 7" !,.,

-1 -0. Askm'Pacific Islande: 3. 1 10 3.1% H~spanic 1.1% 7 y:,

% n= 545, "1 - 5-4i).

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

x-") = .011228, p 2 90. and race ~ ' ( 5 ~ = 1 02659. p 2 90, differences betw eer, the two group^ suggested nca s:gn~ficant d~fferences. In addmon, multl~arlate tests re- w e w q drfferences rn ]terns across h e spkt sample were nor wgnlficant WrIlan's Trace = 03882. p = 94) After these random groups were formed the rnltrai and secondzry CFA studres were cond-acteci Inntlal CFA strtdres were conducted on Group I and secondary CFA \tudles were conducteci or: G r o q 2

Although d l three models assessed appropriateI! deraved solutions on both m r - ?la1 and secondap CFA studies, none of the models assessed had adequate ie\ els of frt (see Table 2 r rn the follow m g . each model 1s revlewed regardmg rtem-factor loadmg estrmates These estamates nnaq be revlewed ~r: Tatrle 3 for ~nrtral CFA stud- ies and In Table 3 far secondap CFA studies

Model I : Theoretically Derived, Three-Factor Solution

Model I , the theoretical, three-factor solution of the OQ, included the 42 items of the OQ that are summed for use in the three scaies that comprise the OQ. Factor I , which inciuded all of rhe Sub-jective Distress scale items, had the foiHowing three highest loading items on the initial CFA study: Item 42 ("l feel blue''). Item 15 {''I feel worthiess"). and Htem 23 C'I feel hopeless about the future"). On the secondary CFA study these three irems loaded hlghest: Item 13 ("1 ireel hopeless about the fu- ture"), >ern 9 ("I feei weak"), and Item 15 ('T feel worthIess").

Factor 2. the IntcrpersonaI Relations subscale, was also included with the i t e m specified in the OQ manual. The items Boading highest on the initial CFA study were Item E 8 ("I feel lonely":, Item 43 (''I am satisfied with my relationships with others"; reverse scc~red), and Item 20 ("1 feel loired and wanied"). The items loading highest on the secorrdary CFA study were I ten I8 ("1 feel lonely"); Item 10 (''1 feei loved and wntecl": reverse scored), and Htem 19 ("I Ihave freqrnenl ~xguments").

Factor 3. the Sociaj Roje Performance scale. was also reviewed, Item 44. i"1 feei angry enough at workkhoo; to do something I mightregret"? was the highest Inad- jng on both modej runs. The second and third highest loading items in the i~iciai CFA study were Item 39 ("I have toc many disagreements a!, worklschoo?") and Item 2 1 ("1 am no1 workingistladying as well as I ased to"). Review of rhe secondary CFA study revealed that Item 28 ("1 am not working!stuciying as weil as 1 used to"j was the second highest loading item on the scale and Item 3 ("I feel stressed at worklschocsl") was the third highest loading.

Revaew of alI the items indicated that only two ( i 4, "I workktudy too much" and 32, "I have rroubie a! work!schooi because drinking or drug use") failed to load on their respective factors sign~ficantly. I t en 113 ioaded very poorly as it had estimated loadings of "06 and .04 on initial and secondary CFA stndjes. respectively. Interest- ingly, review of Tabk 5 indicates that the three factors are very highly correlated across both the initial and secondary runs of this model.

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8 TABLE 3 initiai Conf~rmatory Factor Analysis Study: Generalized Least Squares

Estimates (Completely Standardized Solution]

Modei i . Mode: 2 Modd3: Theorerrcat Three-Factor Two-Fucior Om-Facwr

Solutmc Sofzmon S o h ~ f i m

Factor Facror Factor

Item I 2 ? Irenr i 2 Item I

"Nonsign~ficant factor loading ji.e.. loading is not significantly dBereoi from zero a: the I; = .05 Ievel).

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

TABLE 4 Secondary Conf~rrnatory Factor Analys~s Study: Generaked Least Squares Estimates (Completely Standardized Solution)

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

TABLE 5 Factor Correlations for Theoretically Derived,

Three-Factor Solution Model

Interpersonal Model Run Subiective Distress Relntiorrs

Initid Subjective Distress 1 .OQ - Interpersonal Relations .91 i .O:i Social Role Performance .90 .83

Secmdary Subjective Distress I.00 In:erpersonai Relations .% 1.w Social Role Performance .88 .83

Social Role Perjiormance

Model 2: Two-Factor Solulion

Model 2, which was a two-factor solution that combined the Interpersonal Rela- tions and Social Role Performance subscales of the b>Q into one Life Situation fac- tor. also had a poor global fit. Factor 1 , as before. was a grouping o i all the items from the OQ's original Sub-jective Distress subscale. The three KmSk highly loaded items on the inE tial CFA r.sn were Iteins 4 2 1 5 , and 13. The secondary CFA run pro- duced similar results to those found with Model ! hzving Item!: 23.43, and 15 ioad most highiy. Review of initiai CFA estimated loadings for Factor 2 indicated that the most highly correlated items were 18, 43, and 20. These three i ~ e m s were grocped under the original In~erpersonal Relations scale. The secondaiy CFA study produced results [hat were generally consisterit with this as all of the most Righl y loaded itenrs (1 8, 20. and I!?? ail have interpersonaily oriented content. As was found with Model 1, this modej had factor corre~ararions approaching unity 1.95) on the initial and secondary CFA studies: these factor correBations intimate a single-factor structure. As was folind previously, Item 13 had loadings on Factar 2 that were not significantly different from zero across both groups.

Mode! 3: One-Factor Solctisn

Thss model also ha0 relatively poor fit. Resrrlts of the ;n:~;al @FA rnd~cated that Items -12. 15, and I3 ireverse scored. "I am a happy person") wcre most highly loaded on the srngle factor whereas the Item5 mosi hlghiy loaded on the secondary CFA run were 23,42, and 9. Srrniiar tc fimdrngs on the prewous two modek. Item 14 dsd nor differ srgmficantly from zero w t h regard to factor Joadng

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

In summary. ail of the models evaluated had relatively poor tit. Hnw.e.t.er. kvhen comparing these models against one ano~her \.ia r e ~ y e c t i \ ~ e chi-square stmsrlcs, tiis three-fxtor mode! had a signrficaati!; be~te r fit. albeit srilj pacr, ihan either i3f rhe other models ii.e.. < .!:)I '!. This finding was repiicated ncro.;,> both the i ~ ~ t ; a l and secondary model runs.

DISCUSSION

This srudy was undertaken in order tc? assess the purative factor srructure ::<the OQ. an instrun?reiat that was designed to be rnti!tidlrnensional. SpecLicall>,. thc OQ ,s. a h

construcred with three face-valid conten: domama: subJecr:ve distress. inierper- sonal relations. and socizi role performance. Two of the three models evduared in the study were designed to be n:uitifac:nr~al. Although the r:luIt;drrne:xionaI :nod- eis of the OQ are. appealing as based on rheory. the ~esuitr: ofrhis stud) fniisd to u p - pert them. In general, the goodness-ofifit indexes were margins!. at be>:, across ail of the models. Prior to d x u s s i n g the reasons fnr, and implicatlonh of,. t h z f n d m g , rr is Iclpcrtant ro discuss potentrai causes for [he paor to marginal fit thar was oh- served acrass aIi three n~odeis .

First. the relatively low factor Ioadlnps of several of the itecis wwrsre i i L- the pos- sihdity that :he fir of some izi'the nmdeis tested rn:ght ha1.e been irnpro\wl by either ciiscardii:g same irems or by allowing them t c load on rnors than m e Facrar. For in - srancz. tw.2 items. 15 and 32, loaded below :he .% le.i,el on a ~onsisrenr hasis and were often loaded Seiow statlsrically significant Hzveis. The mos: rroubiesnrne of :hex \vas ltern : &. Alllaough ~t was grossiy nonnalla; disrnbutrd, vs~:h a hkew o f . : 9 and kurtosis oG.37, i t u m the ponrest Irem with respect to item-intzI correiation ( . iJ?, j . Although Iterrs 14 vvas originally intended ri l assess some aspect oi'saciol roie perfarmaaxe, rhese dats lndjcale it is not doing si? significant$. In fact. it does nor appear tc? add significarrrly ti? rhe tntal OQ score. Constrw:~np further rnodels with this ~r! znnsidera~ion could examine rhe effect removal of poorl:, fitring i t e n s might h a w on o,i,eralI model fi:: however. this study was designed to asseas three thenreti-

or ll~erature-based models rather ~ h a n as a model modificat:or, stlid>. Future C F h studies :hat focus or: scale modification of the OQ, measure xlndifkation i.f the OQ. or both could prove i-aluable in this regard.

A second potential reason fi>r :he marginal fir of these models comes frnni Che posslbiiiry rhat [he dlrnens~onaIity nf rhe OQ vanes across the su5groups (e.g., pa- d e ~ s vs. nonpatienrs: n x l e ys. femakj . Related research suggests rhar there n8ere n o srgnificanr differences between rhe relative f3t ni'the three rnodeis e\xiuated i:; this study with regard ie sex. H w w e r . the mdtidimensronal rnodeis appeared ro f:r :he data more appropnatei!, when evduat~mg paraemr grolrps ikfueiler. 1996.i Fur- ther research with this measure must take this finding into accounr as the faz:or structure may \xrp depending or: rhe group assessed a i rh :he OQ

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

Finally, the merhodoIngy employed !i.e., CFA) has limitations that may have In- fluenced the results. For example, with large samp'ies, such as the one utilized in this study. there is zn increased Type I1 error rate.. This is due LC two factors. First, modeks are assessed .i.ia the use of test statistics that create inflated estimates of er- ror with large samples. Second. the CFAprocedure itself, when utilized with a large sample, increases the possibiliry of Type I1 error: "W:th 2 large sample even tr:wii violations of the measurement model assumptiom will Iead to rejectinn of the modei" (Reuaerberg &. Gustafsson, 1902. p. 796). Given this consideration: it would be valuable for this stxiy to be rep!icaied.

l n general, as none of these models fit the data well, these results do not neces- sarily contradicl previous research on [he OQ, which srrggested that the subssales may not be discrete as implied by high internai consrstency and subscale correla- tions (Lamberr er al.. 1994). Addiriond concurrenf vaiid:ty studies by Umphress (1 995) also reported thnt measures that were predicted to correlate most highly with the subscales actually corre'rate2 highesr with the rota1 OQ score or with the Symp- tom Distress subscaie. One interpretation of' these resulx is h i the OQ is more a measure of globai or silbjective distress Ehan of social role and interpersonal func- tionmg. Yet, it is interes:ing to note that when comparing these modeis against (me another via respective chi-square statistics, the three-facior model had a signifi- cantly better fit than either of the other models. Thus, although this study is incon- clusive, further exploratory study n a y yet identify a consistent and valid multidi- inensional structure for. the OQ across a numher of population groups. Toward this end, i t is recommended that further CFA studies conducted on the OQ should allow for the use of empirical (e.g., modification indices) aad rationai manipulation of the item pool so as to ohtain better fitting :nodels.

Even though we failed te support a maItidimensionai model of the OQ with this study, the theory that forms the roundation of the OQ may not be incox-ec: and should not necesslrrily be rejected. Whereas the domams of symptom dis- tress: interpersonal relationships: and social role functioning. as measured with the OQ, appear at the present ume to be so highly correlated that they effectively represent a slngie facror, they also represent distinct and important psycholegi- cal constwcts that merit concern and investigarion by researchers and practitio- ners alike. Thu.~. clinical use of :he OQ may contrnue tn a;tllixe the scales that. have beer, developed. Few problems were obse r~ed with regard to the Item groupings on those scole and the content groupings of these subscakes mzy provide click cians with valuable information regarding various aspects of their patient's lives In a manner clinicians can readily inco:porlrte into treatment.. With [his type of application, cEinicizns cauld develop treatment placs that focus on :hose needs that appear to be most indicaied by the subscaies. Those who wish to use this in- strument to track clir,icaHy significant change in patients should. as [he naafiual indicates, use only a composire score of items from all of the s~rbscales of the

W"

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

En the final ai~alysis, the OQ meets some or the needs thar \i'e:e i n d x x e d I F the In?roducrmn 11s: faris to satisfy nli of then . Ir is z b x i rneascre rhat ha:, a d e q x t e psychometrk prclpertjes and thus it is vziuaSle far merisurnp clinical c?~t;omes. In addidon, the reliability and coiicurren:! vazidity of the ins:rument supge.ir that it is appropriate for ase by the researcher. However. this study failed r:! \.alldate a mc1t;- dimensional structure f ix rhe OQ.

REFERENCES

depression. 4r .c hi;.us f b r C o i ~ m ! f ; . c ~ : i ~ : , i t , p ? . 3 . 5.'.4;.'. R. tu: -1". L:. L E.. & HarnNin. D. L : 1'JGh;. Ind:\idud:~eli outccmc niensures i.f ~nlernai :hang?. \k:h::rlo-

/ c ~ I c ~ L considerattons. Jiwrrtd :!i C:l/;ru!f~rig riid ( ' i !? i i~ ,~2; . . ' \ : .Ji i i i!?i;) . 34, 4 - 5 3 Sonyngc, E R (1993) Unldlmenslotiali:~ nf SCL-96-R scaies in ndolf and adoiescrni cr.i> s:ini~izs.

Jmr?l~i ; ,f c!!?i;<,:i: P?:\i-hiiPgY. 49. 2 i 2-2 I 5. H:cckler. E. 4. i j9931. Applicarions ofcozclrinnce s i rxture ~:~cxki:ng~r: p!.choiog). Causc ri): ci-rnxrr, '

P.rychc!!ncri (i; B:i:;et:r:. In7. 7K-2-3 Burlingcme G kl.. Lnmheri. \?. 4.. Rmingzr. C V'.. Neff. ;.. k Mos~e:. ; t i 995) Pqfnia t lcs ot trxi: .

Ing rnrniii henl:k outcxnrs in ;I rrmageci cx: setting .lo;;i.w! ;!r,+fer;ici:tieo:'iii .i:l'n1it;;>rrL;:,o,! 1':. 225 X i .

Derog:i!is. L, k. ( ?-7:. T k SCi,-9L: i f ~ f i d : . L , N ~ I ~ , :~di>::~:j:::r~~:i~?i Lz~~L!l?!-~>~~ecji~r~.c :<,Y:/~? .T<vi,-9:: Uaitlnmrr: Jahns Ffopkins 'ilrvversiry School of Zled~cinz. CImic2 Pc~chirnlstncs en11

~ e ~ ~ l g a : 1 s , 1.. i? > q83> Sc!,-%!. , i d~~?;?; ; , s : :~~l~;< l?; 5: <~.ri*lg. l2n&',vl~~h ? i i ! < r c ~ - !71~1?~14c?~,~,,l l?i: ?-c1:~5et'i ., e l - sio~r. Birliimo:~: C1inic.i Psychsrnetr~c Research

3ercrgatis. 1- R.. & 3lelisxatos. N i !3X1). The Brici Syni?tom In\vnio::. An ;nirnducior) rrtpoz. .??.- :.h~i;~gic~d I\~.fedr,.;r;r~ i3 95-60?

Eysenck, R. ; !1952> Theefftfec?~ (>f . csvcf1o:herapy: . .4r, evdunlror. .i:~;:r.!trii cv.Ct!r:r!<i!:i!r P w c i ~ : ~ f ~ ~ g : . ,:5,31q 324.

F ' r~yd , .I.. Lnmben. !\I. 1.. & Froyd.:. ; 19%;. 4 r w i c u of practices of ps!.~horherap oarcocie nxasure- nes : Jmnrd @ M e n t d Hedrh. 5. i l - - 15.

Ifnnshome, T. S . ;!9.3?: Psychornetnc propenics and confinnaton facror ;ma!!.sl; of the CCL.A Lon+ i i n s s Scde .fo!arrtai ~ $ P e r s r m i ! ; t v IS.FESS.V!CK 61. 182-i 35.

H o r o w t ~ . L M.. Roscnberg. S . E.. 3n)cr. B , A,. Urtmi?. G , k Viliawnor. \. . S. ( i"86i. ic\-entor?. ,?fin- wrpersonal Problems: Psyhometric pmpsxirs and :iinlcrii appilcatlons. b$~;irri.~i i!! Cor:~iii:,r:p;ir,d C1ini;d F . ~ y c h d i ~ p . 36. 665 P91.

Jh r skog . K., & Sbrbom. 3. : I ?W!. ;!.%EL P: P. pi& I:, r!lc, pmgr im ,iri,!Lll:?i;L~:::i~;r,: : 2nd rd . i Ch:. cago. SPSS.

Dow

nloa

ded

By:

[Brig

ham

You

ng U

nive

rsity

] At:

23:3

3 20

May

200

8

262 MUELI,EIR, LAMBEAT, BURLINGhME

Lmbefl. M. J., Lunnen, K., Urnphress. V., Hmsen, N., B Burlingme. G. (1944) Admin:stmtiur: rind scoring nunuill,fi,r the Outcome Ques:iormoire !0Q 4 S . I i Salt Lake City; LIT. IIlC Center for Health Care Efficacy.

Lambe<. M. J.: Ogies, R., & Masters, K. !i992j. Choosing outcome assessment devices: An organkzs- tianal 'and conceptual scheme. Jorrmal :(f Colmsel:ng and Devekpnenr, 7C. 527-535.

Linden. W., & Wen, P. (1990). Therapy ourcome resexch, health care pok!:. and the continuing lack of accumulated knowiedge. Pro~essicnai P.&dogy: Reseurcir and Pructice, Zi. 482--488

L o e h h , J. C, (1992:. Lotent variable models. An Irirroduction ro,fuctor. pail;, ands:ructurai anulvsis (2nd ed.). Eilisdaie, N. Lawience Eribaum Associates, !nc.

Mneller, R. (1996:. Ci~n.~t.ruci vulidit), cftire Ourcanre Quest;c~r:naire. .4juctrsr uncd.vtlc stu& Unpob- lished docroral dissertatron, Brigham Young Unive:sity. Prcva. UT.

Muiaik. S. A,, James, t. K.. Van Alstine, J . , Bennetl, N.. l h d , S., & Stilwell. C D (1989). E~alcation of goodness-of-fit indices for structural equation models. P.~y9~c~iogical Eu!tetin, 105. 430 4 4 5

?my, 6. 11992). improving psychotherapy services: kppiirations of research. audit and evaluation Britisl: jorrrna! qi Clinical Psyrhcdogy. 31, 3-1 9.

Piersma, H. L.. Boes. J . L.. & Keaume, W. M. (1994). Unidinlensio~:d:ty oi'the Brief Symptom Hnven- t o v ZW) ir? ad& and adoiescent inpatrents. Journal c!f Persortulir) Assesstnenr, 63, 338-344.

Pigoff. H. E. (i9.35). Linkrng outcomes analysis tc i critical clinicai pathways Brhavicral .Yeul!hcure 7, otnorrow, !, 5 9 4 ; .

Reise, S . , Wldnrnan, K.. K: Pugh. R ( 1 991:. Confi:matory facio: analysis and i!e:nresponst. theory: Tw:, approaches h r e x i o i n g nieasurerr~ent invariance. Psychi~lc-pica! Bullerin, ii4. 552'- 566

Reuterberg. S., & Gusiafsson, J ( 1 992). Confirmator) factor analysis and reliability: Testing measure- men: rnodei assumptions Educrzrionu: und I'.:ycldog;cal Meusuwnenr, 5z1 795-81 1

Schwarzaaid, J . , Weisenberg, M.. & Soiornon. Z. ii991:'. Factor invariance of SCL -9s-R: The case of comba; stress reachon Psychoiogzrd Asses.vmerii: .4 Journn; of Consulrirlg utrd Clinical I'syctxrl- o p , 3, 385--3%

Sprelberger. C D (1 9831. il.fotzuu!f!v:he b~ii~r-T~ui~Ar.xieg~Ir,i~enr:)n~ S7APiE'orni f i . Palc Alto, CA. Consuiitrng Psychoiogists Press.

Umphress, V. 1. (lY95':. Clitiic'ul norms u n d ~ w l z d i ~ ~ ofrhe Ourrome Ques:ionnuiw. Unpublished doc- roral dissesation. Brigham Young University, Prow, UT.

Umphress, V. J., lxmbeli, M. J . . Smart, D. W., Sariow. S. 9.. f(r Cious~. 6. C I O F ; ! . Concomnt and con- struct vn!idi:y of the Outcom Questionnaire. Joli,iimni of P.~yrh~~eduat i~na~Asse.r .~me~it , 15.4+55.

Weissman, M . M.. & Bothweli, S. (1976:. Assessment of social adjustment by patienr self-repoi',. Ar- chives qf Ge9ze.d P.~yhiarr?;. 3. 1 1 1 ! -1 I 15.

Yokm, C., & Bertnar,, J . i i 9 8 3 ThrrG-party pymcn: and the outcome of psychotherapy. J:~urr;al qf &onsulri~;,y and .I'/rnr'?iil Psychoiop!.; 5.5, 57 i -576

Reed M. Muelie~ Oregm Health Sciences Un~versrty 3 18 I SW Sam Jackson Park Road Porriand. OR 97203 -30%

Received November 22. 1996 R e w e d ru'ovembe: 4, 1997