TIn

WSa

b

c

a

ARRA

KPSSA

1

pbdaSm

BSI

F

0h

Journal of Anxiety Disorders 28 (2014) 140–147

Contents lists available at ScienceDirect

Journal of Anxiety Disorders

he SPAI-18, a brief version of the Social Phobia and Anxietynventory: Reliability and validity in clinically referred andon-referred samples�

ieke de Ventea,∗, Mirjana Majdandzic a, Marisol J. Vonckenb, Deborah C. Beidel c,usan M. Bögelsa

Research Institute Child Development and Education, University of Amsterdam, Nieuwe Prinsengracht 130, 1018 VZ Amsterdam, The NetherlandsDepartment of Clinical Psychological Science, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The NetherlandsDepartment of Psychology, University of Central Florida, 4000 Central Florida Boulevard, Orlando, FL 32816, United States

r t i c l e i n f o

rticle history:eceived 17 August 2012eceived in revised form 8 May 2013ccepted 10 May 2013

eywords:sychometric propertiesocial anxiety disorderocial phobianxiety disorders

a b s t r a c t

We developed a new version of the Social Phobia and Anxiety Inventory (SPAI) in order to have a briefinstrument for measuring social anxiety and social anxiety disorder (SAD) with a strong conceptual foun-dation. In the construction phase, a set of items representing 5 core aspects of social anxiety was selectedby a panel of social anxiety experts. The selected item pool was validated using factor analysis, reliabilityanalysis, and diagnostic analysis in a sample of healthy participants (N = 188) and a sample of clinicallyreferred participants diagnosed with SAD (N = 98). This procedure resulted in an abbreviated version ofthe Social Phobia Subscale of the SPAI consisting of 18 items (i.e. the SPAI-18), which correlated stronglywith the Social Phobia Subscale of the original SPAI (both groups r = .98). Internal consistency and diag-nostic characteristics using a clinical cut-off score > 48 were good to excellent (Cronbach’s alpha healthy

group = .93; patient group = .91; sensitivity: .94; specificity: .88). The SPAI-18 was further validated in acommunity sample of parents-to-be without SAD (N = 237) and with SAD (N = 65). Internal consistencywas again excellent (both groups Cronbach’s alpha = .93) and a screening cut-off of >36 proved to resultin good sensitivity and specificity. The SPAI-18 also correlated strongly with other social anxiety instru-ments, supporting convergent validity. In sum, the SPAI-18 is a psychometrically sound instrument withgood screening capacity for social anxiety disorder in clinical as well as community samples.

© 2013 Elsevier Ltd. All rights reserved.

. Introduction

Social anxiety is characterised by fear of situations in which aerson feels exposed to and possibly judged by others, and fearsehaving in an improper or embarrassing way. Social anxiety disor-er (SAD), also known as social phobia, is one of the most prevalent

nd most unsettling mental disorders (Furmark, 2002; Stein &tein, 2008). For clinical and research purposes, various assess-ents tools have been developed to measure social anxiety and/or

Abbreviations: BTSQ, Blushing Trembling Sweating Questionnaire; RBS, Riskehaviour Scale; SAD, social anxiety disorder; SIAS, Social Interaction Anxiety Scale;CID, Structured Clinical Interview for DSM-IV; SPAI, Social Phobia and Anxietynventory; SPIN, Social Phobia Inventory; SPS, Social Phobia Scale.� The editorial responsibilities for this manuscript were handled by B. Christopherrueh, Ph.D.∗ Corresponding author. Tel.: +31 20 525 1446; fax: +31 20 525 1200.

E-mail address: w.devente@uva.nl (W. de Vente).

887-6185/$ – see front matter © 2013 Elsevier Ltd. All rights reserved.ttp://dx.doi.org/10.1016/j.janxdis.2013.05.003

SAD, including short screening instruments (e.g. mini Social Pho-bia Inventory (mini SPIN) (Connor, Kobak, Churchill, Katzelnick,& Davidson, 2001)), diagnostic interviews (e.g. Anxiety DisorderInterview Schedule (ADIS) (Di Nardo, Brown, & Barlow, 1994)), andself-report measures. A well-known self report measure that hasa history of extensive psychometric evaluation is the Social Pho-bia and Anxiety Inventory (SPAI), which was developed by Turner,Stanley, Beidel, and Bond (1989). The SPAI assesses an extensiverange of social anxiety symptoms and has proven to possess excel-lent psychometric properties (e.g. Beidel, Turner, Stanley, & Dancu,1989; Turner, Beidel, & Dancu, 1996; Turner et al., 1989). For ourlongitudinal study “Social development of children”, which exa-mines the contribution of predisposition and environmental factorsto the development of social anxiety during early childhood, weneeded a short instrument to screen parents-to-be on SAD, while

preserving assessment of the subjective, behavioural, cognitive,and somatic aspects of social anxiety. We preferred to use the SPAI,because of its conceptual foundation and psychometric quality. Adrawback, however, was that completion of the SPAI takes about

nxiety Disorders 28 (2014) 140–147 141

2S

eC(sBhcGGHgc

a1tSrcbaatfeR(FS(ee

psiwdasSdPL2pfsss

OSSstnSwwova

Table 1Characteristics of sample A presented separately for healthy participants andpatients with SAD.

Healthy (n = 98) Patient (n = 188)

n/M (%/SD) n/M (%/SD)

Gender (male/female) 39/59 (40/60) 89/99 (47/53)Age (years) 34.15 (12.91)c 31.08 (10.11)Education (scale 1–11)a 8.09 (1.86) 7.87 (2.04)Professional level (scale 1–6)b 4.19 (1.64) 3.67 (1.73)

a 1: primary education, to 11: university.b 1: manual labour for which no education is required to 6: labour for which a

W. de Vente et al. / Journal of A

0 min. Hence, we chose to construct an abbreviated version of thePAI.

The SPAI is used in healthy and clinical samples to assess thextent of social anxiety on a continuum (e.g. Beidel, Turner, &ooley, 1993), for screening of SAD using a clinical cut-off scoree.g. Voncken, Bögels, & de Vries, 2003), and as an outcome mea-ure in for example treatment effect studies (e.g. Beidel et al., 1993;ögels, 2006; García-López, Olivares, & Hildago, 2005). The SPAIas been translated into various languages and good psychometricharacteristics were again established (e.g. Bögels & Reith, 1999;arcía-López, Olivares, Hildago, Beidel, & Turner, 2001; Olivares,arcia-Lopez, Hidalgo, & Caballo, 2004; Olivares, García-Lopez,ildalgo, Turner, & Beidel, 1999). The SPAI has also demonstratedood discriminative properties as well as sensitivity to clinicalhange (Beidel et al., 1993; García-López et al., 2005; Peters, 2000).

Also in comparison to other social anxiety questionnaires suchs the Social Interaction Anxiety Scale (SIAS; Mattick & Clarke,998), the Social Phobia Scale (SPS; Mattick & Clarke, 1998), andhe Social Phobia Inventory (SPIN; Connor, Davidson, Churchill,herwood, & Weisler, 2000), the SPAI stands out. First, withegard to content validity, the SPAI examines a broader range oflinically relevant social anxiety symptoms, including subjective,ehavioural, cognitive, and somatic characteristics, while the SIASnd the SPS remain limited to assessing cognitive and subjectivespects, and the SPIN to subjective, behavioural, and somatic symp-oms. Second, also psychometrically, the SPAI demonstrates moreavourable characteristics than the SIAS, the SPS, and the SPIN. Forxample, the SPAI has better discriminative quality (e.g. Bögels &eith, 1999; Peters, 2000) as compared to the SIAS and the SPSe.g. Heinrichs et al., 2002) and to the SPIN (Connor et al., 2000).urthermore, although the internal consistency of the SIAS andPS (Heinrichs et al., 2002; Mattick & Clarke, 1998) and the SPINConnor et al., 2000) are good, the internal consistency of the SPAI isven higher (e.g. Bögels & Reith, 1999; Olivares et al., 1999; Osmant al., 1996).

The SPAI includes two subscales: (1) Social Phobia and (2) Agora-hobia (Turner et al., 1989). The former measures characteristics ofocial anxiety, and the latter assesses fear in situations that are typ-cally associated with agoraphobia, for instance fear for crowds and

aiting in lines. The developers added the Agoraphobia subscale toifferentiate between social anxiety and anxiety that accompaniesgoraphobia. To obtain the SPAI-total score, the Agoraphobia Sub-cale Score should be subtracted from the Social Phobia Subscalecore. Good, or even superior, results in terms of concurrent andiscriminant validity are, however, also obtained with the Socialhobia Subscale Score alone (e.g. Bögels & Reith, 1999; García-ópez et al., 2001; Herbert, Bellack, & Hope, 1991; Olivares et al.,004; Roberson-Nay, Strong, Nay, Beidel, & Turner, 2007). For oururpose to obtain a short instrument with good screening capacityor social anxiety, we reasoned that by leaving out the agoraphobiaubscale we would probably still reach our main objectives, that is, ahort questionnaire with excellent sensitivity and at least adequatepecificity. Consequently, we left out the items on agoraphobia.

Roberson-Nay et al. (2007) and Garcia-Lopez, Hidalgo, Beidel,livares, and Turner (2008) have developed short versions of thePAI consisting of 23 items (in English, for adults) and 16 items (inpanish, for adolescents), respectively, using thorough and soundtatistical techniques including item-response analyses. Despitehese thorough methods, their procedures have one weakness,amely, the original five-factor model underlying the Social Phobiaubscale was not taken into account for item selection. Since weanted to ensure capturing the complete social anxiety construct,

e strived to retain the theoretically sound five-factor structure

f the Social Phobia Subscale, by selecting items representing thearious aspects of social anxiety, that is: (a) distress in social situ-tions (individual and group interactions), (b) anxiety when being

university degree is required.c The healthy group was significantly older than the patient group;

(t(284) = −2.21, p = .028).

the centre of attention, (c) avoidance, (d) cognitive symptoms, and(e) somatic reactions (see Turner et al., 1989).

In sum, the objective of this study was to develop a psychomet-rically sound short instrument to detect social anxiety disorder thatcan also be used to discriminate between individuals on the socialanxiety continuum, and that covers subjective, behavioural, cog-nitive, and somatic domains of social anxiety. The development ofthe abbreviated SPAI consisted of two phases: (I) construction andinitial validation; and (II) secondary validation using a new sam-ple. In phase I, a panel of experts on social anxiety (AS, MV, MW,PE) selected the items from the original SPAI that referred to thecore of social anxiety (disorder) while adhering to the thoroughlyformulated conceptual background. Furthermore, we performed afactor analysis and reliability analyses, and assessed the criterionvalidity on data assembled in a healthy sample and in a sample ofclinic-referred patients with SAD in which the original SPAI wasadministered. In phase II, we validated the abbreviated SPAI byconducting reliability analyses, assessing convergent and diver-gent validity, distributional features, and diagnostic qualities in acommunity sample of parents-to-be with and without SAD.

2. Phase I

2.1. Methods

2.1.1. ParticipantsFor the initial development and validation of the abbreviated

SPAI, datasets of three studies on social anxiety that were con-ducted in the Netherlands using highly similar procedures weremerged. Taken together this sample (sample A) consisted of 188patients with SAD and 98 healthy participants. Data coming fromthe first study (Bögels & Reith, 1999) consisted of 92 clinic-referredpatients with SAD diagnosed using the Structured Clinical Inter-view for DSM-IV (SCID) I (First, Spitzer, Gibbon, & Williams, 1996;Groenestijn, Akkerhuis, Kupka, Schneider, & Nolen, 1997) and 65healthy participants. Data coming from the second study (Vonckenet al., 2003) consisted of 33 healthy individuals. Data comingfrom the third study (Bögels & Voncken, in press) consisted of96 patients with SAD, as diagnosed with the SCID-I (First et al.,1996; Groenestijn et al., 1997). Both clinical samples were recruitedfrom the same ambulatory mental health centre in Maastricht (TheNetherlands) and were highly similar with respect to gender distri-bution, age, education level, and professional level (data availableupon request). Experienced clinical psychologists that were exten-sively trained to administer the SCID performed the diagnosticinterviews. Inter-rater reliabilities from data of the same mentalhealth centre were moderate to excellent for axis-I disorders, with

a Kappa of .83 for SAD (Lobbestael, Leurgans, & Arntz, 2011). Forfurther information on recruitment and selection participants forthese three studies, we refer to Bögels and Voncken (in press),Bögels and Reith (1999), and Voncken et al. (2003). In Table 1,

1 nxiety Disorders 28 (2014) 140–147

cp

22beSS&Ac(oa7ts

2

Stsenh

2

2

twpwsitsT6twtOcnstsfca(Tfafp21(av

Table 2Selected items of the social anxiety subscale of the original SPAI by expertsa denotingthe five main aspects of social anxiety represented in the SPAI.

Aspects Selected items of the original SPAIb

Distress in social situations 4, 9, 12, 13, 14, 15, 16, 23, 27Focus of attention 3, 5, 22Avoidance and escape 7, 8, 24, 25Cognitions 26, 30Somatic reactions 28, 31, 32

a Items selected by at least two experts are listed.b

42 W. de Vente et al. / Journal of A

haracteristics of sample A are presented separately for healthyarticipants and patients with SAD.

.1.2. Materials

.1.2.1. Social Phobia and Anxiety Inventory (SPAI). The SPAI haseen developed to assess social anxiety and social phobia (Turnert al., 1989). The SPAI includes two subscales, the Social Phobiaubscale and the Agoraphobia subscale. For the current study, theocial phobia subscale (official Dutch translation; Scholing, Bögels,

van Velzen, 1995), which consists of 32 items, was included (seeppendix A). Twenty-one items require four separate ratings con-erning the degree of distress in four different social situationsstrangers, authority figures, opposite sex, people in general). Tobtain item scores, these four ratings are averaged. All items have

7-point Likert-scale response format ranging from 1 = ‘never’ to = ‘always’. To obtain a total score range between zero and 192,he number of items (i.e. 32) is subtracted from the summed itemcores.

.1.3. ProcedureIn phase I of the construction of the abbreviated version of the

PAI, an item pool was selected from the Social Phobia Subscale ofhe SPAI (including the subitems referring to four different socialituations, see previous paragraph) based on theoretical arguments,xpert judgements, factor analysis, reliability analyses, and diag-ostic analysis, using a patient sample with a SAD diagnosis and aealthy sample.

.2. Results

.2.1. Preliminary analysesIn order to test the main aspects of social anxiety represented in

he SPAI in Dutch clinical and non-clinical samples, factor analysesere performed. Due to >20% of missing items on the SPAI in threearticipants, the following analyses were based on 186 patientsith SAD and 97 healthy participants. An exploratory factor analy-

is (principal component analysis with varimax rotation) on the 32tems of the Social Phobia Subscale of the SPAI resulted in six fac-ors with Eigenvalues >1 in the healthy sample as well as the patientample and four factors that each explained ≥10% of the variance.he total variance explained was 72% in the patient sample and9% in the healthy sample. The six factor solutions did not result inheoretically sound and clearly interpretable factors. Subsequently,e analysed a forced 5-factor solution, as has been reported by

he developers Turner et al. (1989) and Osman, Barrios, Aukes, andsman (1995). In order to optimise chances to obtain similar out-omes, we excluded items 21 and 29, like Osman et al. (1995) did. Ofote, these items also showed lowest item-total correlations in ourample. In the patient group, this analysis resulted in the followingheoretically sound and clearly interpretable factors: (I) distress inocial situations (Eigenvalue: 6.77; 22.6% variance explained); (II)ocus of attention (Eigenvalue: 4.55; 15.2% variance explained); (III)ognitions (Eigenvalue: 3.90; 13.0% variance explained); (IV) avoid-nce and escape (Eigenvalue: 3.46; 11.5% variance explained); andV) somatic reactions (Eigenvalue: 2.29; 7.65% variance explained).he total variance explained was 70%. In the healthy group, the 5-actor solution did not result in clearly interpretable factors, andgain, only four factors explained ≥10% of the variance each. Aorced 4-factor solution resulted in the following clearly inter-retable factors: I: distress in social situations (Eigenvalue: 6.22;0.7% variance explained); (II) somatic reactions (Eigenvalue: 4.38;

4.6% variance explained); (III) avoidance/escape and cognitionsEigenvalue: 4.18; 13.9% variance explained); and (IV) focus ofttention (Eigenvalue: 3.87; 12.9% variance explained). The totalariance explained was 62%.

Numbers refer to the item numbers of the Social Phobia (SP) Subscale of theoriginal SPAI. Item themes are described in Appendix A.

The extracted five factors in the patient sample, that are simi-lar in content to the factors extracted by Turner et al. (1989) andOsman et al. (1995), were proposed as main aspects (i.e. distress insocial situations, somatic reactions, avoidance/escape, cognitions,and focus of attention) of social anxiety represented in the SPAI tofour experts on social anxiety (AS, MV, MW, PE). The experts wererequested to select the up to five items of the social anxiety sub-scale of the original SPAI that representing these aspects best (i.e.based on face validity). Since inter-item correlations were all >.40and item-total correlations were >.60 for all items (calculated inthe sample as a whole), we did not exclude any items on statisticalgrounds in advance.

2.2.2. Initial item selection by expertsA total of 28 items of the social anxiety subscale of the origi-

nal SPAI was selected by the experts. Since this strategy led to aminimal reduction in items, we chose to select the items that wereselected by at least 2 experts, which resulted in a pool of 21 items.The aspects and selected items are listed in Table 2.

2.2.3. Further analyses of the selected item poolThe item responses ranged from 1 to 7 for all but 1 item, which

ranged between 1 and 6. As a first check on the discriminativeability of the selected item pool, independent t-tests comparedthe group means of the healthy and patient group on each item.Patients scored significantly higher than the healthy participantson all items (all p-values < .001; all Cohen’s d’s > 1), hence all itemswere retained for further analyses. Subsequently, internal consis-tencies were calculated for the subscales representing the socialanxiety aspects. Item 28 (voice leaves me or changes when talkingin social situation) was dropped from the selected item pool, sincethe Cronbach’s alpha’s of the subscale Somatic reactions increased>10% in the healthy as well as the patient group when this itemwas excluded. Factor analyses on the remaining 20 items resultedin a 5-factor solution in the patient group and a 4-factor solutionthe healthy group, similar to the initial factor structure of the SocialPhobia Subscale. In both groups, item 4 (distress when in a socialsituation and expected to participate in certain activity) was allo-cated to the Focus of attention dimension instead of the Distressin social situations dimension, to which it was assigned by theexperts. Hence, for further subscale analysis, item 4 was consid-ered to represent the Focus of attention dimension. Furthermore,items 23 (distress when criticised/turned down) and 27 (antici-patory tension for social situation) loaded on other dimensionsthan the intended Distress in social situations and inconsistently sobetween groups (Cognitions (23 and 27) in the patient group andFocus of attention (23) and Somatic reactions (27) in the healthy

group). Consequently, items 23 and 27 were excluded from the itempool. In sum, a total of 18 items were selected for the final shortversion of the SPAI, which is further referred to as the SPAI-18.

W. de Vente et al. / Journal of Anxiety Disorders 28 (2014) 140–147 143

Table 3Principal-components analysis on the 18 Item-pool of the SPAI and internal consistency of the subscales of the SPAI-18 in sample A.

Factor Number of items Item numbersa Loading (range) Variance explained (%) � b

Healthy group (N = 97)I. Distress in social situations

(Eigenvalue = 3.80)6 9, 12, 13, 14, 15, 16c .67–83 21.1 .87

II. Focus of attention(Eigenvalue = 2.65)

4 3, 4, 5, 22 .60–.81 14.7 .78

III. Avoidance/escape andcognitions(Eigenvalue = 3.58)

6 7, 8, 24, 25, 26d, 30 .47–.83 19.9 .87

IV. Somatic reactions(Eigenvalue = 2.26)

2 31, 32 .88–.90 12.6 .89

Patient group (N = 186)I. Distress in social situations

(Eigenvalue = 4.33)6 9, 12, 13, 14, 15, 16 .62–.86 24.1 .93

II. Focus of attention(Eigenvalue = 2.90)

4 3, 4, 5, 22 .61–.86 16.1 .86

III. Avoidance and escape(Eigenvalue = 2.97)

4 7, 8, 24e, 25 .68–.85 16.5 .86

IV. Cognitions(Eigenvalue = 1.93)

2 26, 30 .80–.81 10.4 .87

V. Somatic reactions(Eigenvalue = 1.88)

2 31, 32 .88–.92 10.7 .90

a Item numbers refer to the original numbers of the Social Phobia Subscale items of the SPAI. Item themes are described in Appendix A.b Cronbach’s alpha.c Item 16 also loaded .41 on factor IV.d Item 26 also loaded .46 on factor I and .43 on factor IV.e

2c

hwS6Asaft.1Pi

gwptSstpitpSsvYiv(fa

SPAI-18, resulting in N = 300 with valid data on both the SPAI-18and the diagnostic status. In Table 5, characteristics of sample B arepresented separately for participants with and without SAD.

Table 4Measures of central tendency, dispersion and distributional characteristics of theSPAI-18 in sample A.

SPAI-18

Healthy (N = 97) Patient (N = 186)

Mean 31.98 71.95Median 32.72 73.40

Item 24 also loaded .46 on factor I.

.2.4. Factor analysis, reliability analysis, and diagnosticharacteristics of the SPAI-18

Factor analysis on the 18 item pool resulted in 4 factors for theealthy group and 5 factors for the patient group (see Table 3),hich is identical to the initial factor solutions on the items of the

ocial Phobia Subscale (see Section 2.2.1). The factors explained8.3% variance in the healthy group and 77.8% in the patient group.ll items had the highest factor loading on the factor they wereelected for by the experts, except again item 4 (see Section 2.2.3),nd 3 items also loaded on another factor, using the criterion of aactor loading > .40. The internal consistency was very good for theotal SPAI-18 (patient group: Cronbach’s alpha = .93; healthy group91) and good to very good for its subscales (see Table 3). The SPAI-8 total score correlated highly with the original score on the Socialhobia Subscale, Pearson’s r = .98, p < .001 in the healthy as well asn the patient group.

The mean score on the SPAI-18 was significantly higher in theroup diagnosed with SAD (M = 71.95, SD = 15.11) than in the groupithout a SAD diagnosis (M = 31.98, SD = 12.57; t(df = 281) = 22.32,

< .001; d = 2.80). In order to assess the ability of the SPAI-18o discriminate between healthy individuals and patients with aAD diagnosis, a receiver operating characteristic (ROC) analy-is was done. The area under the ROC curve, which summariseshe discriminative ability of the SPAI-18, was 0.98 (CI = 0.97–0.99;

< .001). Since the 95% confidence interval did not include 0.5,t can be concluded that the SPAI-18 performs considerably bet-er than chance in discriminating patients from controls. The testerformance indicators that illustrate the capacity to distinguishAD from no-SAD were as follows when using a clinical cut-offcore of >48: sensitivity: .94, specificity: .88, positive predictivealue: .93, negative predictive value: .86; LR+: 4.27; LR−: 0.07;ouden index: .82. Since our goal was to develop a screening

nstrument, we chose a cut-off score by overweighing the rele-

ance of detecting SAD (sensitivity) at the cost of false positivesspecificity). The full range of cut-off scores and associated per-ormance indicators can be obtained on request from the firstuthor.

2.2.5. Descriptive informationMeasures of central tendency and dispersion and distributional

characteristics are presented in Table 4. Since the skewness andkurtosis are <|1|, it can be concluded that the distributional charac-teristics are very good.

3. Phase II

3.1. Methods

3.1.1. ParticipantsFor the secondary validation of the abbreviated SPAI, data were

used from our longitudinal study on the development of social anxi-ety in young children in a sample of 152 parent couples coming fromthe general population (sample B). Information on recruitment,selection of participants, and ethical procedures is provided inMajdandzic, de Vente, and Bögels (2010) and De Vente, Majdandzic,Colonnesi, and Bögels (2011). Information of 2 participants wasmissing on the diagnostic status, assessed using the ADIS (seebelow), and information of 2 other participants was missing on the

Range 4–62 27–103Standard deviation 12.57 15.11Skewness 0.13 −0.32Kurtosis −0.79 −0.39

144 W. de Vente et al. / Journal of Anxiety Disorders 28 (2014) 140–147

Table 5Characteristics of sample B presented separately for participants with and without SAD diagnosis.

Group without SAD (n = 235) Group with SAD (n = 65)

M (SD) M (SD)

Age (years) 32.16 (4.84) 32.05 (6.00)Education (scale 1–8)a 6.69 (1.51) 6.76 (1.33)Professional level (scale 1–11)b 8.52 (2.40)c 7.69 (2.68)Dutch ethnicity (frequency/%) 217 (92%) 62 (95%)

a 1: primary education, to 8: university.nivere gro

3

acsFsw1wpv&(Wt

33t&(pfb

TP

A

e

b 1: manual labour for which no education is required to 11: labour for which a uc The professional level in the group without SAD is significantly higher than in th

.1.2. ProcedureThe abbreviated SPAI was further evaluated using factor

nalyses, reliability analyses, correlational analyses to examineonvergent and divergent validity, and diagnostic analyses in aecond sample consisting of participants with and without SAD.or this study, data collected at the prenatal measurement and theecond post-natal measurement, i.e. when the baby was 1 year old,ere used. At the prenatal measurement, the ADIS (Di Nardo et al.,

994; Wood, Piacentini, Bergman, McCracken, & Barrios, 2002)as administered to assess DSM-IV diagnoses including SAD, andarticipants filled out a questionnaire booklet including the abbre-iated SPAI, the SIAS (Mattick & Clarke, 1998), the SPS (Mattick

Clarke, 1998), the Blushing Trembling Sweating QuestionnaireBTSQ; Bögels & Reith, 1999), and the Risk Behaviour Scale (RBS;

eber, Blais, & Betz, 2002). At the second post-natal measurement,he SPAI-18 was again administered.

.1.3. Measures of convergent and divergent validity

.1.3.1. Social anxiety. In order to examine convergent validity ofhe SPAI-18, social anxiety was measured with the SIAS (Mattick

Clarke, 1998), the SPS (Mattick & Clarke, 1998), and the BTSQBögels & Reith, 1999). The SIAS and the SPS were developed as aair of social anxiety measures. The SIAS was developed to assessear of general social interaction, whereas the SPS to assess fear ofeing scrutinised. Both scales consist of 20 items each that are rated

able 6rincipal-components analysis on the SPAI-18 and internal consistency of the subscales o

Factor Number of items Item numbersa

Group without SAD diagnosisI. Distress in social situations

(Eigenvalue = 4.09)6 9, 12, 13, 14, 15

II. Focus of attention(Eigenvalue = 2.90)

4 3, 4, 5, 22

III; Avoidance/escape(Eigenvalue = 3.51)

4 7, 8, 24c, 25

IV. Cognitions and somaticreactions(Eigenvalue = 2.30)

4 26, 30d, 31, 32

Group with SAD diagnosisI. Distress in social situations

(Eigenvalue = 4.90)6 9, 12, 13, 14, 15

II. Focus of attention(Eigenvalue = 2.50)

4 3, 4, 5, 22

III. Avoidance and escape(Eigenvalue = 3.54)

4 7, 8, 24, 25

IV. Cognitions and somaticreactions(Eigenvalue = 2.41)

4 26e, 30 f, 31, 32

a Item numbers refer to the original numbers of the Social Phobia Subscale items of

ppendix A).b Cronbach’s alpha.c Item 24 also loaded .41 on factor I.d Item 30 also loaded .52 on factor III.e Item 26 also loaded .51 on factor I.

Item 30 also loaded .44 on factor I.

sity degree is required.up with SAD (t(296) = −2.26, p = .026).

on 5-points scales ranging from 0 = ‘not at all true/characteristic ofme’ to 4 = ‘extremely true/characteristic of me’. The BTSQ consists ofsix subscales designed to assess several aspects related to blushing,trembling, and sweating in social situations, including (1) fear, (2)physical symptoms, (3) avoidance, (4) behavioural problems, (5)cognitions, and (6) frequency. For this study, we administered thephysical symptoms subscale, which consists of 13 items referringto body parts that may show red, tremble, or sweat. Items are ratedon a 5-point scale ranging from 0 = ‘not at all applicable’ to 4 ‘verymuch applicable’.

3.1.3.2. Social risk taking. To examine divergent validity, we usedthe social risk taking subscale of the Risk Behaviour Scale (RBS) ofWeber et al. (2002). The RBS consists of 40 items that intend tomeasure the tendency to demonstrate risk behaviours in variousdomains, including gambling, ethical issues, health, social situa-tions, recreation, and finance. Participants rate the likelihood toengage in certain activities or behaviours on 5-point Likert scalesranging from 1 = ‘very unlikely’ to 5 = ‘very likely’. The social risktaking subscale consists of 8 items. Examples are: “Approaching

your boss to ask for a raise” or “Defending an unpopular issue thatyou believe in at a social occasion”. Since social risk taking can beseen as opposite to social anxiety, a negative correlation may beanticipated.

f the SPAI-18 in sample B.

Loading (range) Variance explained (%) ˛b

, 16 .66–78 22.7 .90

.58–.89 16.1 .83

.72–.86 19.5 .88

.43–.89 12.8 .82

, 16 .72–.86 27.2 .94

.47–.90 13.9 .76

.70–.87 19.7 .91

.53–.88 13.4 .80

the SPAI. Items were renumbered successively retaining their original order (see

W. de Vente et al. / Journal of Anxiety Disorders 28 (2014) 140–147 145

Table 7Measures of central tendency, dispersion and distributional characteristics of theSPAI-18 in sample B.

Total (N = 300) SPAI-18Non-SAD(n = 235)

SAD (n = 65)

Mean 32.75 27.80 50.64Median 30.36 27.00 50.75Range 0–95 0–64 16–95Standard deviation 15.49 12.23 12.58Skewness .47 .28 .26Kurtosis .17 −.10 2.04a

o

3

ieofs

3

glo3sfit(hp1acgf

(fcSw(

3

ckag

3

mascan

Table 8Correlation coefficients (Pearson’s r) between the SPAI-18 and convergent and diver-gent instruments.

Correlation coefficients N

Convergent instrumentsSIAS .83*** 289SPS .73*** 289BTSQ – somatic subscale .61*** 289

Divergent instrumentRBS – social risk taking subscale −.25*** 288

a The kurtosis was highly affected by one outlier (score 95). After exclusion of theutlier, the kurtosis was adequate: .66.

.2. Results

In the second validation phase, the SPAI-18 was administeredn its abbreviated form in a new sample. In this manner, possibleffects of fatigue due to the length of the questionnaire and/or itemrder effects that may occur when analysing a selection of itemsrom a larger pool, as was done in phase I, can be excluded, furthertrengthening the validity of the abbreviated version.

.2.1. Factor analysis and reliability analysisFactor analysis of the SPAI-18 resulted in 4 factors for the healthy

roup as well as the patient group (see Table 6), which is simi-ar to the factor solutions on the SPAI-18 in the healthy samplebtained in phase I (see Section 2.2.4). The items on cognitions (26,0), though, loaded together with somatic reactions (21, 32) on aingle factor in the healthy as well as the patient group. The fouractors explained 71.1% of variance in the healthy group and 74.2%n the patient group. All items had the highest factor loading onhe factor they were selected for by the experts, except item 30cognitions in social situations) in the healthy group, which loadedighest (.52) on factor III. For reasons of consistency between sam-les and with other literature (e.g. Olivares et al., 1999; Turner et al.,989), we decided to include item 30 in the factor scale cognitionsnd somatic reactions, for which its factor loading was also suffi-ient (.43). Using the criterion of .40, one other item in the healthyroup and two items in the patient group loaded on more than oneactor (see Table 6).

The internal consistency was excellent for the total SPAI-18Cronbach’s alpha = .93 in both groups) and sufficient to very goodor its subscales (see Table 6). Test–retest reliability was tested byorrelating the SPAI-18 score at the prenatal measurement with thePAI-18 score obtained more than one year later (i.e. when the babyas 1 year old). Pearson’s correlation coefficient was 0.85, p < .001

N = 233), indicating very good test–retest reliability.

.2.2. Descriptive informationMeasures of central tendency and dispersion and distributional

haracteristics are presented in Table 7. Since the skewness andurtosis are <|1|, it can be concluded that the distributional char-cteristics are very good. An exception was the kurtosis in the SADroup, which was about 2, a value still in the acceptable range.

.2.3. Convergent and divergent validityCorrelation coefficients between the SPAI-18 with convergent

easures (SIAS, SPS, BTSQ - physical symptoms subscale) and with divergent measure (RBS – social risk taking subscale) are pre-

ented in Table 8. As was expected, high statistically significantorrelations were obtained between the SPAI-18 and other socialnxiety measures, implying high convergent validity. A modestegative association was found between the SPAI-18 and the social

Note. SIAS, Social Interaction Anxiety Scale; SPS, Social Phobia Scale; BTSQ, Blushing,Trembling, Sweating Questionnaire; RBS, Risk Behaviour Scale.

*** p < .001.

risk taking subscale of the RBS, indicative of adequate divergentvalidity.

3.2.4. Diagnostic characteristicsThe mean score on the SPAI-18 was significantly higher in the

group diagnosed with SAD (M = 50.64, SD = 12.58) than in the groupwithout a SAD diagnosis (M = 27.83, SD = 12.26; t(df = 298) = 13.21,p < .001, d = 1.85). In order to assess the ability of the SPAI-18 to dis-criminate between healthy individuals and participants with a SADdiagnosis, we conducted a receiver operating characteristic (ROC)analysis. The area under the ROC curve was 0.91 (CI = 0.86–0.95;p < .001). The 95% confidence interval did not include 0.5, and there-fore, it is clear that the SPAI-18 performs considerably better thanchance in discriminating participants with SAD from individualswithout SAD diagnosis. Using the previously determined clinicalcut-off score of >48, the test performance indicators were as fol-lows: sensitivity: .68, specificity: .94, positive predictive value: .73,negative predictive value: .90. A more optimal cut-off in this sam-ple was a total score of the SPAI-18 > 36, which can be considered asa screening cut-off for non-clinical samples. The associated perfor-mance indicators were: sensitivity = .91; specificity = .74; positivepredictive value: .48; negative predictive value: .97; LR+: 3.50; LR−:.12; Youden index: .65.

4. Discussion

The SPAI-18 is a fundamentally solid, short version of the SPAIwith good psychometric and diagnostic qualities. Despite the con-siderable reduction of the number of items, the correlation betweenthe SPAI-18 and the Social Phobia Subscale of the original SPAI isvery high. The SPAI-18 also has good distributional features, highinternal consistency, even on a factor scale level, good convergentand divergent validity, excellent test–retest reliability, and goodsensitivity and specificity. Moreover, due to the 60% reduction inthe number of items, the questionnaire can now be completed inabout 8 min, which is a valuable time reduction, when comparedto the 20 min needed to complete the original SPAI.

A strong feature of the SPAI-18 is that the item selection wasbased on the originally defined affective, behavioural, cognitive,and somatic characteristics of social anxiety. These aspects werereflected in the factor structure in the clinically referred sample.In the non-referred samples, though, we found a 4-factor solu-tion to better fit the data. Some divergence in factor solutionscan be considered acceptable given the fact that social anxiety isexpressed differently in healthy individuals and patients, for exam-ple on the characteristic avoidance. For screening purposes, though,one would like to include, rather than exclude, aspects of socialanxiety that may show little variation in healthy individuals, but

are typical of patients (e.g. avoidance), despite the risk of some-what different factor solutions. Both 4- and 5-factor solutions forthe Social Phobia Subscale of the SPAI have been reported by othergroups (e.g. Olivares et al., 1999; Osman et al., 1995, 1996; Turner

1 nxiety

ePiwsOtficnistbsoS1tbsstHeS

srtmooPtroASlcs

Notaicitcbortosir2eBati

46 W. de Vente et al. / Journal of A

t al., 1989). The exact factor extraction of the original SPAI Socialhobia Subscale is somewhat unstable across studies. In some stud-es (Osman et al., 1995, 1996; Turner et al., 1989), separate factors

ere obtained for social anxiety in individual interactions versusocial anxiety in group interactions, which we and others (e.g.livares et al., 1999) did not replicate. In our study, factor struc-

ures of the SPAI-18 also differed slightly between samples. In therst non-clinical sample the avoidance and escape behaviour andognitive symptoms merged into one factor, while in the secondon-clinical sample, the somatic and cognitive symptoms merged

nto one factor. Since the factor structure that we obtained in theecond non-clinical sample was based on a larger N than the solu-ion obtained in the first non-clinical sample, more weight shoulde given to the second outcome. Moreover, a solution in which theomatic and cognitive symptoms merge has also been found byther research groups for the Social Phobia Subscale of the originalPAI (Olivares et al., 1999; Osman et al., 1995, 1996; Turner et al.,989), further strengthening the results. Of note, given the fact thathe cognitive and somatic factors were measured by a small num-ers of items, instability of these factors may occur more easily. Inum, like the original SPAI, the factor structure of the SPAI-18 isomewhat unstable across samples, which may in part be due tohe fact that two factors were measured by only two items each.owever, the same five originally defined aspects of social anxi-ty consistently emerged, supporting the construct validity of thePAI-18 as a comprehensive measure of social anxiety.

Also on other psychometric criteria, the SPAI-18 scores well, andimilar to the Social Phobia Subscale of the original SPAI. First, withespect to distributional features, skewness and kurtosis are closeo zero, indicating normally distributed outcomes, which supports

easuring the intended trait-like outcome of social anxiety. Sec-nd, convergent validity was as good for the SPAI-18 as for theriginal SPAI Social Phobia Subscale (e.g. Bögels & Reith, 1999;eters, 2000). Third, the SPAI-18 demonstrates very good sensi-ivity for detecting social anxiety disorder (>48 to detect clinicallyeferred cases of SAD and >36 to detect non-clinically referred casesf SAD), which is an important feature of a screening instrument.s may be expected, in order to obtain a good sensitivity to detectAD in the community sample, the cut-off score was somewhatower than the optimal clinical cut-off score to detect SAD in thelinical versus healthy sample. Consequently, specificity was alsoomewhat lower, though still sufficient, in the community sample.

Although other groups (Garcia-Lopez et al., 2008; Roberson-ay et al., 2007) have already published about screening versionsf the SPAI (SPAI-23 and SPAI-B, respectively), we are convincedhat the SPAI-18 has advantages over these other measures. First,

drawback of the SPAI-23 (social-phobia subscale) and the SPAI-Bs that the items were not selected to specifically reflect the fiveentral aspects of social anxiety. To illustrate, the majority of thetems of the SPAI-23 and the SPAI-B concern distress in social situa-ions (either individually or in groups) and distress when being theentre of attention. Cognitive and somatic symptoms are assessedy a single item each in both the SPAI-23 and the SPAI-B. More-ver, in contrast to the SPAI-18, the SPAI-23 does not include itemseferring to avoidance or escape of an anxiety provoking situa-ion and the SPAI-B has only one item covering this domain. Inther words, the instruments do not assess behavioural anxietyymptoms specifically. The importance of including behaviour isllustrated by the fact that avoidance is one of the diagnostic crite-ia for social anxiety disorder (American Psychiatric Association,000). Moreover, etiological models assume that the tendency toscape or avoid threat maintains and/or exacerbates anxiety (e.g.

arlow, 2002). In conclusion, the selection of items for the SPAI-23nd the SPAI-B was primarily based on item response characteris-ics. While it should be noted that these instruments do containtems referring to different social anxiety factors as defined by

Disorders 28 (2014) 140–147

the founders Turner et al. (1989), the coverage is not completeand somewhat unbalanced. Consequently, these instruments mayless optimally assess the phenomenology of social anxiety. Anotheradvantage of the SPAI-18 is the retention of subitems referring tospecific others (e.g. authority figures), in order to preserve poten-tial clinically relevant information. Hence, the SPAI-18 may be moreinteresting for use in clinical settings than the SPAI-23 or the SPAI-B.

When considering the results of the present study, it shouldbe noted that the included samples were mainly of Dutch originwith a relatively high educational background, thereby limitinggeneralisation of the results to groups with other backgrounds.Although our results support good reliability, validity, and diag-nostic quality of the SPAI-18 as a screening instrument for socialanxiety disorder, further psychometric analyses are needed toestablish the quality of the instrument. For example, the ability ofthe SPAI-18 to differentiate between SAD and other anxiety disor-ders remains to be assessed. Second, it may be assessed whetherthe SPAI-18, like the original Social Phobia Subscale, is still valid todetect clinical change. Third, future studies may examine whetherthe SPAI-18 on the subscale level provides sufficient clinically validinformation about individual patients’ expressions of social anxi-ety.

In sum, we developed a fundamentally solid, short version of theSPAI, named SPAI-18, which has good psychometric and diagnosticqualities. Due to the substantial reduction of the number of items,the questionnaire can now be completed in about 8 minutes. Thecorrelation between the SPAI-18 and the Social Phobia Subscale ofthe original SPAI is impressively high. This high association and thegood psychometric and diagnostic properties of the SPAI-18 sug-gest that the SPAI-18 can potentially serve other goals, apart fromscreening for social anxiety disorder, such as analysing differentexpressions of social anxiety.

Acknowledgements

This study was funded by an Innovation Research Vidi NWOgrant, number 452-05-345, to the last author. The authors grate-fully acknowledge Dr. Agnes Scholing (University of Amsterdam),Prof. Dr. Michiel Westenberg (Leiden University), and Prof. Dr. PaulEmmelkamp (University of Amsterdam) for their expert judge-ments and selection of the items of the SPAI. We also like to thankSarah de Schutter, Annefleur Visscher, Jasmijn Rahder, and HesterVerwey for their work in the recruitment and data-collection phase.

Appendix A. The SPAI-18: Item selection and reference tonumbering of the Social Phobia (SP) Subscale of the originalSPAI

Item nrSPAI-18

Item nr SP subscaleoriginal SPAI

Item theme

1 3 Centre of attention2 4 Expected to participate in activity3 5 Holding a speech4 7 Avoidance of social situations5 8 Leave social situations6 9 Tension in small groups7 12 Tension in new social situations8 13 Tension when confronted with others9 14 Embarrassing situation10 15 Sharing intimate feelings11 16 Giving opinion12 22 Talk in front of others13 24 Attempt avoiding social situations14 25 Departing from social situations

15 26 Anticipatory cognitions16 30 Disturbing cognitions17 31 Anticipatory somatic symptoms18 32 Somatic symptoms

nxiety

R

A

B

B

B

B

B

B

C

C

D

D

F

F

G

G

G

G

W. de Vente et al. / Journal of A

eferences

merican Psychiatric Association. (2000). Diagnostic and statistical manual of mentaldisorders: DSM-IV-TR. Washington, DC: Author.

arlow, D. H. (2002). Anxiety and its disorders: the nature and treatment of anxietyand panic (2nd ed., pp. ). New York: Guilford Press.

eidel, D. C., Turner, S. M., & Cooley, M. R. (1993). Assessing reliable and clinicallysignificant change in social phobia: validity of the social phobia and anxietyinventory. Behaviour Research and Therapy, 31, 331–337.

eidel, D. C., Turner, S. M., Stanley, M. A., & Dancu, C. V. (1989). The social phobiaand anxiety inventory: concurrent and external validity. Behavior Therapy, 20,417–427.

ögels, S. M. (2006). Task concentration training versus applied relaxation, in com-bination with cognitive therapy, for social phobia patients with fear of blushing,trembling, and sweating. Behaviour Research and Therapy, 44, 1199–1210.

ögels, S. M., & Reith, W. (1999). Validity of two questionnaires to assess social fears:the Dutch Social Phobia and Anxiety Inventory and the Blushing, Trembling andSweating Questionnaire. Journal of Psychopathology and Behavioural Assessment,21, 51–66.

ögels, S. M., & Voncken, M. J. (2013). Mindfulness training versus cognitive therapyfor social anxiety disorder (submitted for publication).

onnor, K. M., Davidson, J., Churchill, L. E., Sherwood, E., & Weisler, R. H. (2000).Psychometric properties of the Social Phobia Inventory (SPIN). British Journal ofPsychiatry, 176, 379–386.

onnor, K. M., Kobak, K. A., Churchill, L. E., Katzelnick, D., & Davidson, J. (2001).Mini-SPIN: a brief screening assessment for generalized social anxiety disorder.Depression and Anxiety, 14, 137–140.

e Vente, W., Majdandzic, M., Colonnesi, C., & Bögels, S. (2011). Intergenerationaltransmission of social anxiety: the role of paternal and maternal fear of negativechild evaluation and parenting behaviour. Journal of Experimental Psychopathol-ogy, 2, 509–530.

i Nardo, P. A., Brown, T. A., & Barlow, D. H. (1994). Anxiety Disorders InterviewSchedule for DSM-IV. Lifetime version (ADIS-IV-L). San Antonio, TX: PsychologicalCorporation.

irst, M. B., Spitzer, R. L., Gibbon, M., & Williams, B. W. (1996). Structured ClinicalInterview for DSM-IV Axis I Disorders, Patient edition (SCID-I/P, version 2.0). NewYork: Biometrics Research Department.

urmark, T. (2002). Social phobia: overview of community surveys. Acta PsychiatricaScandinavica, 105, 84–93.

arcia-Lopez, L. J., Hidalgo, M. D., Beidel, D. C., Olivares, J., & Turner, S. (2008).Brief form of the Social Phobia and Anxiety Inventory (SPAI-B) for adolescents.European Journal of Psychological Assessment, 3, 150–156.

arcía-López, L. J., Olivares, J., & Hildago, M. D. (2005). A pilot study on sensitiv-ity of outcome measures for treatment of generalized social phobia in Spanishadolescents. International Journal of Clinical and Health Psychology, 5, 385–392.

arcía-López, L. J., Olivares, J., Hildago, M. D., Beidel, D. C., & Turner, S. M. (2001).Psychometric properties of the Social Phobia and Anxiety Inventory, the SocialAnxiety Scale for adolescents, the Fear of Negative Evaluation Scale, and theSocial Avoidance and Distress Scale in an adolescent Spanish-speaking sample.Journal of Psychopathology and Behavioural Assessment, 23, 51–59.

roenestijn, M. A., Akkerhuis, G. W., Kupka, R. W., Schneider, H., & Nolen, W. A.(1997). (Dutch translation of the Structural Clinical Interview for DSM-IV AxisI Disorders) Gestructureerd Klinisch Interview voor de vaststelling van DSM-IVAs I Stoornissen, Nederlandse vertaling, SCID-I. Lisse, the Netherlands: Swets &Zeitlinger.

Disorders 28 (2014) 140–147 147

Heinrichs, N., Hahlweg, K., Fiegenbaum, W., Frank, M., Schröder, B., & von Witzleben,I. (2002). Validität und Reliabilität der Social Interaction und Anxiety Scale (SIAS)und der Social Phobia Scale (SPS) [Validity and reliability of the Social Interactionand Anxiety Scale (SIAS) and the Social Phobia Scale (SPS)]. Verhaltenstherapie,12, 26–35.

Herbert, J. D., Bellack, A. S., & Hope, D. A. (1991). Concurrent validity of the social pho-bia and anxiety inventory. Journal of Psychopathology and Behavioral Assessment,13, 357–368.

Lobbestael, J., Leurgans, M., & Arntz, A. (2011). Inter-rater reliability of the StructuredClinical Interview for DSM-IV Axis I Disorders (SCID I) and Axis II Disorders (SCIDII). Clinical Psychology and Psychotherapy, 18(1), 75–79.

Majdandzic, M., de Vente, W., & Bögels, S. (2010). Rearing histories of individualswith and without social anxiety who become first time parents. Anxiety Stressand Coping, 23, 243–258.

Mattick, R. P., & Clarke, J. C. (1998). Development and validation of measures ofsocial phobia scrutiny fear and social interaction anxiety. Behaviour Researchand Therapy, 36, 455–470.

Olivares, J., García-Lopez, L. J., Hildalgo, M. D., Turner, S. M., & Beidel, D. C. (1999).The Social Phobia and Anxiety Inventory: reliability and validity in an adolescentSpanish population. Journal of Psychopathology and Behavioural Assessment, 21,67–78.

Olivares, J., Garcia-Lopez, L. J., Hidalgo, M. D., & Caballo, V. (2004). Relationshipsamong social anxiety measures and their invariance: a confirmatory factor anal-ysis. European Journal of Psychological Assessment, 20, 172–179.

Osman, A., Barrios, F. X., Aukes, D., & Osman, J. R. (1995). Psychometric evaluationof the Social Phobia and Anxiety Inventory in college students. Journal of ClinicalPsychology, 51, 235–243.

Osman, A., Barrios, F. X., Haupt, D., King, K., Osman, J. R., & Slavens, S. (1996).The Social Phobia and Anxiety Inventory: further validation in two nonclinicalsamples. Journal of Psychopathology and Behavioural Assessment, 18, 35–47.

Peters, L. (2000). Discriminant validity of the Social phobia and Anxiety Inventory,the Social Phobia Scale (SPS) and the Social Interaction Anxiety Scale (SIAS).Behaviour Research and Therapy, 38, 943–950.

Roberson-Nay, R., Strong, D. R., Nay, W. T., Beidel, D. C., & Turner, S. M.(2007). Development of an abbreviated Social Phobia and Anxiety Inventory(SPAI) using item response theory: the SPAI-23. Psychological Assessment, 19,133–145.

Scholing, A., Bögels, S., & van Velzen, C. (1995). Angst in sociale situaties (SPAI-N).Authorized translation of the Social Phobia and Anxiety Inventory developed byS. M. Turner, C. V. Dancu, D. C. Beidel (1988).

Stein, M. B., & Stein, D. J. (2008). Social anxiety disorder. Lancet, 371, 1115–1125.Turner, S. M., Stanley, M. A., Beidel, D. C., & Bond, L. (1989). The Social Phobia

and Anxiety Inventory: construct validity. Journal of Psychological and BehavioralAssessment, 11, 221–234.

Turner, S. M., Beidel, D. C., & Dancu, C. V. (1996). Social Phobia and Anxiety Inventorymanual. North Tonawanda, NY, USA: Multi Health Systems.

Voncken, M. J., Bögels, S. M., & de Vries, K. (2003). Interpretation and judgmentalbiases in social phobia. Behaviour Research and Therapy, 41(12), 1481–1488.

Weber, E. U., Blais, A. R., & Betz, N. E. (2002). A domain-specific risk-attitude scale:measuring risk perceptions and risk behaviours. Journal of Behavioral Decision

Making, 15, 263–290.

Wood, J. J., Piacentini, J. C., Bergman, R. L., McCracken, J., & Barrios, V. (2002). Concur-rent validity of the Anxiety Disorders Section of the Anxiety Disorders InterviewSchedule for DSM-IV: child and parent versions. Journal of Clinical Child andAdolescent Psychology, 31, 335–342.