Patient and Therapist Perspectives on AllianceDevelopment: Therapists’ Practice Experiencesas Predictors

Helene A. Nissen-Lie,1* Odd E. Havik,2 Per A. Høglend,3 Michael Helge Rønnestad1

and Jon T. Monsen1

1Department of Psychology, University of Oslo, Oslo, Norway2Department of Clinical Psychology, University of Bergen, Bergen, Norway3 Institute of Clinical Medicine, University of Oslo, Oslo, Norway

Research has shown that the therapist’s contribution to the alliance is more important for the outcomethan the patient’s contribution (e.g., Baldwin, Wampold, & Imel, 2007); however, knowledge is lackingabout which therapist characteristics are relevant for alliance building and development. The objectiveof this study was to explore the development of the working alliance (using the Working AllianceInventory), rated by both patients and therapists as a function of therapist in-session experiences.The therapist experiences were gathered by means of the Development of Psychotherapists CommonCore Questionnaire (Orlinsky & Rønnestad, 2005). Data from the Norwegian Multisite Study of theProcess and Outcome of Psychotherapy (Havik et al., 1995) were used. Multilevel growth curve analysesof alliance scores from Sessions 3, 12, 20 and 40 showed that the therapist factors predicted workingalliance levels or growths differently, depending on whether the alliance was rated by patients or bytherapists. For example, it emerged that therapists’ negative reactions to patients and their in-sessionanxiety affected patient-rated alliance but not therapist-rated alliance, whereas therapist experiences of flow(Csikszentmihalyi, 1990) during sessions impacted only the therapist-rated alliance. The patterns observed inthis study imply that therapists should be particularly aware that their negative experiences of therapy arenoticed by, and seem to influence, their clients when they evaluate the working alliance through the courseof treatment. Copyright © 2014 John Wiley & Sons, Ltd.

Key Practitioner Message:• The findings of this study suggest that the working alliance is influenced by therapists’ self-reported prac-

tice experiences, which presumably are communicated through the therapists’ in-session behaviours.• The study found a notable divergence between practice experiences that influenced the therapists and

those that influenced the patients when evaluating the working alliance.• Specifically, practitioners’ self-reported difficulties in practice, such as their negative reactions to

patients and their in-session anxiety, affected patient-rated alliance but not therapist-rated alliance,whereas therapist experiences of ‘flow’ during sessions impacted only the therapist-rated alliance.

• Practitioners should note that patient alliance ratings were more likely to be influenced by therapists’negative practice experiences than by positive ones.

• The divergence in the patient and therapist viewpoints has potential implications for therapist trainingand supervision and everyday self-reflection.

Keywords: Therapist Effects, Therapist Factors, Alliance Development, Patient and Therapist Perspectives,Multilevel Growth Curve Modelling

INTRODUCTION

Empirical findings demonstrate that the therapist’s contri-bution to the therapist–patient alliance is more important

for the outcome than the patient’s contribution (e.g.,Baldwin et al., 2007; Del Re et al., 2012). However, the roleof therapist characteristics in influencing the alliance hasbeen the focus of fewer studies and is less well understoodthan the role of patient characteristics. Moreover, existingstudies typically measure alliance quality only at singlepoints in time rather than following the development ofthe alliance across the course of treatment. Indeed, those

*Correspondence to: Helene A. Nissen-Lie, University of Oslo, P.O.Box 1094, Blindern, 0317 Oslo, Norway.E-mail: h.a.nissen-lie@psykologi.uio.no

Clinical Psychology and PsychotherapyClin. Psychol. Psychother. (2014)Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/cpp.1891

Copyright © 2014 John Wiley & Sons, Ltd.

investigations that do explore alliance development ratherthan one-point evaluations demonstrate closer links be-tween therapist-rated alliance development and outcome,compared with patient-rated alliance development andoutcome (Kivlighan & Shaughnessy, 1995; Kramer et al.,2008). This is at odds with the established view that thepatient’s perception of the alliance is more valuable thanthe therapist’s.The aim of the present study was to explore how thera-

pist factors contribute to alliance quality over time, asperceived by both patients and therapists.

Perspectives on the Working Alliance

The working alliance in contemporary research is com-monly conceptualized in line with Bordin’s (1979, 1994)model as consisting of an emotional bond between thera-pist and patient (reflecting mutual trust, liking and appre-ciation), as well as agreement about the tasks and goals ofthe treatment. In a research synthesis, Horvath, Del Re,Flückiger, and Symonds (2011) concluded that the impactof the alliance on outcome ‘is ubiquitous irrespective ofhow the alliance is measured, from whose perspective itis evaluated, when it is assessed, the way the outcome isevaluated, and the type of therapy involved. The qualityof the alliance matters’ (p. 13).Despite the explicit or implicit ideal of the therapist

being attuned to his or her patients when evaluating thealliance, the research literature indicates that the ‘client–therapist perspective divergence may be the rule ratherthan the exception’ (Tryon, Blackwell, & Hammel, 2007,p. 630). Horvath (2000) pointed to the different observa-tional stances and roles of patients and therapists to expli-cate this divergence. Patients are influenced by theirprevious interpersonal experiences that become repre-sented in, or transferred to, the therapeutic relationship,whereas therapists rate the alliance according to theirtheoretical assumptions and clinical experience (Horvath,2000; Kramer et al., 2008). Although such a distinction isclearly relevant, therapists may also be influenced by theirown relationship patterns and by their personal issues,which, in conjunction with their professional expertise,probably shape their perception of the alliance. This prop-osition was examined in the present study.

Therapist Characteristics Impacting theWorkingAlliance

In their reviews on therapist characteristics and theworking alliance, Ackerman and Hilsenroth (2001, 2003)summarized the finding that therapists perceived by theirpatients as being empathic, warm, trustworthy, confident,open, honest and interested achieved higher patient-ratedalliance scores than therapists who came across as dis-tanced, rigid, critical, tense and distracted. In addition,

interventions such as structuring therapeutic work toomuch or too little, inappropriate transference interpreta-tions, silence or self-disclosures associated negatively withalliance ratings, whereas, e.g., facilitating in-session emo-tional expression seems to have a beneficial effect on theworking alliance.With regard to more trait-like therapist dispositions, one

study exploring therapist attachment orientations usingthe Adult Attachment Interview methodology found thatanxiously attached therapists quite unexpectedly receivedhigher patient-rated alliance scores after the first therapysession. However, over time, the effect of this dimensionwas reversed (Sauer, Lopez, & Gormley, 2003). Perhapsas a result of their heightened attachment sensitivity, thesetherapists may be more able to perceive and respond tosubtle variations in their patients’ initial interpersonalbehaviours, but this underlying apprehension turns outto have less positive effects in the long run. In a relatedstudy, Dinger, Strack, Sachsse, and Shauenburg (2009)used the Adult Attachment Interview conducted by expertinterviewers to assess the impact of therapist attachmentrepresentations on therapy process and outcome. Theyfound that more preoccupied therapists achievedpoorer alliances, but only so with more interpersonallydistressed patients.In their investigations examining the same patient and

therapist sample used in the current study, Hersoug,Høglend, Monsen, and Havik (2001) and Hersoug et al.(2009) observed a negative impact of therapists’ self-reported interpersonal problems of the cold/detached kind,as measured by the 64-item Inventory of InterpersonalProblem (IIP-64; Horowitz, Alden, Wiggins, & Pincus,2000), on both patient-rated and therapist-rated alliance.Hersoug and colleagues also discovered that some of thetherapist characteristics examined exerted a differentialimpact on the working alliance, depending on the rater’sperspective. For instance, patient alliance evaluations wereinfluenced by therapists’ reports of their mother’s care(measured by the Parental Bonding Instrument) and bytheir professional post-graduate training, whereas therapistevaluations of the alliance were not influenced by eithervariable (Hersoug et al., 2009). Also, a recent study suggeststhat the working alliance is influenced by the therapists’quality of life, but in divergent wayswhen rated by patientsor by therapists (Nissen-Lie, Havik, Høglend, Monsen &Rønnestad, 2013). Therapists’ personal burdens were foundto be strongly and inversely related to the growth of thealliance as rated by patients but were unrelated to thera-pist-rated alliance. Conversely, therapists’ personal satis-factions were positively associated with therapist-ratedalliance growth but were unrelated to patient ratings ofthe alliance. These dimensions were stable over a periodof 6 years.Another study on therapist feelings found that thera-

pists’ emotional reactions towards clients measured by

H. Nissen-Lie et al.

Copyright © 2014 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. (2014)

the Ratings of Emotional Attitudes to Clients by Treaters(REACT) scale (Najavits et al., 1994) were strongly relatedto various process measures such as the working alliance,in cases of clients suffering from alcohol or drug depen-dence (Wolff & Hayes, 2009). Therapists with negativeemotional reactions towards clients received lower allianceratings from their clients. These findings supplement theobservations of the seminal Vanderbilt studies (e.g., Henry,Schacht, & Strupp, 1990), in which therapists’ negativeinterpersonal behaviours, such as subtly hostile and con-trolling statements, were highly correlatedwith the numberof self-blaming statements by the patients. The Vanderbiltstudies also found that therapists with disaffiliativeintrojects tended to engage in a higher level of problematicinterpersonal processes during sessions, which in turn wereassociated with poorer outcome in patients.

Therapists’ Self-reports: Findings from the Society forPsychotherapy Research (SPR) CollaborativeResearch Network

The present study forms part of the large-scale Interna-tional Study of the Development of Psychotherapists(ISDP) run by the SPR Collaborative Research Network(Orlinsky & Rønnestad, 2005). This programme hasdeveloped a comprehensive survey instrument, theDevelopment of Psychotherapists Common Core Questionnaire(DPCCQ) (Orlinsky et al., 1999), to measure aspects oftherapists’ self-assessed professional and personal func-tioning. Recently, the instrument has been implementedto study some therapist characteristics in relation topsychotherapy processes and outcomes in a variety oftreatment settings. In one of these studies, a clinical trialof short-term versus long-term therapies provided topatients suffering from depressive and/or anxiety disor-ders in Finland, Heinonen et al. (2013) investigatedtherapist characteristics measured by the DPCCQ as predic-tors of the formation and development of patient-rated andtherapist-rated working alliances. Their findings indicatethat a number of the concepts that the DPCCQ measuresare relevant in determining alliance quality. One of theseconcepts, therapists’ self-rated basic interpersonal skills,was found to predict the formation of better patient-ratedalliances in both short-term and long-term therapies,whereas other factors were found to relate differently tothe alliance, depending on treatment length. Therapists’self-assessed interpersonal styles, characterized by activelyengaging and encouraging patients, were associated witha better alliance in short-term treatments but were relatedto alliance deterioration in long-term therapies, such aspsychoanalysis. Furthermore, therapists’ professional self-confidence consistently predicted their alliance ratings butwas not predictive of patient alliance ratings.

Previous studies in a Norwegian naturalistic outpatientsetting also found strong positive and negative associationsbetween therapists’ subjective experiences of therapy (partof their ‘work involvement styles’ gathered from theDPCCQ scales) (Orlinsky & Rønnestad, 2005) and earlypatient-rated alliance (Nissen-Lie, Monsen, & Rønnestad,2010), as well as patient outcome (Nissen-Lie, Monsen,Ulleberg & Rønnestad, 2013). Even if the second-orderfactor scales of the work involvement styles did not predictearly alliance, some of their sub-dimensions did. Forexample, one factor denoting therapist difficulties in prac-tice such as lack of empathy and tolerance of the emotionalneeds of patients, irritability towards patients and so on,called ‘negative personal reaction’ (NPR), was negativelyrelated to alliance quality as rated by the patients. Bycontrast, difficulties called ‘professional self-doubt’ (PSD)were associatedwith higher patient alliance ratings andmorechanges in interpersonal distress in patients. Analyses of thetemporal stability of these factors showed that theywere sur-prisingly stable across a 6-year time span, indicating to someextent that they are more trait-like dispositions, rather thancontext dependent (Nissen-Lie et al., 2010). In sum, the find-ings of these recent studies suggest that therapists rate theirpractice experiences in reliable ways and that these DPCCQfactors measure traits that are relevant for the way theyworkwith patients in the therapy room. The current study extendsthe previous study on early alliance by analysing the work-ing alliance over time (i.e., alliance development) and by in-cluding therapist alliance ratings to compare with patientalliance ratings.

Research Questions

The current investigation examined the following researchquestions:

1. Is there a relationship between alliance quality over thecourse of therapy and the following therapist self-reported characteristics: (a) work involvement styles;(b) difficulties in practice; and (c) in-session feelings?

2. Do these relationships depend on whether the allianceis rated by patients or by therapists?

METHODS

Design

The current study used data from the Norwegian MultisiteStudy of the Process and Outcome of Psychotherapy(NMSPOP) (Havik et al., 1995). The NMSPOP is a naturalis-tic outpatient psychotherapy study following 370 patientstreated at 15 public outpatient clinics at eight different siteswithin the public healthcare system in Norway. The

Therapist Factors and Alliance Development

Copyright © 2014 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. (2014)

inclusion policy was liberal: only patients with serioussubstance abuse, acute crises requiring hospitalization andpsychoses were excluded from the study.

Participants

PatientsThe patients in the NMSPOP (n=370) were referred to or-

dinary public outpatient clinics for non-selected types ofclinical symptoms and disorders. They suffered relativelysevere and complex mental disturbances (half the samplesuffered from personality disorders, and 63% were severelydistressedwith a Global Severity Index (GSI) ≥ 0.97; Tingey,Lambert, Burlingame, & Hansen, 1996). The analyses of pa-tient-rated alliance scores presented later were conducted ona subsample of patients (n=227) who had provided aminimum of three measurements of alliance ratings(Section on Procedure), in order to allow for growth curvemodelling (Hox, 2002, 2010).

In this study sample of 227 patients, there were 169(74.4%) women and 58 (25.6%) men. Ages ranged from18 to 65years with a mean of 35.7years (standard deviation[SD] =9.5years). Their most frequent Axis 1 diagnoses wereanxiety disorders (67%) and affective disorders (55.9%), and50.2% filled the criteria for at least one personality disorder(29.2% from Cluster A, 20.4% from Cluster B and 84.1% fromCluster C; 2.7% were non-specified personality disorders1).The level of psychosocial functioning at baseline, asmeasuredby the Global Assessment of Functioning scale (Endicott,Spitzer, Fleiss, & Cohen, 1976), ranged from 20 to 85, with amean of 57.6 (SD=8.9). The number of therapy sessions re-ceived by this subsample of patients ranged from 20 sessionsto a maximum of 364, with a mean of 74 (SD=65.2). Forty-eight per cent of patients received between 20 and 40 sessions,whereas the remaining 52% received more than 40 sessions.

As with the patient-rated alliance scores (n= 227), thetherapist-rated alliance scores for a given patient thatcounted at least three or more ratings (up to a maximumof four) were selected to model growth curves. The num-ber of patients for whom at least three therapist-rated alli-ance scores had been obtained was 162. The therapistsbegan rating the alliances about a year into the project,which explains why a lower number of alliance ratingswere acquired from the therapists than from the patients.

TherapistsThe therapist sample in the study comprised 70 psycho-

therapists: 46 psychologists, 14 psychiatrists, eightphysiotherapists and two psychiatric nurses. Their meanlevel of experience in practising psychotherapy was10.0 years (SD= 6.57 years), ranging from 0 to 28 years.

The therapists’ average patient caseload was almost fivepatients, ranging from 1 to 10. A 5-point Likert scale wasused to measure the degree to which the therapists wereinfluenced by various theoretical orientations, rangingfrom 1 (not at all) to 5 (very greatly). A large proportionof the therapists in the sample (78.3%) reported apsychoanalytic/psychodynamic salient orientation of fouror more. Further, a considerable proportion reported to besaliently influenced by other theories, mainly humanistic(31%) and cognitive (29.4%) orientations.

Treatments

The treatments offered were non-manualized and represen-tative of psychotherapy as practised in outpatient clinics inmental health institutions in Norway. Seven of the eightsites offered open-ended individual therapy, and oneoffered time-limited (40 session) psychodynamic psycho-therapy sessions. Two sites provided affect consciousnesstreatments (Monsen & Monsen, 1999). The treatments wereinfluenced mainly by psychodynamic treatment models,although the therapists were influenced by a variety oftreatment orientations (Section on Therapists).

Measures

Therapist MeasuresWork involvement styles. The model for work involvementstyles is divided into two broad dimensions: ‘healinginvolvement’ and ‘stressful involvement’. Healing involve-ment comprises the following first-order factors fromvarious DPCCQ subscales: ‘invested relational agency’,‘affirming’ and ‘accommodating’ interpersonal styles, in-session feelings of ‘flow’, use of ‘constructive coping’ andpossession of ‘basic relational skills’. Stressful involvement,however, is composed of the following factors: frequentdifficulties, in-session feelings of ‘anxiety’ and ‘boredom’and unconstructive coping called ‘avoiding therapeuticengagement’ (Orlinsky & Rønnestad, 2005). Both healinginvolvement and stressful involvement factors yieldedexcellent internal consistencies in the present sample(α=0.87 and 0.81, respectively). The work involvementstyle factors demonstrated good convergent and discrimi-nant validity in terms of meaningful associations withexternal correlates such as the extensiveness of therapeuticexperience, theoretical breadth, level of perceived worksupport, experience of personal therapy and of supervision,as well as various therapist characteristics (Orlinsky &Rønnestad, 2005; Zeek et al., 2012).

Difficulties in practice. Therapists’ experiences of difficul-ties were measured using 21 statements describing variouschallenges in psychotherapy work, identified in a thoroughqualitative research process conducted in communion by a1Note that an overlap between clusters was possible.

H. Nissen-Lie et al.

Copyright © 2014 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. (2014)

group of clinicians (Davis et al., 1987). The statements followthe question, ‘Presently, how often do you feel…’, andresponses were made on a 6-point Likert-type scale rangingfrom 0 (never) to 5 (very often). Former factor analyses on thepresent sample (Nissen-Lie et al., 2010) established twofactors from the scale, referred to as PSD and NPR, as inthe large-scale international study (Orlinsky & Rønnestad,2005). PSD includes nine items that reflect doubts aboutprofessional efficacy in treating patients, such as ‘Lackingin confidence that you might have a beneficial effect on apatient’ and ‘Unsure how best to deal effectively with apatient’. NPR comprises six items reflecting deficient empa-thy and hostile feelings towards patients, for instance‘Irritatedwith a patient who is actively blocking your effort’and ‘Unable to have much real empathy for a patient’sexperiences’. Both the PSD and NPR obtained excellent reli-ability estimates in the current sample of therapists (α=0.90and 0.84, respectively) (Nissen-Lie et al., 2010).

In-session feelings. Therapists’ self-reported feelings inpsychotherapy sessions were measured using a 12-itemquestionnaire containing single-worded items based onthe theory of Csikszentmihalyi (1990) in response to thequestion: ‘Lately, how often in the work with clients haveyou felt…?’ These items were rated on a Likert-type scalefrom 0 (never) to 4 (very often). As with the other relevantscales, this scale also underwent factor analyses (usingthe principal axis extraction method with promax rota-tion2) on the present therapist sample. This procedureuncovered a three-factor structure, which correspondedwell with the ISDP findings (Orlinsky & Rønnestad,2005), except that one item (challenged) could not beincluded due to a too low and mixed loading pattern. Thisfactor solution explained 55.2% of the total variance in theitem scores. The three factors were labelled as in the ISDP:anxiety (containing the following items: overwhelmed,

pressured, anxious and trapped), flow (inspired, stimulatedand engrossed) and boredom (bored, absent, inattentive anddrowsy). The alpha scores of these three factors were0.80, 0.62 and 0.60, respectively.The therapist factors used in the study have been repro-

duced across different samples (Orlinsky & Rønnestad,2005). This measurement equivalence seems to reflect theconstruct validity of the various DPCCQ scales. Severalof these factors demonstrate predictive power in psy-chotherapy process and outcome obtained from variousobservational perspectives (e.g., Heinonen et al., 2013;Nissen-Lie et al., 2010; Zeek et al., 2012).Table 1 presents descriptive statistics and inter-correlations

of the therapist variables used in this study.

Working alliance. The short version of the Working AllianceInventory (WAI), in both the patient-rated and therapist-rated versions (Horvath & Greenberg, 1989), comprising12 items (Tracey & Koskotovic, 1989), was used to evaluatetheworking alliance in this study. TheWAIwas constructedto measure the degree of collaboration between therapistand patient in terms of agreement about the tasks and goalsof therapy and the quality of the emotional bond betweenthe two, in accordance with the three aspects of Bordin’s(1979) alliance model. In the WAI, respondents are askedto rate the extent to which they feel certain statementsdescribing the relationship and collaboration are true (e.g.,‘I believe my therapist likes me, or I am confident in mytherapist’s ability to help me’) on a 7-point Likert scale from1 (never) to 7 (always). Both the longer 36-item version andthe shorter 12-item version of the WAI scale have demon-strated temporal stability and high internal consistencyand have been shown to possess good convergent anddiscriminant validity. The reliable association with psycho-therapy outcome has been suggested to be an indicator ofthe external or criterion validity of the WAI (Horvath &Greenberg, 1989; Horvath, 1994).The scale’s construct validity has been found to correspond

to Bordin’s original tripartite alliance model (Horvath &Greenberg, 1989); however, the three dimensions show

2This method was chosen to allow for expected inter-correlationsamong the resulting factors.

Table 1. Descriptives and inter-correlations of therapist factors

Therapist factors M SD Min Max 1 2 3 4 5 6 7

1. Healing involvement 10.97 1.41 7.12 14.40 —2. Stressful involvement 3.72 1.45 0.36 8.27 �0.21 —3. PSD 1.24 0.70 0.11 3.78 �0.38** 0.83** —4. NPR 0.80 0.55 0.00 3.33 �0.18 0.61** 0.58** —5. Flow 2.39 0.45 1.33 3.00 0.67** �0.22 �0.37** �0.15 —6. Boredom 0.86 0.38 0.00 1.5 �0.15 0.60** 0.35** 0.16 �0.17 —7. Anxiety 0.84 0.55 0.00 2.75 �0.10 0.81** 0.72** 0.46** �0.10 0.25* —

N= 70.PSD=professional self-doubt. NPR=negative personal reaction.**Correlation is significant at the 0.01 level (two tailed).*Correlation is significant at the 0.05 level (two tailed).

Therapist Factors and Alliance Development

Copyright © 2014 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. (2014)

strong scale inter-correlations. To test the factor structure ofthe WAI, Tracey and Koskotovic (1989), employing confir-matory factor analyses, showed that the best model fit ofWAI scores was achieved for a hierarchical two-level factorstructure with a higher-order global alliance factor and threefirst-order factors (corresponding to task, goal and bond).The existence of a general second-order alliance dimensionhas led to a common procedure of employing an averagedsum score as the alliance index in several studies, includingthis one.

Patient Control Variable

Baseline interpersonal problemsPatients’ level of interpersonal problems prior to treat-

ment represented the patient control measure in the study.Including this variable in the models permitted control-ling for caseload differences between therapists on adimension that has been found to negatively influencepatients’ ability to form an alliance (Nissen-Lie et al.,2010; also Constantino & Smith-Hansen, 2008). The effectof the therapist variables on alliance was thus tested fora patient with an average level of interpersonal distress.Interpersonal distress was assessed using the circumplexversion of the IIP-64 (Horowitz et al., 2000). Each itemwas rated on a 5-point scale ranging from 0 (not at all) to4 (extremely). A total interpersonal distress score in thepatient sample was calculated from the sum of all itemsat baseline (IIP pre-score). This score is regarded as a validmeasure of self-reported global interpersonal distress(Gurtman & Balakrishnan, 1998). The IIP-64 has demon-strated satisfactory psychometric properties in terms oftest–retest reliability, internal consistency and constructvalidity (Horowitz et al., 2000), which have been repli-cated for the Norwegian translation of the IIP-64 (Monsen,Havik, Eilertsen, & Hagtvet, 2006).

Procedure

The enrolment of patients in the NMSPOP began in 1995and continued until August 2001. By the end of 2005, alltreatments in the project were terminated. The patientswere informed, recruited and assessed by trained coordi-nators (clinical psychologists or psychiatrists) at each site.The participating therapists were the ordinary staffmembers at the clinics involved, who agreed to take partin this longitudinal study. The patients were assigned tothe therapists by the local co-coordinator as they werereferred to the outpatient clinics. This assignment wasbased purely on the availability of the therapists, andpatients were screened only in terms of exclusion criteria.The diagnostic assessments were conducted using theStructured Clinical Inventory of the Diagnostic and Statisti-cal Manual of Mental Disorders fourth edition (Axes I and II)

interviews (Spitzer et al., 1994; First et al., 1994), based onthe Diagnostic and Statistical Manual of Mental Disordersfourth edition criteria (American Psychiatric Association,1994). The patients also completed a core battery of self-reporting questionnaires (measuring symptoms, inter-personal problems, self-image and so on) at pre-treatment,during the treatment process, at the end of treatment andduring follow-up. The participation in the study wasbased on informed and signed consent, and the projectwas approved by the Regional Committee for MedicalEthics in Norway.The present study investigated patient-rated and therapist-

rated working alliance (WAI-P and WAI-T) as dependentvariables. The working alliance was rated at the end ofSessions 3, 12 and 20, and every 20th session thereafter. Inthe present investigation, WAI ratings were included up toSession 40, so the longitudinalWAI-P andWAI-Twere basedon a maximum of four ratings (corresponding to the ratingsat the 3rd, 12th, 20th and 40th sessions).The therapists completed the DPCCQ self-report survey

a maximum of six times during the project period, at1-year intervals. Their responses to the second admin-istration of the DPCCQ instrument were used in theanalyses since some of the variables (e.g., coping strategiesthat are part of the work involvement styles) were notincluded in the first version. Note that there was no sys-tematic time relationship between the alliance ratings andthe therapists’ responses to the DPCCQ, owing to the con-tinuing inclusion of patients and therapists into the project,so that the individual treatment processes had differentstarting points, unrelated to the DPCCQ administrations.

Data Analyses

Multilevel ModellingDue to the hierarchical (nested) structure of the data in

which repeated working alliance measurements (Level 1)were nested within patients (Level 2) who were nestedwithin therapists (Level 3), three-level growth curve anal-yses were employed to investigate the research questions.Multilevel modelling (MLM) accounts for the dependenceof residuals due to covariance between the levels in thedata (Lutz et al., 2007; Tasca & Gallop, 2009). Further,MLM has the advantage of partitioning and explainingvariance at different levels and is robust in allowing formissing observations or unequally distributed measure-ments available per level (Hox, 2010).

In the current MLM growth curve procedure, a‘time’ variable (modelled as a fixed slope across thefour measurements) was entered at Level 1. This timevariable was transformed into its natural logarithm(‘logtime’) in order to achieve a better fit of the model tothe data, on the basis of the change in the goodness-of-fitindex (�2 log-likelihood). The log transformation of timeis typical in psychotherapy research (Tasca & Gallop, 2009)

H. Nissen-Lie et al.

Copyright © 2014 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. (2014)

and was used because a steeper slope in alliance growthwas observed at the beginning of treatment compared withlater in therapy. After entering ‘logtime’ as a predictor atLevel 1, the patient control variable (IIP pre-score) wasentered at Level 2, followed by the therapist predictors(Level 3), including their interactionwith time to investigatethe rate of change explained by a given predictor (Hox,2002; Singer & Willett, 2003; Tasca & Gallop, 2009). Thetherapist factors were tested in three different multilevelmodels (Figure 1), examining their effect on the WAI-Pand WAI-T intercepts or slopes, consecutively. The firstmodel investigated thework involvement styles, the secondincluded therapist difficulties (PSD and NPR) and the thirdmodel explored the three factors, anxiety, flow and bore-dom, reflecting therapists’ in-session feelings.

The predictors were entered only as fixed effects. Allexplanatory variables were grand mean centred, asrecommended in the literature (e.g., Hox, 2002; Singer,1998). The logtime variable was also centred, with the result

that the intercept in themodels reflected the level of alliancein the middle of the measurement waves (i.e., betweenSessions 12 and 20). Centring of explanatory variables isused to reduce multi-collinearity and to aid the interpreta-tion of the findings, particularly concerning interactioneffects (Hox, 2002). All models were estimated using themaximum likelihood estimation procedure (as recom-mended by, e.g., Hox, 2002), since the primary focus wasmodelling fixed effects, and there was a relatively largenumber of groups (70 therapists) (Maas & Hox, 2005). Thesoftware Linear Mixed Models (IBM, SPSS version 20.0)was used to conduct the MLM growth curve analyses.

RESULTS

Table 2 presents an overview of the descriptive statis-tics of the patient and therapist alliance scores (WAI-Pand WAI-T) at various time points.

Figure 1. Diagram of the three multilevel models predicting patient-rated and therapist-rated Working Alliance Inventory (WAI-Pand WAI-T, respectively) consecutively. IIP-64 = 64-item Inventory of Interpersonal Problem. NPR = negative personal reaction.PSD = professional self-doubt

Therapist Factors and Alliance Development

Copyright © 2014 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. (2014)

Multilevel Growth Curve Modelling of WAI-P

Table 3 presents the results of the MLM growth curvemodels predicting patient-rated alliance (WAI-P) andtherapist-rated alliance (WAI-T).

Model 1 (WAI-P): Work Involvement StylesHealing involvement and stressful involvement did not

predict the level of alliance as rated by patients (BHealing =�0.03, p> 0.05; BStressful =�0.04, p> 0.05); but stressfulinvolvement negatively influenced growth in WAI-P(BStressful * Logtime =�0.11, p< 0.05).

Model 2 (WAI-P): Difficulties in PracticeProfessional self-doubt positively predicted the alliance

level (BPSD=0.35, p< 0.05) and negatively predicted alliancegrowth (BPSD*LogtimeE=�0.33, p< 0.01). NPR demonstrateda clear negative effect on alliance levels (BNPR=�0.58,p< 0.01) but did not affect the WAI-P slope (BNPR*LogtimeE=0.03, p> 0.05).

Model 3 (WAI-P): In-session FeelingsOnly therapists’ anxiety demonstrated a significant ef-

fect on the WAI-P slope (BAnxiety * Logtime =�0.28, p< 0.05).The other in-session feelings (i.e., flow and boredom) didnot affect the intercept or slope of WAI-P.

Multilevel Growth Curve Modelling of WAI-T

Model 1 (WAI-T): Work Involvement StylesThe work involvement styles showed no significant asso-

ciation with either WAI-T levels (BHealing=�0.08, p> 0.05;BStressful =�0.08, p> 0.05) or growth (BHealing *Logtime= 0.07,p> 0.05; BStressful * Logtime=�0.04, p> 0.05).

Model 2 (WAI-T): Difficulties in PracticeThe PSD and NPR were not associated with WAI-T

levels (BPSD =�0.15, p> 0.05; BNPR =�0.24, p> 0.05) norwith the rate of change (BPSD * Logtime = 0.03, p> 0.05;BNPR * Logtime = 0.03, p> 0.05).

Model 3 (WAI-T): In-session FeelingsOnly flow had a positive effect on WAI-T levels (BFlow =

0.31, p< 0.05) but was unrelated to the rate of change ofthe alliance (BFlow * Logtime = 0.40, p> 0.05). Therapistlevels of anxiety and boredom in sessions wereunrelated to therapist-rated alliance (BAnxiety = 0.20, p> 0.05;BAnxiety *Logtime = 0.16, p> 0.05; BBoredom=�0.17, p> 0.05;BBoredom*Logtime=�0.08, p> 0.05).

DISCUSSION

From previous research, we know that the therapists’ con-tribution to the alliance is more important for treatmentoutcome than that of the patients (Baldwin et al., 2007;Del Re et al., 2012). In order to investigate which therapistfactors influence the development of the working allianceand whether these factors depend on the rater ’s

Table 2. Descriptives of alliance ratings provided by patients(WAI-P) and therapists (WAI-T)

Session N Min Max Mean SD

WAI-P 3 227 2.33 7.00 5.10 1.0512 222 3.00 7.00 5.26 0.9120 222 1.67 7.00 5.35 0.9740 148 1.25 7.00 5.43 0.92

WAI-T 3 152 2.25 6.67 4.92 0.8112 156 2.83 6.75 5.12 0.7720 153 3.75 6.67 5.29 0.7040 92 3.17 6.92 5.32 0.67

WAI-P = patient-rated Working Alliance Inventory. WAI-T = therapist-rated Working Alliance Inventory.

Table 3. Results of multilevel modelling growth curveanalyses: therapists’ practice experiences as predictors ofpatient-rated alliance (WAI-P) and therapist-rated alliance(WAI-T)

Models

WAI-P WAI-T

Estimate (SE) Estimate (SE)

Model 1: work involvement stylesHealing involvement �0.03 (0.05) 0.08 (0.06)Healing involvement * slope �0.001 (0.03) 0.07 (0.06)Stressful involvement �0.04 (0.06) �0.08 (0.06)Stressful involvement * slope �0.11* (0.04) �0.07 (0.08)

Model 2: difficulties in practicePSD 0.32* (0.14) �0.15 (0.16)PSD * slope �0.33** (0.11) 0.03 (0.20)NPR �0.58** (0.14) �0.24 (0.18)NPR * slope �0.03 (0.13) �0.20 (0.23)

Model 3: in-session feelingsFlow 0.32 (0.16) 0.31* (0.15)Flow * slope 0.10 (0.12) 0.40 (0.21)Boredom 0.00 (0.18) �0.17 (0.17)Boredom* slope �0.20 (0.13) �0.08 (0.25)Anxiety �0.11 (0.13) 0.20 (0.12)Anxiety * slope �0.28** (0.09) �0.16 (0.17)

N= 70. Estimation method: maximum likelihood.PSD=professional self-doubt. NPR= negative personal reaction. SE =standard error. WAI-P = patient-rated Working Alliance Inventory.WAI-T = therapist-rated Working Alliance Inventory.*p≤ 0.05, **p≤ 0.01, p≤ 0.001.Bold characters indicate significant effects.

H. Nissen-Lie et al.

Copyright © 2014 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. (2014)

perspective, the current study explored three factors de-scribing therapist self-assessed experiences of therapeuticpractice: their work involvement styles, difficulties inpractice and in-session feelings (Heinonen et al., 2013;Orlinsky & Rønnestad, 2005). Some of these factorsdisplayed notable associations with working alliancedevelopment, but their impact depended on whether thealliance was rated by patients or by therapists. The fol-lowing discussion will focus on how this discrepancycan provide insight into the nature of the alliance betweenpatient and therapist and its working mechanisms.The findings of this study point to the idea previously

suggested, namely that not only patients but also thera-pists rate the alliance according to dispositions withinthemselves (Horvath, 2000). This may be illustrated bythe finding that in-session feelings of flow were found topredict only therapists’ alliance ratings and not those ofthe patients. It seems that when therapists feel stimulated,inspired, focussed or even engrossed in therapeutic ses-sions, it seems to enhance their own view of the alliance,without actually influencing their patients. Yet, in con-trast, therapists’ difficulties in finding something to likeor respect in a patient, being unable to tolerate a patient’semotional needs and feeling irritated or frustrated(i.e., NPR) seem to be communicated somehow to patients,since such feelings showed a negative association withalliance quality when the alliance was rated by patients. Aplausible explanation for this is that such feelings compro-mise the therapists’ ability to connect emotionally and gainthe trust and confidence of their patients. Further, thenegative effect of therapists’ NPR corresponds to theunfavourable effect of their negative emotional reactions to-wards patients observed in the study mentioned earlier(Wolff &Hayes, 2009). Those authors reminded us that evena small amount of negativity in therapists can harm therapyprocesses considerably.The aforementioned Vanderbilt studies (e.g., Henry

et al., 1990) demonstrated that therapists’ self-directedhostility and lack of resistance in responding to challeng-ing patients in a hostile or controlling manner distin-guished therapists in good versus poor alliance withpatients (Horvath & Bedi, 2002). The findings of the pres-ent study seem to corroborate the destructive effect oftherapists’ disaffiliative feelings towards patients. Thetherapists’ negative reactions possibly tap into feelingsthat might be understood as countertransference feelings.Therapists’ stable negative feelings towards patientsmight represent their own ‘blind spots’ (Freud, 1912):problematic aspects of themselves they do not see butare noticeable to their patients. This notion resonates withanother finding of the present study. Therapists’ feelingsof being anxious, overwhelmed and pressured (‘in-sessionanxiety’) displayed an unfavourable effect on the allianceas rated by patients but did not affect the therapists’ per-ception of the alliance.

The mechanism by which therapists’ negative feelingsinfluence the ways in which patients experience theworking alliance is an issue of speculation. Although itmay seem likely that those therapists who reported beingmore anxious and struggling with negative feelingstowards patients expressed these emotions in ways thatnegatively influenced the alliance, there may be an alter-native mechanism at play, involving a more reciprocalprocess. Despite an attempt to reduce the impact of thepatients’ level of relational functioning by controlling fortheir pre-treatment interpersonal distress, patients whowere particularly avoidant, rigid or hostile may haveelicited more tension and negative reactions in therapists,thus fuelling a poorer alliance.The therapists’ tendency to doubt themselves in their

professional work, as configured in the PSD factor, didnot influence therapist-rated alliance. In line with previ-ous research that found that this factor relates positivelyto early patient-rated alliance (Nissen-Lie et al., 2010)and to patient outcome (Nissen-Lie, Havik, Høglend,Monsen and Rønnestad, 2013), PSD was found to be pos-itively associated with alliance levels. However, in thecurrent study, PSD demonstrated a negative associationwith alliance growth when the alliance was rated bypatients. Taken together, these results can be interpretedin light of the previously mentioned findings of Saueret al. (2003), namely that anxiously attached therapistsreceived higher initial alliance ratings from their patientsbut that this was reversed in the longer term. We mayspeculate as to whether the positive effect of PSD on initialalliance ratings is due to PSD reflecting a self-criticalevaluation and sensitivity towards the needs of thepatient, which is appreciated by patients, particularlyduring alliance building. Over time, however, the tendencyto doubt oneself may give the impression that the therapistlacks self-efficacy, thus compromising the confidence thatthe patient feels in the therapist. We should not forget,however, that PSD was positively associated with a reduc-tion in interpersonal distress for patients during and afterpsychotherapy (Nissen-Lie, Havik, Høglend, Monsen andRønnestad, 2013). This issue requires further clarificationin future research.

Limitation of Design, Measurements and Inferences

When studying predictors of the working alliance asevaluated by patients and therapists, one runs into thequestion of whether alliance ratings reflect a more generalpropensity within the individual rater to perceive inter-play with others in a more positive or negative light ratherthan provide a picture of the ongoing patient–therapistdynamic. Our design does not permit disentangling thetwo, but the findings seem to suggest that subjective states

Therapist Factors and Alliance Development

Copyright © 2014 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. (2014)

and traits of therapists and patients determine allianceratings, rendering psychotherapy a truly intersubjectiveenterprise.Moreover, the associations between therapist work ex-

periences and alliances observed in this study were notcontextualized in specific psychotherapy patient–therapistcases. Therapists most likely behave and feel differentlywith each patient, but the therapist factors in this studytap a cross section of their feelings across all their patients.The establishment of several reliable and robust associa-tions between these generalized self-reports and theworking alliance indicates that these factors, at least tosome extent, reflect more trait-like, as opposed to state-like, self-appraisals. Several of these therapist factorsdemonstrate temporal stability across a 6-year measure-ment period (Nissen-Lie, Havik, Høglend, Monsen andRønnestad, 2013). Nonetheless, we cannot exclude thepossibility that the associations found might have beenmulti-directional and/or mediated by unknown variables.Other methodological problems include the fewer

alliance ratings obtained from therapists compared withthe numbers obtained from patients. This was due to thefact that the therapists only started to evaluate the alli-ances a year into the NMSPOP project when they startedto rate the alliance with new patients enrolled in the study.Selecting only the therapies with at least three alliance

measures might have excluded the least successful thera-pies that had not provided sufficient data. In turn, thismay have influenced the associations between therapistfactors and alliance scores found in the study and limitedtheir generalizability. Also, given that the therapists wereself-selected in terms of participating in the study, andtherefore open to becoming a research subject on a poten-tially threatening topic such as monitoring their therapyprocesses, the present findings might not be generalizableto other therapists who did not opt to be enrolled in thisstudy (Heinonen et al., 2013).

Concluding Remarks

One discernible pattern in the findings of the presentstudy is that patient alliance ratings were more likely tobe influenced by therapists’ negative practice experiences(i.e., therapists’ stressful involvement, in-session anxietyand difficulties in practice) than by positive ones. How-ever, these aspects did not influence the therapists whenthey rated the alliances (also Nissen-Lie, Havik, Høglend,Monsen and Rønnestad, 2013). This bias is noteworthyand should attract the attention of psychotherapyteachers, supervisors and practising therapists. Fromthese findings, one may conjecture that therapists’negative emotional states or traits are more noticeable topatients because they reflect an interpersonal responsestyle that deviates from what patients most need in a

therapist: a supportive and non-judgmental person whoprovides them with a secure relationship from which toexplore painful and challenging material.

REFERENCESAckerman, S. J., & Hilsenroth, M. J. (2001). A review of therapist

characteristics and techniques negatively impacting the thera-peutic alliance. Psychotherapy: Theory, Research, Practice, Training,38, 171–185.

Ackerman, S. J., & Hilsenroth, M. J. (2003). A review of therapistcharacteristics and techniques positively impacting the thera-peutic alliance. Clinical Psychological Review, 23, 1-33.

American Psychiatric Association. (1994). Diagnostic andStatistical Manual of Mental Disorders (4th edn) (DSM-IV).Washington, DC: American Psychiatric Association.

Baldwin, S. A., Wampold, B., & Imel, Z. E. (2007). Untangling thealliance–outcome correlation: Exploring the relative impor-tance of therapist and patient variability in the alliance. Journalof Consulting and Clinical Psychology, 75, 842–852.

Bordin, E. (1979). The generalizability of the psychoanalyticconcept of the working alliance. Psychotherapy: Theory, Researchand Practice, 16, 252–260.

Bordin, E. (1994). Theory and research on the therapeutic workingalliance: New directions. In A.O. Horvath, & L.S. Greenberg(Eds.), The working alliance: Theory, research and practice (pp.13–38). New York: John Wiley & Sons, Inc.

Constantino, M., & Smith-Hansen, L. (2008). Patient interper-sonal factors and the therapeutic alliance in two treatmentsfor bulimia nervosa. Psychotherapy Research, 18(6), 683–698.

Csikszentmihalyi, M. (1990). Flow: The psychology of optimal expe-rience. New York: Harper & Row.

Davis, J.D., Elliot, R.,Davis,M.L., Binns,M., Francis,V.M.,&Schröder,T. A. (1987). Development of a taxonomy of therapists’ difficulties.Initial report. British Journal of Medical Psychology, 60, 109–119.

Del Re, A. C., Flückiger, C., Horvath, A. O., Symonds, D., &Wampold, B. E. (2012). Therapist effects in the therapeuticalliance–outcome relationship: A restricted-maximum likeli-hood meta-analysis, Clinical Psychology Review, 32, 642–649.

Dinger, U., Strack, M., Sachsse, T., & Shauenburg, H. (2009). Thera-pists’ attachment, patients’ interpersonal problems and alliancedevelopment over time in inpatient psychotherapy. Psychother-apy: Theory, Research, Practice, Training, 46(3), 277–290.

Endicott, J., Spitzer, R. I., Fleiss, J. L., & Cohen, J. (1976). The GlobalAssessment Scale: A procedure for measuring overall severity ofpsychiatric disturbance.Archives of General Psychiatry, 33, 766–771.

First, M. B., Spitzer, R. L., Gibbon, M., Williams, J. B. W., &Benjamin, L. S. (1994). Structured clinical interview for theDSM.IV axis II personality disorder (SCID-II) (version 2.0). NewYork: New York State Psychiatric Institute.

Freud, S. (1912). Papers on technique. The dynamics of trans-ference. Standard Edition, 12, 97–108.

Gurtman, M. B., & Balakrishnan, J. D. (1998). Circular measure-ment redux: The analysis and interpretation of interpersonal cir-cle profiles. Clinical Psychology, Science and Practice, 5(3), 344–360.

Havik, O. E., Monsen, J. T., Høglend, P., von der Lippe, A. L.,Lyngstad, G., Stiles, T. (1995). The Norwegian Multisite Studyon Process and Outcome of Psychotherapy (NMSPOP). Re-search protocol available from Odd E. Havik, Department ofClinical Psychology, University of Bergen, Christiesgt. 12,5015 Bergen, Norway.

H. Nissen-Lie et al.

Copyright © 2014 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. (2014)

Heinonen, E., Lindfors, O., Härkänen, T., Virtala, E., Jääskeläinen,T., & Knekt, P. (2013). Therapists’ professional and personalcharacteristics as predictors of working alliance in short-termand long-term psychotherapies. Clinical Psychology and Psycho-therapy doi: 10.1002/cpp.1852. [Epub ahead of print].

Henry, W. P., Schacht, T. E., & Strupp, H. H. (1990). Patient andtherapist introject, interpersonal process and differential out-come. Journal of Consulting and Clinical Psychology, 58, 768–774.

Hersoug, A. G., Høglend, P., Havik, O. E., von der Lippe, A., &Monsen, J. T. (2009). Therapist characteristics influencing the qual-ity of alliance. Clinical Psychology and Psychotherapy, 16(2), 100–110.

Hersoug, A. G., Høglend, P., Monsen, J. T., & Havik, O. E. (2001).Quality of working alliance in psychotherapy. Therapistvariables and patient/therapist similarity as predictors. Journalof Psychotherapy Practice and Research, 10, 205–216.

Horowitz, L. M., Alden, L. E., Wiggins, J. S., & Pincus, A. L.(2000). Inventory of interpersonal problems. San Antoni, TX: Psy-chological Corporation.

Horvath, A. O. (1994). Empirical validation of Bordin’s pantheo-retical model of alliance: The Working Alliance Inventory per-spective. In A. O. Horvath, & L. S. Greenberg (Eds.), Theworking alliance: Theory, research and practice (pp. 109–128).New York: John Wiley & Sons, Inc.

Horvath, A. O. (2000). The therapeutic relationship: From trans-ference to alliance. Journal of Clinical Psychology, 56 (2), 163–173.

Horvath, A. O., & Bedi, R. P. (2002). The Alliance. In J. C.Norcross (Ed): Psychotherapy relationships that work. Therapistcontributions and responsiveness to clients (pp. 37–70). New York:Oxford University Press.

Horvath, A. O., & Greenberg, L. S. (1989). Development andvalidation of the Working Alliance Inventory. Journal ofCounseling Psychology, 36, 233–233.

Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011).Alliance in individual psychotherapy. Psychotherapy, 48(1), 9–16.

Hox, J. (2002). Multilevel analysis. Techniques and applications.Mahwah, New Jersey: Lawrence Erlbaum Associates.

Hox, J. (2010). Multilevel analysis. Techniques and applications (2ndedn). Mahwah, New Jersey: Lawrence Erlbaum Associates.

Kivlighan, D. M., & Shaughnessy, P. (1995). Analysis of the devel-opment of the working alliance using hierarchical linearmodeling. Journal of Counseling Psychology, 47, 362–371.

Kramer, U., de Roten, Y., Beretta, V., Michel, L., & Despland, J-N.(2008). Patient’s and therapist’s views of early alliance buildingin dynamic psychotherapy: Patters and relation to outcome.Journal of Counseling Psychology, 55 (1), 89–95.

Lutz, W., Leon, S. C., Martinovich, Z., Lyons, J. S., & Stiles, W. B.(2007). Therapists effects in outpatient psychotherapy: A three-level growth curve approach. Journal of Counseling Psychology,54, 32–39.

Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes formultilevel modeling. European Journal of Research Methods ofthe Behavioral and Social Sciences, 1(3), 86–92.

Monsen, J. T., & Monsen, K. (1999). Affects and affect conscious-ness—a psychotherapy model integrating Silvan Tomkins’ af-fect and script theory within the framework of selfpsychology. In A. Goldberg (Ed.), Pluralims in self psychology.Progress in self psychology, Vol. 15, (pp. 287–306). Hillsdale NJ:The Analytic Press.

Monsen, J. T., Havik, O. E., Eilertsen, D. E., & Hagtvet, K. (2006).Circumplex structure and personality disorder correlates of theinterpersonal problems circumplex model (IIP-C). PsychologicalAssessment, 18, 165–173.

Najavits, L. M., Griffin, M. L., Luborsky, L., Frank, A., Weiss, R.D., Liese, B. S., Thompson, H., Nakayma, E., Siqueland, L.,Daley, D., & Onken, L. S. (1994). Therapist emotional reac-tions to substance abusers: A new questionnaire and initialfindings. Psychotherapy, 32, 669-677.

Nissen-Lie, H. A., Havik, O. E., Høglend, P. A., Monsen, J. T., &Rønnestad, M. H. (2013). The contribution of therapists’ per-sonal life quality to the development of the working alliance.Journal of Counseling Psychology, 60(4), 483–495.

Nissen-Lie, H. A., Monsen, J. T., & Rønnestad, M. H. (2010).Therapist predictors of early patient-rated working alliance:A multilevel approach. Psychotherapy Research, 20(6), 627–646.

Nissen-Lie, H. A., Monsen, J. T., Ulleberg, P., & Rønnestad, M. H.(2013). Psychotherapists’ self-reports of their difficulties andinterpersonal functioning in practice as predictors of patientoutcome. Psychotherapy Research, 23(1), 86-104.

Orlinsky, D. E., & Rønnestad, M. H. (2005).How psychotherapists de-velop: A study of therapeutic work and professional growth. Washing-ton, DC: American Psychological Association.

Orlinsky, D. E., Ambühl, H., Rønnestad, M. H., Davis, M.,Willutzki, U., Botermans, J. F., Dazord, A., Cierpka,M., Aapro,N., Buchheim, P., Bae, S., Davidson, C., Friis-Jorgensen, E.Joo, E., Kalmykova, E., Meyerberg, J., Northcut, T., Parks,B., Scerb, E., Schröder, T., Shefler, G., Stiwne, D., Stuart,S., Tarragona, M., Branco Vasco, A., & Wiseman, H. (1999).Development of psychotherapists: concepts, questions, andmethods of a collaborative international study. PsychotherapyResearch, 9, 127–153.

Sauer, E. M., Lopez, F. G., & Gormley, B. (2003). Respective contri-bution of therapist and client adult attachment orientation tothe development of the early working alliance: A preliminarygrowth modeling study. Psychotherapy Research, 13, 371–382.

Singer, J. D. (1998). Using SAS PROC MIXED to fit multilevelmodels, hierarchical models and individual growth models.Journal of Educational and Behavioural Statistics, 59, 323–355.

Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analy-sis. Modeling change and event occurrence. New York: OxfordUniversity Press.

Spitzer, R. L., Gibbon, M., Skodol, A. E., Williams, J. B. W., &First, M. B. (Eds.), (1994). DSM-IV casebook: A learningcompanion to the Diagnostic and Statistical Manual of Mental Dis-orders (4th edn). Washington DC: American PsychiatricAssociation.

Tasca, G.A., & Gallop, R. (2009). Multilevel modeling of longitu-dinal data for psychotherapy researchers: I. The basics. Psycho-therapy Research, 19(4-5), 429–437.

Tingey, R., Lambert, M. J., Burlingame, G. M., & Hansen, N. B.(1996). Assessing clinical significance. Proposed extensions tomethod. Psychotherapy Research, 6 (2), 109-123.

Tracey, T. J., & Koskotovic, A. M. (1989). Factor structure of theWorking Alliance Inventory. Journal of Consulting and ClinicalPsychology, 57, 207–210.

Tryon, G. S., Blackwell, S. C., & Hammel, E. F. (2007). A meta-analytic examination of client–therapist perspectives of theworking alliance. Psychotherapy Research, 17, 629–642.

Wolff, M. C., & Hayes, J. A. (2009). Therapist variables: Predictorsof process in the treatment of alcohol and other drug problems.Alcoholism Treatment Quarterly, 27(1), 51–65.

Zeek, A., Orlinsky, D. E., Hermann, S., Joos, A., Wirsching, M.,Weidmann,W., & Hartmann, A. (2012). Stressful involvement inpsychotherapeutic work: Therapist, client and process correlates.Psychotherapy Research, 22(5), 543–555.

Therapist Factors and Alliance Development

Copyright © 2014 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. (2014)