Psychotherapy process measure research and the evaluation of psychotherapy orientation: A narrative...

Journal of Psychotherapy Integration, Vol. 9, No. 3, 1999

Psychotherapy Process Measure Research and theEvaluation of Psychotherapy Orientation: ANarrative Analysis

Heidi Levitt1,3 and Lynne Angus2

This article discusses the need for a systematic method that enables researchersto evaluate integrative therapy approaches using a range of therapy processmeasures. The Narrative System Process Coding (NPCS; Angus, Hardtke, &Levitt, 1996) is proposed as such a method, and is applied with the Experienc-ing Scale (Klein, Mathieu, Gendlin, & Keisler, 1970) and the Levels ofClient Perceptual Processing (LCPP; Toukmanian, 1986) to three brief goodoutcome integrative therapy modalities to illustrate this need. The study foundhigher Experiencing Scale scores to be most strongly related to an experientialapproach to therapy and to the NPCS internal narrative processes. HigherLCPP scores were most strongly related to the NPCS reflexive narrativeprocess and to a perceptual-processing approach. The discussion initiates adiscourse on the importance of explicating process measures' origins whencomparing different therapy approaches in order to allow for the meaningfulconsolidation of process research findings.

KEY WORDS: process measure; psychotherapy orientation; integrative psychotherapy; narra-tive; psychotherapy research.

Over the last decade, the role of the co-constructive process of narra-tion in therapy has been increasingly focused upon by psychotherapy prac-titioners, researchers, and theoreticians (e.g., Goncalves, 1994; Neimeyer,1994; Polkinghorne, 1988; Rennie, 1994; Sarbin, 1982; White & Epson,

1Department of Psychology, University of Memphis, Memphis, Tennessee.2Psychology Department, 297 Behavioural Science Building, York University, 4700 KeeleStreet, North York, Ontario, M3J 1P3, Canada.

3Correspondence should be directed to Heidi Levitt, Department of Psychology, Campus Box526400, University of Memphis, Memphis, Tennessee 38152-6400.

279

1053-0479/99/0900-0279$16.00/0 © 1999 Plenum Publishing Corporation

280 Levitt and Angus

1990). Within these approaches, psychotherapy is seen as a narrative projectthat entails the articulation and formulation of clients' life stories such thatit facilitates the emergence of more coherent personal narratives.

The term "narrative" has come to have different usages and implica-tions in psychotherapy literature (see Angus & Hardtke, 1994, and Vogel,1994). Narrative approaches to psychotherapy process research may beparticularly appropriate for the description and identification of therapeuticprocesses that span different therapeutic modalities (e.g., Angus & Hardtke,1994; McLeod & Balamoutsou, 1986; Levitt, Hardke, & Angus, 1993). TheNarrative Process Model (Angus, Levitt, & Hardtke, in press) views allpsychotherapies as incorporating processes of telling narratives, of experi-encing and evoking emotions associated with narratives, and of reflectingon the personal meanings embedded in these narratives.

The Narrative Process Coding System (Angus, Hardtke, & Levitt,1996) was developed as heuristic tool, from this model, which unitizes andcategorizes psychotherapy text, enabling one to explore within-therapyprocess. This narrative approach can be particularly useful when assessingintegrative or differing therapeutic approaches by process measures as,instead of using units that prize interventions from one therapeutic orienta-tion, the system creates common units based on narrative processes, inher-ent in the telling of personal experience, which cut across psychotherapyorientation. An illustration of its application to three different integrativetherapies in conjunction with two therapy process measures is presentedin this article.

Psychotherapy Process Measure Research and IntegrativeTherapy Approaches

The dominant method of psychotherapy evaluation used by psycho-therapy process researchers has been the application of psychotherapyprocess scales and measures (e.g., the Client Vocal Quality Scale (CVQ;Rice, Koke, Greenberg, & Wagstaff, 1979), The Experiencing Scale (Klein,Mathieu, Gendlin, & Kiesler, 1970), and the Core Conflictual RelationalThemes (CCRT; Luborsky & Crits-Christoph, 1990) to psychotherapeutictranscripts. These are nominal or rank-ordered scales that assess specificelements of the therapeutic process. Raters are trained to objectively andreliably apply the measures to the sessions in order to ensure the replicabil-ity of the findings. These raters are usually external to the therapy in thesense that they are not the client nor the therapist.

Psychotherapy process measures, such as the Experiencing Scale orthe CVQ, largely began as a method to test Rogerian theory (Hill &

Process Measure Research and Psychotherapy Orientation 281

Corbett, 1992). Since this time an array of process measures have beendeveloped, tracking a wide variety of psychotherapy discourse characteris-tics. Generally, these rating scales have been designed to evaluate variablesor events (often designated in degrees of "good process") that reflect theassumptions underlying a particular therapeutic approach. For instance,the Experiencing Scale and the CVQ, derived from client-centered thera-pists, rate the client's emotional involvement in therapy discourse while theCCRT, by a psychodynamic-client-centered therapist, identifies developingtransferential themes.

Scoring procedures for the various measures often differ, and whenrating systems specify different units of transcript text for evaluation, itbecomes difficult to integrate findings arising from the application of avariety of measures. As these units of analysis and definitions of processare often derived from the therapeutic orientation of the scale originator,questions can be raised as to how process measures can be used on integ-rative transcript data sets, or on sets that arise from therapeutic modalitiesother than the founding approach of the process measures, and of how theresults of these studies can best be interpreted. For instance, a unitizationsystem may fragment tasks (e.g., a dream sequence) that it does not recog-nize as meaningful, but that might have been left intact if the unitizingsystem had been derived from the principles of another therapy.

These concerns are particularly important when using a variety ofrating systems on different types of therapies. For instance, the Levels ofClient Perceptual Processing Scale (LCPP; Toukmanian, 1986,1990,1992)uses scoring units thats divide transcript units according to shifts in theclient's perceptual processing level while the Referential Activity Scale(Bucci, 1987) uses "idea units" based on perceived shifts in content. Thecomparison of process measure results is often difficult as a researcher doesnot have a common set of data from which different ratings can be comparedand contrasted. Consequently, research data generated from different pro-cess measures become difficult to integrate and the information researchersgather is often left unaccumulated and fragmented.

Past research studies evaluating different integrative therapies havebeen hampered by measures that create units based in the conceptualframework of one therapy, potentially biasing findings toward that therapy(often a single-orientation approach). As process measure research hasbecome foundational in areas of both psychotherapy process and outcomeresearch, biases in these efforts could potentially distort understandings ofdifferences between therapeutic orientations.

Integrative therapy researchers have been calling for methods that canbe used to assimilate research, to study cases from different perspectives,and to start building frameworks upon which the findings from different


approaches can be applied clinically (e.g., Beitman, 1994; Goldfried, 1991;Glass, Norcross, Arn Koff, & Lambert, 1993). Elliott (1991) and Russelland Staszewski (1988) argue for the selection of a semantic unit in orderto integrate findings from different therapies. Our ability to meaningfullycompare results from different therapies and process measures as wellas our ability to communicate and compare research findings is severelyhampered by the absence of a common process of unitization.

The Narrative Process Coding System (NPCS; Angus, et al., 1996) isone response to these concerns. This system, based on the Narrative ProcessModel (see Angus et al., in press; Angus & Hardtke, 1994), uses a semanticrationale to divide therapy transcripts first by topic or theme units and thenby narrative process.

Three narrative processes are identified. (1) External narrative se-quences are focused on event descriptions. A client may relay a story abouta confrontation at work—for instance, describing the setting and the verbalexchange. (2) Internal narrative sequences are focused on the client's expe-riential state. This client may continue on to discuss how upsetting thisconfrontation was and how it affected his/her emotional state. (3) Reflexivenarrative sequences are indicative of analytical or evaluative processes. Forinstance, this client may shift into an analysis of how this confrontationmakes sense in the larger scope of the client's history or current life events.

The three narrative process types are seen as integral to all forms oftherapy, and are not priorized one over the other. As the smallest units inthe system include at least four sentences, they provides enough data tobe meaningfully rated by most process measures. This system has beenfound to be useful in the comparison of different therapeutic approaches,topic themes, metaphor themes, and therapy process measures (e.g., Hard-tke, 1996; Goncalves, Korman, & Angus, in press; Levitt, Hardtke, & Angus,1993; Levitt, Korman, & Angus, 1997, in press).

Present Study

This study's primary goal was to explore the NPCS in relation to threeintegrative therapeutic modalities and two standardized rating systems. TheExperiencing Scale (Klein et al, 1970) and the LCPP (Toukmanian, 1986)were applied to three good-outcome brief therapy dyads: a perceptualprocessing dyad, an experiential dyad (process experiential), and a psycho-dynamic-client-centered dyad. The NPCS was used to systematically seg-ment the psychotherapy sessions into units of transcript for further ratingon the Experiencing Scale and LCPP.

Three good outcome dyads were chosen so that prototypical exemplars

https://www.researchgate.net/publication/232520495_Research_issues_in_psychotherapy_integration?el=1_x_8&enrichId=rgreq-a36094b2-5c33-4b2e-b51c-e951e3d286fe&enrichSource=Y292ZXJQYWdlOzIyNTI4MDE0MDtBUzoxNzk3NzUyNjgyMDQ1NDRAMTQxOTg3MzE3Mjc0Ng==


of each therapy could be examined. Each of the three dyads examined isrepresentative of a different therapeutic approach. Standardized assessmentmeasures as well as client and therapist subjective impressions were usedto evaluate outcome overall for the therapy dyads. Adherence measuresand therapist subjective impression were used to assess the degree to whichthe three therapy dyads exemplified their espoused therapeutic approaches.

Instead of focusing on selected in-session events, this project adoptsa naturalistic methodological approach to the extent that it examines eachsession in its entirety. The Experiencing Scale, LCPP, and the NPCS areused to provide a description of the dyads and to produce hypotheses aboutthe therapeutic processes at play. The therapy dyads are examined in termsof early (sessions 2, 3, 4), middle (middle three sessions), and late stages(last three sessions) to allow for a differentiating examination of therapeuticprocess over time.

METHOD

Therapists

The three therapies are eclectic or integration-based approaches versussingle-approach forms of humanistic, cognitive, or psychodynamic-client-centered therapies. Good outcome dyads were chosen for inclusion in thisstudy so that a paradigmatic example for each therapeutic approach couldbe compared and contrasted.

All therapists had at least 20 years experience in conducting therapy.As well, measures of therapeutic integrity, or adherence, were used toensure that the therapies chosen were indeed representative of their respec-tive treatments. Additionally, the dyads were all considered to be goodoutcomes by their therapists and to be typical of the therapists' therapeu-tic orientation.

Perceptual-Processing Therapy

The perceptual processing therapist was the originator of this approach,which works toward the development of more productive cognitive sche-mas. It integrates information processing and client-centered philosophiesand interventions. The perceptual processing theory holds that clients arein need of broadened perceptual bases for processing and gathering infor-mation (Toukmanian, 1986, 1992). The perceptual processing therapist at-tempts to focus clients' attention on certain processing patterns so that


increasingly functional and flexible perceptual processes can be developed.The Treatment Integrity Measure of Perceptual Processing (Toukmanian,unpublished manuscript) was used to verify that this therapy was an exem-plar of perceptual-processing therapy.

Process-Experiential Therapy

The founder of this approach was the therapist for this study. Theprocess-experiential orientation is emotionally focused and directed towardthe evocation and restructuring of emotional schemes (Greenberg, Rice, &Elliott, 1993). Different client-centered and gestalt interventions are inte-grated to facilitate clients' emotions and assist them in reorganizing andreconstructing these schemes. The Experiential Therapy Adherence Mea-sure (Greenberg & Goldman, 1992) was used to determine that this therapywas representative of process-experiential therapy.

Psychodynamic-Client-Centered Therapy

The therapist for this approach has had extensive training and practice.This therapy integrated client-Centered and psychodynamic approaches totreatment. The psychodynamic approach predicts that psychological diffi-culty is rooted in unconscious inner conflict (e.g., Arlow, 1989). Transfer-ence analysis, dream analysis, and the interpretation of client's behaviorare interventions that distinguish this therapeutic approach. Although nointegrative measure was available to assess whether this approach wasintegrative, the Dynamic Adherence Measure was used to assess the psycho-dynamic aspect of this therapy (Goldman, 1991), illustrating that this ther-apy did utilize some dynamic interventions but also identifying frequentreflective responses, supporting the client-centered aspect of this integ-rative approach.

Clients

All participants were self-referred female clients selected by their ther-apist for participation in this study. Although all were in brief therapy, thelength of their therapeutic relationships differed. One client participatedin 15 sessions of experiential therapy, another client had 10 sessions ofcognitive therapy, while the third was involved in 20 sessions of psychody-namic-client-centered therapy. All clients were selected by their therapist


for participation in this project and their consent was obtained. None ofthe subjects presented with psychosis, brain damage, psychiatric history,or suicidal ideation, nor did they require psychiatric medication.

Transcripts

The therapy sessions were recorded and then transcribed prior to thisstudy. Three initial, three middle, and final three sessions were selected torepresent the beginning, middle, and end phases of therapy. The first sessionwas excluded, however, as it was thought that it may be atypical, includinghistory-taking or other assessment procedures not representative of thetherapeutic approach. Of the twelve perceptual processing transcripts, ses-sions 2-10 were analyzed; of the fifteen process-experiential transcripts,sessions 2-4, 8-10, and 13-15 were included in this analysis; and of thetwenty psychodynamic-client-centered transcripts, sessions 2-4, 10-12, and18-20 were analyzed.

Measures

Outcome Measures

For each of the three therapy dyads included in this study, standardizedself-report measures were used to evaluate client outcome.

Perceptual Processing In this therapy, the Beck Depression Inventory(Beck, Ward, Mendelson, Mock, & Erbaugh, 1961), the Tennessee SelfConcept Scale (Fitts, 1965), the Social Adjustment Scale (Weissman &Bothwell, 1976), the Eight role Category Questionnaire, and the Interper-sonal Problem Checklist were used.

Process-Experiential Therapy The SCL-90 (Deogratis, 1977), the BeckDepression Inventory, and the Inventory of Interpersonal Problems (Horo-witz, Rosenberg, Baer, Ureno, & Villasneno, 1988) were used to classifythis as a good outcome therapy.

Psychodynamic-Client-Centered Therapy For the dynamic therapy, theBehaviour System Index (Deogratis, 1977), the Social Adjustment Scale,and the Beck Depression Inventory were used to assess outcome. On theBasis of pretherapy and posttherapy change scores, each of the dyads wereidentified as having achieved a significant positive therapeutic outcome.


Psychotherapy Process Measures

The Experiencing Scale The Experiencing Scale (Klein et al., 1970) isa seven-level measure of the client's involvement in therapy. It examinesthe construct of "experiencing," which is defined as "the extent to whichinner referents become the felt data of attention, and the degree to whichefforts are made to focus on, expand, and probe those data" (Klein, Ma-thieu-Coughlan, & Kiesler, 1986). The Experiencing Scale is derived fromGendlin's experiential and Roger's client-centered theories, and is one ofthe first widely used therapeutic process measures (Hill & Corbett, 1993).

As one of the standard measures of therapeutic process, the Experienc-ing Scale has been used to validate therapy process measures under develop-ment (e.g., Maher, Stalikas, Boissoneault, & Tauer, 1990; Sherman & Skin-ner, 1988). Klein et al. (1986) reviewed the literature on this measure,finding that Experiencing repeatedly has been associated with high-qualitytherapist interventions from different types of therapy (i.e., client-centered,gestalt, and psychodynamic approaches), illustrating the construct validityof Experiencing. They also found Experiencing to be associated with finaltherapeutic outcome, especially when considering the degree of change inExperiencing over a therapy or when examining the Experiencing ratingswithin the later sessions of treatment.

The Experiencing Scale (Klein et al., 1970) evaluates clients' ability toexplore their inner state and their ability to be guided by an inner referent.The first three levels of this scale reflect an increasing referencing to innerstate within the therapy discourse. A shift takes place at the fourth levelwhere the clients' focus of discussion becomes their experiential state.Levels five, six, and seven reflect the identification, exploration, and resolu-tion of issues relating to the clients' experiential state.

In terms of applying the Experiencing Scale, units of transcript areassigned both a mode rating, which is representative of the most frequentrating within a unit, as well as a peak rating, which indicates the highestexperiencing level within a unit of text. Klein et al. (1970, 1986) suggest thatraters make "running ratings," listening for shifts in level of Experiencingoccurring in the therapy session. The peak rating is the highest score inthe selected segment and the mode is the most frequent or representativerating of experiencing within the segment. What constitutes a "segment,"however, is left undefined. The question is left unanswered as to how ratersare to identify uniform units or segments in the therapy sessions, givendifferent therapeutic strategies. This lack of clarification regarding a system-atic unitizing method may create difficulties when attempting to comparestudies using the Experiencing Scale.

Upon assessing the reliability of the Experiencing ratings in this study,


using three randomly selected sessions of the perceptual processing treat-ment (69 narrative sequences), the rater achieved an overall percentageagreement level of 96% for modal ratings and 80% for peak ratings withan independent rater.

Levels of Client Perceptual Processing. Theoretically, the LCPP (Touk-manian, 1986) is based on an information processing approach to cognitive-affective phenomena and was developed from examining complete therapysessions. It categorizes the client's manner of processing information intoseven levels. The first two are those of "undifferentiated statements," whichare abstract and superficial, and "elaborations," which provide a moredetailed account of an episode. The next three categories represent differen-tiation with either an external, analytical, or internal focus. In the sixthcategory, reevaluation, different options or considerations, relevant to theproblem at hand, are explored. Integration is the last category in whichdifferentiated aspects of a topic are used to construct a new perspective.The validity of this scale has been discussed in relation to the construct itproposes (Toukmanian, 1986).

The LCPP uses "scoring units," each of which fall within one LCPPcategory. Shifts between units occur when the LCPP category shifts. Aswith the Experiencing Scale, this approach is limited by the absence of areliable and independent method to the segmenting therapy session tran-scripts. Interrater agreement using Cohen's kappa has ranged from .37 to.78 in the categorization of preunitized text (e.g., Brown, 1984; Sinclair,1990; Toukmanian, 1986; Zink, 1990; Jackson, 1993). The rater of this studyand an independent rater achieved a correlation of .82 for modal ratingsand .87 for peak LCPP ratings. The construct underlying this scale hasdemonstrated validity in studies that have found high LCPP scores to beassociated with positive therapeutic outcome (see Toukmanian, 1986).

Narrative Process Coding System

This classification system provides standardized units of transcripttext for the application of psychotherapy process measures. Initially,therapy session transcripts are divided into topic segments according tochanges in the theme of the therapy dialogue. Psychotherapeutic textis then subdivided according to shifts between the three narrative processtypes previously described. The average unit in this study was 30 sentencesin length.

Several tests of interrater reliability were conducted in order toassess the coding of the NPCS rater for this study. The rater from thisstudy and an independent rater unitized and coded a session from the


perceptual processing dyad examined in this study (23 narrative se-quences), and were found to have an agreement level of 89% in identifyingtopic segments and an agreement level of 83% in identifying and classifyingnarrative sequences. Then, using 50 narrative sequences from a client-centered and a cognitive therapy, these raters were able to identify topicsegments with an overall agreement level of 84% and an interrateragreement of 88% (Cohen's kappa. 75) on the identification of narra-tive sequences.

As the NPCS derives the topic segments directly from the client-therapist dialogue, its process allows the themes that are identified botha precision and an inherent content validity often not found in thematicrating measures that may force the transcript data into preconceptualizedtheme categories (e.g., Richards & Longborg, 1996).

The NPCS narrative processes have been found to have two formsof construct validity. First, the patterns of narrative processes identifiedin different therapeutic orientations were found to be theoreticallyconsistent with the processes prescribed by the therapy orientation beingexamined (e.g., a cognitive therapy session was found to be characterizedby significantly more reflexive sequences than a client-centered or agestalt session; Levitt, Hardtke, & Angus, 1993). Second, psychotherapyprocess measures, which were conceptually similar to a narrative process,identified sequences of that narrative processes with significantly higherratings than the sequences of the other two narrative processes (e.g.,the Referential Activity Scale [Bucci, 1985], which prizes verbal detailand descriptiveness, was found to rate external narrative sequences assignificantly higher than sequences of the other two narrative processes;Levitt, 1993).

Procedure

The Narrative Process Coding System was applied for the purposeof comparing the process measure ratings on the different brief therapyapproaches. All transcripts were divided into topic segments using theNPCS and then further subdivided into narrative sequences. These narrativesequences were assigned a narrative code: external, internal, or reflexive.

The Experiencing Scale and the LCPP were applied to the therapytranscripts such that each narrative sequence received a modal and peakrating on both scales. Analyses of variance and log-linear analyses wereemployed in the investigation into the ways good outcome therapy differsin narrative code and process measure ratings.


RESULTS

The Narrative Process Coding System and the Therapeutic Approaches

In comparing the number of narrative sequences identified in thetherapy transcripts, difference between the three therapy approaches wereevident. The log-linear analysis confirmed that the three therapeutic ap-proaches were significantly different in terms of both the number of identi-fied narrative sequences (df = 2, x2 = 11.61, p = .003) and the type ofnarrative sequences (external, internal or reflexive units) identified in thesessions drawn from the three therapy dyads (df= 4, x2 = 25.40, p = .0001;see Table I).

In terms of narrative sequence types, it seems that the three therapiesdistinguish themselves by their narrative processes (see Levitt et al., 1993,for a more comprehensive discussion of the patterns in narrative sequence).The perceptual-processing dyad is predominantly characterized by the re-flexive code (54% of narrative sequences), the psychodynamic-client-cen-tered is characterized by the external code (54% of narrative sequences),and the experiential therapy is distinguished by the higher frequency ofinternal narrative sequences (29%) in comparison with the other two thera-pies (perceptual-processing: 10%; psychodynamic-client-centered: 6%).

The overall number of narrative sequences identified in the perceptual-processing (162) and psychodynamic-client-centered (170) sessions werequite similar; however, the process-experiential therapy had more se-quences (227), indicating that there was more shifting between internal,external, and reflexive processes in this therapy; perhaps a result of activeinterventions such as chairing work. There were no significant differencesbetween numbers of topic segments (perceptual processing sessions: 107;

Table I. Percentage of the Three Narrative Sequence Types by Therapeutic Orientation

Therapy

Percep.-processing

Process-experiential

Psychodynamic-C-C

Period

123Total123Total123Total

Total Seq.

624555

162918254

227664757

170

% External

472731361932262553555454

% Internal

130709103624242908000906

% Reflexive

406760544544504639453740


process-experiential sessions: 112; psychodynamic-client-centered ses-sions: 113).

The Experiencing Scale and the Therapeutic Approaches

In order to compare the three dyads in terms of levels of Experiencing,the narrative sequences from the NPCS analyses were used as the unit ofanalysis. Across the three therapies, Experiencing Scale modal and peakratings were found to have significant differences using a repeated measuresanalysis of variance (ANOVA; df=l, F = 75.53, p = .0001; see Table II),with the highest ratings in the process-experiential approach, the lowest inthe perceptual-processing therapy, and the psychodynamic-client-centeredscores falling between the two groups.

The process-experiential approach differed significantly from the othertwo approaches in terms of Experiencing Scale ratings. The average modalrating given to the nine sessions was 2.76 (SD = 0.92) with an averagepeak rating of 3.47 (SD = 1.12). In the initial phase (sessions 2-4) theaverage mode was 2.93 (SD = 0.95) and the peak was 3.64 (SD = 1.04).The mode rating then dropped to 2.59 (SD = 0.86) along with the peakthat fell to 3.34 (SD = 1.08) in the middle phase (middle 3 sessions) oftreatment. The final phase (last three sessions) had an average experiencingmodel rating of 2.73 (SD = 0.94) with a peak of 3.38 (SD = 1.27).

The psychodynamic-client-centered therapy increased slightly both inmid- and high-experiencing as the sessions progressed. The overall meanmodal score was 2.49 (SD = 0.99) with a peak score of 3.14 (SD = 1.08).The modal ratings moved from an average of 2.45 (SD = 0.95; phase 1)down to 2.30 (SD = 0.69; phase 2) and then up to 2.68 (SD = 1.20; phase3) over the three stages of therapy analyzed. The peak ratings followed asimilar pattern moving from a score of 3.08 (SD = 0.98) down to 3.00(SD = 0.93) and then rising to 3.32 (SD = 1.27) over the three phasesof therapy.

Within the perceptual processing therapy, the overall mode and peakmean ratings were 2.28 (SD = 0.89) and 3.02 (SD = 1.02), respectively.

Table II. Experiencing Scale Ratings and the Therapeutic Approaches

Therapeutic approach

Perceptual-processingProcess-experientialPsvchodynamic-C-C

Overall

Mode

2.282.762.49

Peak

3.023.473.14

Phase one

Mode

2.112.932.45

Peak

2.793.643.08

Phase two

Mode

2.092.592.30

Peak

2.873.343.00

Phase three

Mode

2.642.732.68

Peak

3.403.383.32


Although the percentages of higher Experiencing Scale rating (ratings of5-7) were low during the first two phases of the treatment, they rose inthe last phase of the therapy. The mean model rating moved from 2.11(SD = 0.32) in the initial phase of treatment to 2.09 (SD = 0.29) in midther-apy then to 2.64 (SD = 1.42) in the final phase. Similarly, the mean peakratings increased from an initial score of 2.79 (SD = 0.73) to a midtherapymean of 2.87 (SD = 0.55) and then to 3.40 (SD = 1.44) in the last phaseof the brief therapy.

Experiencing Scale Ratings within Narrative Sequences

When examining the Experiencing Scale ratings contextualized bynarrative sequence type, external, internal, or reflexive, certain patternsare apparent within therapeutic modalities, while others seem to be evidentacross the sessions from all three dyads. Using a repeated-measures AN-OVA, the combined Experiencing Scale ratings of the three dyads werefound to be significantly different across the three types of narrative processsequences (df = 1, F = 415.02, p = .0001; see Table III). As ratings acrossnarrative sequence type were not significant when examined in interactionwith therapeutic modality, a decision was made to collapse ratings acrosstherapy in Table III.

Across the three therapeutic dyads, using a post hoc ANOVA, theinternal sequences were found to be significantly different from reflexiveand external sequences in terms of modal Experiencing scores (df = 4,F = 3.14, p = .03), but not in terms of peak Experiencing scores. Sincepeak ratings represent the highest score within a sequence, however brief,they may be less sensitive to sustained differences which may exist betweennarrative codes.

The external narrative sequences Experiencing Scale received ratingsthat fell primarily into levels two or three. The mean modal Experiencingrating for these sequences was 2.13 and the mean peak rating was 2.73.Only seven external sequences received ratings at level four or above froma total of 245 external narrative sequences. All of these ratings were peak

Table III. Experiencing Scale Ratings Within Narrative Sequence Type

Narrative sequence type

External narrative sequencesInternal narrative sequencesReflexive narrative sequence

Mean modal rating (SD)

2.13 (0.48)3.45 (0.69)2.55 (1.09)

Mean peak rating (SD)

2.73 (0.72)3.85 (0.73)3.43 (1.27)


ratings identified in the psychodynamic-client-centered therapy—the ther-apy characterized by a predominance of external narrative sequences.

The internal narrative sequences were mainly characterized by ratingsat levels 3 and 4. The mean mode rating was 3.45, while the mean peakrating was 3.85. There were some peak ratings at level six in the process-experiential therapy, the therapy most differentiated by a high number ofinternal narrative sequences.

The mean ratings for reflexive sequences were 2.55 for the mode and3.43 for the peak. These ratings seemed to be dichotomized. Lower modeand peak ratings of levels two and three as well as higher ratings of fiveand six were common while midrange ratings (level four) were much lesscommon. Eighty-nine percent of the five and six level modal ratings and81% of the peak ratings at the same levels occurred within the reflexivenarrative process type.

The LCPP and the Therapeutic Approaches

The units from the NPCS analyses were used for the application ofthe LCPP to the sessions from the three therapy dyads. An examinationof the levels of the LCPP revealed different patterns across therapeuticapproach. The results of a repeated-measures ANOVA indicated that theLCPP mode and peak scores for each therapy dyad were significantlydifferent from one another (df = 1, F = 51.98, p = .0001; see Table IV).The perceptual-processing therapy had the highest levels of LCPP for bothmode and peak ratings, followed by the process-experiential therapy, andthe psychodynamic-client-centered sessions obtained the lowest ratingsoverall.

For the set of nine perceptual processing sessions rated in this study,the mean mode rating was 3.49 (SD = 1.61) and the mean peak rating was4.77 (SD = 1.47). Over the sessions, the LCPP ratings were fairly constant,rising slightly from the initial phase of therapy to the final phase of therapy.In the initial phase the mean modal rating was 3.0 (SD = 1.27), increasingto 3.9 (SD = 1.32) and then lowering slightly to 3.7 (SD = 1.89). The mean

Table IV. Levels of Client Perceptual Processing and the Therapeutic Approaches

Therapeutic approach

Perceptual-processingProcess-experientialPsychodynamic-C-C

Overall

Mode

3.493.422.70

Peak

4.774.283.80

Phase one

Mode

3.003.042.60

Peak

4.083.983.72

Phase two

Mode

3.093.352.82

Peak

5.274.183.86

Phase three

Mode

3.734.132.71

Peak

5.154.913.84


peak rating in the initial phase was 4.08 (SD = 1.08); this rose to 5.27 (SD= 1.39) in the middle phase and then fell slightly to 5.15 (SD = 1.65) inthe final phase of this therapy.

Compared to the perceptual processing dyad, the process-experientialtreatment had lower LCPP ratings across the nine sessions with a meanmodal rating of 3.42 (SD = 1.51) and a mean peak rating of 4.28 (SD =1.47). The initial phase was given a mode rating of 3.04 (SD = 1.37) andpeak of 3.98 (SD = 1.28). These scores rose in the middle phase to a meanmodal score of 3.35 (SD = 1.51) and a mean peak score of 4.18 (SD =1.52). During the final three sessions, the mode increased to 4.13 (SD =1.49) and the peak to 4.91 (SD = 1.50).

The psychodynamic-client-centered therapy achieved the lowest rat-ings of LCPP scores with an overall modal rating of 2.7 (SD = 1.42) anda peak of 3.8 (SD = 1.69). These ratings were fairly consistent over thesessions examined. Initially, the mean modal score was 2.61 (SD = 1.32).During the middle phase it rose slightly to 2.82 (SD = 1.40), then fell againto 2.71 (SD = 1.57) during the final three sessions. The mean peak scoresassigned to the three groupings of sessions were very similar, beginning at3.72 (SD = 1.66), moving to a mean score of 3.86 (SD = 1.39) in the middlephase and ending with a mean of 3.84 (SD = 1.95) in the past phaseof treatment.

The LCPP within Narrative Sequences

Using a repeated-measures ANOVA, the combined scores of the threedyads were found to differ significantly in relation to narrative process type(df = 1, F = 386.79, p = .0001; see Table V). It appeared that in terms ofnarrative sequence type, LCPP ratings were highest in the reflexive code.There were no significant interactions found when analyzing narrative se-quence type in interaction with therapeutic modality for either the internalor reflexive sequences, but there is a significant therapy by narrative se-quence interaction for the external sequence type (df = 2, F = 5.10, p =.01). The most common modal score for all therapies within the externalsequences was level 2, elaboration. The peak scores, however, reveal a

Table V. The Levels of Client Perceptual Processing Within Narrative Sequence Type

Narrative sequence type

External narrative sequencesInternal narrative sequencesReflexive narrative sequences

Mean modal rating (SD)

2.41 (1.11)3.51 (1.66)3.76 (1.54)

Mean peak rating (SD)

3.61 (1.56)4.35 (1.46)4.76 (1.45)


different pattern, with the most common peak score in the perceptual-processing and the experiential therapies being level 4, differentiation withan analytic focus, while the most common peak score in the dynamic therapyremains at level 2.

Reflexive narrative sequences obtained the highest ratings on thisprocess measure (mean modal score = 3.76, mean peak score = 4.76) arecharacterized by processes such as discrimination of thoughts and feelings,self-evaluation, and integration—processes that are prized by the LCPP.Category four, differentiation with analytic focus, was therefore the mostfrequent modal rating for the reflexive sequences in both the perceptual-processing and process-experiential approaches. In the psychodynamic-client-centered dyads's reflexive sequences, category four was almost asfrequent as category six, reevaluation.

The mean scores for internal sequences (mean modal score 3.51, peak4.35) were somewhat lower than means for the reflexive sequences. Itseemed that the different therapeutic approaches had quite dissimilar LCPPratings within this narrative sequence type. The cognitive therapy hadlevel 2, or elaboration, as the most common category; in the experientialapproach, level 5, differentiation with internal focus, was the most common;and the psychodynamic-client-centered had level 6, reevaluation as themost frequently level rated. As the LCPP scale identifies types of cognitiveprocessing, internal sequences are found throughout the scale, in relationto their level of differentiation.

The Experiencing Scale and the LCPP

The two scales seemed to have a low positive correlation to one an-other. Within the perceptual-processing dyad, the modes of the Experienc-ing Scale and the LCPP had a Pearson correlation coefficient of .38 (n =158, p = .0001, r2 = .14) and a peak correlation of .37 (n = 158, p = .0001,r2 = .14). These results support previous research findings that comparedthese two measures on psychotherapy transcripts (Shirley, 1983; Toukman-ian, 1986). The correlations were slightly higher within the psychodynamic-client-centered therapy (mode .41 [n = 163, p = .0001, r2 = .17], peak .42[n = 163, p = .0001, r2 = .18]) and lower within the process-experientialtherapy (.12 [n = 225, p = .07, r2 = .01] and peak .17 [n = 225, p = .01,r2 = .03]). Although it seems that there is some relation in the ways thatthese measures identify positive therapeutic process, their ratings indicatethat they are fundamentally centered on different aspects of this process.


DISCUSSION

Study Limitations and Strengths

As this study focuses on one dyad from each therapeutic orientation,strong conclusions cannot be made from this data. Although findings fromthe NPCS analyses appeared consistent with the theory of therapy putforward by each approach, it is difficult to separate the effects of thetherapist and client upon each therapy. This study was meant, however, togenerate grounded hypotheses regarding the use of process measures onintegrative therapies and the potential of the NPCS as a research tool whenstudying integrative therapies. These hypotheses are being validated bylater studies (e.g., Hardtke, 1996) and will continue to be examined infuture research.

The Choice of a Narrative Unit

This study presents an example of a comparative exploration of psycho-therapies using different process measures. The adoption of a narrativeunit allowed for a semantically derived segmentation of text rooted in thetherapy narration and not based upon a model of process used by theprocess measures in question. As the NPCS uses narrative markers tosegment therapy transcripts, it has a semantically driven method of dividingtext instead of one based upon models that prize specific types of therapeuticprocess. Consequently, it can be used to describe, compare, and unitize bothintegrative and single-approach therapies as well as ratings from processmeasures derived from their philosophies. This study illustrated how thisunitization can provide a common unit that can allow for the consolidationof research from different integrative and single-orientation therapy ap-proaches and their process measures.

Initiating a Discourse on Process Measure Origin and Application

In this study, each set of sessions from the integrative therapeuticapproaches examined emphasized different types of narrative processingduring therapy sessions. Considering both the findings from the NPCS andthe process measure ratings, it appears that each therapy dyad used adifferent path to achieve a successful outcome: the perceptual-processingtherapist, a reflexive path; the process-experiential therapist, an internalpath; and the psychodynamic-client-centered therapist, an external path.


The results from this study suggest that each process measure mayidentify good therapeutic process in terms of the narrative process typethat can be thought to be most distinguishing of its own therapy of origin.As the principles behind each process measure evolved from the study ofprocesses characteristic of the therapy of origin, this correspondence maybe expected. Consequently, the highest processing scores for the measureswere found in the therapy sessions conducted in the approach closest tothat of the measure's origin and that had the highest number of thosenarrative sequences.

For instance, ratings from the LCPP, derived from a perceptual-pro-cessing orientation, were highest within the reflexive narrative sequences,the dominant narrative sequence type identified within the perceptual-processing treatment. The Experiencing scale, derived from client-centeredand Experiential therapy, rated internal sequences as most indicative ofhigh levels of client processing and "good process." These sequences werefound to be most common within the process-experiential therapy sessions.Similarly, in a study (Levitt, 1993) examining the Referential Activity Scale(Bucci, 1987), a dynamic measure, external narrative sequences were foundto have the highest scores on this psychodynamic-based scale and to bemost common in the psychodynamic-client-centered therapy sessions pre-sented in this paper. The origin of therapy process measures and the relationof this origin to expected findings, however, are often not discussed inresearch articles examining psychotherapy process.

The tentative findings presented in this paper are important as theyinitiate a discourse on the development and application of psychotherapyprocess measures. Researchers may wish to take care to explicate the originof each therapy approach examined in relation to the psychotherapy processmeasures they select for evaluation. For instance, it may not be meaningfulto evaluate a therapy predicated upon being externally focused by thedegree that it is internal, nor to judge an orientation with a reflexive ap-proach by an externally focused measure or unitizing system. This explica-tion may help distinguish "good" process from "desired" process, a particu-larly important distinction for the evaluation of integrative psychotherapies,which may entail more varied processes.

If an understanding of the relation between therapeutic orientationand therapy process measures is developed, psychotherapy researchers mayopt to use rating systems in a more targeted fashion when comparingtherapies. The application of process measures via a system of unitizinglike the NPCS may allow researchers to contrast integrative therapies, anddifferent therapeutic approaches, with the same process measures or tocontrast sequences more likely to be directed toward similar therapeuticgoals. For instance, although a cognitive therapy may be found to have


more reflexive sequences and a psychodynamic treatment may be foundto be focused on external dialogue, while within internal sequences boththerapies may have the expression of feeling as a focus or a goal. Byapplying a measure such as the Experiencing Scale to the internal sequencesof both therapies, a researcher can evaluate all the sequences that havethis common direction instead of contrasting the text of entire sessions, mostof which may be directed toward goals not valued by this process measure.

For example, one approach to identifying a good-outcome therapywould be to look for internal sequences rated high on the ExperiencingScale, reflexive sequences high on the LCPP, and external sequences ratedhigh on a measure such as the Referential Activity Scale. As well, a strategysuch as this one may provide a more economical way of studying therapyas one would not need to apply the process measure in a haphazard wayacross the entire transcript, but could focus on the units most relevant tothe task in question. This suggestion is one of many that may be proposedin response to the initiation of a dialogue on the application of processmeasures to therapies of different origins or to integrative therapies.

Implications for Clinical Training and Supervision

In terms of clinical training and supervision, the exploration of narra-tive process can be of use in identifying narrative processes most appropriatefor any one therapy task. Through researching narrative processes, integ-rative clinicians can be taught to initiate different narrative processes thatmay assist them toward the later implemention of different strategies intherapy. For instance, if therapists wish to heighten Experiencing they maybegin by initiating internal narrative processing to deepen client emotion,or they may shift to reflexive narrative processing if they feel they wish toraise a client's LCPP level. An understanding of the type of narrativediscourse in which different interventions are embedded could guide theimplementation of different therapy interventions in a more directed andmeaningful fashion.

ACKNOWLEDGMENTS

We would like to express our gratitude to Leslie Greenberg, KarenHardtke, Sandra Jackson, and Shake Toukmanian for their assistance withthis project. A previous version of this paper was presented at the Societyfor Psychotherapy Research 1994 Conference in York, England.


REFERENCES

Angus, L. E., & Hardtke, K. K. (1994). Narrative processes in psychotherapy. CanadianPsychology, 35, 190-203.

Angus, L. E., Hardtke, K. K., & Levitt, H. M. (1996). The Narrative Process Coding System:Revised version. Unpublished manuscript. York University, North York, Ontario, M3J1P3.

Angus, L., Levitt, H., & Hardtke, K. (in press). Narrative processes and psychotherapeuticchange: An integrative approach to psychotherapy research and practice. Journal ofClinical Psychology.

Arlow, J. (1989). Psychoanalysis. In R. Corsini & D. Wedding (Eds.), Currentpsychotherapies(4th ed.; pp. 197-238). Itasca, IL: F. E. Peacock Publishers.

Beck, A., Ward, C., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuringdepression. Archives of General Psychiatry, 4, 561-571.

Beitman, B. D. (1994). Stop exploring—Start defining the principles of psychotherapy integra-tion: Call for a consensus conference. Journal of Psychotherapy Integration, 4, 203-228.

Brown, H. D. (1984). Counsellor and client in-therapy verbal behaviour, Master's thesis, YorkUniversity, Toronto, Ontario.

Bucci, W. (1987). A manual for the coding of referential activity. Unpublished manuscript,Adelpni University: Garden City, NY.

Derogatis, L. (1977). SCL-90-R manual. Clinical Psychometrics Research Unit, John HopkinsUniversity School of Medicine.

Elliott, R. (1991). Five dimensions of therapy process. Psychotherapy Research, 1, 92-103.Fitts, W. (1965). Manual: Tennessee Self Concept Scale. Los Angeles, CA: Western Psychologi-

cal Services.Glass, C. R., Norcross, J. C., Arnkoff, D. B., & Lambert, M. J. (1993). Research directions for

psychotherapy integration: A roundtable. Journal of Psychotherapy Integration, 3, 91-131.Goldfried, M. (1991). Research issues in psychotherapy integration. Journal of Psychotherapy

Integration, 1, 5-25.Goldman, R. (1991). The validation of the Experiential Therapy Adherence Measure. Master's

thesis, Department of Psychology, York University, Toronto, Ontario.Goncalves, O. F. (1994). From epistemological truth to existential meaning in cognitive narra-

tive psychotherapy. Journal of Constructivist Psychology, 7, 107-118.Concalves, O. F., Korman, Y., & Angus, L. (in press). Constructing psychopathology from a

cognitive narrative perspective. In R. A. Neimeyer & J. O. Raskin (Eds.), Constructions ofdisorder: meaning-making frameworks for psychotherapy. Washington, DC.: APA Press.

Greenberg, L., & Goldman, R. (1992, April). Experiential therapy adherence measure. Paperpresented at Society for Psychotherapy Integration Conference, San Diego, CA.

Greenberg, L., Rice, L., & Elliott, R. (1993). Facilitating emotional change: The moment-by-moment process. New York, The Guilford Press.

Hardtke, K. (1996). Characterizing therapy focus and exploring client process: Investigatingtherapeutic modalities from a narrative approach. Master's thesis, Department of Psychol-ogy, York University, Toronto, Ontario.

Hill, C., & Corbett, M. (1992). A perspective on the history of process and outcome researchin counseling psychology. Journal of Counseling Psychology, 40, 3-25.

Horowitz, L. M., Rosenberg, S. E., Baer, B. A., Ureno, G., & Villasneno, V. S. (1988). Inventoryof interpersonal problems: Psychiatric properties and clinical application. Journal ofClinical and Consulting Psychology, 56, 885-892.

Jackson, S. (1993). A multilevel analysis of clients' self narratives in psychotherapy. Master'sthesis, Department of Psychology, York University, Toronto, Ontario.

Klein, M. H., Mathieu-Coghlan, P. L., & Gendlin, E., & Kiesler, D. J. (1970). The ExperiencingScale: A research and training manual (2 vols.). Madison: Wisconsin Psychiatric Institute,Bureau of Audio-Visual Instruction.

Klein, M. H., Mathieu-Coghlan, P. L., & Kiesler, D. J. (1986). The Experiencing Scales. In


L. Greenberg & M. Pinsoff (Eds.), The psychotherapeutic process: A research handbook(pp. 21-71). New York: The Guilford Press.

Korman, Y. (1995). Freezing to melting: Metaphor themes in good outcome psychotherapy.Honour's thesis, Department of Psychology, York University, Toronto, Ontario, M3J 1P3.

Korman, Y., & Angus, L. E. (under revision). From losing to winning: An inductive analysisof metaphor themes in psychotherapy. Submitted to the Journal of Psychotherapy Research.

Levitt, H. (1993). A comparative analysis of the Narrative Process Coding System and threestandardized psychotherapy process measures: A multi-modal analysis. Master's thesis.Department of Psychology, York University, Toronto, Ontario, M3J 1P3.

Levitt, H. M., Hardtke, K. K., & Angus, L. E. (1993, June). Two approaches to analysingnarratives in the Gloria sessions: A comparative analysis. In L. Angus (Moderator),Two approaches to analysing narrative(s) in the Gloria sessions: A comparative analysis.Symposium conducted at the Society for Psychotherapy Research Conference, Pitts-burgh, PA.

Levitt, H., Korman, Y., & Angus, L. (1997). A metaphor analysis in a good and a pooroutcome treatment of depression: "Unloading burden" versus "Being burdened." Psico-logia: Teoria, Investigao e Pratica, 1, 329-346.

Levitt, H., Korman, Y., & Angus, L. (in press). A metaphor analysis in treatments of depression:metaphor as a marker of change. Counselling Psychology Quarterly.

Levitt, H. M., Angus, L. E. & Hardtke, K. K. (under revision). An inquiry into psychotherapeu-tic orientation and the question of client change: A narrative approach.

Luborsky, L., & Crits-Christoph, (1990). Understanding transference: The CCRT method. NewYork: Basic Books.

Maher, A., Stalikas, A., Boissoneault M., & Tauer, K. (1990). A scale for assessing degreeof strength of client feeling. Canadian Journal of Counselling, 24, 107-116.

McLeod, J., & Balamoutsou, S. (1986). Representing narrative process in therapy: Qualitativeanalysis of a single case. Counselling Psychology Quarterly, 19, 61-76.

Neimeyer, R. A. (1994). The role of client-generated narratives in psychotherapy. Journal ofConstructivist Psychology, 7, 229-242.

Polkinghorne, D. (1988). Narrative knowing and the human sciences. Albany: State Universityof New York Press.

Rennie, D. L. (1994). The functions of story-telling in psychotherapy: The client's subjectiveexperience. Psychotherapy, 31, 234-243.

Rice, L. N., Koke, C. J., Greenberg, L. S., & Wagstaff, A. K. (1979). Manual for Client VocalQuality. York University, Toronto, Ontario.

Richards, P. S., & Lonborg, S. D. (1996). Development of a method for studying thematiccontent of psychotherapy sessions. Journal of Consulting and Clinical Psychology,64(4), 701-711.

Russell, R. L., & Staszewski, C. (1988). The unit problem: Some systematic distinctions andcritical dilemmas for psychotherapy process research. Psychotherapy, 25, 191-200.

Sarbin, T. (1982). Narrative psychology: The storied nature of human conduct, New York:Praeger Publishers.

Sherman, E., & Skinner, K. W. (1988). Client language and clinical process: A cognitive-semantic analysis. Clinical Social Work Journal, 16, 391-405.

Shirley, M. (1983). Three measures of client experiential process: Their relationship to eachother and client cognitive differentiation. Unpublished Master's Thesis, York University,Toronto, Ontario.

Sinclair, L. (1990). Metaphor and client perceptual processing. (Doctoral dissertation, Depart-ment of Psychology, York University, Toronto, Ontario) Dissertation Abstracts Interna-tional, 51, 460813.

Toukmanian, S. (1986). A measure of client perceptual-processing. In L. Greenberg & M.Pinsoff (Eds.), The psychotherapeutic process: A research handbook (pp. 107-130). NewYork: The Guilford Press.

Toukmanian, S. (1990). A schema-based information processing perspective on client changein experiential therapy. In J. Lietar, J. Rombauts, & R. Van Balen (Eds.), Client-centred


and experiential psychotherapy in the nineties (pp. 309-326). Leuven, Belgium: LeuvenUniversity Press.

Toukmanian, S. (1992). Studying the client's perceptual processes and their outcomes intherapy., In S. Toukmanian & D. Rennie (Eds.), Psychotherapy process research (pp.77-107). Newbury Park, CA: Sage Publications.

Toukmanian, S. (unpublished manuscript). Treatment Integrity Measure of Perceptual Pro-cessing. York University, Toronto, Ontario, M3J 1P3.

Vogel, D. (1994). Narrative perspectives in theory and therapy. Journal of ConstructivistPsychology, 7, 243-261.

Weissman, M., & Bothwell, S. (1976). Assessment of social adjustment by patient self report.Archives of General Psychiatry, 33, 1111-1115.

White, M., & Epson, D. (1990). Narrative means to therapeutic ends. New York: W. W. Norton.Zink, D. (1990). Change in the context of perceptual-processing therapy: Process outcome

research. Master's thesis, Department of Psychology, York University, Toronto, Ontario.

Psychotherapy process measure research and the evaluation of psychotherapy orientation: A narrative...

Documents

Transcript of Psychotherapy process measure research and the evaluation of psychotherapy orientation: A narrative...