Dr. J H Smith Is it Time for a Paradigm Shift in Psychology 2014

17
PARADIGM SHIFT 1 | Page IS IT TIME FOR A PARADIGM SHIFT IN PSYCHOLOGY? John H. Smith PhD Peter Bauer LCSW Veterans Administration Medical Center San Antonio, Texas

Transcript of Dr. J H Smith Is it Time for a Paradigm Shift in Psychology 2014

PARADIGM SHIFT

1 | P a g e

IS IT TIME FOR A PARADIGM SHIFT IN PSYCHOLOGY?

John H. Smith PhD

Peter Bauer LCSW

Veterans Administration Medical Center

San Antonio, Texas

PARADIGM SHIFT

2 | P a g e

ABSTRACT

The purpose of this paper is to suggest that a paradigm shift is needed in the field of

Psychology due to the recent attempts to define what is “scientific” from what is not. In light of

ours society’s notable emphasis on science, designating certain work as “scientific” carries

considerable weight. Thus, if certain terms are used to designate certain work as scientific then

works not so designated are seen as “unscientific” and of less value—even before a thorough

reading has been attempted. Terms such as Evidence Supported Therapies (EST’s), Evidence

Based Therapies (EBT’s) and evidenced based practices (EBP’s) are defined and clarified. A

Paradigm Shift is presented which will encourage responsible research, training, and practice,

without undue and unproductive acrimony.

KEYWORDS: evidence supported theories, evidence based theories, evidence based practices

PARADIGM SHIFT

3 | P a g e

IS IT TIME FOR A PARADIGM SHIFT IN PSYCHOLOGY?

The evidence-based therapies (EBTs) can include but are not limited to those based on

randomized controlled trials (RCTs). To avoid confusion, since different authors have used

various terminologies, the authors have chosen to follow the distinctions made by Wachtel

(2010). The terms empirically supported or empirically validated are usually used to denote only

those treatments deemed to be “scientific” (i.e. based on the use of studies employing (RCTs).

Evidence-based practices (EBPs) are generally broader, and include treatments which may or

may not be based on RCTs, but are based on recognized scientific evidence. Despite the conflict

in the field of psychology regarding what constitutes scientific evidence, there are four areas of

general agreement: The importance of evidence: educating the patient on the nature of

problems: learning basic coping skills: and symptom reduction.

Evidentiary support is especially important in light of the findings regarding the negative

effects of psychological treatments (Barlow, 2010). According to Barlow (2010), the negative

effects of some treatments are occasionally concealed in the “average improvement scores.”

Thus, treatments can clearly do harm to patients, so carefully identifying treatments with good

supporting evidence is critical. Tragically therapies have led to patient deaths (e.g. re-birthing so

the risk of employing unsupported treatments is not simply an academic one (Barlow 2010).

In general, if patients is more aware of the nature and course of their problem areas, they

likely will derive greater benefit from therapy. Learning basic coping skills is often helpful in

dealing with the identified problems e.g. anger management. Symptom reduction, while not

necessarily the ultimate goal of treatment, can be highly significant in improving the quality of

life for the patient.

PARADIGM SHIFT

4 | P a g e

The above noted general areas of agreement, however, mask some important areas of

differences re: what constitutes evidence. The issue of what constitutes acceptable evidence has

a major bearing on the types of research conducted, and the areas deemed sufficiently important

to be the focus of research. In addition, student training and the general conduct of clinical

practice are also deeply impacted by the prevailing views of what constitutes acceptable

evidence. A discussion of these areas of difference is given below.

IMPORTANT AREAS OF DIFFERENCE

In recent years, the field of psychology has attempted to identify and promote evidence

supported therapies (EST) (American Psychological Association [APA], 2006). While this goal

is positive in that it attempts to encourage care based on best evidence, there is a clear downside.

Studies have shown such lists of ESTs generally rely on “gold standard” studies using RCTs

(Wachtel 2010: Westen, Novotny & Thompson-Brenner, 2004). Treatments are then

dichotomously listed as supported or not supported. Studies dispute this, pointing out that RCTs

apply in some areas and not well in others (Westen et al., 2004). This application of RCTs in

some cases was found to be completely invalid. Westen et al. go on to argue for “a shift from

validating treatment packages to testing intervention strategies (our emphasis—by Westen) and

theories of change that clinicians can integrate into empirically informed therapies.” Treatment

packages are essentially equivalent to treatment manuals. Intervention strategies are based on

broad, theoretically consistent approaches which allow for more clinical judgment and flexibility

on the part of the therapist. This approach is further supported by work in Norcross (2011). The

single-minded insistence on RCTs as a “gold standard” has, in part, had the effect of encouraging

a false polarization within psychology: those that treat based on “evidenced” and those that do

not.

PARADIGM SHIFT

5 | P a g e

One example of an approach that has been criticized due to lack of “evidence” (and

therefore not qualified as an EST) is psychodynamic psychotherapy (PP). PP has been

vehemently criticized in many settings (Knekt et al., 2008a; Shedler, 2010), with claims that it

does not have a solid evidentiary base, or, an anecdotally as an “outmoded” approach (see

Wachtel, 2010). However, Leichsenring and Rabung (2008) provide substantial evidence for the

efficacy of this psychodynamic approach to treatment (see also Shedler, 2010). Unfortunately,

there appears to have developed an ideological war, where evidence based on other than RCTs is

seen as insufficient. The treatments that do not meet the “gold standard” (RCTs) are deemed to

be (by implication) unscientific, and likely to be outmoded and indefensible. The great wealth of

evidence traditionally gained from systematic observations, patient and client reports, surveys,

correlational data, and naturalistic observational studies is often discarded or ignored in favor of

RCTs. However, evidence gained by these traditional methods can greatly assist in

understanding which treatment interventions (not treatment packages) would be or were most

effective (Westen et al., 2004). For example, in treating depression, a person may also suffer

from co-morbid substance abuse disorder, and have a diagnosed personality disorder, which

further complicates treatment. The treatment package (treatment manual) for depression likely

does not fully address these complicating factors. However, treatment interventions which

address the substance abuse and personality disorder factors, guided by an overall theoretical

understanding of the case, would allow the therapist to address these factors. The substance

abuse and personality disorder can complicate the depression, but may be somewhat independent

of the depression in their expression and effects on the person treated. Despite the fact that some

manualized therapies allow for some flexibility, focusing on the treatment interventions, and not

PARADIGM SHIFT

6 | P a g e

treatment manuals, allows for more fully addressing complicating factors, while still treating the

depression in a theoretically consistent manner.

POSSIBLE DANGERS OF TOO NARROW A DEFINITION OF “EVIDENCE”

In short, the narrowing view of what constitutes “scientific evidence” has encouraged the

field splitting off into warring theoretical halves. The theoretical camps seem to ignore or to heed

Luborsky, Singer, and Luborsky’s (1975) encouragement to focus on what is done in the therapy

session (see also Seligman, 1995). As had been noted previously, the effects of investigator

allegiance to particular theoretical stances often all but guarantee that the approach preferred by

the experimenter will be “found” to be the most effective (Luborsky et al., 1999).

The narrowing focus on what constitutes “evidence” has much more than theoretical

importance. As a practical matter, grants often appear to be made more on the basis of a rigid

insistence on RCTs (Westen et al., 2004). Also, student training in many places has focused

narrowly on approaches which are deemed to meet the “gold standard” that typically involves

short term, cognitive behavioral therapies, and manualized therapies that seek to minimize

individual clinical judgments (Shedler, 2010; Wachtel, 2010; Westin et al., 2004). Thus, the

scope of scientific inquiry is unnecessarily narrowed while a concomitant narrowing in the

breadth of clinical training is occurring (M. Lambert, personal communication, August, 2008).

This means that useful intervention strategies may be discarded as non-gold standard. Often,

clinicians in training are essentially groomed to be technicians who execute manualized therapies

for supposedly specific disorders (e.g., depression).

ASSUMPTIONS OF MANUALIZED THERAPIES

While manual therapies can be helpful (Westen et al., 2004), and can be more easily

studied due to their specificity, unfortunately they make some highly untenable assumptions.

PARADIGM SHIFT

7 | P a g e

Examples of untenable assumptions made by manualized therapies are as follows: psychological

processes are highly malleable; most patients have one primary problem or can be treated as if

they do; the pragmatics of DSM-IV-TR diagnosis; the problem of co-morbidity and its different

role in research versus clinical practice; psychological symptoms can be understood and treated

in isolation form personality dispositions (see Westen et al., 2004). In brief, however, the

malleability assumption is revealed in the typically brief periods of treatment (6-16 sessions).

Most patients may present with one primary problem, but increasingly complex problems

typically emerge, making the second assumption most unsupportable. The pragmatics of DSM-

IV-TR diagnosis may also present difficulties as a committee arrived at these diagnoses and are

not pure or confined to one category (e.g., note problem of co-morbidity; multiaxial system).

One person can have multiple diagnoses. How many manuals are needed to treat these multiple

disorders that first appeared to be one disorder? In short, assumptions that, for example,

depression is a unitary disorder or is a single disorder that can be treated as if the person did not

have more than one disorder seem especially untenable (Parker & Fletcher, 2007). Evidence

shows that many persons presenting for care are not fully aware of what the main problem is, and

often more than one significant problem emerges, possibly complicated by co-morbid substance

abuse or Axis-II personality disorders (Westen et al., 2004). Thus, to the extent that students are

being trained as if presenting one presenting problem was sufficient, they are being trained to

treat disorders not people who exist.

As noted above, most people present for care with more than one significant (and likely

related) problem. The DSM-IV-TR is purposely constructed in a multidimensional (multiaxial)

manner so that the complexity of the persons’ difficulties can be more fully represented and

recognized in treatment. Thus, if students are trained only in the ESTs, their capacity to work

PARADIGM SHIFT

8 | P a g e

with complex multifaceted cases will be markedly limited. While allowing for “clinical

judgment” may introduce unwanted uncertainties in research studies, reducing professionals to

the role of technicians dispensing pre-programmed procedures is clearly not the answer. If the

patient is depressed, anxious, and exhibits some psychotic episodes, is the treating person

supposed three separate manuals (one each for depressions, anxiety, psychotic episodes)? If so, is

one assuming that all conditions are independent and unrelated? This is highly questionable

assumption. Given the sheer number of DSM-IV-TR diagnoses, the number of manuals could

rapidly become overwhelming (Westen et al., 2004). As noted by Luborsky, Barber, Siquelend,

McLellan, and Woody (1997), “the psychotherapist matters.” Thus, attempts to minimize the role

and judgment of the psychotherapist may be counterproductive to good treatment.

EFFECTIVENESS OF SHORT TERM THERAPIES WITH INCREASING CASE COMPLEXITIES

In addition to the above considerations, manualized therapies are often brief, and used

where a maximum of about 10-12 sessions are planned. While the manualized therapies are

helpful for more delimited problems (Corey, 2009), as the complexities of the problems increase,

shorter-term therapies often prove insufficient (Anderson & Lambert, 2001; Hansen & Lambert,

2003; Knekt et al., 2008b; Westen et al., 2004). For example, treatment of post-traumatic stress

disorder, to chronic and severe PTSD (APA, 2000). It is also clear that all those exposed to

traumas do not develop a form of PTSD. The reasons why PTSD may not always develop are

complex and not fully understood. It is possible that such protective factors as pre-existing

positive coping mechanisms and abilities, good support systems, and a minimum history of pre-

existent traumas may be involved. Given the range of disorders currently included under the

general rubric of PTSD, and arguments advanced by Herman (1992) that Disorders of Extreme

Stress (DESNOS) are qualitatively different from other forms of PTSD, it is clear that a

PARADIGM SHIFT

9 | P a g e

prescribed “treatment package” is not the way to go (especially in light of previously cited

evidence by Westen et al., 2004). The challenge of understanding and treating PTSD has been

great, and has spawned numerous treatments, which are often tied to learning theory and

conditioning models. Examples of such treatment approaches tied to learning theory and

conditioning models are: Cognitive Behavior Therapy (CBT); Interpersonal Therapy (IPT);

Dialectical Behavioral Therapy (DBT); and Prolonged Exposure Therapy (PET) that were

developed by Foa, Hembree, and Rothbaum (2007) and rely on a deconditioning type model as

does Eye Movement Desensitization and Reprocessing (EMDR) (Corey, 2009). The various

treatments listed vary considerably in the degree of evidence effectiveness. For example, some

articles are severely critical of EMDR (Rothmayer, 2003); but the CBT treatments are usually

prominently listed as among the “first line” treatments for PTSD (Department of Veterans

Affairs/Department of Defense, 2004). An interesting debate has been occurring regarding the

assumption made by many that trauma-focused therapies are necessarily the best treatments for

PTSD (Wampold, 2001). Furthermore, in Wampold’s work disputing the superiority of trauma-

focused therapies for PTSD has been contested by several authors (Bisson & Andrew, 2007;

Ehlers et al., 2010).

One of the treatments that has had great success in reducing severe anxiety and fear in

PTSD has been Prolonged Exposure Therapy (PET). However, unfortunately, many patients

drop out of PET prematurely, not being willing to undergo the considerable stress and anxiety

that this procedure typically entails (Corey, 2009). However, persons who have experienced a

single incident of trauma such as rape may qualitatively differ from soldiers who are repeatedly

traumatized during one, two, or more tours of duty in Iraq or Afghanistan. While not minimizing

the long-term effects of even one highly traumatic experience, it may be that repeated traumas

PARADIGM SHIFT

10 | P a g e

experienced over a prolonged period of time could produce highly severe and chronic PTSD,

especially when care is delayed and/or is insufficient (see Veterans for Common Sense v. U.S.

Department of Veterans Affairs). As noted by Westen et al., as the complexity of the disorder

increases, STT prescribed care may well prove to be inadequate. This is especially true in light of

the marked co-morbidity of substance abuse disorders (Westen et al., 2004). Thus, depending on

how soon after the trauma(s) treatments are initiated, severity, etc., a variety of treatment

approaches or strategies may prove fruitful without resorting to prescribed “treatment packages”

which are “evidence based”. Given the previously cited problems in how evidence is gathered

(i.e., uneven appropriateness or total inappropriateness of randomized controlled trials) the

results are variably applicable methods lead to variably supported methods of care. Stated

somewhat differently, if the gathering of evidence is not fully valid in the manner in which it is

executed, then the treatments based on such evidenced are not fully valid either.

CURRENT AND PROPOSED PARADIGM SHIFT

The current paradigm seems to be that only ESTs (usually based on randomized

controlled trials) should be used, and that those not following this definition are unscientific and

are practicing outside the “Best Practice Guidelines” (see APA Task Force, 2007). The proposed

paradigm shift is that given the complexity of people, a variety of EBPs should be respected,

considered, and utilized. The EBPs can include but are not limited to those based on randomized

controlled trials. The proposed paradigm shift encourages openness to a variety of evidence-

supported approaches, without resorting to a certified list based only on RCTs for “evidence”.

The hope is that a variety of EBPs will be available, with data on effectiveness relative to the

complexity of the disorder(s) treated, and evidence including that based on RCTs but not limited

to same. The available evidence supporting care can be both researched and practiced without

PARADIGM SHIFT

11 | P a g e

undue acrimony among the professionals attempting to advance the cause of the science of

psychology and its application to clinical treatment and care.

Students in training to become effective psychologists doing clinical treatment should be

encouraged and taught to be open minded and informed about various treatments available, and

their respective evidentiary bases. The persons who come to psychologists for treatment deserve

to be considered for a wide range of EBPs which may be most helpful to them, with this

consideration not limited by narrow conceptions of what constitutes evidence-based treatment.

The following describes three therapy scenarios in which increase flexibility in approach to

treatment would be needed. Person A presents with depression clearly related to a specific

environmental event such as job less. In this case, a basic manualized therapy approach to

treatment of depression may be sufficient and effective. Person B presents for treatment with the

above noted description, but there is complicating abuse of substances, which does not appear to

be entirely related to the job loss. In this case, a theoretically more complex situation may well

obtain, e.g., the person’s depression may be found to pre-date significantly the job loss. Person C

presents for care with symptoms of what appears to be situational depression. However, as

further history is obtained, it becomes clear that there is a long history of depression which

considerably pre-dates the reported job loss, and appears to be part of a long established family

pattern of depression. In addition, the person admits to a much longer history of substance abuse,

and repeated and prolonged episodes of depression even when substances were not abused.

Further exploration reveals some very significant and self-defeating patterns of personality

dysfunction.

In considering the above examples where treatment of depression is involved, it is clear

that the degree of complexity of the care needed has markedly increased from example one to

PARADIGM SHIFT

12 | P a g e

example three. In order to address adequately the increasingly complex treatment needs of the

person presenting for care, the therapist must have the training and the freedom to modify a

manualized approach to treatment of depression, and must be able to address complex care needs

in a theoretically consistent manner involving much clinical judgment and skill.

The proposed paradigm shift is suggested as a means of restoring a more balanced and

accurate (humble?) stance regarding the complexities involved in research, training and practice.

It may be well for people to heed the often quoted warning of Santayana (1905) that those who

fail to learn the lessons of history are doomed to repeat them.

DISCUSSION

This paper supports the use of EBTs, not treatment based simply on theoretical notions. Further,

psycho-educational approaches and materials are seen as helpful in a variety of therapy

situations. The authors support a more open and inclusive definition of EBT, and assert that too

narrow a view of what is defined as “evidence” has led to unproductive and damaging disputes,

which have harmed research, training, and clinic practice.

PARADIGM SHIFT

13 | P a g e

REFERENCES

Abbass, A.A. (2003). The cost-effectiveness of short-term dynamic psychotherapy. Expert

Review of Pharmacoeconomics and Outcome Research 3, 535-539. doi:

10.1586/14737167.3.5.535.

Anderson, M., & Lambert, M. (2001). A survival analysis of clinically significant change in

outpatient psychotherapy. Journal of Clinical Psychology, 57, 875-888. doi:

10.1002/jclp.1056

American Psychological Association (2006). Evidence based practice in psychology. The

American Psychologist, 61 (4), 271-285. doi: 10 1037/0003-066X.61.4.271.

American Psychological Association. Task Force on Serious Mental Illness and Sever Emotional

Disturbance (2007). Catalog of clinical training opportunities: Best practices for

recovery and improved outcomes for people with serious mental illness. Retrieved from

www.apa.org/practice/resources/grid/catalog.pdf

American Psychiatric Association (2000). Diagnostic and statistical manual of mental disorders

(4th ed., text revision). Washington, DC: author.

Barlow, D.H. (2010). Negative effects from psychological treatments: A perspective. American

Psychologist, 65 (1), 13-20.

Bisson, J. & Andrew, M. (2007). Psychological treatment of post-traumatic stress disorder

(PTSD). Cochrane Database of Systematic Reviews. 2007 (3) 1-109. doi:

10.1002/14651858,CD003388.Pub3.

Corey, G. (2009). Theory and practice of counseling and psychotherapy (8th ed.). Belmont, CA:

Brooks/Cole.

PARADIGM SHIFT

14 | P a g e

Courtois, C., Ford., J.D., Van der Kolk, B.A., & Herman, J.L. (2009). Treating complex

traumatic stress disorders: An Evidence based Guide. New York, NY: Gilford Press.

Department of Veterans Affairs and Department of Defense. (2004). VA/DOD clinical guidelines

for the management of post-traumatic stress. Retrieved online from

http://www.healthquality.va.gov/ptsd/ptsd.pdf

Ehlers, A., Bisson, J., Clark, D.M., Creamer, M., Pilling, S., Richards, D……Yule, W. (2010).

Do all psychological treatments really word the same in post-traumatic stress disorder?

Clinical Psychology Review, 30, 269-276. doi: 10.1016/j.cpr.2009.12.001

Foa, E., Hembree, E., & Rothbaum, B.O. (2007). Prolonged exposure therapy for PTSD:

Emotional processing of traumatic experiences. New York, NY: Oxford University

Press.

Hansen, N.B., & Lambert, M. (2003). An evaluation of the dose-response relationship in

naturalistic treatment settings using survival analysis. Mental Health Services Research 5

(1), 1-12.

Herman, J.I. (1992). Trauma and recovery: The aftermath of violence—from domestic abuse to

political terror. New York, NY: Basic Books.

Knekt, P., Lindfors, O., Harkanen, T., Vallkiski, M., Virtala, E., Laaksonen, M.A…..The

Helsinki Psychotherapy Study Group (2008a). Randomized trial on the effectiveness of

long-and short-term psychodynamic psychotherapy and solution-focused therapy on

psychiatric symptoms on a 3-year follow-up. Psychological Medicine, 38, 689-703. doi:

10.1017/S003329270700164X.

PARADIGM SHIFT

15 | P a g e

Knekt, P., Lindfors, O., Laaksonen, M.A., Raitasalo, R., Haaramo, P., Jarbikiski, A., & The

Helsinki Psycholtherapy Study Group (2008b). Effectiveness of short-term and long-term

psychotherapy on workability and functional capacity: A randomized clinical trial on

depressive and anxiety disorders. Journal of Affective Disorders, 107, 95-106. doi:

10.1016/j.jad.2007.08.005.

Leichsenring, F., & Rabung, S. (2008). Effectiveness of long-term psychodynamic

psychotherapy: A meta-analysis. The Journal of the American Medical Association, 300,

(13), 1551-1565.

Luborsky, L., Singer, B., Luborsky, L. (2975). Comparative studies of psychotherapy: is it true

that “everyone has won and all must have prizes?” Archives of General Psychiatry, 32,

995-1008.

Luborsky, L., Barber, J.P., Siqueland, L., McLellan, A.T., & Woody, G. (1997). Establishing a

therapeutic relationship with substance abusers. In L.S. Onken, J.D. Blaine, & J.J. Boren

(eds.). Beyond the Therapeutic Alliance: Keeping the Drug Dependent Individual in

Treatment (pp. 233-244). Washington DC: U.S. Government Printing Office.

Luborsky, L., Diguer, L., Seligman, D.A., Rosenthal, R. Kraus, E.D., Johnson, S….Schweizer,

E. (1999). The researches own therapy allegiances: A “wild card” in comparisons of

treatment efficacy. Clinical Psychology: Science and Practice, 6, 95-106. doi:

10.1093/clipsy.6.1.95.

Lynch. D., Laws, K.R., & McKenna, P.J. (2010). Cognitive behavioural therapy for major

psychiatric disorder: Does it really work? A meta-analytical review of well-controlled

trials. Psychological Medicine, 40, 9-24. doi: 10.1017/Soo3329170900590X.

National Academies (2007, October 19). Effectiveness of most PTSD therapies is uncertain.

PARADIGM SHIFT

16 | P a g e

Science Daily. Retrieved may 28, 2011 from http://www.science

daily.com/releases/2007/10/07101823518.htm.

Norcross, J.C. (ed.). (2011). Psychotherapy relationships that work: Evidence based

responsiveness (2nd ed.) New York, NY: Oxford University Press.

Office of the Chief of Public Affairs Press Release (2010). Army health promotion, risk,

reduction, and suicide prevention (Report 2010). Retrieved from

http://usarmy.vo.llnwd.net/el/HPRRSP/HP-RR-SPReport2010_v00.pdf

Parker, G. & Fletcher, K. (2007). Treating depression with the evidence-based psychotherapies:

A critique of the evidence. Acta Psychiatrica Scandinavica, 115, 352-359.

Rothmayer, C. (2003). Treating Post-Traumatic Stress Disorder (PTSD) with FMDR. Retrieved

from Vanderbilt University, Health Psychology Home Page website:

healthpsych.psy.vanderbilt.edu/EMDR_PTSD.htm.

Santayana, G. (1905). Life of Reason: Reason in Common Sense. New York, NY: Charles

Scribner’s Sons.

Seligman, M.E.P. (1995). The effectiveness of psychotherapy: The consumer reports study.

American Psychologist, 50, 965-974. doi: 10.1037/0003-066X.50.12.965.

Shedler, J. (2010). The efficacy of psychodynamic psychotherapy. American Psychologist, 65,

98-109. doi: 10.1037/a0018378.

Solomon, S.D. & Johnson, D.M. (2002) Psychosocial treatment of post-traumatic stress disorder:

A practice-friendly review of outcome research. Journal of Clinical Psychology, 58, 947-

959. doi: 10.1002/jclp.10069.

Veterans for Common Sense v. U.S. Department of Veterans Affairs. No. 08-16728 D.C. No.

3:07-cv-03758-SC-OPINION (2011).

PARADIGM SHIFT

17 | P a g e

Wachtel, P.L. (2010). Beyond “ESTs”: Problematic assumptions in the pursuit of evidence

based practice. Psychoanalytic Psychology, 27 (3), 251-272.

Wampold, B.E. (2001). The Great Psychotherapy Debate (2nd ed.): Models, Methods, and

Findings, Mahwah, NJ: Lawrence Erlbaum.

Westen, D. & Morrison, K. (2001). A multidimensional meta-analysis of treatments for

depression, panic, and generalized anxiety disorder: An empirical examination of the

status of empirically supported therapies. Journal of Consulting and Clinical Psychology,

69, 875-899. doi: 10.1037/0022-006X.69.6.875.

Westen, D., Novotny, C.M., & Thompson-Brenner, H. (2005). The empirical status of

empirically supported psychotherapies: Assumptions, findings and reporting in

controlled clinical trials. Psychological Bulletin, 130, 631-663. doi: 10.1037/0033-

2909.130.4.631.