Comparative Effectiveness of Holding Therapy with Aggressive Children

11
Comparative Effectiveness of Holding Therapy with Aggressive Children Robin Myeroff, PhD Gary Mertlich, PhD University of Toledo Jim Gross, MA ABSTRACT: This study was undertaken to assess the effects of holding therapy on children who have a history of aggressive and delinquent behaviors. The study design was a prospective, pre-post, quasi-experimental controlled study. The subjects were re- cruited through the Attachment Center at Evergreen, Colorado. Eligible subjects were adopted children between the ages of 5-14 years, living in the present adopted home for at least one year. All children had a history of aggressive and delinquent behaviors, which prompted the contact with Evergreen. Findings resulted in significant decrease in the outcome variable for the treatment group within this study. KEY WORDS: Achenbach; Childhood Aggression; Treatment; Holding Therapy; Adop- tion; Foster Care. Young aggressive children can commit such defiant and destructive acts as lying, stealing, vandalism, fire setting, and running away. 1 While it is well accepted that aggressive and antisocial behaviors in childhood are related (violent, criminal outcomes), 2,3,4 it is only over the last ten years that an increase in adolescent violent acts (142% increase in murder and manslaughter) has been identified. 5 Conse- quently, aggression in children has been posted as a major public health concern. 6 Aggressive behaviors are included under a wide variety of psychi- atric terms ranging from attention deficit hyperactivity through oppo- sitional defiant, conduct, and attachment disorders. 7 Distribution of these disorders is estimated to be between 2 to 16 percent of children within the general population. 7 It is not surprising that researchers Received December 20, 1997; For Revision April 7, 1998; Accepted September 22, 1998. Address correspondence to Robin Myeroff, PhD, 3800 Park East Dr., Suite 150, Beachwood, OH 44122. Child Psychiatry and Human Development, Vol. 29(4), Summer 1999 © 1999 Human Sciences Press, Inc. 303

Transcript of Comparative Effectiveness of Holding Therapy with Aggressive Children

Comparative Effectiveness of Holding Therapywith Aggressive Children

Robin Myeroff, PhDGary Mertlich, PhD

University of ToledoJim Gross, MA

ABSTRACT: This study was undertaken to assess the effects of holding therapy onchildren who have a history of aggressive and delinquent behaviors. The study designwas a prospective, pre-post, quasi-experimental controlled study. The subjects were re-cruited through the Attachment Center at Evergreen, Colorado. Eligible subjects wereadopted children between the ages of 5-14 years, living in the present adopted homefor at least one year. All children had a history of aggressive and delinquent behaviors,which prompted the contact with Evergreen. Findings resulted in significant decreasein the outcome variable for the treatment group within this study.

KEY WORDS: Achenbach; Childhood Aggression; Treatment; Holding Therapy; Adop-tion; Foster Care.

Young aggressive children can commit such defiant and destructiveacts as lying, stealing, vandalism, fire setting, and running away.1

While it is well accepted that aggressive and antisocial behaviors inchildhood are related (violent, criminal outcomes),2,3,4 it is only overthe last ten years that an increase in adolescent violent acts (142%increase in murder and manslaughter) has been identified.5 Conse-quently, aggression in children has been posted as a major publichealth concern.6

Aggressive behaviors are included under a wide variety of psychi-atric terms ranging from attention deficit hyperactivity through oppo-sitional defiant, conduct, and attachment disorders.7 Distribution ofthese disorders is estimated to be between 2 to 16 percent of childrenwithin the general population.7 It is not surprising that researchers

Received December 20, 1997; For Revision April 7, 1998; Accepted September 22,1998.

Address correspondence to Robin Myeroff, PhD, 3800 Park East Dr., Suite 150,Beachwood, OH 44122.

Child Psychiatry and Human Development, Vol. 29(4), Summer 1999© 1999 Human Sciences Press, Inc. 303

Child Psychiatry and Human Development

have identified such behavioral problems as the major impetus forchildren's mental health referrals. Numerous interventions have beenattempted to treat aggressive behavior8 including individual, group,residential, behavioral, and psychopharmacological measures.9,10,1,11

However to date few interventions have been widely successful.Recently, holding therapy, originally used with autistic children, is

now being recognized as useful with aggressive behaviors in children.Based on attachment theory holding therapy in part attempts to re-pair the postulated disruption that occurred in the formative yearsbetween the infant and primary caregiver.12,13 A conceptualization ofattachment theory explains how this can occur. After a child experi-ences a repeated number of parenting inconsistencies he will inter-nalize the negative input from the primary caregiver and act this outthrough abusive and aggressive behaviors toward others.

One way that holding therapy impacts on the original disruptedcycle of attachment is by creating a representation of a healthy at-tachment cycle for the child.14,15 This occurs in the treatment by mod-eling the healthy attachment cycle in which the child will receive pos-itive input from the therapist and care giver by way of eye contact,physical holding, and cognitive restructuring. The positive input fromthe therapist and parent assists the child in attaching to the adoptiveparent which, will decrease aggression. The child can now internalizepositive input from the environment, curbing the tendency towardsdestructive behavior. During the points in the session when the childbecomes activated with anger or despair the parent and/or therapistcontinues to contain the child physically and assists him in cogni-tively understanding and self regulating his emotions. This replicatesthe healthy attachment cycle beginning with the child becomingaroused and the caregiver offering positive input by way of physicalholding, soothing, eye contact, and, adds in helping to articulate thechild's internal struggle. As the child begins to internalize this pro-cess after many hours and days of intense contact with the parentand therapist, internal reorganization begins to take place. When thechild's anger is met with love and understanding from the therapistand caregiver, the aggression can then be libidinized with boundariesand not leaving the child with unbound and destructive anger.16 Thecontainment and self-regulation of aggression is more manageable forthe child and destructiveness decreases. Simultaneously, the child in-ternalizes the adopted mother and begins to trust her. The relation-ship between the child and the mother begins to develop as the at-tachment becomes increasingly more secure. This allows for the

304

Robin Myeroff, Gary Mertlich, and Jim Gross

development of a sense of remorse in the child, and results in a de-crease in aggressive and delinquent behaviors.16 The shift in the childbased on the exposure to the healthy attachment cycle, allows for anincrease in secure attachment and, therefore, a decrease in aggressiveand delinquent behaviors.

Methods

Subjects

The total sample consisted of 23 subjects. Subjects were recruited from thereferral populations at the Attachment Center at Evergreen, Colorado. Crite-ria for participation included the requirement that the child's adoptive par-ents had contacted the Attachment Center at Evergreen between 1996 and1997. To be considered for treatment the child had to show evidence of de-structive behaviors and difficulty attaching to their parents. All children werebetween the ages of 4 and 14 years at the time of contact. Every child hadexperienced at least one other type of therapeutic intervention prior to at-tending the Attachment Center under the care of a professional medical doc-tor or mental health worker. All children were living in the home at the timeof contact and returned home after treatment.

The sample consisted of 23 children, 17 males and 6 females, ranging from5 to 14 years of age. Subjects were either in the treatment or comparisongroup because of the timing and or the ability to pursue therapy during theprojected course of the study. Due to the strict inclusion criteria, only 46% ofthe treatment population of The Attachment Center at Evergreen were in-volved as subjects. The two groups were similar in terms of distribution ofage, gender, and race (Table 1). They were also similar in regard to the num-ber of pre-adoption placements as identified by the intake data.

Measure

The Child Behavior Checklist" is a widely used parent report measure de-signed to assess behavior problems in children 4 to 16-years of age. The CBCLdepicts the child's behavioral pattern across both broadband (Internalizingand Externalizing) and narrowband syndromes. A higher score of either 1 or 2indicates more behavioral problems. The 2 problem scales used in this studyare aggression and delinquency. Construct validity is supported by correlatesof CBCL scales with significant associations with analogous scales on theQuay-Peterson18 Revised Behavior Problem Checklist and the Connors19 Par-ent Questionnaire. Criterion-related validity is supported by the ability of theCBCL's quantitative scale scores to discriminate between referred and non-referred children after demographic effects were parceled out. The Cronbach'sor reliability alpha range measures were .74 for delinquent behaviors and .92for aggressive behaviors for boys age 4-11 years, and .83 for delinquent be-haviors and .92 for aggressive behavior for boys age 12-18 years. The rangefor girls ages 4-11 years is .73 for delinquent behaviors and .92 for aggressive

305

306 Child Psychiatry and Human Development

Table 1Demographic VariablesDescriptive Statistics

Categorical Variables:

Variable

Gender

Race

Pre-Adopt.Placement

TreatmentGroup

MaleFemaleCaucasianAsianMix1-3 years

4-6 years7—9 years10 and above

Continuous Variables:

Variable

Age

Income

Group

TreatmentComparisonTreatmentComparison

N

102

10208

301

N

12111211

%

83.316.783.316.70.0

66.7

25.00.08.3

Mean

8.369.163.55.92

ComparisonGroup

N

74

10016

410

%

63.636.490.90.09.1

54.5

36.49.10.0

Range

(6 , 12)(6 , 11)(3,5)(2 ,6)

behaviors. Girls age 12-18 years shows .92 for delinquent behaviors and .92for aggressive behaviors.

The study design was a prospective two group pre-post-design of conveni-ence.

Recruitment Procedures

Subjects were recruited based on a parent-initiated phone call to the At-tachment Center at Evergreen. A description of the therapy was offered to theparent and a series of screening questions about the child's early history andpresent level of functioning were conducted to determine the potential fortreatment. Subjects volunteered to be a part of the study based on the knowl-edge that a study was being conducted to test the effects of this holding treat-ment and that the decision to participate would not have any affect on theirfuture treatment at the Attachment Center. The comparison group for thestudy was comprised of families who did not attend the Attachment Centerdue to time restraints or finances and parents who were information seeking.

Robin Myeroff, Gary Mertlich, and Jim Gross

Their lack of attendance was not due to the condition of the child or parent.Prior to treatment parents were mailed the CBCL for the mothers to fill outone week prior to treatment (time 1). After the treatment was completed theparents were mailed the CBCL for the mother to fill out six weeks post treat-ment (time 2). The comparison groups were mailed the measures at the sameintervals as the treatment groups.

Independent Variable-Holding Therapy

The following description of the holding therapy conducted at Evergreen iscondensed from a procedure manual written by Levy & Orlans.14 The treat-ment at the Attachment Center at Evergreen is a two-week therapy modeloften referred to as "a two-week intensive." The referred child, parents, andtreatment team, consisting of one therapist and the treatment foster mother,are all present for 30 hours of therapy. This breaks down to three consecutivehours of therapy daily. The 30 treatment hours are broken down to threehours per day for 10 consecutive working days. Each family entering treat-ment at Evergreen are assigned to a therapeutic parent who houses the childfor the two weeks of treatment. This means that the interactions the parentand child have together are the three hours during the actual treatment time,weekends, and certain times during the two weeks when the parent and childhave interactions for limited amounts of time. All therapists and therapeuticparents are trained systematically at the Attachment Center.

The therapy consists of four basic techniques which include cognitive re-structuring, psycho-dramatic reenactment, inner child metaphor, and thera-peutic holding. The therapeutic holdings are designed to imitate the infant-nurturing position on a couch. The child lies across the therapist's lap withher head resting on a pillow. This allows for close proximity, eye contact, andphysical restriction.

Each session follows as closely as possible the session outline, which will bediscussed in an abbreviated fashion in a session-by-session format. All inter-ventions occur in sequence but may be delayed or accelerated depending onthe family dynamics and the strengths of the child. Each session begins witha meeting of all participants with the exception of the child. At this time thechild is waiting in a separate room. All sessions after the first session areconducted using the holding technique.

Session one begins with a history-taking interview and assessment of theparents and child. The parents and the child contract verbally with the treat-ment team, entering the treatment based on the mutually agreed upon tech-niques and goals for the ten days.

Session two includes rapport building between client and therapist and pro-viding a cognitive framework for the treatment. This encompasses a descrip-tion of the first year life cycle and how infants develop trust. A review of thechild's early history with both birth parents and other foster placements isreviewed in light of the trust cycle explored above. Also included in the secondsession is a review of treatment rules and specific behaviors and changesexpected of the child.

Session three focuses on the resistance of the client and the controllingbehaviors displayed both in the therapeutic process and in the adoptive home.Validation and support are offered as the assistance of conscious connections

307

308

between past and present are verbalized and understood. The child is encour-aged to release anger, sadness, fear, or rage. The child's perceptions of herselfand the adult caregivers in her life are identified.

Session four begins with the child expressing thoughts and feelings aboutthe therapy including the therapeutic parents. Issues of attachment and emo-tional traumas are investigated as they relate to the child's experiences inearly life. The child is assisted to correctly identify feelings and begin ver-balizing these feelings.

Session five begins with the treatment team and parents present discussingthe previous day's and evening's events. Parenting techniques and skills arereviewed in light of the therapeutic parents' report of the child's behavior. Thesecond half of the session is spent in a therapeutic hold, continuing the dis-cussion of the early history of the child with the child.

Sessions six, seven, and eight are the middle phases of treatment allowingfor a more in-depth focus on emotional aspects of the early traumatic experi-ences. Psychodramatic reenactment is utilized at this time. The treatmentteam role-plays significant people in the child's past allowing for a gradualprogression into the events of the past and the ability to confront and expresswhat is needed leading the child to an interpersonal sense of mastery. Thisalso allows for revisions of old self perceptions and fantasies about self andpast significant figures.

The inner child metaphor is also utilized during these sessions as the childis asked to visualize herself in the past and, while being held, is asked aseries of questions about that early time and how those experiences and feel-ings relate to her present relationships.

In addition to these techniques mother-child exercises are repeated manytimes including holding, covering with blankets, and feeding with a bottle.

Session nine includes exploration of any birth father issues that may bepresent. The adoptive father now holds the child, as psycho-dramatic reenact-ment is utilized to provoke and resolve these father issues. The process ofgrief and mourning is explored in relation to the many losses experienced bythese children. This process allows for cognitive restructuring through thedialogue with the role-played birth parents.

Reunification with the adoptive family occurs at this point and the childleaves to spend the night with the adoptive parents instead of the therapeuticparent.

Session ten begins with a review of the prior night and interactions arediscussed. A complete review of the entire ten sessions takes place with every-one on the treatment team including the child. Family members talked abouttheir learning experiences during this time, and a specific follow up plan isthen developed.

Results

Data analysis consisted of two-tailed independent and paired t teststo discern any between and within group significant differences. Thet test was employed in this case as opposed to analysis of variance

Child Psychiatry and Human Development

Robin Myeroff, Gary Mertlich, and Jim Gross 309

(ANOVA), because the groups were so small (n < 50) that use of AN-OVA would have been inappropriate.20 A two tailed t test with thisgroup size is a more robust statistic.20 In addition, equivalent non-parametric tests were performed and validated the following findings.

In the treatment group, aggression scores before intervention (timeone) ranged from 69 to 91, with a mean of 80.75, and post interven-tion (time two) aggression scores ranged from 51 to 94 with a mean of66.27 (SD = 13.87). A paired two tailed t test found a significantdifference between the pre- and post-intervention scores (t = 4.26; df= 10; p < .002). In the comparison group, aggression scores rangedfrom 56 to 92 at time one with a mean of 74.27 (SD = 12.78), whiletime two scores ranged from 58 to 92 with a mean of 74.33 (SD =14.18). A paired two tailed t test showed no significant difference be-tween scores at time one and time two (t = — .58; df = 8; p = .579).Table 2 summarizes the within group differences in aggression scores.

Similar results were obtained with the two groups on delinquencyscores. The treatment group delinquency scores at time one, (prein-tervention) ranged from 62 to 84 with a mean of 72.83 (SD = 6.74),and time two, (postintervention) scores ranged from 50 to 86 with amean of 65.82 (SD = 10.89). Again paired two tailed t test demon-strated significant difference between the pre and post interventionscores for delinquency (t = 2.37; df = 10; p < .04). With the compari-son group, time one delinquency scores ranged from 59 to 84, mean of70.37 (SD = 8.27), and time two scores ranged from 54 to 81 with amean of 69.89 (SD = 9.64). Correspondingly, paired t test showed nosignificant difference between delinquency scores between time oneand time two (t = .20; df = 8; p = .85). Table 3 presents the withingroup findings for delinquency scores.

Comparison of pretest aggression sum minus post-test aggressionsum scores corroborated a significance between the treatment and

Table 2Statistical Analysis for Aggression Scores

Group

Comparisonpre time1post time2Treatmentpre timelpost time2

N

119

1211

Mean

74.2774.33

80.7566.27

SD

12.7814.18

6.7713.87

t df

-.58 8

4.26 10

p value

0.579

.001

comparison groups with an independent two tailed t test (t = 3.57; df= 18; p < .003). In a like manner t test results for pretest delin-quency sum minus post-test delinquency sum scores also demon-strated a significant difference between treatment and comparisongroups (t = 2.46; df = 18; p < .04). The results indicate a betweengroup difference on both aggression and delinquency for the compari-son and treatment groups over the course of the study.

Discussion

Efforts to address the problem of aggression and behavioral diffi-culties range from hospital based21,22 and community based programs.23

Because studies show that children with high rates of antisocialbehaviors are likely to continue these behaviors into adulthood,24,25 itbecomes increasingly important to find treatment that can impactthese behaviors.24,25 Heretofore, there have been no quantitativestudies which verify the effectiveness of holding therapy. The purposeof the present study was to investigate the relationship between hold-ing therapy and later aggressive behaviors as compared to childrenwith a similar profile who did not receive holding therapy.

The significant decreases in the outcome variables for the treat-ment group in this study may be due to the holding therapy basedupon the theory that disruption in the formative years has a criticalimpact on the attachment between infant and the primary caregiver.12,13

The comparison group by contrast did not receive any interventionand exhibited no significant changes over time for either the aggres-sion (p = .81) or delinquent (p = .99) scores. Children with highaggression are significantly more likely to have high delinquencyscores in the treatment and comparison groups, respectively, (treat-

Child Psychiatry and Human Development310

Table 3Statistical Analysis for Delinquency Scores

Group

Comparisonpre time 1post time2Treatmentpre time 1post time2

N

119

1211

Mean

70.3769.89

72.8365.82

SD

8.279.64

6.7410.89

t df

.2 8

2.37 10

p value

.85

.04

Robin Myeroff, Gary Mertlich, and Jim Gross

ment group rho = .84, p = <0.001; comparison group; rho = 0.61,p = 0.05). It may be that as one score decreases due to an effectiveintervention, so will the other score.

Delinquency tended to have even more of a significant decrease forthe treatment groups, (both separately and comparing the twogroups), may be because the questionnaire alluded to more of a char-acterological component for the questions about delinquency (fire set-ting, running away, truancy or lack of guilt) than for the questionsreferring to aggression (arguing, destructive action to self or property,temper tantrums). According to our conceptual framework the devel-opment of the child's attachment towards the parent simultaneouslyincreases the capability to feel remorse and the capacity for self regu-lation.

Suggestions for Further Research

Further investigation and replication are warranted in order to ex-tend knowledge and the effects of holding therapy on the specialneeds adopted population. Replicating these findings in a controlledmultimodel experimental study could provide information in order toevaluate the effectiveness of different forms of treatment.

Differences in the specific form of early abuse such as physical andsexual abuse and neglect, along with other possible extraneous vari-ables may offer insight into the reasons some children clearly benefitfrom this treatment while others do not seem to make any progress.Attachment therapy as it is practiced at the Attachment Center andother facilities across the country is a controversial and provocativetreatment. Comparative research into the different forms of holdingtherapy could result in new parameters for the treatment.

Summary

This study examined the effects of attachment therapy as per-formed by the Attachment Center at Evergreen on aggressive chil-dren between the ages of 5 and 14 years. The hypothesis that holdingtherapy will reduce aggressive behaviors in the special needs adoptedpopulation was supported.

Significant differences in the reduction of aggression and delin-quency scores in children who underwent a 2 week treatment pro-gram at the Attachment Center at Evergreen were found in this

311

312

study. Delinquency scores decreased across time for the treatmentgroup. The comparison group, who were eligible for treatment at theAttachment Center based on specific criteria but, did not receive anyintervention exhibited no significant changes over time for either ag-gression or delinquency scores as reported from the Child BehaviorChecklist.

Given that childhood aggressive behaviors can become increasinglymore violent and destructive, often at the expense of others includingfamily members, animals, and peers, a strong and effective interven-tion is necessary in order to assist these individuals in decreasingboth the intensity and frequency of their aggressive behaviors. Hold-ing therapy addresses the issues of attachment, early wounding andaggression through the breakdown of psychological defenses in thecontext of the adoptive family.

This finding indicates the importance of using holding therapy asone component of an intervention for children with aggressive anddelinquent behaviors between the ages of 5 and 14 years. Furtherinvestigation and replication are warranted in order to extend knowl-edge of holding therapy on this and other populations.

References

1. Kazdin, A.E., Siegel, T.C., & Bass, D: Cognitive problem-solving skills training andparent management training in the treatment of antisocial behavior in children. JConsult and Clinical Psychol, 60, 733-747, 1992.

2. Robins, L.N: Longitudinal studies: Sturdy childhood predictors of adult antisocialbehavior. Psychological Medicine, 8, 611-622, 1978.

3. Garrison, S.T., & Stolberg, A.L: Modification of anger in children by affective imag-ery training. J Abnorm Child Psychol, 11, 115-130, 1983.

4. Lochman, J.E., White, K.J., & Wayland K.K: Cognitive-behavioral assessment andtreatment with aggressive children. In P. C. Kendall (Eds.) Child and AdolescentTherapy: Cognitive-Behavioral Procedures. New York: Guilford Press, 1991.

5. Federal Bureau of Investigation (1993). Uniform Crime Report for the UnitedStates 1992. Washington, D.C.: U.S. Department of Justice.

6. Walker, B., Goodwin, N.J., & Warren, R.C: Violence: A challenge to the publichealth community. Journal National Medical Association, 84, 490-496, 1992.

7. American Psychiatric Association DSM-IV: Diagnostic and Statistical Manual ofMental Disorders. 4th ed. Washington, DC: The Association, 1994.

8. Kazdin, A.E: Treatment of Antisocial Behavior in Children and Adolescents. Home-wood, IL: Dorsey Press, 1985.

9. Werry, J.S., & Wollersheim, J.P: Behavior therapy with children and adolescents: Atwenty-year overview. J Am Acad Child Adolesc Psychiatry, 28, 1-18, 1989.

10. Sanchez, L.E., Armenteros, J.L., Small, A.M., Campbell, M., & Adams P.B.: Placeboresponse in aggressive children with conduct disorder. Psychopharmacology Bulle-tin, 30, 209-213, 1994.

Child Psychiatry and Human Development

Robin Myeroff, Gary Mertlich, and Jim Gross 313

11. Slee, P.T. Family climate and behavior in families with conduct disordered chil-dren. Child Psychiat Hum Developm, 26, 255-266, 1996.

12. Bowlby, J: Attachment and Loss Vol. LII: Sadness and Depression. New York: BasicBooks, 1980.

13. Egeland, B, & Sroufe, A: Attachment and Early Maltreatment. Child DevelopmVol. 52: 44-52, 1981.

14. Levy, T, & Orlans, M: Intensive Short-Term Therapy with Attachment DisorderedChildren. Unpublished Manuscript, 1995.

15. Myeroff, R: Comparative Effectiveness With The Special Needs Adoptive Population.Unpublished dissertation, Union Institute, Cincinnati, 1997.

16. Barrett, T. Supporting drive fusion: Mitigating Destructive Aggression In Infants,Toddlers And Preschoolers. J Child Anal, Vol. 6, 128-151, 1995.

17. Achenbach, T, & Edelbrock, C: Manual for the Child Behavior Checklist and Re-vised Child Behavior Profile. University of Vermont, Burlington, 1983.

18. Quay, H.C., & Peterson, D.R: Interim Manual for the Revised Behavior ProblemChecklist. University of Miami, Coral Gables, 1983.

19. Conners, C.K: Rating Scales for use of drug studies with children. Psychophar-macology Bulletin: Pharmacotherapy with children. Washington DC: U.S. Govern-ment Printing Office, 1973.

20. Munroe, B., & Page, E.B. Statistical Methods for Health Care Research, 2nd ed.Philadelphia: J.B. Lippincott, 1993.

21. Lock, J & Strauss, G: Psychiatric hospitalization of adolescents for conduct disor-der. Hospital and Community Psychiatry, Vol 45 (9) 925-928, 1994.

22. Malone, R., Luebbert, J., Pena, A., Biesecker, K: The Overt Aggression Scale in astudy of lithium in aggressive conduct disorder. Psychopharmacology Bull, Vol 30(2) 215-218, 1994.

23. Offord, D., & Bennett, K: Conduct Disorder: Long-term outcomes and interventioneffectiveness. J Am Acad Child Adolesc Psychiatry: Oct. Vol 33 (8) 1069-1078,1994.

24. Robins, L., & Price, R: Adult disorders predicted by childhood conduct problems:Results from the NMH Epidemiological Attachment Area Project. Psychiatry, Vol54, 116-132, 1991.

25. Loeber, R: Antisocial behavior: More enduring than Changeable? Special Section. JAm Acad Child Adolesc Psychiatry, Vol 30, 393-397, 1991.