Pediatric Medical Complexity Algorithm: A NewMethod to Stratify Children by Medical Complexity

WHAT’S KNOWN ON THIS SUBJECT: Quality measures developedby the Pediatric Quality Measures Program are required to assessdisparities in performance according to special health care needstatus. Methods are needed to identify children according to levelof medical complexity in administrative data.

WHAT THIS STUDY ADDS: The Pediatric Medical ComplexityAlgorithm is a new, publicly available algorithm that identifies thesmall proportion of children with complex chronic disease inMedicaid claims and hospital discharge data with good sensitivityand good to excellent specificity.

abstractOBJECTIVES: The goal of this study was to develop an algorithm basedon International Classification of Diseases, Ninth Revision, ClinicalModification (ICD-9-CM), codes for classifying children with chronicdisease (CD) according to level of medical complexity and to assessthe algorithm’s sensitivity and specificity.

METHODS: A retrospective observational study was conducted among700 children insured by Washington State Medicaid with $1 SeattleChildren’s Hospital emergency department and/or inpatient en-counter in 2010. The gold standard population included 350 childrenwith complex chronic disease (C-CD), 100 with noncomplex chronicdisease (NC-CD), and 250 without CD. An existing ICD-9-CM–basedalgorithm called the Chronic Disability Payment System wasmodified to develop a new algorithm called the Pediatric MedicalComplexity Algorithm (PMCA). The sensitivity and specificity of PMCAwere assessed.

RESULTS: Using hospital discharge data, PMCA’s sensitivity for cor-rectly classifying children was 84% for C-CD, 41% for NC-CD, and 96%for those without CD. Using Medicaid claims data, PMCA’s sensitivitywas 89% for C-CD, 45% for NC-CD, and 80% for those without CD.Specificity was 90% to 92% in hospital discharge data and 85% to91% in Medicaid claims data for all 3 groups.

CONCLUSIONS: PMCA identified children with C-CD (who have accessedtertiary hospital care) with good sensitivity and good to excellentspecificity when applied to hospital discharge or Medicaid claimsdata. PMCA may be useful for targeting resources such as carecoordination to children with C-CD. Pediatrics 2014;133:e1647–e1654

AUTHORS: Tamara D. Simon, MD, MSPH,a,b Mary LawrenceCawthon, MD, MPH,c Susan Stanford, MPH, MSW,b JeanPopalisky, DNP, RN,b Dorothy Lyons, MPA,c Peter Woodcox,BA,c Margaret Hood, MD,a Alex Y. Chen, MD, MS,d and RitaMangione-Smith, MD, MPH,a,b for the Center of Excellenceon Quality of Care Measures for Children with ComplexNeeds (COE4CCN) Medical Complexity Working GroupaDepartment of Pediatrics, University of Washington/SeattleChildren’s Hospital, Seattle, Washington; bSeattle Children’sResearch Institute, Seattle, Washington; cResearch and DataAnalysis Division, Washington Department of Social and HealthServices, Olympia, Washington; and dDepartment of Pediatrics,Children’s Hospital Los Angeles, Keck School of Medicine at theUniversity of Southern California, Los Angeles, California

KEY WORDSadministrative data, claims data, children, medical complexity,sensitivity, specificity, stratification, validation

ABBREVIATIONSC-CD—complex chronic diseaseCD—chronic diseaseCDPS—Chronic Illness and Disability Payment SystemCOE4CCN—Center of Excellence on Quality of Care Measures forChildren with Complex NeedsCRG—Clinical Risk GroupED—emergency departmentICD-9-CM—International Classification of Diseases, Ninth Revi-sion, Clinical ModificationNC-CD—noncomplex chronic diseasePMCA—Pediatric Medical Complexity AlgorithmPQMP—Pediatric Quality Measures ProgramSCH—Seattle Children’s HospitalWA-Medicaid—Washington State Medicaid

(Continued on last page)

PEDIATRICS Volume 133, Number 6, June 2014 e1647

ARTICLE

by guest on June 21, 2016Downloaded from

InMarch2011, the Centers forMedicare&Medicaid Services and the Agency forHealthcare Research and Quality part-nered to fund 7 Centers of Excellencethat constitute the Pediatric QualityMeasures Program (PQMP) mandatedby the 2009 Child Health InsuranceProgram Reauthorization Act.1 Thecharge to the PQMP is to develop newquality of care measures and/or en-hance existing measures for children’shealth care across the age spectrum.2,3

The Center of Excellence on Quality ofCare Measures for Children with Com-plex Needs (COE4CCN) was chargedwith identifying and/or developinga valid method to assess disparities incare according to level of medicalcomplexity for children with specialhealth care needs.

COE4CCN initially considered severalmethods to classify children accordingto level of medical complexity butchose to focus on algorithms that useInternational Classification of Dis-eases, Ninth Revision, Clinical Modifi-cation (ICD-9-CM), codes. Althoughlimited by reliance on billing data,4

approaches based on ICD-9-CM codesare relatively inexpensive to usecompared with survey methods, canbe used on existing administrativedata with relative efficiency, and arefavored by states and insurance plans.Although several proprietary ICD-9-CM–based algorithms might be usedfor these purposes, the mandate wasto identify or develop a tool that wasopen access and publicly available forwidespread use in quality measure-ment. After reviewing and evaluatingseveral existing algorithms,5–14 wechose to modify the Chronic DisabilityPayment System (CDPS) algorithm.12

The objectives of the present studywere to develop the modified versionof CDPS and assess its sensitivity andspecificity for correctly identifyingchildren with varying levels of medicalcomplexity.

METHODS

This development and validation re-search was undertaken in 5 stages:(1) development of consensus defi-nitions for 3 levels of medical com-plexity; (2) selection of an existingICD-9-CM algorithm; (3) modificationof the algorithm to conform to theconsensus medical complexity defi-nitions; (4) selection of a gold stan-dard population of children throughmedical record review; and (5) eval-uation of the modified algorithm’ssensitivity and specificity when ap-plied to the gold standard pop-ulation. All study procedures werereviewed and approved by the in-stitutional review boards of SeattleChildren’s Research Institute andWashington State.

Developing Consensus Definitions

To develop an algorithm that wouldfacilitate assessment of disparitiesaccording to special health care needstatus, COE4CCN developed consensusdefinitions for 3 levels of medicalcomplexity: children with complexchronic disease (C-CD), children withnoncomplex chronic disease (NC-CD),and children without chronic disease(CD) (Table 1). The COE4CCN MedicalComplexity Working Group developedthe first draft of these consensusdefinitions after review and discus-sion of 2 previously published carecoordination conceptual frameworksand their accompanying definitionsfor levels of medical complexity.15,16

The working group is composed ofindividuals with expertise in inpatientand outpatient management of chil-dren with C-CD and/or NC-CD. The en-tire COE4CCN then provided reviewand feedback on the draft consensusdefinitions. COE4CCN includes 43 rep-resentatives from 2 state Medicaidagencies, Family Voices, pediatricnursing, hospital medicine, and out-patient primary care, as well as pedi-

atric health services research. Thefinal consensus definitions incor-porated the center-wide feedback.

Algorithm Identification

After evaluating several existingalgorithms,5–14 we chose to modifyCDPS,12 which is a diagnosis-basedrisk adjustment model that usesICD-9-CM codes to assess risk of in-curring high costs.17 CDPS was se-lected because it has the mostcomprehensive listing of ICD-9-CMcodes among the algorithms evalu-ated, is publicly available, and, incontrast to most other algorithms,includes important codes for mentalhealth conditions.

Algorithm Modification

We developed and evaluated a novelalgorithm for this study, the PediatricMedical Complexity Algorithm (PMCA).PMCA represents a modification ofCDPS that conforms to the COE4CCNconsensus definitions for medicalcomplexity. CDPS modification involvedremoving several types of ICD-9-CMcodes, including those consistent withadult illness (eg, myocardial infarc-tion), related to childbirth, consistentwith acute illness (eg, acute otitismedia), and representing pediatricchronic conditions that are most oftenmild in severity (eg, eczema, myopia,iron deficiency anemia). Excluding codesfor conditions with a substantial pro-portion of mild disease (eg, eczema)from PMCA reduces the potential foroverestimation of disease burden in thepopulation (ie, false-positive findings).However, this process may result inchildrenwithmore severe forms of suchdiseases not being captured or correctlyclassified (ie, false-negative findings).

Further modifications included theaddition of “flags” to each retained ICD-9-CM code by 2 authors (T.D.S. andR.M.-S.). Conditions associated with

e1648 SIMON et alby guest on June 21, 2016Downloaded from

deteriorating health and an increasedrisk of shorter life expectancy inadulthood (defined as death in thefourth to fifth decade [eg, cystic fibro-sis, complex congenital heart disease,malignancy]) were flagged as pro-gressive; when consensus was not im-mediately reached, life expectancydata for the condition were reviewed,and a final decision was made. Bodysystem flags were also assigned topermit body system counts and sub-sequent classification to NC-CD (1 bodysystem) or C-CD ($2 body systems). Afull list of the ICD-9-CM codes includedin PMCA, and their progressive andbody system flags, is provided in Sup-plemental Table 5. SAS programmingcode (SAS Institute, Inc, Cary, NC) was

subsequently developed for PMCA tocategorize children into the 3 levels ofmedical complexity based on adjudi-cated claims.

To capture data on children with C-CDbased on technology dependence for.6 months (Table 1), we adapteda previously developed set of tech-nology assistance ICD-9-CM codes18

and tested PMCA’s sensitivity andspecificity with and without these ad-ditional codes.

Identifying a Gold StandardPopulation: Classifying Children byMedical Complexity By UsingMedical Record Review

Children 0 to 18 years old, insuredby Washington State Medicaid (WA-

Medicaid), and seen at Seattle Chil-dren’s Hospital (SCH) for $1 emer-gency department (ED) visit and/orinpatient stay in 2010 were potentiallyeligible for the study. To oversamplechildren in the C-CD group, thesechildren were categorized into 1 of 9mutually exclusive risk groups by us-ing 3M Clinical Risk Group (CRG) soft-ware (St Paul, MN)19 applied to 4 years(2007–2010) of SCH ED, inpatient, andday surgery administrative data. AfterCRG categorization, a sample of 1000children was randomly selected, withoversampling (n = 500) for childrenwith lifelong chronic conditions (CRGgroups 5b, 6, 7, 8, and 9).

A trained nurse researcher (J.P.)blinded to CRG categorization made

TABLE 1 COE4CCN Consensus Definitions of 3 Levels of Medical Complexity

Condition Description Potential Examplea

Children with C-CDa

Significant chronic conditions in $2 body systems: Type 1 diabetes and static encephalopathy; type 1 diabetes and depression;developmental delay and chronic pulmonary conditionsSignificant chronic condition is defined as a physical, mental or

developmental condition that can be expected to last at least a year, willuse health care resources above the level for a healthy child, requiretreatment of control of the condition, and the condition can be expected tobe episodically or continuously debilitating.

Body systems include: cardiac, craniofacial, dermatologic, endocrinologic,gastrointestinal, genetic, genitourinary, hematologic, immunologic,mental health, metabolic, musculoskeletal, neurologic, ophthalmologic,otologic, pulmonary/respiratory, and renal.

OR A progressive condition that is associated with deteriorating health witha decreased life expectancy in adulthood.

Muscular dystrophy, cystic fibrosis, paraplegia, quadriplegia, malignancy

OR Continuous dependence on technology for at least 6 months. Tracheostomy 6 ventilator assistance, renal dialysis, gastrostomy tube,cerebrospinal fluid shunt

OR Malignancies: progressiveormetastaticmalignancies that affect life function.Exclude those in remission for .5 years.

Lymphoma, leukemia, brain tumor

Children with NC-CDChronic conditions that last at least 1 year: These conditions are commonlylifelong but can be episodic with periods of good health between episodes.They include physical, developmental, ormental health conditions thatmaypersist into adulthood butmay also resolve either secondary to the naturalhistory of the disease or as a result of surgical intervention. Theseconditions involve a single body system, are not progressive, can varywidely in severity, and result in highly variable health care utilization.

Type 1 diabetes, atrial septal defect, asthma, depression, attention-deficit/hyperactivity disorder

Children without CDAcute nonchronic conditions: A physical, developmental or mental healthcondition that is not expected to last .1 year. These children maytemporarily (for ,1 year) use health care resources above the normallevel for a healthy child.

Ear infection, pneumonia, diarrhea and dehydration, bronchiolitis

Healthy: No acute or chronic health conditions. These children do not usehealth care resources above the normal level for a healthy child.

NA

NA, not applicable.a The examples used in this document to illustrate definitions of medical complexity and chronicity are intended to demonstrate characteristics specified in the definition/descriptions. It is notour intention to imply that specific diseases and conditions are by default linked to the categories that they were used to illustrate.

ARTICLE

PEDIATRICS Volume 133, Number 6, June 2014 e1649by guest on June 21, 2016Downloaded from

assignments into 1 of 3 levels ofmedical complexity (Table 1) byreviewing all available SCH electronicmedical records. When level assign-ment was unclear, cases were re-viewed by a panel of physicians (T.D.S.,A.Y.C., M.H., and R.M.-S.) also blinded toCRG categorization, and assignmentswere made by consensus. Among thesample of 1000 randomly selectedchildren, medical records were re-viewed until the target gold standardpopulation of 700 children was as-sembled. The target population in-cluded 350 children with C-CD, 100with NC-CD, and 250 without CD. Thesesample sizes were determined a priorito allow for stable estimates of PMCA’ssensitivity and specificity for correctlyclassifying patients into the 3 levels ofcomplexity.

Almost all (699 of 700) of the goldstandard population children weresuccessfully matched in the WA-Medicaid claims database. Twentyindividuals .18 years old were ex-cluded because they were not eligiblefor WA-Medicaid for substantial por-tions of the study period and had in-complete claims data. One child havingonly secondary Medicaid coverage wasalso excluded. The final WA-Medicaidstudy sample numbered 678, whereasall 700 children were included in theSCH study sample.

To determine the representativeness ofthe gold standard population, charac-teristics for the 678 study childrenwerecomparedwith the overall WA-Medicaid–insured child population from 2009 to2011.

Algorithm Evaluation

Three versions of the PMCA SAS codewere developed to characterize thetiming and frequency of coded con-ditions from administrative data: theleast, more, and most conservativeversions described in Table 2. Childrenin the sample had up to 3 years of dataavailable for analysis in both the SCHand WA-Medicaid claims databases, 1year before and 1 year after the yearof their hospitalization or ED visit(ie, January 1, 2009–December 31, 2011).All children were included regardlessof how much data they had available tocontribute to the analysis. All childrenin both the SCH and WA-Medicaidsamples had at least 1 claim in 2010that represented the ED and/or in-patient encounter making them eli-gible for gold standard populationselection.

We determined PMCA’s sensitivity andspecificity for correctly classifyingchildren into the 3 levels of complexityby using SCH discharge and WA-Medicaid claims data. SCH data in-cluded administrative claims from

inpatient, ED, and day surgery encoun-ters. WA-Medicaid data included allinpatient and outpatient claims pro-vided to the state. We also evaluatedthe performance of 3 different ver-sions of the PMCA SAS code describedin Table 2.

Afterapplying thePMCASAScode toSCHdischarge and WA-Medicaid claimsdata, we examined cases misclassifiedby PMCA to determine if there werepatterns that might inform futuremodifications to the algorithm.

RESULTS

Characteristics of the gold standardpopulation were compared with theoverall WA-Medicaid–insured pop-ulation of children (Table 3). Thestudy sample was younger and moreracially and ethnically diverse. A dis-proportionate number of study chil-dren had fee-for-service coverage(47% vs 20%) and prolonged Medic-aid eligibility (56% vs 46% with 36months).

Using WA-Medicaid data, 536 (79%) ofthe study sample had 3 years of claimsdata to contribute to the analyses, 122(18%) had 2 years, and only 20 (3%) hadjust 1 year. In contrast, using SCH data,152 (22%)of thesamplehad3years, 252(36%) had 2 years, and 296 (42%) had 1year of claims data to contribute toanalysis.

TABLE 2 Classification of Disease Complexity Based on Progressive and Body System Flags and Frequency of Encounters Observed in AdministrativeData

Category Least Conservative Version More Conservative Version Most Conservative Version

C-CDProgressive condition $1 claim $1 claim $1 claimMalignancy $1 claim $1 claim $1 claimOther $1 claim per body system for

2 different body systems duringthe measurement perioda

$2 claims per body system for2 different body systems duringthe measurement perioda

$1 claim per body system per year for2 different body systems during themeasurement perioda

NC-CD$1 claim for a single body system not

flagged as progressive duringthe measurement perioda

$2 claims for a single body systemnot flagged as progressive duringthe measurement perioda

$1 claim per single body system per yearnot flagged as progressive during themeasurement perioda

Without CDNone of the above during the

measurement periodaNone of the above during the

measurement periodaNone of the above during the

measurement perioda

a In the current study, a 3-year measurement period was used (January 1, 2009–December 31, 2011).

e1650 SIMON et alby guest on June 21, 2016Downloaded from

Optimal performance conditions forPMCA in hospital discharge and Med-icaid claims data are shown in Table 4.Using up to 3 years of hospital dis-

charge data and the least conservativeversion of the PMCA code (Table 2), al-gorithm sensitivities for correctlyclassifying children were 84% for chil-

dren with C-CD, 41% for children withNC-CD, and 96% for children withoutCD. Using up to 3 years of WA-Medicaidclaims data and the more conservativeversion of the PMCA code, sensitivitieswere 89% for children with C-CD, 45%for children with NC-CD, and 80% forchildren without CD. Specificity wasgood to excellent in both hospital(90%–92%) and Medicaid (85%–91%)data for all 3 levels of complexityaccording to the least and more con-servative PMCA code versions, re-spectively. The most conservativeversion of the PMCA code resulted inthe highest rate of misclassification.Addition of the technology assistancecodes did not improve PMCA’s sensi-tivity or specificity regardless of whichversion of the code was used (data notshown).

We identified 3 patterns of mis-classification in which PMCA (appliedto either hospital discharge or Med-icaid claims data) categorized chil-dren in the gold standard populationas having no CD who were determinedto have NC-CD according to medicalrecord review. The first pattern in-volved children with mild episodicconditions that were counted by themedical record reviewer as chronicbut are not included in PMCA (eg,eczema, headache). The second pat-tern involved children with conditionsthat had largely resolved (eg, epilepsybut seizure-free for 2 years). For thesecases, there was evidence for thecondition in themedical record, but nohealth care utilization/claims for itduring the study period (January 1,2009–December 31, 2011). The thirdpattern involved children with mentalhealth conditions or developmentaldelay, conditions that are commonlyundercoded in claims data.3,20

Children determined to have NC-CDaccording to medical record reviewwho were misclassified by PMCA asC-CD in both hospital and Medicaid

TABLE 3 Demographic and Enrollment Characteristics of the Gold Standard Sample (n = 679) andthe Overall WA-Medicaid Population Between 2009 and 2011 (n = 829 012)

Characteristic Gold Standard Samplea Overall Medicaid Populationb

Age as of December 31, 2010,1 yc 54 (8%) 47 718 (6%)1–4 yc 257 (38%) 205 469 (25%)5–9 yc 146 (22%) 217 360 (26%)10–14 yc 119 (17%) 191 674 (23%)15–19 yc 103 (15%) 166 791 (20%)

Male genderd 361 (53%) 416 628 (50%)Race/ethnicityWhitec 252 (37%) 413 784 (50%)Hispanic 145 (21%) 201 025 (24%)Other/mixedc 170 (25%) 139 792 (17%)African Americanc 89 (13%) 48 399 (6%)Asian 23 (3%) 26 032 (3%)

Written languageEnglishc 497 (73%) 651 207 (79%)Otherc 182 (27%) 177 805 (21%)

Coverage for majority of eligibility monthsFee-for-servicec 318 (47%) 169 708 (20%)Managed carec 361 (53%) 659 304 (80%)

Months of medical assistance eligibility 2009–2011,12 moc 17 (3%) 43 707 (5%)13–24 moc 125 (18%) 185 536 (22%)25–35 moc 158 (23%) 219 559 (27%)36 moc 379 (56%) 380 210 (46%)

PMCA more conservative approach designationC-CDe 329 (47%) 51 851 (6%)NC-CDe 100 (14%) 134 764 (16%)Without CDe 249 (36%) 608 966 (74%)Not classifiablee 1 (0%) 33 431 (4%)

Data are presented as n (%).a Demographic data were available for 679 children and claims data for 678, resulting in different n values for the text andtable.b We used 3 years of claims data from 2009 to 2011 for children aged,19 years on July 1, 2010, with aminimum eligibility of 1month in 2010 and 2 months in 2009–2011.c Statistically significant based on the confidence interval for a proportion applied to the gold standard sample.d Two children had missing gender data.e Statistical tests were not performed because we deliberately oversampled children with C-CD for the gold standardpopulation.

TABLE 4 Sensitivity and Specificity for PMCA in SCH Discharge Data and WA-Medicaid Claims Data

Approach Algorithm C-CD NC-CD Without CD

Sensitivity Specificity Sensitivity Specificity Sensitivity Specificity

SCH (n = 700) n = 350 n = 350 n = 100 n = 100 n = 250 n = 250Least PMCAa 84 (80–88) 92 (89–94) 41 (32–51) 92 (85–96) 96 (93–98) 90 (86–93)More PMCA 73 (68–77) 96 (93–98) 19 (13–28) 94 (88–97) 100 (99–100) 72 (66–77)Most PMCA 66 (61–71) 96 (93–98) 1 (0–5) 99 (95–100) 100 (99–100) 57 (51–63)

WA-Medicaid (n = 678) n = 329 n = 329 n = 100 n = 100 n = 249 n = 249Least PMCA 91 (87–94) 79 (74–83) 37 (28–47) 88 (80–93) 69 (63–74) 94 (90–96)More PMCAa 89 (85–92) 85 (81–89) 45 (36–55) 91 (84–95) 80 (75–85) 91 (87–94)Most PMCA 78 (73–82) 90 (86–93) 16 (10–24) 95 (89–98) 95 (92–97) 75 (69–80)

The n values are the number of children from the gold standard sample included in the designated category. Data are givenas % (95% confidence interval).a Recommended algorithm.

ARTICLE

PEDIATRICS Volume 133, Number 6, June 2014 e1651by guest on June 21, 2016Downloaded from

data often had chronic disease in 2systems with resolved disease in atleast 1 system (eg, asthma, repairedventricular septal defect). In suchcases, the resolved condition was notcounted by the medical record re-viewer but did result in claims thatwere detected by using PMCA.

DISCUSSION

To enable assessment of disparities incare according to special health careneed status, we developed a novel al-gorithm by modifying an existing ICD-9-CM–based algorithm (CDPS) to alignwith the COE4CCN consensus defini-tions for 3 levels of medical complex-ity. PMCA exhibited good sensitivityfor correctly categorizing childrenwith C-CD, excellent sensitivity forcorrectly categorizing children with-out CD, but poor sensitivity forcorrectly categorizing children withNC-CD. For optimal identification ofthese 3 groups of children using toMedicaid claims data, we recommendusing the more conservative version ofthe PMCA code and up to 3 years ofclaims data when available. For hospi-tal discharge data (limited to ED, in-patient, and day surgery claims), werecommend using the least conserva-tive version of the PMCA code and up to3 years of data when available. (ThePMCA SAS code and documentation areavailable from the authors upon re-quest.)

In hospital discharge data, ICD-9-CMcodes for chronic disease are rela-tively infrequent because they areassigned only at the time of an ED,inpatient, or day surgery encounter. Asa result, the least conservative versionof PMCA, in which only 1 use of anincluded ICD-9-CM code is requiredduring a 3-year time period (Table 2),resulted in the best performance. Incontrast, Medicaid claims data cap-ture far more health care utilization;therefore, the more conservative

version of PMCA, in which 2 uses of anincluded ICD-9-CM code are requiredduring a 3-year time period, resultedin the best performance.

Children with NC-CD are the most dif-ficult to correctly identify by usingadministrative data. These conditionsmay be episodic for a given child andcan encompass a wide range of se-verity, from mild to severe. This vari-ation results in highly fluctuatinghealth care utilization over time withinthe NC-CD group.With hospital dischargedata, PMCA misclassified children withNC-CD into both thewithout-CD andC-CDgroups. Children with NC-CD who weremisclassified according to PMCA aswithout CD in hospital data often hadepisodic disease, developmental con-cerns, or a resolving problem thatrequired outpatient care. Althoughelectronic medical records used forthis study included outpatient, inpa-tient, and ED encounters, the hospital-based discharge data consisted of ED,inpatient, and day surgery claims. SCHoutpatient claims were not available,thus constraining PMCA’s ability to cor-rectly detect conditions being man-aged exclusively in that setting. Dueto the relative infrequency of encoun-ters in the SCH discharge data, weelected to use the least conservativeversion of PMCA (Table 2). However, thisapproach unfortunately increased thelikelihood that false-positive findingswould be observed in the C-CD groupbecause only 1 claim for each of 2separate body systems is requiredduring the measurement period forclassification into this group. Thismethod potentially results in childrenwho have a history of chronic diseasein 2 body systems but resolved diseasein at least 1 (NC-CD) being incorrectlyclassified according to PMCA as havingC-CD.

In Medicaid claims data, PMCA morecommonly misclassified children withNC-CD as having C-CD. Because our

medical record review focused onvisits occurring in the SCH outpatientand inpatient settings, substantialportions of a child’s care occurringin settings outside of SCH were notavailable to the reviewer; it is possible,therefore, that relevant conditionswere not captured when categorizingchildren in the gold standard pop-ulation. In contrast, when PMCA wasapplied to Medicaid claims data,health care utilization both within andoutside of SCH were available to in-form categorization. Replication ofthis study in a system in which allmedical records data are available forreview could potentially address thislimitation.

We purposefully oversampled chil-dren with C-CD for the gold standardpopulation (350 of 700); this pop-ulation therefore differs from theoverall state Medicaid population. Asexpected with large numbers of chil-dren with C-CD, the study sample hadmore fee-for-service coverage andprolonged Medicaid eligibility. In ad-dition, the sample was younger andmore racially and ethnically diversethan the state Medicaid population.Overall, 6% of children insured by WA-Medicaid from 2009 to 2011 wereclassified with C-CD, 16% with NC-CD,and 74% without CD.

As the Patient Protection and Afford-ableCareAct is implemented,Medicaidand the health care system increas-ingly need strategies to allocate re-sources. Children with C-CD are mostlikely to benefit fromcare coordinationand other resources, and accurateidentification of this group is critical.21

These children may suffer the worstquality of care for many of the mea-sures under development by the PQMP.Use of PMCA will allow us to addressthe legislative mandate to assess dis-parities by using special health careneed status and further test the hy-pothesis that children with C-CD

e1652 SIMON et alby guest on June 21, 2016Downloaded from

experience poorer quality of care thaneither children with NC-CD or healthychildren.

Thepresent studyhadseveral additionallimitations. Approaches relying on sec-ondary data containing ICD-9-CM codesare only as good as the completeness ofthe original sources. Therefore, childrenwho do not interact with the health caresystem will lack the necessary in-formation todefine their level ofmedicalcomplexity, and children enrolled in$2health plans will have incomplete en-counter information in both data sets.Furthermore, encounters for conditionsthat do not result in a claim will not becaptured in administrative data. Anymanual assessment of medical recordsdata are subject to human error; how-ever, the reviewer for the present studyis experienced in this type of review22

and brought cases in which she wasunsure to the physician panel for eval-

uation. Assignment of the body systemand progressive flags to ICD-9-CM codesin PMCA was based on the clinical ex-perience of 2 authors; others may thusdisagree with our designations ofwhether a condition is progressive orwith body system assignments for con-ditions. Because this study targetedidentification of children with C-CD, ourgold standard population was drawnfrom a tertiary care hospital and wasnot representative of WA-Medicaid–insured children. Further validationwork in other populations of children,including health systems in whichmost children primarily access out-patient care, is needed. We also an-ticipate the need for further changesto PMCA to ensure compatibility withfuture widespread adoption of In-ternational Classification of Diseases,10th Revision, Clinical Modification,codes.

CONCLUSIONS

Despite its limitations and potentialfuture revisions, PMCA is a new, publiclyavailable algorithm that identifiedchildren with C-CD (who have accessedtertiary hospital care) with good sen-sitivity and good to excellent specificitywhen applied to either hospital dis-charge or Medicaid claims data. Ashealth care reform is implemented, useof PMCA will be critical to targetresources and services such as carecoordination to children with the mostneeds.

ACKNOWLEDGMENTSThe COE4CCN Medical Complexity Work-ing Group also included Kim Arthur,Penny Knapp, Michael Murphy, JustineNelson, Jeff Tenney, and Chuan Zhou.We appreciate feedback on the manu-script provided by Doug Opel.

REFERENCES

1. Dougherty D, Schiff J, Mangione-Smith R. TheChildren’s Health Insurance Program Reau-thorization Act quality measures initiatives:moving forward to improve measurement,care, and child and adolescent outcomes.Acad Pediatr. 2011;11(suppl 3):S1–S10

2. Mangione-Smith R, Schiff J, Dougherty D.Identifying children’s health care qualitymeasures for Medicaid and CHIP: anevidence-informed, publicly transparentexpert process. Acad Pediatr. 2011;11(suppl 3):S11–S21

3. Zima BT, Murphy JM, Scholle SH, et al. Na-tional quality measures for child mentalhealth care: background, progress, andnext steps. Pediatrics. 2013;131(suppl 1):S38–S49

4. McCarthy EP, Iezzoni LI, Davis RB, et al. Doesclinical evidence support ICD-9-CM di-agnosis coding of complications? MedCare. 2000;38(8):868–876

5. Feudtner C, Christakis DA, Connell FA. Pe-diatric deaths attributable to complexchronic conditions: a population-basedstudy of Washington State, 1980-1997. Pe-diatrics. 2000;106(1 pt 2):205–209

6. Feudtner C, Hays RM, Haynes G, Geyer JR,Neff JM, Koepsell TD. Deaths attributed to

pediatric complex chronic conditions: na-tional trends and implications for sup-portive care services. Pediatrics. 2001;107(6). Available at: www.pediatrics.org/cgi/content/full/107/6/E99

7. Feudtner C, Silveira MJ, Christakis DA.Where do children with complex chronicconditions die? Patterns in WashingtonState, 1980-1998. Pediatrics. 2002;109(4):656–660

8. Feudtner C, Christakis DA, Zimmerman FJ,Muldoon JH, Neff JM, Koepsell TD. Char-acteristics of deaths occurring in chil-dren’s hospitals: implications for supportivecare services. Pediatrics. 2002;109(5):887–893

9. Feudtner C, DiGiuseppe DL, Neff JM. Hos-pital care for children and young adults inthe last year of life: a population-basedstudy. BMC Med. 2003;1:3

10. Perrin JM, Kuhlthau K, McLaughlin TJ, EttnerSL, Gortmaker SL. Changing patterns ofconditions among children receiving Sup-plemental Security Income disability benefits.Arch Pediatr Adolesc Med. 1999;153(1):80–84

11. Kuhlthau KA, Beal AC, Ferris TG, Perrin JM.Comparing a diagnosis list with a surveymethod to identify children with chronic

conditions in an urban health center.Ambul Pediatr. 2002;2(1):58–62

12. University of California, San Diego. ChronicIllness and Disability Payment System.Available at: http://cdps.ucsd.edu/index.html. Accessed April 15, 2013

13. van der Lee JH, Mokkink LB, GrootenhuisMA, Heymans HS, Offringa M. Definitionsand measurement of chronic health con-ditions in childhood: a systematic review.JAMA. 2007;297(24):2741–2751

14. Fishman PA, Shay DK. Development andestimation of a pediatric chronic diseasescore using automated pharmacy data.Med Care. 1999;37(9):874–883

15. Antonelli RC, McAllister JW, Popp J. MakingCare Coordination a Critical Component ofthe Pediatric Health System: A Multidisci-plinary Framework. New York, NY: Common-wealth Fund; 2009

16. McDonald KM, Schultz E, Albin L, et al. CareCoordination Atlas Version 3 (Prepared byStanford University under subcontract to Battelleon Contract N. 290-04-0020). AHRQ Publication No.11-0023. Rockville, MD: Agency for HealthcareResearch and Quality. November 2010

17. Kronick R, Gilmer T, Dreyfus T, Lee L. Im-proving health-based payment for Medicaid

ARTICLE

PEDIATRICS Volume 133, Number 6, June 2014 e1653by guest on June 21, 2016Downloaded from

beneficiaries: CDPS. Health Care FinancRev. 2000;21(3):29–64

18. Berry JG, Hall DE, Kuo DZ, et al. Hospital utili-zation and characteristics of patients experi-encing recurrent readmissions within children’shospitals. JAMA. 2011;305(7):682–690

19. Hughes JS, Averill RF, Eisenhandler J, et al.Clinical Risk Groups (CRGs): a classification

system for risk-adjusted capitation-basedpayment and health care management.Med Care. 2004;42(1):81–90

20. Kuhlthau K, Jellinek M, White G, Vancleave J,Simons J, Murphy M. Increases in behav-ioral health screening in pediatric care forMassachusetts Medicaid patients. ArchPediatr Adolesc Med. 2011;165(7):660–664

21. Cohen E, Kuo DZ, Agrawal R, et al. Childrenwith medical complexity: an emergingpopulation for clinical and research ini-tiatives. Pediatrics. 2011;127(3):529–538

22. Neff JM, Clifton H, Park KJ, et al. Identifyingchildren with lifelong chronic conditions forcare coordination by using hospital dischargedata. Acad Pediatr. 2010;10(6):417–423

(Continued from first page)

Dr Simon helped design the study, modified the Chronic Illness and Disability Payment System algorithm to develop the Pediatric Medical Complexity Algorithm,reviewed cases in developing the gold standard population, and drafted the initial manuscript; Dr Cawthon helped design the study, supervised the analyses inMedicaid data, reviewed and revised the manuscript, and takes responsibility for the integrity of the Medicaid data and accuracy of the Medicaid data analysis; MsStanford conducted the analyses in hospital discharge data, critically reviewed the manuscript, and takes responsibility for the integrity of the hospital data andaccuracy of the hospital data analysis; Ms Popalisky performed chart review to identify the gold standard population, discussed cases with the physician panel, andcritically reviewed the manuscript; Ms Lyons and Mr Woodcox conducted the analyses in Medicaid claims data and critically reviewed the manuscript; Drs Hood andChen reviewed cases in developing the gold standard population and critically reviewed the manuscript; and Dr Mangione-Smith conceptualized and designed thestudy, modified the Chronic Illness and Disability Payment System algorithm to develop the Pediatric Medical Complexity Algorithm, reviewed cases in developing thegold standard population, and reviewed and revised the manuscript. All authors approved the final manuscript as submitted.

The contents of this article are solely the responsibility of the authors and do not necessarily represent the official view of the sponsors. None of the sponsorsparticipated in design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of themanuscript.

www.pediatrics.org/cgi/doi/10.1542/peds.2013-3875

doi:10.1542/peds.2013-3875

Accepted for publication Feb 27, 2014

Address correspondence to Tamara D. Simon, MD, MSPH, Seattle Children’s Research Institute, Building 1, M/S JMB-9, 1900 Ninth Ave, Seattle, WA 98101. E-mail:tamara.simon@seattle childrens.org

PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275).

Copyright © 2014 by the American Academy of Pediatrics

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

FUNDING: This study was done under funding from a cooperative agreement with the Agency for Healthcare Research and Quality and Centers for Medicare andMedicaid Services, grant #U18HS020506, part of the Children’s Health Insurance Program Reauthorization Act Pediatric Quality Measures Program. Dr Simon is alsosupported by award K23NS062900 from the National Institute of Neurological Disorders and Stroke. Funded by the National Institutes of Health (NIH).

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

e1654 SIMON et alby guest on June 21, 2016Downloaded from

DOI: 10.1542/peds.2013-3875; originally published online May 12, 2014; 2014;133;e1647PediatricsMangione-Smith

Dorothy Lyons, Peter Woodcox, Margaret Hood, Alex Y. Chen and Rita Tamara D. Simon, Mary Lawrence Cawthon, Susan Stanford, Jean Popalisky,

Medical ComplexityPediatric Medical Complexity Algorithm: A New Method to Stratify Children by

  

ServicesUpdated Information &

/content/133/6/e1647.full.htmlincluding high resolution figures, can be found at:

Supplementary Material

html/content/suppl/2014/05/08/peds.2013-3875.DCSupplemental.Supplementary material can be found at:

References

/content/133/6/e1647.full.html#ref-list-1at:This article cites 18 articles, 6 of which can be accessed free

Citations /content/133/6/e1647.full.html#related-urls

This article has been cited by 1 HighWire-hosted articles:

Subspecialty Collections

/cgi/collection/practice-based_learning_-_development_subPractice-Based Learning & Development

/cgi/collection/administration:practice_management_subAdministration/Practice Managementthe following collection(s):This article, along with others on similar topics, appears in

Permissions & Licensing

/site/misc/Permissions.xhtmltables) or in its entirety can be found online at: Information about reproducing this article in parts (figures,

Reprints /site/misc/reprints.xhtml

Information about ordering reprints can be found online:

rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.Grove Village, Illinois, 60007. Copyright © 2014 by the American Academy of Pediatrics. All and trademarked by the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elkpublication, it has been published continuously since 1948. PEDIATRICS is owned, published, PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly

by guest on June 21, 2016Downloaded from

DOI: 10.1542/peds.2013-3875; originally published online May 12, 2014; 2014;133;e1647PediatricsMangione-Smith

Dorothy Lyons, Peter Woodcox, Margaret Hood, Alex Y. Chen and Rita Tamara D. Simon, Mary Lawrence Cawthon, Susan Stanford, Jean Popalisky,

Medical ComplexityPediatric Medical Complexity Algorithm: A New Method to Stratify Children by

  

  /content/133/6/e1647.full.html

located on the World Wide Web at: The online version of this article, along with updated information and services, is

 

of Pediatrics. All rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.Boulevard, Elk Grove Village, Illinois, 60007. Copyright © 2014 by the American Academy published, and trademarked by the American Academy of Pediatrics, 141 Northwest Pointpublication, it has been published continuously since 1948. PEDIATRICS is owned, PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly

by guest on June 21, 2016Downloaded from