96qs

8/8/2019 96qs

1/8

Special articles

Publicly disclosed information about the quality ofhealth care: response of the US public

E C Schneider, T Lieberman

AbstractPublic disclosure of information about thequality of health plans, hospitals, and doc-tors continues to be controversial. The USexperience of the past decade suggeststhat sophisticated quality measures andreporting systems that disclose infor-mation on quality have improved theprocess and outcomes of care in limitedways in some settings, but these eVorts

have not led to the consumer choicemarket envisaged. Important reasons forthis failure include limited salience ofobjective measures to consumers, thecomplexity of the task of interpretation,and insuYcient use of quality results byorganised purchasers and insurers toinform contracting and pricing decisions.Nevertheless, public disclosure may moti-vate quality managers and providers toundertake changes that improve the deliv-ery of care. EVorts to measure and reportinformation about quality should remainpublic, but may be most eVective if theyare targeted to the needs of institutionaland individual providers of care.

(Quality in Health Care 2001;10:96103)

Keywords: public disclosure; quality of health care;quality improvement

Background

Since the first publication of hospital mortalityrates in 1986 by the federal agency that admin-isters the US Medicare insurance programme,public disclosure of information about thequality of health plans, hospitals, and doctorshas provoked controversy.1 Publicly disclosedperformance reports, sometimes called reportcards, are one manifestation of a health caremarketplace in which competing providers

would measure and report information aboutthe quality of care they oVer.2 According to theproponents of market theory, US employers,who pay a major share of health care costs inthe USA, would purchase prepaid comprehen-sive care through health plans rather thanreimbursing providers for each service.3 Em-ployers would use report cards to choose highquality health plans at a reasonable cost.Insured employees would select health plans,hospitals, and doctors using report card infor-mation about providers quality, price, andaccessibility. As a result, health plans, hospitals,and doctors with the best performance records

would gain market share by improving thequality of services and becoming more eY-cient.4 The allure of a market based onconsumer choice is such that other nationsare now trying to emulate its principles as theyreform their own health systems.5

Ironically, the USA has not itself realised thisideal consumer choice market. Performancemeasures have not moved the marketplace andconsumers have not flocked to better perform-

ing health plans, hospitals, or physicians.Instead, while quality and performance resultshave not dramatically influenced the choices ofindividual or group purchasers, they appear tohave stimulated some improvements in healthcare, probably by revealing poor qualitypractices to organisational managers. Theactions of these managers may have improvedsome elements of clinical care for thoseenrolled in managed care between 1996 and1999. For example, the number of childrenreceiving varicella immunisation has increasedfrom 40% to 64%, the percentage of heartattack victims receiving beta-blocker medi-cation increased from 62% to 85%, and the

percentage of adolescents receiving measles,mumps, and rubella immunisation has in-creased from 52% to 59%.6

In the past decade, coalitions of large corpo-rate employers using their purchasing leveragehave stimulated an expansion in the number ofprogrammes that measure and publicly dis-close information about quality.7 In 1992 Con-sumer Reports was the first organisation to ratehealth plans by asking a large number ofsubscribers about their experiences in managedcare. Subscribers rated overall satisfaction aswell as a variety of dimensions of service qualitysuch as waiting times for non-emergencyappointments, waiting times on the telephone,and in doctors surgeries. At the same time anon-profit organisation, the National Com-mittee for Quality Assurance (NCQA), began toaccredit health plans and to assess theirperformance using a standardised set of mea-sures known as HEDIS, the Health PlanEmployer Data and Information Set.

HEDIS was designed primarily to gauge theunderuse of medical services in managed careplans. In developing and expanding HEDISover the years the NCQA has convened repre-sentatives of employers, health plans, and con-sumers to guide policy on performancemeasurement. According to the NCQA model,

Quality in Health Care 2001;10:9610396

Division of General

Medicine and Primary

Care, Department of

Medicine, Brigham

and Womens Hospital

and Harvard Medical

School and

Department of Health

Policy and

Management, Harvard

School of Public

Health, Boston,Massachusetts 02115,

USA

E C Schneider, instructor

Center for Consumer

Health Choices,

Consumers Union,

New York, USA

T Lieberman, director

Correspondence to:Dr E C [email protected]

Accepted 28 March 2001

www.qualityhealthcare.com

8/8/2019 96qs

2/8

employers would oVer their employees choiceamong only the health plans with the best per-formance records. In choosing from prese-lected health plans, employees would usereport cards to discriminate further. Thefederal government also contributed by financ-ing development of the Consumer Assessmentof Health Plans (CAHPS), a standardisedsurvey of health plan members that providesdata for plan comparisons. The government

also required the health plans serving Medicarebeneficiaries with which it had a contract toreport HEDIS and CAHPS data.

This proliferation of programmes hasspawned concerns about adverse eVects ofpublicly disclosing information about healthcare quality. Some observers have worriedabout the potential for performance results tomislead patients, because the measures addressonly a small fraction of the care provided bydoctors and may not be suYciently standard-ised or adjusted for important diVerences inthe health risks of providers populations.812

Also, providers could ignore unmeasuredaspects of practice and concentrate on meas-

ures that show them in a favourable light orgame measurement systems by refusing tocare for the most ill patients or by recordinginaccurate data.13 The expense of generatingrigorous performance reports might divertresources from patient care.14 Others have wor-ried that two key audiencespurchasers andpatientswould be unable to use this novelinformation to aid in the decision making andchoice of providers.1518 Providers might be-come indiVerent or hostile to performancemeasurement. For example, NCQA has facedconsiderable diYculty convincing employersthat there is a business case for improvingquality through report cards. Report cardsdont translate into a market advantage, says

Allan Greenberg, former CEO of Harvard Pil-grim Health Care. The small and mid-sizeemployers are not involved in that movement.The larger ones do care, but will they pay morefor good report cards? No.19

In this contentious environment US policymakers have gained considerable knowledgeabout the eVects of a public disclosurestrategy.20 This review describes the lessons ofthe public disclosure movement in the USAand focuses on the eVects of disclosure on theprocesses and outcomes of health care, theeVects on choices by individual and group pur-chasers, the eVects on health plans selection ofproviders, and the response of the media to

demands for information about quality. Wethen speculate about the value of public disclo-sure and the future directions that it mighttake.

Evidence for the eVectiveness of publicdisclosureMuch of what we know about the eVects ofpublic disclosure of information about healthcare quality is based on experience with fourprogrammes: (1) disclosure of risk adjustedhospital mortality rates, (2) disclosure of riskadjusted mortality rates after coronary arterybypass graft surgery, (3) disclosure of health

plan performance using HEDIS measures ofthe quality of preventive care, and (4) disclo-sure of patient and enrollee experiences withthe quality of health plan and hospital careusing a variety of surveys.

To evaluate the impact of public disclosure,researchers have looked for evidence of directand indirect eVects of the information. Draw-ing on market theory, some researchers havehypothesised that, in the wake of disclosure,

individuals choose higher quality providersleading to measurable improvements in popu-lation health outcomes. They also suggest that,for this to occur, a series of measurablemediating events are necessary: purchasers andconsumers should have access to report cards,obtain them, interpret them correctly, andswitch providers or health plans, appropriatelyincreasing the market shares of high qualityproviders. Other researchers have postulatedthat providers will use performance reports toimprove practices even if consumers do not usethem to make decisions. We review existingresearch, explore some of the institutionalresponses, and describe the influence of the

media on the demand for publicly disclosedinformation about quality over the past decade.

EVect on processes and outcomes of care

A handful of studies have attempted to evaluatethe impact of public disclosure programmes onthe processes and outcomes of care. TheNCQA has found that health plans haveincreased the delivery of preventive servicesthat were the subject of measurement and pub-lic disclosure. In general, observational studiesof outcomes have compared mortality ratesand other clinical outcomes (caesareansection rates and vaginal birth after caesareanrates) before and after statewide or local publicdisclosure programmes began.2125 All these

studies found small improvements in measuredoutcomes; however, none involved a controlgroup that did not receive public information,so the relationship between improved out-comes and public disclosure remains specula-tive. For instance, rates of mortality for coron-ary artery bypass graft surgery appeared to bedeclining over time at similar rates in neigh-bouring states that did not have publicdisclosure programmes, suggesting that im-provements by providers, rather than selectionof high quality surgeons by consumers, wasmore likely to explain the decline.26 27

There are many reasons why processes andoutcomes might be slow to change in response

to new information. Barriers such as lack ofawareness of the information, a lack of salienceor interpretability, or the inability to switchproviders because of constraints related toinsurance or travel distance could interferewith the use of information about quality. Pur-chasers might have existing contracts withselected providers that cannot be alteredimmediately. Poorly performing hospitalsmight be able to maintain the flow of patientsby reducing their prices and thereby retainingcontracts with purchasers. For the last 6 years,New York State has published risk adjusteddata on mortality rates after cardiac surgery. In

Publicly disclosed information about the quality of health care 97


8/8/2019 96qs

3/8

two studies, managed care organisations didnot appear to contract with hospitals that hadlower mortality rates. Indeed, they may havebeen more likely to contract with hospitals thathad higher mortality rates, presumably becauseof pre-existing contracts or because they wereable to obtain lower prices.28 29

Barriers to consumer use of information

In light of the diYculty of interpreting broader

changes in processes or outcomes of care, manystudies have examined the direct eVects of dis-closing information to purchasers and consum-ers. A growing body of literature suggests thatneither purchasers nor consumers make use ofpublicly disclosed information about quality.20

This failure appears to be related to the short-comings of report cards, the complexity of thetask for individual consumers of obtaining andcomparing information and, perhaps mostimportant, the limited salience of the infor-mation.17

Much time, eVort, and money has beenexpended to identify the best formats fordisclosing information about qualityfor in-

stance, whether numbers, stars, bars, circles, orgraphs are most informative. Despite this, mostreport cards are not useful documents. Con-sumer Reports, the well known independentmagazine that tests and rates products andservices, examined 30 report cards intended tohelp employees and Medicare and Medicaidrecipients choose a health plan. All failed to tellconsumers how to use the data or how to maketrade oVs. Most did not present price infor-mation that would help users to make a tradeoV between the premium required and thequality of services. Many report cards over-whelmed consumers with too much infor-mation and oVered explanations that wereconfusing or misleading. Sometimes unrelated

dimensions were mixed up and rolled into asingle measure that obscured the significanceof each. The report cards did not rank plans ormake judgements, but left it to consumers tofigure out how to interpret the data. For politi-cal reasons, sponsors of report cards have hesi-tated to oVer such judgements.19

EVorts to enhance presentation formats maywell be a smokescreen that obscures a morefundamental problemnamely, the relativeweakness of quantitative data compared withpersonal past experience, recommendationsfrom trusted professionals, or recommenda-tions from informal networks of family andfriends. Consumers have not perceived report

card information to be useful because the indi-cators are not salient. They do not view allquality indicators as equally useful, and theyfrequently do not share the assumptions thatunderpin quality indicators.30 To date, thedevelopers of HEDIS and CAHPS haveconsidered their measures to be importantproxies for the overall quality of healthplansan assumption that remains untested.No published studies have assessed whether ornot health plans with higher HEDIS orCAHPS scores also have better clinical out-comes or provide more appropriate medicalcare.

It is important to note that HEDIS assessesunderuse, but not overuse or misuse, whichare equally important dimensions of quality.Neither HEDIS nor CAHPS measures theappropriateness of care (whether patients willreceive the right diagnosis and the right treat-ment at the right time). The rate of mammo-graphy in a health plan does not tell patientswhether they will get appropriate follow uptreatment for breast cancer. Nor do HEDIS or

CAHPS tell patients facing potential backsurgery whether the doctor accurately deter-mined their need for the operation or whetherthe back surgeon has performed the procedureoften enough to qualify as competent. Misuseand overuse are highly salient to patients butare still unmeasured.

In the absence of salience, quality indicatorsdo not outweigh other factors such as price orthe recommendations of family and friends. Ina controlled experiment in which subjects werepaid to review simulated quality informationon health plans and then choose from amongthem, consumers were willing to choose lowprice, high quality health plans, but tended to

be more influenced by the benefits availableand the price. A large percentage of subjectswere willing to keep a more expensive healthplan with more benefits even if it had lowerquality ratings.31

Assuming the availability of valid, reliable,and credible report cards, there are many otherreasons why consumers might fail to use themto drive competition and improve quality.Firstly, consumers must believe that qualityvaries. If they do not, then they will not seekinformation to learn about diVerences in qual-ity. Secondly, consumers must believe that theyhave a choice of health plans, doctors, and hos-pitals. They must have timely access to

information about quality for real time decisionmaking, they must trust the source of theinformation, and they must be able to interpretit correctlythat is, they must understand theinformation suYciently to be able to make abetter choice.

Failure to satisfy even one of these precondi-tions can render a report card useless as a deci-sion tool. For example, in a national surveyfewer than 50% of Americans believed thatthere are big diVerences in quality amonghealth care providers such as health plans, hos-pitals, and primary care providers.18 Less thanone third believed that there are big diVer-ences in quality among specialists. Thissuggests that about half of all Americans wouldnot even seek information about quality diVer-ences. Another study reported that only aminority of Americans have a choice amonghealth plans.32 Those without a choice wouldbe unlikely to make use of health plan reportcards. Less than one third of Americansreported that they had seen any information onhealth care quality. Even though the vastmajority believed the information would beuseful, very few reported using it.18 A relatedseries of studies in which clinical investigatorscarefully interviewed individuals about thecontent of report cards revealed that, in the

98 Schneider, Lieberman


8/8/2019 96qs

4/8

context of health care, consumers did not eas-ily understand information about quality andeven had problems interpreting price infor-mation.30

Failure of highly salient information toreach patients

Even under the best conditions when infor-mation is highly salient, it is not clear that con-sumers use it. In Pennsylvania, a state with a

public disclosure programme that lists hospitalspecific and surgeon specific risk adjustedmortality rates, cardiac surgery patients whohad recently undergone coronary artery bypassgraft surgery did not obtain or use theinformation. Fewer than 20% of the patientswere aware of the mortality rates, fewer than12% reported that they saw the informationbefore having an operation, and fewer than 1%reported that the information had influencedtheir choice of hospital or surgeon.33 The stateagency had engaged in extensive disseminationeVorts that included newspaper stories andpublic service announcements in the media. Inaddition, it made the reports available free by

mail as well as on a web site (www.phc4.org).In general, most patients accepted informationabout quality from their referring cardiologist.A related study found that cardiologists rarelyinformed patients of the existence of the riskadjusted operative mortality rates.13

The situation was similar for information onthe quality of long term care facilities (nursinghomes). Among readers of Consumer Reportswho had recent experience with long term care,only 6% had used the results of comprehensivestate surveys detailing lapses in quality of nurs-ing homes. The reports are published to assistpeople in choosing a long term care facility.34

Can information on quality reshape

public opinion?By shaping the opinions of relatives, friends, ortrusted clinician advisors, publicly disclosedquality information could conceivably alter thechoices patients make without their knowl-edge.18 35 This eVect, which would driveconsumers to vote with their feet, should bemeasurable as a rise in market share ofhospitals or health plans identified as highquality performers and a shrinking of marketshare among those identified as low qualityperformers.

Few studies have examined the possibilitythat market shares change in the wake of pub-lic disclosure. After the Health Care Financing

Administration (HCFA) published risk ad-justed hospital mortality data in 1986, onestudy found no change in occupancy ratesamong hospitals.36 As in Pennsylvania, NewYorks cardiac surgery reporting programmedisseminates results annually, publishing themin major newspapers and posting them on itsweb site (www.health.state.ny.us). Even thoughrisk adjusted mortality rates after cardiacsurgery declined rapidly in New York Stateonce the public reporting programme began,surgical volumes did not rise or fall signifi-cantly at hospitals with high and low mortalityrates.25 Another study of the New York

programme suggested that surgeons with lowermortality after cardiac surgery might haveincreased their personal market share of surgi-cal operations after public disclosure began,but the reasons for this growth remaineduncertain.37

Role of corporate purchasers in selectinghigh quality providers

Given the challenges faced by individualconsumers, some researchers have advocatedthat purchasers interpret information on qual-ity and guide consumers towards higher qualityhealth plans and hospitals using contractingleverage or financial incentives.38 This ap-proach, sometimes labelled value based pur-chasing, includes selective contracting withhigh quality, low cost providers, availability ofcomparative performance reports (or reportcards) at the time of open enrolment whenemployees are selecting insurance plans, andperformance based premium sharing in whichthe employer subsidises a greater percentage ofthe insurance premium for high quality health

plans, making these arrangements less costly tothe employee than lower quality health plans.39

Research suggests, however, that few pur-chasers or employer coalitions use informationabout quality in a meaningful way. Surveys ofemployers in 1996 and 1997 showed that onlya third were aware of a health plan accredita-tion programme oVered by the NCQA. Only9% of employers insisted that the health planshould pass accreditation, and only 1% pro-vided HEDIS data to employees.40 Only 15%of these firms thought it was very importantto provide HEDIS data to employees. Largeremployers have been the main proponents ofhealth care performance measurement andpublic disclosure. They were more likely than

smaller employers to use performance stand-ards in contracting decisions with health plans.

Health plans with poorer scores on variousdimensions of quality have not withered away.Contrary to conventional market theory, someplans with the worst scores for consumer satis-faction have indeed become dominant in themarketplace. Cigna, Humana, and US Health-care (now Aetna US Healthcare) have beenranked towards the bottom in Consumer Reportsrankings in 1992, 1996, and 1999, and we haveno reason to believe there will be a change inthe next report due in the autumn of 2001.19 41 42 One reason for the lower satisfac-tion scores for US Healthcare may rest with the

plans tighter control of medical services. Thisenabled the plan to control costs and pass onthe savings to employers in the form of lowerpremiums which is what employers demanded(Dr Carol Diamond, former President of USQuality Algorithms, personal communication).Furthermore, many health plans thrive eventhough they have refused to allow publicdisclosure of their HEDIS performance. Formany years the Cigna health plan did not allowthe NCQA to release its performance data tothe public. In fact, 21 out of 54 health plansrated by Consumer Reports in 1999 refused toallow disclosure of their HEDIS performance



8/8/2019 96qs

5/8

by NCQA.19 These 21 plans scored signifi-cantly lower than the other plans in theConsumer Reports study. In addition, healthplans that reported HEDIS indicators to theNCQA, but refused to allow the NCQA to dis-close them publicly, had lower scores on theHEDIS indicators than other health plans.43

Cigna, Humana, and what is now Aetna USHealthcare are among the most successfulmajor national health plans, suggesting that

factors other than patient satisfaction influencean employers decision to oVer particularhealth plans. Prompt payment of claims, largenetworks, and low prices are paramount inemployers decisions.40 At the same time, USHealthcare was an innovator in quality im-provement programmes but, according to Dia-mond, the marketplace did not reward the planfor those eVorts. High ratings and highsatisfaction do not translate into increasedenrolment or financial success. Harvard Pil-grim Health Care exhibited both high HEDISscores43 and excellent satisfaction scores,19 42

but in early 2000 the plan had plunged intofinancial diYculty.

In the USA, smaller employers (with fewerthan 50 employees) represent 96% of the jobmarket. Among these employers, quality ofcare and accreditation ranked fifth and sixthout of six factors considered in buying healthinsurance coverage for their employees. Price,benefits packages, and physician availabilityranked higher.44 Some employers have pooledtheir buying power by forming or joiningpurchasing coalitions.45 Whether these coali-tions use information about quality is not clear.Among employers that were members of healthcare purchasing coalitions, most stated thatthey were responsible for the quality of healthplans oVered to employees, but the primaryfactors considered were the geographical cov-

erage oVered by health plans and access ofmembers to services.46

Some purchasing coalitions have been eVec-tive in negotiating price discounts with healthplans, in disseminating information aboutquality (primarily HEDIS data and satisfactionsurveys), and in improving quality.47 In particu-lar, the Pacific Business Group on Healthincorporated publicly disclosed HEDIS infor-mation into contract negotiations and put morethan $8 million in premium payments in a riskpool to be paid only to health plans thatreached performance targets. It appeared thatHEDIS data were a valuable part of the processof improving care.48 In a highly structured

regional purchasing arrangement throughwhich a purchasing coalition buys health careservices from care systems with non-overlapping providers (as opposed to healthplans that may include many of the same pro-viders in each of their networks), the BuyersHealth Care Action Group (BHCAG) demon-strated that care systems with higher publiclydisclosed satisfaction scores appeared to begaining enrolment compared with those withlower satisfaction scores.49 50

Few studies have examined whether or notperformance reports provided by employershave had an impact on employee selection of

health plans. In one large company, employeeswere less likely to select health plans with highsatisfaction ratings but were more likely toselect health plans with high preventive servicescores.51 Other aspects of health plan perform-ance had no relationship to the choice of healthplan, but the higher the premium, the lower theprobability that employees would select aparticular plan. In a related study thereappeared to be no relationship between the

publicly disclosed HEDIS scores of a healthplan and its share of enrolment when control-ling for other factors, which suggests that theinformation did not encourage enrolment inhigh quality health plans.52

It has also been suggested that health plansmight use information about quality on behalfof their members to select specialists and hos-pitals for their networks. However, two recentstudies suggest that health plans did not userisk adjusted mortality data to obtain cardiacsurgery services at hospitals with lower mor-tality rates and, instead, may have contractedselectively with hospitals that had highermortality rates.28 29 Some hospitals have de-

manded non-steerage clauses in health plancontracts to prevent selective contracting.

The media and demand for publicdisclosure of information on quality ofhealth care

By 1996 when Consumer Reports surveyed sub-scribers about their health plans for the secondtime, new raters had entered the field. Theavailability of HEDIS scores from the NCQA,as well as data on satisfaction,enabled US Newsand World Report and Newsweek magazines tocompile their own ratings of health plans. Inthe mid 1990s news organisations rushed topublicise the initial release of HEDIS data. In

many instances they made inferences whichmay have gone beyond what the data wouldsupport. In 1996 Consumer Reports noted thatthe numbers from HEDIS data were insuY-cient to measure quality. US News publishedannual ratings up to 1999 and then stoppedbecause the chief editor believed that consum-ers in most areas had no choice (AveryComarow, Editor,Americas Best Hospitals, per-sonal communication). Newsweek continues topublish ratings to coincide with the annualrelease of NCQA data. The NCQA publishesinformation about quality as part of a productcalled Quality Compass, of which an interactiveversion is available on its website (www.nc-

qa.org). The current version of Quality Com-pass combines the results of the NCQAaccreditation surveys with HEDIS measures.The results are presented in five groups: accessand service, qualified providers, stayinghealthy, getting better, and living with illness.Many dimensions are rolled into one singlecomposite represented by stars. Underlyingdata are not readily available for consumerswho wish to probe behind the combinedscores. The release of the NCQA datagenerates a lot of press attention from localnewspapers and television stations which pub-licise the relevant results for their geographical



8/8/2019 96qs

6/8

areas. Some states such as Maryland, New Jer-sey, New York, and Pennsylvania also makeavailable report cards giving HEDIS scores forhealth plans operating in their states. Newspa-pers sometimes publish information from thesereport cards. The eVects of these eVorts remainlargely unstudied.

US News continues to publish an annualranking of hospitals. Using data such asmortality rates, ratings provided by top special-

ists, and data from surveys conducted by theAmerican Hospital Association, the report liststhe 173 best hospitals in the country.53

Although one study confirmed that highlyranked hospitals were more likely to providecardiac care consistent with guidelines forpatients with myocardial infarction, there hasbeen little or no evidence that patients areselecting hospitals according to the US Newsranking. Patients most often go to the hospitalrecommended by their doctor, and locationmay be the most important factor used inselecting a hospital. Even when health planshave tried to steer members to particularhospitals that were centres of excellence for a

particular procedure or treatment, patientshave refused to go,arguing that health plans aresending them to these hospitals to save money(Dr Andrew Weisenthal, Kaiser Permanente,personal communication).

US News has no quantitative data on how orwhether its rankings are used. According toAvery Comarow, Editor of Americas Best Hos-

pitals, US News receives twice as many callsfrom hospitals and health systems as it doesfrom consumers. Hospitals and health systemswant to learn what they can do to be listed. Itappears that hospitals will go to great lengths toappear on these lists, says Comarow, Whatstarted out as something to bring service toconsumers has turned into a marketing tool.

The media have played a leading role inpromoting the use of public information onquality, but it is not clear that consumers aredemanding such information.

Resistance of providers to performancereports

During the past few years providers haveincreased their resistance to further attempts tomake public the data on quality. For example,Consumer Reports has had diYculty in gettinghealth plans to provide data to augment surveyresults from its members. Those that doparticipate often give skimpy answers thatappear as if public relations oYcials have

prepared them or they delay participation untilthe data are no longer relevant. Some providersare simply reluctant to participate in voluntaryeVorts to improve quality. The CaliforniaHealthcare Foundation and the CaliforniaInstitute for Health System Performance, anaYliate of the California Healthcare Associ-ation, are sponsoring a project called PatientEvaluation and Performance in California Hos-pitals. Some 300 hospitals have been asked tofield a survey to patients inquiring about theirhospital stay. The data from a standardisedinstrument are to be made public later this year.Only 100 hospitals representing 50% of the

beds have participated (Ann Monroe, Director,Quality Initiative, California Healthcare Foun-dation, personal communication).

Conclusions and future directions

The US experience with public disclosure is inmany ways paradoxical. Consumers express adesire for more information about qualitywhile, at the same time, few are able to under-stand or use it. Producers of report cards con-

tinue to refine measures and create newpresentation formats for patients and purchas-ers even though there is no evidence that thesegroups choose health plans or providers basedon report card information. Despite the factthat consumers and purchasers do not use theinformation to guide their selection of provid-ers, outcomes and performance appear to beimproving in some areas. If public disclosure ofreport cards has an eVect, it is because disclo-sure focuses health plans, hospitals, and otherproviders to address issues of health care qual-ity that may in the past have been ignored. 24 5456

By standardising the measurement of quality,HEDIS has created expectations about per-

formance and helped organisations to identifygaps in care. This may be its greatest contribu-tion. Provider organisations have made eVortsto measure and increase performance rates byusing measures for internal monitoring and toset financial rewards and penalties for provid-ers.57 58 Report cards on rates of mortality aftercardiac surgery have directed hospital managersand surgeons to problem areas, forcing them toexamine the process of surgical care and toimprove it. Public disclosure appears to galva-nise providers into action, either because theyperceive a threat to their reputations or becauseit is an opportunity to market their excel-lence.20 27

Will the sleeping giant of consumer choice

awaken if we discover the right qualitymeasures, the right report formats, the rightdissemination strategy, or the right methods foreducating consumers to use report cards?Would measures that evaluate individual physi-cians on a variety of performance dimensionsbe more useful to consumers? We do not thinkso. Evidence suggests that voluntary reportingby health plans and hospitals is inadequate,that purchasers are losing interest, that costsare too high, and provider resistance too formi-dable.

Except in the case of mandates from publicinsurance programmes, health plans and hospi-tals have not been required by law or regulation

to report performance data for public disclo-sure. While many health plans and hospitals doreport data voluntarily or at the request of cor-porate purchasers, in 1996 65% of health plansdid not participate in the NCQAs HEDISreporting and resistance to reporting may begrowing.59 In Cleveland, Ohio the ClevelandHealth Quality Choice programme, one of themore ambitious and successful pioneeringeVorts to report hospital quality, ceased opera-tions when a large number of hospitals refusedto report performance data.60

Some groups such as the Leapfrog group, anorganisation of employers, frustrated with their



8/8/2019 96qs

7/8

inability to obtain useful data on performanceare asking instead for other data on which torate quality. They have abandoned perform-ance measures in favour of structural meas-ures such as the presence of automated drugentry ordering systems in hospitals or intensivecare staYng ratios. While such informationmay be useful to assure a minimum standard ofquality in a resource constrained system, thisseems to reject the notion of accountability and

competition based on performance.The health care system simply cannot aVord

the cost of making a quality measurement setsuYciently encyclopaedic to meet the needs ofevery patient needing care for every condition.We have probably reached the end of eVorts toimprove presentation formats for informationabout quality. Even if we perfected collectionand formatting, dissemination would continueto be a formidable hurdle. Geography andtravel costs constrain consumers choiceamong providers for specialised health services.Consumers are unlikely to have the time,money, or inclination to obtain the specialisedtraining that would make them expert in the

interpretation of dozens of measures of quality.Providers have found it easy to underminethe information on quality if it bites them. InPennsylvania some cardiac specialists per-suaded patients that hospitals with low mor-tality rates following cardiac surgery were notbetter, but instead were unwilling to operate onthe most ill patients who were more likely todie. Reports on nursing home facilities thatdisclose the results of state inspections arerequired by law to be posted in every nursinghome across the country. This is an excellentsource of consumer information that details thequality of care given in a facility. However, astudy by Consumer Reports in 1995 found thatthe reports of these inspections were unavail-

able, hidden, out of reach, out of date, orunreadable in 37 out of 52 nursing homes.61

Many have suggested that the internet couldmake the consumers task more manageable. Inview of the scant use of information on qualitythat we cite, the proliferation of health web sitesthat serve consumers may seem paradoxical,but it is not. Consumers actively seek infor-mation about diseases that they have and treat-ments they are considering. This is not a newphenomenon, but has existed since before theearly days of organised medicine. The internetprovides information primarily about diseasesand treatments, but not about performance.The suspect credibility of many web sponsors

makes it unclear whether the web can oV

erunbiased and believable information on thequality of competing provider organisations.

Finally, all eVorts to encourage consumers toselect providers are aimed at those fortunateAmericans who have health insurance. Theuninsured endure severe quality problems inhealth care, and performance measures do notaddress their problems.62 Outside the Medicaidprogramme, which insures some of the poorand has moved beneficiaries into managed carewhile oVering report cards based on HEDISand CAHPS measures, there is no targeteddissemination of report cards to the poor or

uninsured, nor is it clear how they could beuseful in the absence of a range of care options.Community health centres, an importantresource for poor and uninsured patients, havebeen unable to muster the resources forperformance measurement.

A sense of urgency about quality of care andthe safety of health care systems in the USA isfuelling continued interest in public reportingon quality and has led to proposals to report

physicians error rates.63 New reporting initia-tives that extend HEDIS and CAHPS toaddress the care of patients insured bytraditional insurance (rather than managedcare) are under way and new uses of the infor-mation for public health monitoring are ofgreat interest.6468 Medical professional socie-ties such as the American Society for ClinicalOncology and the American Diabetes Associ-ation are sponsoring initiatives to develop newquality measures in their areas for treatmentand management of chronic disease. It may bethat medical professionals are beginning to seethe value of performance measurement toguide quality improvement. It is possible that

patients may be more interested in measuresthat assess the care of chronic conditions.Although consumers may not use publicly

disclosed information on quality, we believethat quality measurement should continue andthat results should be made available to theinterested public. Keeping in mind that per-formance reports are not the engine that willdrive consumers to choose high quality provid-ers, organisations such as the NCQA have animportant leadership role and should continueto expand the dialogue among stakeholders,including purchasers, consumers, and provid-ers. These organisations should continue theremarkable voluntary eVort to expand andrefine measures, to reduce the costs of

measurement, to make data collection moreeYcient, and to understand the use of reportcardsnot as a guide for consumer choice butas a way to bring about system-wide qualityimprovement.11

Voluntary eVorts will not, however, beadequate to assure widespread quality im-provement. Some form of legal mandate maybe necessary to ensure that all providers collectperformance data and participate in qualityimprovement. Information about quality mustcontinue to be publicly available. If it is not,there will be no assurance that provider institu-tions will pay attention to medical care quality.In the USA publicly disclosed information

about health care quality can be used to estab-lish a public consensus about the need forquality improvement and the targets for thatimprovement. Such information will also pro-vide a necessary check on the activities of alargely profit making health care industry inwhich investors goals may supersede the focuson quality. But public disclosure of informationabout the quality of health care is a weak strat-egy for ensuring quality.

T Lieberman is a contributing editor to the Columbia JournalismReview and a board member of the National Committee forQuality Assurance, Washington, DC.



8/8/2019 96qs

8/8

The views expressed here are solely those of the authors and donot necessarily represent those of their aYliated institutions.

1 Epstein A. Performance reports on quality: prototypes,problems, and prospects. N Engl J Med 1995;333:5761.

2 Bodenheimer T. The American health care system:physicians and the changing medical marketplace.N Engl J

Med1999;340:5848.3 Enthoven AC. The history and principles of managed com-

petition. Health AVairs 1993;12(Suppl):2448.4 Bodenheimer T. The American health care system: the

movement for improved quality in health care. N Engl JMed1999;340:48892.

5 Marshall M, Shekelle P, Leatherman S, et al. Public disclo-

sure of performance data:learning from the US experience.Quality in Health Care 2000;9:537.6 National Committee for Quality Assurance. The state of

managed care quality. Report ed: ncqa.org, 2000.7 Epstein AM.Rolling down the runway:the challenges ahead

for quality report cards. JAMA 1998;279:16916.8 Kassirer JP. The quality of care and the quality of measuring

it. N Engl J Med 1993;329:12635.9 McNeil BJ, Pedersen SH, Gatsonis C. Current issues in

profiling quality of care. Inquiry 1992;29:298307.10 Robinson JC, Gardner LB. Adverse selection among multi-

ple competing health maintenance organizations. Med Care1995;33:116175.

11 Schneider EC, Riehl V, Courte-Wienecke S, et al. Enhanc-ing performance measurement: NCQAs road map for ahealth information framework. National Committee forQuality Assurance. JAMA 1999;282:118490.

12 Green J, Wintfeld N. Report cards on cardiac surgeons.Assessing New York States approach. N Engl J Med 1995;332:122932.

13 Schneider EC, Epstein AM. Influence of cardiac-surgeryperformance reports on referral practices and access tocare. N Engl J Med 1996;335:2516.

14 McGlynn EA. Six challenges in measuring the quality ofhealth care. Health AVairs 1997;16:721.15 Edgman-Levitan S, Cleary P. What information do

consumers want and need? Health AVairs 1996;15:4256.16 Hibbard JH, Jewett JJ. What type of quality information do

consumers want in a health care report card? Med Care ResRev 1996;53:2847.

17 Hibbard J, Slovic P, Jewett J. Informing consumer decisionsin health care: implications from decision-making research.The Milbank Quarterly 1997;75:395414.

18 Robinson S,Brodie M. Understanding the quality challengefor health consumers: the Kaiser/AHCPR survey. JCAHO

J Qual Improv 1997;23:23944.19 Lieberman T. How does your HMO stack up? Rating the

raters. Consumer Rep 1999;64:23.20 Marshall MN,Shekelle PG, Leatherman S, et al. The public

release of performance data: what do we expect to gain? Areview of the evidence. JAMA 2000;283:186674.

21 Rosenthal GE, Quinn L, Harper DL. Declines in hospitalmortality associated with a regional initiative to measurehospital performance. Am J Med Qual 1997;12:10312.

22 Peterson E, DeLong E, Jollis J, et al. The eVects of NewYorks bypass surgery provider profiling on access to careand patient outcomes in the elderly. J Am Coll Cardiol1998;32:9939.

23 Mennemeyer ST, Morrisey MA, Howard LZ. Death andreputation: how consumers acted upon HCFA mortalityinformation. Inquiry 1997;34:11728 (in Macintosh PCU-NIX TextHTML format).

24 Longo DR, Land G,Schramm W, et al. Consumer reports inhealth care: do they make a diVerence in patient care?

JAMA 1997;278:157984.25 Hannan EL, Kilburn H Jr, Racz M, et al. Improving the out-

comes of coronary artery bypass surgery in New YorkState. JAMA 1994;271:7616.

26 Ghali W, Ash A, Hall R, et a l . Statewide qualityimprovement initiatives and mortality after cardiac surgery.

JAMA 1997;277:37982.27 OConnor GT, Plume SK, Olmstead EM, et al. A regional

intervention to improve the hospital mortality associatedwith coronary arter y bypass graft surgery.JAMA 1996;275:8416.

28 Mukamel DB, Mushlin AI, Weimer D, et al. Do qualityreport cards play a role in HMOs contracting practices?Evidence from New York State. Health Serv Res 2000;35:31932.

29 Erickson LC, Torchiana DF, Schneider EC, et al. The rela-

tionship between managed care insurance and use oflower-mortality hospitals for CABG surgery. JAMA 2000;283:197682.

30 Hibbard J, Jewett J, Engelmann S, et al. Can Medicare ben-eficiaries make informed choices? Health AVairs 1998;17:18193.

31 Spranca M, Kanouse DE, Elliott M, et al. Do consumerreports of health plan quality aVect health plan selection?Health Serv Res 2000;35:93347.

32 Gawande AA, Blendon R, Brodie M, et al. Does dissatisfac-tion with health plans stem from having no choices? Health

AVairs 1998;17:18494.33 Schneider EC, Epstein AM. Patient use of public perform-

ance reports: a survey of cardiac surgery patients. JAMA1998;279:163842.

34 Lieberman T. The consumer reports complete guide to healthservices for seniors. New York: Three River Press, 2000:21213.

35 Luft HS, Garnick DW, Mark DH, et al. Does quality influ-ence choice of hospital? JAMA 1990;263:2899906.

36 Vladeck BC, Goodwin EJ, Myers LP, et al. Consumers andhospital use: the HCFA death list. Health AVairs 1988;7:1225.

37 Mukamel DB, Mushlin AI. Quality of care informationmakes a diVerence: an analysis of market share and pricechanges after publication of the New York State cardiacsurgery mortality reports. Med Care 1998;36:94554.

38 Enthoven AC. Multiple choice health insurance: the lessonsand challenge to employers. Inquiry 1990;27:36873.

39 Agency for Health Care Policy and Research (AHCPR).Theory and reality of value-based purchasing. AHCPR publi-cation no. 98-0004, 1997.

40 Gabel J, Hunt K, Hurst K. When employees choose healthplans do NCQA accreditation and HEDIS data count?New York: The Commonwealth Fund, 1998: 125.

41 Lieberman T. Health care in crisis: are HMOs the answer?Consumer Rep 1992;57:519.

42 Lieberman T. How good is your health plan? Consumer Rep1996;61:28.

43 National Committee for Quality Assurance (NCQA). Qual-ity compass national averages, 1997 and 1998 data.Washington: NCQA, 1999.

44 Legnini MW, Rosenberg LE, Perry MJ, et al. Where doesperformance measurement go from here? Health AVairs2000;19:1737.

45 Long SH, Marquis MS. Pooled purchasing: who are theplayers? Health AVairs 1999;18:10511.

46 Lo Sasso AT, PerloV L, Schield J, et al. Beyond cost:responsible purchasing of managed care by employers.Health AVairs 1999;18:21223.

47 Maxwell J, Briscoe F, Davidson S, et al. Managed competi-tion in practice: value purchasing by fourteen employers.Health AVairs 1998;17:21626.

48 SchauZer HH, Brown C, Milstein A. Raising the bar: theuse of performance guarantees by the Pacific BusinessGroup on Health. Health AVairs 1999;18:13442.

49 Knutson D. Case study: the Minneapolis Buyers HealthCare Action Group. Inquiry 1998;35:1717.

50 Christianson J, Feldman R, Weiner JP, et al. Early experiencewith a new model of employer group purchasing in Minne-sota. Health AVairs 1999;18:10014.

51 Chernew M, Scanlon DP. Health plan report cards andinsurance choice. Inquiry 1998;35:922.

52 Scanlon DP, Chernew M. HEDIS measures and managedcare enrollment. M ed Car e Res Rev 1999;56(Suppl2):6084.

53 Best hospitals finder. US News. usnews.com, 2000.54 Bentley JM, Nash DB. How Pennsylvania hospitals

responded to publicly released reports on coronary arterybypass graft surgery. JCAHO J Qual Improvement 1998;24:409.

55 Dziuban SW Jr, McIlduV JB, Miller SJ, et al. How a NewYork cardiac surgery program uses outcomes data. AnnThorac Surg 1994;58:18716.

56 Rosenthal GE, Hammar PJ, Way LE, et al. Using hospital

performance data in quality improvement: the ClevelandHealth Quality Choice experience. Joint Commission J QualImprovement1998;24:34760.

57 Grumbach K, Osmond D, Vranizan K, et al. Primary carephysicians experience of financial incentives in managed-care systems. N Engl J Med 1998;339:151621.

58 Kerr EA, Mittman BS, Hays RD, et al. Quality assurance incapitated physician groups. Where is the emphasis? JAMA1996;276:12369.

59 Farley D, McGlynn E, Klein D. Assessing quality inmanaged care: health plan reporting of HEDIS perform-ance measures. New York: The Commonwealth Fund,1998.

60 Greene J. Report cards. Clevelands Phoenix (news). Hospi-tals & Health Networks 1999;73:14.

61 Lieberman T. Nursing homes: when a loved one needs care.Consumer Rep 1995;60:51828.

62 Lieberman T. Second class medicine. Consumer Rep2000;65:42.

63 Chassin MR, Galvin RW. The urgent need to improvehealth care quality. JAMA 1998;280:10005.

64 Miller T, Leatherman S. The National Quality Forum: ame-too or a breakthrough in quality measurement and

reporting? Health AVairs 1999;18:2337.65 McGee J, Kanouse DE, Sofaer S, et al. Making surveyresults easy to report to consumers: how reporting needsguided survey design in CAHPS. Consumer Assessment ofHealth Plans Study. Med Care 1999;37:MS3240.

66 Carman KL, Short PF, Farley DO, et al. Epilogue: early les-sons from CAHPS demonstrations and evaluations.Consumer Assessment of Health Plans Study. Med Care1999;37:MS97105.

67 Zaslavsky AM, Landon BE, Beaulieu ND, e t al . Howconsumer assessments of managed care vary within andamong markets. Inquiry 2000;37:14661.

68 Jencks SF, Cuerdon T, Burwen DR, et al. Quality of medicalcare delivered to Medicare beneficiaries: a profile at stateand national levels. JAMA 2000;284:16706.



96qs

Documents

Transcript of 96qs