Influence of educational, audit and feedback, system based, and incentive and penalty interventions...

Clin Chem Lab Med 2015; 53(2): 157–183

Review

Daniel M. Kobewka*, Paul E. Ronksley, Jennifer A. McKay, Alan J. Forster and Carl van Walraven

Influence of educational, audit and feedback, system based, and incentive and penalty interventions to reduce laboratory test utilization: a systematic review

DOI 10.1515/cclm-2014-0778Received July 29, 2014; accepted August 25, 2014; previously published online September 27, 2014

Abstract: Laboratory and radiographic tests are often ordered unnecessarily. This excess testing has financial costs and is a burden on patients. We performed a systematic review to determine the effectiveness interventions to reduce test utilization by physicians. The MEDLINE and EMBASE data-bases were searched for the years 1946 through to September 2013 for English articles that had themes of test utilization and cost containment or optimization. Bibliographies of included papers were scanned to identify other potentially

relevant studies. Our search resulted in 3236 articles of which 109 met the inclusion criteria of having an interven-tion aimed at reducing test utilization with results that could be expressed as a percent reduction in test use relative to the comparator. Each intervention was categorized into one or more non-exclusive category of education, audit and feed-back, system based, or incentive or penalty. A rating of study quality was also performed. The percent reductions in test use ranged from a 99.7% reduction to a 27.7% increase in test use. Each category of intervention was effective in reducing test utilization. Heterogeneity between interventions, poor study quality, and limited time horizons makes generaliza-tions difficult and calls into question the validity of results. Very few studies measure any patient safety or quality of care outcomes affected by reduced test use. There are numerous studies that use low investment strategies to reduce test uti-lization with one time changes in the ordering system. These low investment strategies are the most promising for achiev-able and durable reductions in inappropriate test use.

Keywords: audit and feedback; education; laboratory uti-lization; reducing; test use.

IntroductionHealthcare spending in has increased in both relative and absolute terms over the past several decades [1, 2]. This increase has been seen in many countries and will be unsustainable if the trend continues at the present rate [3]. Laboratory and radiographic testing is a promising target for reducing spending because many of the tests ordered by physicians are suspected to be unnecessary; in some studies, 95% of tests performed are inappropriate as judged by criteria of redundancy or their probability of adding value to patient care [4].

*Corresponding author: Dr. Daniel M. Kobewka, Departments of Medicine and Epidemiology and Community Medicine, Ottawa Hospital, Civic Campus, 1053 Carling Avenue, Ottawa, ON K1Y 4E9, Canada, E-mail: [email protected]; and Department of Medicine, The Ottawa Hospital, University of Ottawa, Ottawa, Ontario, Canada; and Department of Epidemiology and Community Medicine, University of Ottawa, Ottawa, Ontario, CanadaPaul E. Ronksley: Ottawa Hospital Research Institute, Department of Clinical Epidemiology, Ottawa, Ontario, CanadaJennifer A. McKay: Department of Epidemiology and Community Medicine, University of Ottawa, Ottawa, Ontario, CanadaAlan J. Forster: Department of Medicine, The Ottawa Hospital, University of Ottawa, Ottawa, Ontario, Canada; Department of Epidemiology and Community Medicine, University of Ottawa, Ottawa, Ontario, Canada; Ottawa Hospital Research Institute, Department of Clinical Epidemiology, Ottawa, Ontario, Canada; Performance Measurement, The Ottawa Hospital, Ottawa, Ontario, Canada; and Institute for Clinical Evaluative Sciences, Toronto, Ontario, CanadaCarl van Walraven: Department of Medicine, The Ottawa Hospital, University of Ottawa, Ottawa, Ontario, Canada; Department of Epidemiology and Community Medicine, University of Ottawa, Ottawa, Ontario, Canada; Ottawa Hospital Research Institute, Department of Clinical Epidemiology, Ottawa, Ontario, Canada; and Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada

UnauthenticatedDownload Date | 5/19/16 2:24 AM

158 Kobewka et al.: Systematic review of interventions to reduce test use

The perceived prevalence of inappropriate laboratory utilization has prompted many attempts to reduce test utilization. Several systematic and narrative reviews have been published on this topic [5–8]. The most recent review, published in 1998, used a behavioral framework to clas-sify interventions and found that targeting multiple behav-ioral factors was more successful than targeting a single factor [8]. However, this review only dichotomized study outcomes by statistical significance, making it difficult to understand the utility of these interventions. Other reviews used categories of education, audit and feedback or other but were unable to make generalizations about which strategy is most effective because of differences within each category of intervention, a wide range of effects, and lack of a common measure [5, 6]. To date, no review has quantitatively compared the influence of various interven-tions on test utilization reduction. In addition, previous reviews have not examined investments required to imple-ment these interventions. For these reasons, we performed a systematic review to determine the effectiveness of all interventions to reduce test utilization by physicians.

Methods

Data sources and searches

We performed a systematic review following a predeter-mined protocol in accordance with published guidelines for reporting of systematic reviews of randomized con-trolled trials [9]. We identified all potentially relevant articles published in English by searching Medline (1946 through September 2013) and Embase (1947 through Sep-tember 2013). Searches were enhanced by systematically scanning bibliographies of identified articles and relevant review articles as well as articles deemed by PubMed to be related to the included studies. To search electronic databases, we used the strategy recommended for sys-tematic reviews of interventional studies [10] and speci-fied two comprehensive search themes (Online-Appendix 1). Theme 1 identified relevant terms related to laboratory utilization. Theme 2 related to optimization and cost con-tainment. We then combined the two datasets using the Boolean operator ‘and’ and limited the intersection to human studies.

Study selection

Relevant articles were selected from the retrieved studies using a two-phase process. Abstracts were first reviewed

for eligibility by three researchers (DK, PR, JM). All abstracts reporting on the effect of an intervention on laboratory utilization were selected for full text review. This initial stage was intentionally liberal; we discarded only abstracts that clearly were not reporting inventions aimed at reducing test utilization. Full text articles were then assessed by one reviewer (DK) with verification by two reviewers (PR, JM) to determine if the study met the specified inclusion criteria. Inclusion criteria included: 1) study population had to include physicians; 2) the aim of the intervention was the modification of test utilization; 3) a comparator arm (either standard care or no intervention) was needed; and 4) the study had to quantify laboratory test utilization with and without the intervention so that a percent change could be calculated. All study designs were considered within this review.

Intervention classification

Interventions used to effect physician laboratory order-ing practices were categorized into one or more of the following non-exclusive categories:1. Educational interventions in which appropriate test

ordering (including the distribution of guidelines) was taught to physicians;

2. Audit and feedback interventions in which physicians were presented their test utilization compared to their previous utilization or peer utilization or the total costs of the tests they ordered;

3. System-based interventions involving one-time, permanent changes to test ordering processes including: order form modifications; computer order entry systems with rules disallowing test ordering in specific circumstances; and clinical decision support systems (CDSS) in which an interactive computer system forces physicians to integrate previous knowledge about the patient and/or the medical literature into the test ordering process; and

4. Incentive or penalty interventions in which physicians received rewards or punishments for certain test ordering practices. For each intervention, we also determined whether the targeted physicians were actively engaged during the intervention’s development [8].

Data extraction and quality assessment

For each study, we extracted information on publication year, country of origin, study design (randomized two-arm


Kobewka et al.: Systematic review of interventions to reduce test use 159

trials, before and after studies, prospective cohort), the healthcare providers targeted and the tests targeted for reduction. We recorded characteristics of the study intervention(s) along with its duration. We assessed study quality using an adapted version of the Effective Practice and Organization of Care (EPOC) Cochrane guidelines for interventional studies [11]. In particular, we assessed whether: patients and providers were similar across inter-vention and comparator groups; a randomized control group was used within the study, there was sufficient detail to describe the intervention, there was risk of contamina-tion between the intervention and comparator groups, and whether a time-series analysis was performed. Lastly, we assessed whether studies measured any patient safety outcome that could be affected by reducing test utilization.

Statistical analysis

The unit of analysis was the intervention, with some studies having more than one intervention. The effective-ness of interventions was summarized using the percent relative reduction in test volume for the intervention group relative to the comparator group. This was calculated as:

−(# of tests in comparator group) (# of tests in interventiongroup)(# of tests in comparator group)

Given the variability in the types of interventions, duration of interventions, targeted tests, and patient populations, we did not conduct a meta-analysis. Instead, a descriptive analysis of the relative reductions was per-formed using medians and inter-quartile ranges (IQR) to summarize effects across the various intervention types. We also documented when these reductions in a particu-lar study were statistically significant. Results were strati-fied by categories of interventions (educational, system based, audit/feedback and incentive and penalty) and were visualized using box plots. Stratifications were then conducted by additional interventional factors within each category and by measures of study quality and sta-tistical significance. The effect of physician involvement was also measured across interventions. Finally sensitiv-ity analyses were conducted to assess the effect of inter-ventions that targeted four or more tests.

In order to highlight the interventions with the best return on investment we included a narrative review of interventions with the largest reductions in test utilization as well as the most effective system-based interventions tar-geting four or more tests. All analyses were performed using Stata version 13.0 (Stata Corp., College Station, TX, USA).

ResultsOur search generated 3236 unique citations. Following the initial screening and full-text review, 109 unique studies that included 119 interventions were included in the sys-tematic review (Figure 1) [12–121]. Study characteristics and brief descriptions of each intervention are shown in Table 1. Ninety-three studies (85.3%) used before-after designs, 2 (1.8%) were non-randomized prospective trials, and 14 (12.8%) were randomized controlled trials. Studies were published between 1974 and 2013 with 61 studies (56.0%) performed in North America, 27 (24.8%) in Europe, 11 (10.1%) Australasia, and 10 (9.2%) in other regions. Studies used very limited time horizons with only 15 of 109 studies (13.8%) measuring intervention effects beyond 1 year.

Study quality

Study quality is summarized in Online-Appendix 2. Of the 109 studies, 14 (12.8%) had a randomized design and 37 (33.9%) provided evidence that patients were similar between groups. Time-series analysis was performed in only 11 (10.1%) studies. Interventions were described

adequately to be reproducible in 88 (80.7%) studies. Test utilization was reported with the patient or the encoun-ter as the unit of analysis in 79/109 (72.5%) studies with the rest using the physician or the group of physicians working that location as the unit of analysis.

Intervention components

Of 119 interventions, 51 (42.9%) had an educational com-ponent, 54 (45.4%) had a system-based component, 41 (34.5%) had an audit feedback component, with only one (0.8%) using incentive or penalty. Physicians were involved in the development of interventions in only 17 (14.3%) interventions. Thirty-two (26.9%) interventions were multifaceted, being classified into more than one of the four main intervention categories. A measure of patient safety or quality of care was included in 13 (11.9%) studies.

Intervention effects

The percent relative reductions in laboratory utilization ranged from a +99.7% (a reduction in the test volumes)



3236 citations (Medline: 1554; EMBASE: 1633) identified fromelectronic searches bibliography searching

272 studies that passed initialscreen and were retrieved forfurther scrutiny

109 studies includedin systematic review

49 relevant articles identifiedthrough bibliographic searchof included papers

163 studies excluded:- No intervention to reduce utilization- Did not report on outcomes of interest

2964 citations excluded (Studies did notreport on interventions related tolaboratory utilization, or did not containoriginal data)

Figure 1 Selection of articles in systematic review. The search was performed in September of 2013.

to –27.7% (an increase in test volume). Table 2 dis-plays the percent relative reductions in test volume for each intervention and their sub-categories. Interven-tions with an educational component had the highest median relative reduction in test volume at 34.5% (IQR 16.5–49.0) while audit feedback or system-based inter-ventions had a relative reduction of 22.0% (8.6–34.6) and 22.2% (3.6–68.3), respectively (Figure 2). The single incentive and penalty intervention had a 5.8% relative reduction.

Thirty-three (27.7%) of the interventions targeted three or fewer tests, while 86 (72.3%) targeted four or more tests. The median relative reduction for interventions that tar-geted three or fewer tests was 40.2% (IQR 23.7–52.8) while that targeting four or more tests was 18.8% (IQR 8.2–28.5) (Figure 3). Thirty interventions (25.2%) used multiple interventions; these multifaceted interventions had larger reductions in test use with a median relative reduc-tion in test volume of 32.7% (IQR 15.1–47.5) versus 21.4% (IQR 9.5–33.3) for interventions that were classified into a single category. Interventions for which physicians were engaged during the intervention development had similar median relative reductions as interventions without phy-sician input.

Finally, we found important differences in the rela-tive reductions across quality indicators (Table 3). Most notably, the effect of interventions was less if a study reported a reduction in test utilization per patient rather than reduction per institution or physician group and studies with a concurrent control group had smaller reductions. Unexpectedly there were larger relative reduc-tions in studies that had a risk of contamination between the experimental and control groups.

Interventions with the largest effects

We found extensive heterogeneity in the components that made up each intervention, how interventions were imple-mented, the study setting, and the tests that were targeted for reduction. Meaningful generalizations across interven-tion types is therefore difficult; so the interventions with the largest relative reductions in test volume are described here (Table 1). The studies by Froom et al. and Lewand-rowski et al. attained the largest relative reductions with both targeting a single test. Froom et al. changed labo-ratory policy so that urine microscopy was done only if specifically requested by the physician; this resulted in



Tabl

e 1

Stud

y ch

arac

teris

tics

and

desc

riptio

n of

inte

rven

tions

.

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Amuk

ele

2011

US

A

Phys

icia

ns

wor

king

in

surg

ery,

he

mat

olog

y, a

nd

inte

rnal

med

icin

e

Si

ngle

te

achi

ng

hosp

ital

Th

e or

der f

orm

was

mod

ified

with

som

e te

sts

bein

g bu

ndle

d an

d ot

hers

bei

ng

unbu

ndle

d. P

hysi

cian

s w

ho w

ere

high

us

ers

wer

e ed

ucat

ed a

bout

coag

ulat

ion

test

s an

d an

alg

orith

m w

as d

istri

bute

d

Be

fore

-afte

r

X

X

PT, P

TT, T

T an

d fib

rinog

en

77.8

Arch

amba

ult

2011

US

A

All p

hysi

cian

s

Sing

le

teac

hing

ho

spita

l

ES

R te

stin

g w

as n

o lo

nger

per

form

ed

Befo

re-a

fter

X

ESR

and

CRP

47

.5

Atta

li 20

06

Isra

el

Resi

dent

ph

ysic

ians

Si

ngle

te

achi

ng

hosp

ital

Ed

ucat

iona

l lec

ture

abo

ut e

xces

sive

an

d in

appr

opria

te te

stin

g; te

sts

wer

e un

bund

led

on th

e re

ques

t for

m; a

sen

ior

phys

icia

n ga

ve fe

edba

ck a

bout

ord

erin

g pr

actic

es

Be

fore

-afte

r

X

X

X

Al

l blo

od te

sts

34

.5a

Bara

zzon

i 200

2

Switz

erla

nd

All p

hysi

cian

s se

eing

pat

ient

s pr

e-op

erat

ivel

y fo

r ele

ctiv

e pr

oced

ures

6

com

mun

ity

hosp

itals

w

ith s

urge

ry

faci

litie

s

Al

l sta

keho

lder

s w

ere

invo

lved

in

a gu

idel

ine

crea

tion

proc

ess.

The

guid

elin

es w

ere

then

ado

pted

as

hosp

ital

polic

y

Be

fore

-afte

r

X

Al

l pre

-ope

rativ

e te

sts

26

.1a

Bare

ford

199

0

USA

Al

l med

ical

sta

ff:

cons

ulta

nts

and

juni

or h

ouse

sta

ff

Si

ngle

co

mm

unity

ho

spita

l

M

onth

ly u

sage

sta

tem

ents

wer

e di

strib

uted

to p

hysi

cian

s. G

uide

lines

on

app

ropr

iate

test

ing

wer

e di

strib

uted

an

d so

me

test

s w

ere

canc

eled

if

inap

prop

riate

Be

fore

-afte

r

X

X

X

CB

C, E

SR a

nd P

T

62.1

Baric

chi 2

012

Ita

ly

Gene

ral

prac

titio

ners

4

prim

ary

care

clin

ics

Te

stin

g al

gorit

hms

for 7

com

mon

clin

ical

sc

enar

ios

wer

e de

velo

ped.

Phy

sici

ans

wer

e th

en e

duca

ted

abou

t the

alg

orith

ms

and

told

to u

se th

em

Be

fore

-afte

r

X

X

All b

lood

test

s

4.7

Barie

199

5

USA

Al

l med

ical

sta

ff in

th

e IC

U

Sing

le

teac

hing

ho

spita

l

Te

stin

g pa

thw

ays

wer

e in

trodu

ced

and

audi

t and

feed

back

was

don

e m

onth

ly

Befo

re-a

fter

X

X

Al

l blo

od te

sts

and

ches

t ra

diog

raph

s

34.6

Bate

s 19

99

USA

Al

l phy

sici

ans

Si

ngle

te

achi

ng

hosp

ital

De

finiti

ons

of a

redu

ndan

t tes

t wer

e cr

eate

d. Th

e co

mpu

ter o

rder

ent

ry

syst

em cr

eate

d re

min

ders

whe

n te

sts

wer

e or

dere

d th

at w

ere

redu

ndan

t

Ra

ndom

ized

tri

al

X

El

ectro

lyte

s, u

rinal

ysis

, di

goxi

n, to

bram

ycin

, am

inop

hylin

e, va

ncom

ycin

, an

d ge

ntam

ycin

leve

ls,

urin

e, sp

utum

and

stoo

l cu

lture

s, C

lost

ridiu

m d

iffic

ile

test

ing,

fibr

in sp

lit p

rodu

cts

0.

1



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Bate

s 19

97

USA

Al

l phy

sici

ans

Si

ngle

te

achi

ng

hosp

ital

Co

sts

wer

e di

spla

yed

for e

ach

test

that

w

as o

rder

ed.

Be

fore

-afte

r

X

Al

l blo

od te

sts

and

the

35

mos

t com

mon

radi

olog

ic

test

s

4.

1

Bere

nhol

tz 2

001

US

A

Phys

icia

ns

wor

king

in a

su

rgic

al IC

U

Si

ngle

te

achi

ng

hosp

ital

Cl

inic

al p

athw

ays

wer

e cr

eate

d fo

r va

rious

sur

gica

l pro

blem

s. A

ppro

pria

te

labo

rato

ry te

stin

g w

as s

ugge

sted

by

the

path

way

Be

fore

-afte

r

X

Al

l lab

orat

orie

s in

clud

ed in

th

e pa

thw

ay

–6.8

Berw

ick

1986

US

A

35 in

tern

ists

Si

ngle

hea

lth

man

agem

ent

orga

niza

tion

Do

ctor

s m

et to

dis

cuss

the

use

of

part

icul

ar te

sts

and

jour

nal a

rtic

les

wer

e di

strib

uted

on

the

use

of th

ese

test

sCl

inic

ians

rece

ived

indi

vidu

aliz

ed

feed

back

on

thei

r rat

es o

f tes

t ord

erin

g co

mpa

red

to th

eir p

eers

and

thei

r ran

k re

lativ

e to

thei

r pee

rsPh

ysic

ians

wer

e ra

nked

by

the

prop

ortio

n of

test

s th

ey o

rder

ed th

at

wer

e ab

norm

al. E

ach

phys

icia

n w

as th

en

give

n th

eir r

ank

Be

fore

-afte

r

Befo

re-a

fter

Befo

re-a

fter

X X X

ESR,

T4, s

erum

glu

cose

, he

tero

phile

ant

ibod

y te

stin

g, A

NA, S

LE

prep

arat

ion,

RF,

liver

en

zym

es, c

hole

ster

ol

leve

ls, r

adio

grap

hs o

f pe

riphe

ral b

ones

and

ch

est,

ECGs

0.

2

14.2

a

–4.1

Boon

-Fal

leur

19

95

Belg

ium

Ph

ysic

ians

w

orki

ng o

n liv

er a

tra

nspl

ant u

nit

Pe

diat

ric li

ver

unit

A

rule

bas

ed d

ecis

ion

supp

ort

appl

icat

ion

deci

ded

on th

e ne

xt d

ay’s

la

bora

tory

test

s ba

sed

on p

revi

ous

resu

lts

Be

fore

-afte

r

X

Al

l blo

od te

sts

24

.9

Buck

ingh

am

1994

Sc

otla

nd

All p

hysi

cian

s

All h

ospi

tals

in

a s

ingl

e ci

ty

Ju

stifi

catio

n fo

r tes

ts o

rder

ed h

ad to

be

give

n. S

peci

fics

on h

ow th

is w

as d

one

are

not g

iven

Be

fore

-afte

r

X

Al

l bio

chem

ical

blo

od te

sts

1.

4

Bunt

ing

2004

Ca

nada

20

0 ph

ysic

ians

w

ho o

rder

ed th

e m

ost t

ests

in 1

997

Si

ngle

pr

ovin

ce

Feed

back

on

test

usa

ge w

as g

iven

four

tim

es o

ver a

2-y

ear p

erio

d

Rand

omiz

ed

cont

rolle

d tri

al

X

All b

lood

test

s

7.8a

Burn

ett 1

991

Au

stra

lia

All p

hysi

cian

s re

ques

ting

bloo

d w

ork

from

a s

ingl

e m

edic

al la

bora

tory

Si

ngle

priv

ate

med

ical

la

bora

tory

Al

l lab

orat

ory

test

s w

ere

unbu

ndle

d so

th

ey h

ad to

be

orde

red

indi

vidu

ally

Be

fore

-afte

r

X

Al

l blo

od te

sts

18

.8a

Cald

eron

-M

arga

lit 2

005

Is

rael

Al

l phy

sici

ans

Si

ngle

te

achi

ng

hosp

ital

Se

vera

l int

erve

ntio

ns in

clud

ing

man

dato

ry co

nsul

tatio

n pr

ior t

o or

derin

g,

educ

atio

n, re

stric

tion

or e

limin

atio

n of

va

rious

test

s

Be

fore

-afte

r

X

X

All b

lood

test

s

19.0

a



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Cars

on 1

995

US

A

All p

hysi

cian

s

Sing

le

teac

hing

ho

spita

l

An

alg

orith

m th

at a

utom

atic

ally

re

stric

ted

fibrin

deg

rada

tion

prod

uct

test

ing

and

inst

ead

used

D-d

imer

test

ing

for s

uspe

cted

DIC

Be

fore

-afte

r

X

D-

Dim

er a

nd fi

brin

de

grad

atio

n pr

oduc

ts

48.3

a

Cart

er 2

002

US

A

Phys

icia

ns

send

ing

spec

imen

s to

the

lab

Si

ngle

te

achi

ng

hosp

ital

Al

l tes

t req

uest

s w

ere

revi

ewed

by

a m

ore

seni

or p

hysi

cian

Be

fore

-afte

r

X

Al

l lab

orat

orie

s th

at

cost

> $7

5 US

D

23.0

Chen

200

3

USA

Al

l Phy

sici

ans

orde

ring

sele

cted

te

sts

Si

ngle

te

achi

ng

hosp

ital

A

rem

inde

r on

the

half-

life

of v

ario

us

anti-

epile

ptic

dru

gs w

as s

how

n on

th

e co

mpu

ter w

hen

repe

at le

vels

wer

e re

ques

ted

in a

sho

rt tim

e pe

riod

Be

fore

-afte

r

X

X

Anti

epile

ptic

dru

g le

vels

23

.4a

Chu

2012

Au

stra

lia

Med

ical

stu

dent

s an

d re

side

nts

Si

ngle

te

achi

ng

hosp

ital

Se

lect

ed te

sts

wer

e re

stric

ted

and

requ

ired

a se

nior

sta

ff m

embe

r to

sign

off

on th

em p

rior t

o be

ing

orde

red

Be

fore

-afte

r

X

Co

agul

atio

n te

sts,

thyr

oid

func

tion

test

s, E

SR a

nd

d-di

mer

7.

5a

Chu

1996

US

A

Phys

icia

ns ta

king

ca

re o

f tra

uma

patie

nts

Si

ngle

te

achi

ng

hosp

ital

A

liber

al a

dmis

sion

alg

orith

m w

ith

exte

nsiv

e te

stin

g w

as co

mpa

red

to

phys

icia

ns o

nly

orde

ring

test

s th

ey

thou

ght n

eces

sary

Be

fore

-afte

r

X

He

mat

olog

y, ch

emis

try

and

coag

ulat

ion

prof

ile,

urin

alys

is, b

lood

gas

, blo

od

alco

hol l

evel

, typ

e an

d sc

reen

68

.3

Cohe

n 19

82

USA

Al

l phy

sici

ans

wor

king

on

the

teac

hing

uni

ts

4

clin

ical

te

achi

ng

units

at

a si

ngle

te

achi

ng

hosp

ital

Th

e co

st o

f all

imag

ing

test

s or

dere

d w

ere

give

n to

eac

h te

am w

eekl

y

The

cost

of a

ll bl

ood

test

s or

dere

d w

ere

give

n to

eac

h te

am w

eekl

y

Ra

ndom

ized

co

ntro

lled

trial

Rand

omiz

ed

cont

rolle

d tri

al

X X

CXR,

CT

scan

s, U

S an

d liv

er

scan

sAl

l blo

od te

sts

12

.9

24.7

a

Da

vido

ff 19

89

USA

24

med

ical

in

tern

s en

terin

g a

univ

ersi

ty tr

aini

ng

prog

ram

Si

ngle

te

achi

ng

hosp

ital

Le

ctur

es o

n pr

obab

ility

and

test

ch

arac

teris

tics

wer

e co

mpa

red

to

plac

ebo

lect

ures

on

econ

omic

s an

d co

st

cont

rol

Ra

ndom

ized

co

ntro

lled

trial

X

All b

lood

test

s

16.5

a

Dets

ky 1

986

Ca

nada

Al

l atte

ndin

g ph

ysic

ians

20

teac

hing

ho

spita

ls

in a

sin

gle

prov

ince

Re

side

nts

and

inte

rns

wen

t on

strik

e fo

r 7

days

in 1

980

Be

fore

-afte

r

X

Al

l blo

od te

sts

8.

3a



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Dick

inso

n 19

87

USA

Al

l phy

sici

ans

in

the

inst

itutio

n

Sing

le

teac

hing

ho

spita

l

Th

e nu

mbe

r of u

nnec

essa

ry te

sts

was

m

easu

red

and

show

n to

dep

artm

ent

head

s

Be

fore

-afte

r

X

Al

l blo

od te

sts

24

.5a

Dixo

n 19

74

USA

In

tern

s w

orki

ng o

n th

e w

ards

Si

ngle

te

achi

ng

hosp

ital

In

tern

s an

d re

side

nts

wer

e lim

ited

to 8

te

sts

each

Be

fore

-afte

r

X

Al

l blo

od ch

emis

try te

sts

66

.7a

Dow

ling

1989

US

A

Resi

dent

s in

a

fam

ily h

ealth

ce

nter

Si

ngle

fam

ily

heal

th ce

nter

Le

ctur

es a

nd g

uide

lines

on

appr

opria

te

test

ing

wer

e di

strib

uted

. Aud

it an

d fe

edba

ck o

f tes

t usa

ge w

as p

erfo

rmed

re

peat

edly

Be

fore

-afte

r

X

X

CBC

and

TSH

47

.5a

Dura

nd-Z

ales

ki

1993

Fr

ance

Al

l phy

sici

ans

at

the

hosp

ital

Si

ngle

te

achi

ng

hosp

ital

Th

e la

bora

tory

requ

isiti

on w

as ch

ange

d so

that

the

purp

ose

of e

ach

tum

or m

arke

r w

as in

clud

ed o

n th

e re

quis

ition

Be

fore

-afte

r

X

Tu

mor

mar

kers

23

.9a

Durie

ux 2

003

Fr

ance

Al

l phy

sici

ans

at

the

hosp

ital

Si

ngle

te

achi

ng

hosp

ital

Th

e la

bora

tory

requ

isiti

on w

as ch

ange

d so

that

the

purp

ose

of e

ach

tum

or m

arke

r w

as in

clud

ed o

n th

e re

quis

ition

. The

pu

rpos

e fo

r eac

h te

st, i

.e.,

scre

enin

g,

follo

w-u

p or

dia

gnos

is w

as e

xpla

ined

on

the

requ

isiti

on

Be

fore

-afte

r

X

X

CEA,

alp

ha fe

to-p

rote

in,

CA19

-9

56.5

a

Eccl

es 2

001

En

glan

d an

d Sc

otla

nd

Gene

ral

Prac

titio

ners

M

ultip

le

prim

ary

care

cl

inic

s

Gu

idel

ines

on

whe

n to

ord

er lu

mba

r sp

ine

and

knee

X-ra

ys w

ere

circ

ulat

ed

alon

g w

ith fe

edba

ck o

n th

e nu

mbe

r of

requ

ests

to e

ach

phys

icia

n

Ra

ndom

ized

co

ntro

lled

trial

X

X

Kn

ee a

nd lu

mba

r spi

ne X

R

40.8

Eise

nber

g 19

77

USA

Re

side

nt

phys

icia

ns

Sing

le

com

mun

ity

hosp

ital

Th

ere

was

an

educ

atio

n pr

ogra

m w

ith

a w

eekl

y le

ctur

e; re

sults

from

a s

ingl

e au

dit o

n ut

ility

of t

estin

g w

ere

circ

ulat

ed

alon

g w

ith a

mem

oran

dum

from

the

med

ical

dire

ctor

ask

ing

for c

aref

ul

cons

ider

atio

n of

test

usa

ge

Be

fore

-afte

r

X

PT

T

42.1

a

Emer

son

2001

US

A

All p

hysi

cian

s w

orki

ng a

t the

m

edic

al ce

nter

Si

ngle

te

achi

ng

hosp

ital

La

bora

tory

requ

isiti

on w

as re

desi

gned

w

ith s

ome

test

s un

bund

led

and

casc

ades

im

plem

ente

d fo

r thy

roid

func

tion

and

anem

ia

Be

fore

-afte

r

X

Al

l blo

od te

sts

4.

7a

Ever

ett 1

985

US

A

Firs

t yea

r res

iden

ts

Sing

le

teac

hing

ho

spita

l

Re

side

nts

wer

e ta

ught

abo

ut la

bora

tory

ut

iliza

tion

usin

g sp

ecifi

c cas

es

Rand

omiz

ed

cont

rolle

d tri

al

X

All b

lood

test

s an

d m

icro

biol

ogy

–1

2.3



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Ever

ett 1

983

US

A

Firs

t yea

r res

iden

ts

train

ing

in in

tern

al

med

icin

e

Si

ngle

te

achi

ng

hosp

ital

Au

dit a

nd fe

edba

ck o

f cos

ts w

as

perfo

rmed

wee

kly,

cost

edu

catio

n ne

ws

lette

rs w

ere

dist

ribut

ed a

nd co

sts

of

test

s w

ere

put i

n pa

tient

char

ts. T

here

w

ere

one-

on-o

ne m

eetin

gs o

f fac

ulty

with

re

side

nts

to d

iscu

ss co

sts

Ra

ndom

ized

co

ntro

lled

trial

X

X

Al

l blo

od te

sts

15

.1a

Feld

kam

p 19

96

USA

Al

l phy

sici

ans

in

heal

th ce

nter

Si

ngle

te

achi

ng

hosp

ital a

nd

22 s

atel

lite

clin

ics

Al

gorit

hms

wer

e de

velo

ped

for t

hyro

id

func

tion

test

ing

and

impl

emen

ted

by th

e la

bora

tory

Be

fore

-afte

r

X

Th

yroi

d fu

nctio

n te

stin

g

21.6

Feld

man

201

3

USA

Al

l pro

vide

rs w

ho

orde

r lab

orat

ory

test

s th

roug

h a

com

pute

rized

or

der e

ntry

sys

tem

Si

ngle

te

achi

ng

hosp

ital

Co

st o

f eac

h te

st w

as d

ispl

ayed

whe

n it

was

ord

ered

Ra

ndom

ized

co

ntro

lled

trial

X

X

61

dia

gnos

tic b

lood

test

s

8.6a

Fine

gan

2005

Ca

nada

Al

l phy

sici

an

train

ees

in a

ho

spita

l

Si

ngle

te

achi

ng

hosp

ital

Ea

ch a

nest

hetis

t sel

ecte

d te

sts

to

be p

erfo

rmed

pre

oper

ativ

ely

at th

eir

disc

retio

n. Th

e co

mpa

rato

r was

sur

gery

sp

ecifi

c pro

toco

ls th

at w

ere

bein

g us

ed

Be

fore

-afte

r

All b

lood

test

s

3.6

Fong

200

8

Aust

ralia

Al

l phy

sici

ans

wor

king

in th

e em

erge

ncy

room

Si

ngle

te

achi

ng

hosp

ital

A

clin

ical

dec

isio

n ru

le w

as u

sed

inst

ead

of p

hysi

cian

dis

cret

ion

Be

fore

-afte

r

X

CT

hea

d

–27.

7a

Fow

kes

1986

Un

ited

King

dom

Al

l res

iden

ts

wor

king

on

the

med

ical

uni

t

Si

ngle

te

achi

ng

hosp

ital

A

guid

elin

e on

test

ing

was

dis

tribu

ted

and

wee

kly

mee

tings

wer

e he

ld w

ith

med

ical

sta

ff to

dis

cuss

thei

r use

of t

ests

in

the

prev

ious

wee

k

No

n-ra

ndom

ized

co

ntro

lled

trial

X

X

Al

l blo

od te

sts

63

.4a

Frie

dman

201

0

USA

Ph

ysic

ians

taki

ng

care

of c

hild

ren

post

pro

cedu

re

Si

ngle

te

achi

ng

hosp

ital

A

set o

f sta

ndar

d as

sess

men

t and

m

anag

emen

t pla

ns w

ere

deve

lope

d an

d im

plem

ente

d af

ter c

erta

in p

roce

dure

s

Be

fore

-afte

r

X

Al

l blo

od te

sts

and

imag

ing

15

.6

Froo

m 2

012

Is

rael

Al

l phy

sici

ans

Re

gion

al

labo

rato

ry

serv

ing

a si

ngle

Hea

lth

Man

agem

ent

Orga

niza

tion

Ur

ine

mic

rosc

opy

was

no

long

er d

one

auto

mat

ical

ly in

stea

d th

e ph

ysic

ian

had

to s

peci

fical

ly re

ques

t it

Be

fore

-afte

r

X

Ur

ine

for d

ip a

nd

mic

rosc

opy

99

.7a



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Gam

a 19

91

Unite

d Ki

ngdo

m

Gene

ral i

nter

nist

s

Sing

le

com

mun

ity

hosp

ital

Da

ta o

n pe

rson

al a

nd p

eer e

xpen

ditu

re

of la

bora

tory

reso

urce

s w

as g

iven

to

phys

icia

ns m

onth

ly fo

r 12

mon

ths

Ra

ndom

ized

co

ntro

lled

trial

X

Clin

ical

chem

istry

24

.6a

Gam

a 19

92

Unite

d Ki

ngdo

m

Phys

icia

ns ta

king

ca

re o

f inp

atie

nts

Si

ngle

co

mm

unity

ho

spita

l

Da

ta o

n pe

rson

al a

nd p

eer c

linic

al

chem

istry

exp

endi

ture

per

pat

ient

was

pr

ovid

ed to

phy

sici

ans

mon

thly

for 1

2 m

onth

s

Be

fore

-afte

r

X

Cl

inic

al ch

emis

try a

nd

hem

atol

ogy

27

.0a

Godd

ard

2011

Ire

land

Al

l phy

sici

ans

wor

king

in th

e IC

U

Sing

le

teac

hing

ho

spita

l

Al

l rec

urrin

g or

ders

for b

lood

wor

k w

ere

disc

ontin

ued;

inst

ead

each

test

had

to b

e or

dere

d as

a s

ingl

e oc

curr

ence

Be

fore

-afte

r

CBC,

ure

a, e

lect

roly

tes,

co

agul

atio

n, L

FTs,

m

agne

sium

, Ca,

PO4

, al

bum

in, C

RP

33

.1

Gold

en 1

987

US

A

All p

hysi

cian

s

Outp

atie

nt

depa

rtmen

t of

a s

ingl

e te

achi

ng

hosp

ital

Th

e co

st o

f tes

ts b

illed

to th

e pa

tient

was

ch

ange

d so

that

ord

erin

g in

divi

dual

test

s w

as ch

eape

r tha

n or

derin

g th

e w

hole

pa

nel

Be

fore

-afte

r

X

Al

l blo

od ch

emis

try te

st

33.5

a

Gortm

aker

198

8

USA

Al

l phy

sici

ans

wor

king

in th

e ho

spita

l

Si

ngle

co

mm

unity

ho

spita

l

Ni

ne s

taff

mee

tings

wer

e he

ld to

dis

cuss

co

st is

sues

. Dat

a on

exc

ess

usag

e w

as s

ent t

o al

l doc

tors

. All

phys

icia

ns

reac

hed

a co

nsen

sus

on w

hen

each

test

w

as co

nsid

ered

inap

prop

riate

Be

fore

-afte

r

X

BU

N, C

BC, c

hem

istry

pr

ofile

, CK,

crea

tinin

e,

elec

troly

tes,

fast

ing

and

rand

om b

lood

sug

ar,

hem

atoc

rit, P

T, S

GOT,

ur

inal

ysis

14

.0a

Griv

ell 1

981

Au

stra

lia

All p

hysi

cian

s ca

ring

for

inpa

tient

s

Si

ngle

te

achi

ng

hosp

ital

Ph

ysic

ians

rece

ived

a re

port

ever

y 4

wee

ks o

n th

e ty

pe a

nd n

umbe

r of t

ests

or

dere

d. P

hysi

cian

s w

ere

also

sho

wn

whe

re th

ey ra

nk in

com

paris

on to

thei

r pe

ers

Be

fore

-afte

r

X

Al

l blo

od te

sts

0.

0

Groo

pman

199

2

USA

Al

l med

ical

st

aff o

rder

ing

labo

rato

ries

in

the

ER

Si

ngle

te

achi

ng

hosp

ital

Te

sts

wer

e ta

ken

off t

he s

tand

ard

requ

isiti

on a

nd h

ad to

be

writ

ten

in b

y ha

nd

Be

fore

-afte

r

X

PT

/PTT

61

.6a

Gros

s 19

88

USA

Al

l phy

sici

ans

taki

ng ca

re o

f in

patie

nts

Si

ngle

te

achi

ng

hosp

ital

Gu

idel

ines

on

whe

n to

ord

er b

lood

cu

lture

s w

ere

dist

ribut

ed

Befo

re-a

fter

X

Bloo

d cu

lture

s

75.0

a



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Hasc

hke-

Bech

er

2009

Au

stria

No

doc

tors

wer

e ta

rget

ed, i

t was

a

labo

rato

ry

inte

rven

tion

Si

ngle

te

achi

ng

hosp

ital

A

deci

sion

rule

was

crea

ted

to d

eter

min

e w

hen

the

labo

rato

ry w

ould

pro

cess

liq

uor d

iagn

ostic

s

Be

fore

-afte

r

X

Li

quor

dia

gnos

tics

28

.6a

Hutto

n 20

09

Unite

d Ki

ngdo

m

All e

mer

genc

y ro

om a

nd m

edic

al

unit

phys

icia

ns

Si

ngle

te

achi

ng

hosp

ital

Re

ques

ts fo

r CRP

from

the

ER h

ad to

be

app

rove

d by

a s

enio

r phy

sici

an.

Juni

or s

taff

wer

e to

ld to

lim

it CR

P us

e.

Last

ly re

peat

test

ing

with

in 2

4 h

was

di

sallo

wed

Be

fore

-afte

r

X

X

CRP

85

.0a

Jelin

ek 1

990

Au

stra

lia

All p

hysi

cian

s

Sing

le

teac

hing

ho

spita

l

Gu

idel

ines

on

whe

n ab

dom

inal

film

s co

uld

be o

rder

ed w

ere

circ

ulat

ed a

nd

all r

eque

sts

had

to b

e ru

n by

a s

enio

r ph

ysic

ian

Be

fore

-afte

r

X

X

Abdo

min

al p

lain

film

s

52.2

Kelly

199

8

Aust

ralia

Al

l phy

sici

ans

in

the

ER

Sing

le

com

mun

ity

hosp

ital

Gu

idel

ines

on

who

sho

uld

rece

ive

a bl

ood

cultu

re w

ere

dist

ribut

ed

Befo

re-a

fter

X

Bloo

d cu

lture

s

53.0

Kum

wila

isak

20

08

USA

Al

l phy

sici

ans

wor

king

in th

e su

rgic

al IC

U

Si

ngle

te

achi

ng

hosp

ital

Gu

idel

ines

rega

rdin

g w

hen

certa

in b

lood

te

sts

shou

ld b

e us

ed w

ere

intro

duce

d at

a

staf

f mee

ting

and

sent

out

by

emai

l. A

sess

ion

was

repe

ated

for n

ew re

side

nts

ever

y m

onth

Be

fore

-afte

r

X

Al

l blo

od te

sts

20

.8a

Laro

cque

199

4

Cana

da

Surg

eons

and

su

rgic

al h

ouse

st

aff

Si

ngle

te

achi

ng

hosp

ital

Gu

idel

ines

on

whi

ch te

sts

wer

e ap

prop

riate

for d

iffer

ent m

edic

al

cond

ition

s w

ere

post

ed o

n th

e w

ards

and

di

strib

uted

as

pock

et ca

rds

Be

fore

-afte

r

X

Pr

e-op

scr

eeni

ng te

sts

incl

udin

g bl

ood

wor

k, C

XR

and

ECGs

10

.1a

Levi

ck 2

013

US

A

Clin

icia

ns o

rder

ing

the

BNP

test

A

heal

th

netw

ork

incl

udin

g 2

hosp

itals

an

d se

vera

l co

mm

unity

he

alth

ce

nter

s

Co

mpu

ter o

rder

ent

ry s

yste

m d

ispl

ayed

a

war

ning

whe

n a

repe

at B

NP w

as b

eing

or

dere

d

Be

fore

-afte

r

X

BN

P

22.2

a



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Lew

andr

owsk

i 19

94

USA

Al

l phy

sici

ans

wor

king

at t

he

hosp

ital

Si

ngle

te

achi

ng

hosp

ital

A

guid

elin

e on

app

ropr

iate

test

ing

was

dev

elop

ed. A

ll re

ques

ts w

ere

then

re

view

ed b

y la

bora

tory

sta

ff an

d th

e or

derin

g ph

ysic

ian

was

cont

acte

d if

the

requ

est w

as n

ot in

acc

orda

nce

with

the

guid

elin

e

Be

fore

-afte

r

X

X

LD-is

ozym

es

99.6

Man

cuso

199

9

USA

Al

l phy

sici

ans

at

the

hosp

ital

Si

ngle

sp

ecia

lized

or

thop

edic

su

rger

y ho

spita

l

Gu

idel

ines

wer

e pu

blis

hed

sugg

estin

g se

lect

ive

test

ord

erin

g pr

eope

rativ

ely

Be

fore

-afte

r

X

Pr

e-op

erat

ive

bloo

d w

ork,

EC

G, C

XR,

30

.0a

Mar

tin 1

980

US

A

Resi

dent

s an

d m

edic

al s

tude

nts

Si

ngle

te

achi

ng

hosp

ital

Th

ere

was

a 1

h le

ctur

e on

labo

rato

ry

cost

s. M

oney

was

giv

en to

the

resi

dent

s if

thei

r lab

orat

ory

use

decr

ease

d

No

n-ra

ndom

ized

co

ntro

lled

trial

X

X

Al

l rad

iolo

gic a

nd b

lood

te

sts

5.

8a

Ther

e w

as a

1-h

lect

ure

on la

bora

tory

co

sts.

Sen

ior p

hysi

cian

s th

en m

et w

eekl

y w

ith re

side

nts,

they

revi

ewed

case

s an

d su

gges

ted

chan

ges

in te

st o

rder

ing

prac

tices

.

X

X

–1

7.8a

Mar

ton

1985

US

A

Resi

dent

s an

d m

edic

al s

tude

nts

Si

ngle

te

achi

ng

hosp

ital

A

man

ual a

bout

ratio

nal t

est o

rder

ing

was

giv

en to

the

train

ees

Ra

ndom

ized

co

ntro

lled

trial

X

All b

lood

test

s

18.2

Feed

back

on

labo

rato

ry u

sage

and

cost

s in

curr

ed w

as s

how

n to

trai

nees

alo

ng

with

thei

r usa

ge co

mpa

red

with

pee

rs

X

33.3

A m

anua

l abo

ut ra

tiona

l tes

t ord

erin

g w

as g

iven

to th

e tra

inee

s. Th

ey a

lso

rece

ived

feed

back

on

labo

rato

ry u

sage

, co

sts

incu

rred

and

thei

r usa

ge co

mpa

red

with

pee

rs

X

X

40

.9a

May

200

6

USA

Al

l doc

s w

orki

ng

on in

patie

nt

serv

ice

Si

ngle

te

achi

ng

hosp

ital

Du

plic

ate

orde

rs w

ere

dele

ted

and

all

bloo

d w

ork

orde

rs e

xpire

d in

24

h. B

lood

w

ork

wou

ld o

nly

be d

raw

n ev

ery

4, 6

or

12 h

. Fut

ure

orde

rs co

uld

only

be

for a

si

ngle

occ

urre

nce

Be

fore

-afte

r

X

Al

l blo

od te

sts

11

.5a



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Meh

ari 1

997

Ne

w

Zeal

and

Ph

ysic

ians

w

orki

ng in

ICU

Si

ngle

te

achi

ng

hosp

ital

A

guid

elin

e w

as p

lace

d on

eac

h pa

tient

s ch

art s

tatin

g w

hen

it is

app

ropr

iate

to d

o ea

ch te

st

Be

fore

-afte

r

X

CB

C, co

agul

atio

n te

sts,

Cr

, Na,

K, u

rea,

calc

ium

, gl

ucos

e, M

g, P

O4, A

BGs

21

.2

Men

g 20

06

Cana

da

All p

hysi

cian

s

3 te

achi

ng

hosp

itals

A

polic

y w

as cr

eate

d re

gard

ing

who

is

allo

wed

to o

rder

trop

onin

test

ing

and

how

man

y re

peat

s ca

n be

ord

ered

Be

fore

-afte

r

X

Tn

I

2.9

Mer

lani

200

1

Switz

erla

nd

Doct

ors

resi

dent

s an

d nu

rses

w

orki

ng in

the

inte

nsiv

e ca

re u

nit

Sing

le

teac

hing

ho

spita

l

A

guid

elin

e on

whe

n ar

teria

l blo

od g

ases

ca

n be

don

e w

as ci

rcul

ated

Be

fore

-afte

r

X

X

ABG

41

.5a

Mey

er 2

010

US

A

Gyne

colo

gic

onco

logi

sts

and

othe

r phy

sici

ans

Si

ngle

te

achi

ng

hosp

ital

A

mul

tidis

cipl

inar

y te

am cr

eate

d a

guid

elin

e an

d th

en cr

eate

d a

com

mun

icat

ion

plan

to d

isse

min

ate

it

Be

fore

-afte

r

X

Pa

p te

sts

31

.2a

Miy

akis

200

6

Gree

ce

All p

hysi

cian

s w

orki

ng a

t the

ho

spita

l

Si

ngle

te

achi

ng

hosp

ital

Th

e re

sults

of a

n au

dit o

n te

st u

se w

ere

pres

ente

d al

ong

with

stra

tegi

es fo

r re

duci

ng u

tiliz

atio

n

Be

fore

-afte

r

X

X

25 la

bora

torie

s he

mat

olog

y an

d ch

emis

try te

sts

13

.5a

Mor

ris 1

992

US

A

All p

hysi

cian

s w

orki

ng a

t the

ho

spita

l

Si

ngle

te

achi

ng

hosp

ital

On

ly o

ne s

tool

sam

ple

for p

aras

ites

was

ac

cept

ed b

y th

e la

bora

tory

inst

ead

of

thre

e

Be

fore

-afte

r

X

St

ool f

or o

va a

nd p

aras

ite

33.3

Mut

imer

199

2

Unite

d Ki

ngdo

m

All p

hysi

cian

s w

orki

ng in

the

liver

uni

t

Li

ver u

nit

at a

sin

gle

teac

hing

ho

spita

l

La

bora

tory

test

s to

be

orde

red

for t

he

next

day

wer

e de

term

ined

by

prot

ocol

ba

sed

on p

atie

nt ch

arac

teris

tics

and

prev

ious

test

ing

Be

fore

-afte

r

X

He

mat

olog

y, b

ioch

emis

try,

imm

unol

ogy,

mic

robi

olog

y an

d cr

oss

mat

chin

g

9.

3a

Nard

ella

199

5

USA

An

esth

etis

ts a

nd

surg

eons

Si

ngle

te

achi

ng

hosp

ital

Gu

idel

ines

abo

ut a

ppro

pria

te p

re-

oper

ativ

e te

stin

g w

ere

dist

ribut

ed a

nd

disc

usse

d at

sta

ff m

eetin

gs

Be

fore

-afte

r

X

CB

C, P

T, P

TT, b

leed

ing

time,

BU

N, C

r, Gl

ucos

e AL

T AS

T,

elec

troly

tes

61

.8a

Neils

on 2

004

US

A

Resi

dent

s,

phys

icia

ns a

nd

nurs

e pr

actit

ione

rs

who

use

d co

mpu

ter o

rder

en

try

Si

ngle

te

achi

ng

hosp

ital

Re

curr

ing

test

s ha

d to

be

conf

irmed

da

ily, m

any

test

s w

ere

limite

d to

a s

ingl

e oc

curr

ence

and

pre

viou

s re

sults

wer

e di

spla

yed

whe

n or

derin

g te

sts

Be

fore

-afte

r

X

Al

l blo

od te

sts

and

imag

ing

64

.2a

Nels

on 1

978

US

A

All p

hysi

cian

s w

ho w

ork

at th

e ho

spita

l

Si

ngle

co

mm

unity

ho

spita

l

Th

ere

was

a 2

0-m

in g

rand

roun

ds

pres

enta

tion

on B

12 te

stin

g.

Sugg

estio

ns w

ere

give

n on

whe

n to

ord

er

this

test

Be

fore

-afte

r

X

B1

2 an

d fo

late

45

.3



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Nigh

tinga

le 1

994

Unite

d Ki

ngdo

m

Resi

dent

s w

orki

ng

on th

e liv

er u

nit

Li

ver u

nit

at a

sin

gle

teac

hing

ho

spita

l

A

com

pute

rized

pro

toco

l was

crea

ted

that

su

gges

ted

test

s fo

r the

nex

t day

bas

ed

on cu

rren

t lab

orat

ory

valu

es a

nd p

atie

nt

char

acte

ristic

s

Be

fore

-afte

r

X

Al

l blo

od te

sts

23

.7

Nire

l 201

1

Isra

el

All p

hysi

cian

s w

orki

ng in

the

heal

th s

yste

m

Pr

imar

y ca

re

clin

ics

An

onl

ine

med

ical

reco

rd w

as in

trodu

ced

so th

at o

ld la

bora

tory

test

s co

uld

be

acce

ssed

Be

fore

-afte

r

X

Al

l blo

od te

sts

3.

6a

An o

nlin

e m

edic

al re

cord

was

intro

duce

d so

that

old

radi

olog

y te

sts

coul

d be

ac

cess

ed

Be

fore

-afte

r

X

Al

l rad

iolo

gy te

sts

12

.3

Page

ler 2

013

US

A

Phys

icia

ns

wor

king

in

pedi

atric

ICU

Si

ngle

te

achi

ng

hosp

ital

A

rule

pre

vent

ing

repe

atin

g te

stin

g w

as

crea

ted

in th

e co

mpu

ter o

rder

ent

ry

syst

em. I

t had

to b

e ov

errid

den

to o

rder

re

peat

test

ing

Be

fore

-afte

r

X

CB

C, ch

emis

try a

nd

coag

ulat

ion

37

.7a

Page

ler 2

009

US

A

Not r

epor

ted

Si

ngle

te

achi

ng

hosp

ital

Re

curr

ing

orde

rs fo

r CXR

wer

e no

long

er

allo

wed

and

inst

ead

had

to b

e re

-ord

ered

da

ily

Be

fore

-afte

r

X

Ch

est X

R

10.0

Pate

l 200

5

Unite

d Ki

ngdo

m

All p

hysi

cian

s w

orki

ng in

the

hosp

ital

Si

ngle

te

achi

ng

hosp

ital

Ur

ine

was

not

cultu

red

if th

e ur

inal

ysis

w

as n

orm

al

Befo

re-a

fter

X

Urin

e cu

lture

4.

0

Pilo

n 19

97

Cana

da

Phys

icia

ns a

nd

nurs

es w

orki

ng in

th

e IC

U

Si

ngle

te

achi

ng

hosp

ital

A

guid

elin

e ab

out w

hen

ABGs

sho

uld

be u

sed

was

crea

ted.

It w

as d

istri

bute

d on

poc

ket c

ards

and

taug

ht d

urin

g ed

ucat

ion

sess

ions

Be

fore

-afte

r

X

AB

Gs

36.7

a

Pop

1989

Ne

ther

land

Ge

nera

l pr

actit

ione

rs

All p

hysi

cian

s us

ing

the

sam

e la

bora

tory

in

one

city

Tw

o tim

es p

er y

ear p

hysi

cian

s w

ere

give

n a

repo

rt of

test

s th

ey h

ad o

rder

ed a

nd

whe

ther

they

wer

e ap

prop

riate

or n

ot.

Phys

icia

ns w

ere

also

info

rmed

of t

heir

redu

ndan

t tes

ts

Be

fore

-afte

r

X

BU

N, C

r, liv

er e

nzym

es,

WBC

, Hgb

, ESR

, glu

cose

36

.2

Pow

er 1

999

Au

stra

lia

Surg

ical

Res

iden

ts

in p

re-o

p cl

inic

Si

ngle

te

achi

ng

hosp

ital

Pu

blis

hed

guid

elin

es o

n te

st o

rder

ing

wer

e di

strib

uted

. A m

axim

um n

umbe

r of

test

s pe

r pat

ient

was

inst

itute

d an

d te

st

orde

ring

was

revi

ewed

by

a co

nsul

tant

Be

fore

-afte

r

X

X

X

CB

C, co

agul

atio

n te

sts,

ur

ea, L

FTs,

CXR

, ECG

37

.1



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Pow

les

2009

Un

ited

King

dom

Al

l phy

sici

ans

in re

gion

usi

ng

the

univ

ersi

ty

labo

rato

ry fo

r PSA

te

stin

g

Si

ngle

te

achi

ng

hosp

ital

Pr

osta

te-s

peci

fic a

ntig

en w

as ta

ken

off

the

requ

est f

orm

and

had

to b

e w

ritte

n in

by

hand

Be

fore

-afte

r

X

PS

A

17.3

a

Prat

200

9

Fran

ce

All p

hysi

cian

s w

orki

ng in

ICU

Si

ngle

te

achi

ng

hosp

ital

A

guid

elin

e w

as cr

eate

d on

whe

n to

pe

rform

eac

h of

the

com

mon

blo

od te

sts.

Th

e co

st o

f eac

h te

st w

as a

lso

show

n on

th

e gu

idel

ine.

Teac

hing

was

don

e on

how

to

redu

ce te

st u

tiliz

atio

n

Be

fore

-afte

r

X

CX

R an

d da

ily ro

utin

e BW

49

.0a

Pysh

er 1

999

US

A

All p

hysi

cian

s or

derin

g te

sts

at

this

labo

rato

ry

Si

ngle

te

achi

ng

hosp

ital

Pr

edef

ined

chem

istry

pan

els

wer

e el

imin

ated

from

the

requ

isiti

on fo

rms.

In

stea

d ea

ch te

st h

ad to

be

orde

red

sepa

rate

ly

Be

fore

-afte

r

X

Al

l blo

od ch

emis

try te

sts

32

.7a

Ram

oska

199

8

USA

Ph

ysic

ians

and

re

side

nts

wor

king

in

the

emer

genc

y de

partm

ent

Si

ngle

te

achi

ng

hosp

ital

Gr

aphs

of l

abor

ator

y ut

iliza

tion

and

cost

s of

labo

rato

ry te

sts

at th

e ho

spita

l was

di

spla

yed

on a

bul

letin

boa

rd

Be

fore

-afte

r

X

Al

l blo

od te

sts

17

.8a

Ratn

aike

199

3

Aust

ralia

Al

l phy

sici

ans

wor

king

in th

e ca

rdia

c car

e un

it

Si

ngle

te

achi

ng

hosp

ital

Gu

idel

ines

wer

e cr

eate

d by

the

hosp

ital

and

issu

ed to

the

card

iolo

gy u

nit.

No

furt

her d

etai

ls a

bout

the

guid

elin

es w

ere

give

n

Be

fore

-afte

r

X

Al

l blo

od te

sts

57

.2

Rhyn

e 19

79

USA

Fa

mily

doc

tors

Th

e fa

mily

m

edic

ine

prog

ram

as

soci

ated

w

ith o

ne

univ

ersi

ty

A

char

t aud

it of

test

ord

erin

g w

as d

one

and

the

resu

lts w

ere

pres

ente

d to

m

edic

al s

taff

Be

fore

-afte

r

X

X

Thyr

oid

func

tion

test

s

36.4

a

Robe

rts

1993

Ca

nada

Al

l phy

sici

ans

wor

king

at t

he

hosp

ital

Si

ngle

te

achi

ng

hosp

ital

Se

rum

osm

olal

ity w

as re

mov

ed fr

om

the

requ

isiti

on a

nd b

lood

coun

t di

ffere

ntia

l was

not

don

e if

whi

te ce

ll co

unt w

as n

orm

al. A

pro

toco

l was

use

d to

det

erm

ine

whe

n el

ectro

lyte

s sh

ould

be

chec

ked.

Dai

ly C

XRs

wer

e no

long

er

allo

wed

Be

fore

-afte

r

X

X

Bloo

d ga

ses,

ser

um

osm

olal

ity, C

BC, N

a, K

, gl

ucos

e, C

XR, a

nd E

CG

18

.1a



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Rose

nblo

om

2005

US

A

All p

hysi

cian

s w

orki

ng o

n se

lect

ed in

patie

nt

units

Si

ngle

te

achi

ng

hosp

ital

M

agne

sium

coul

d on

ly b

e or

dere

d on

ce w

ith n

o re

curr

ence

and

a s

cree

n di

spla

ying

old

resu

lts a

nd e

duca

tion

on

whe

n m

agne

sium

sho

uld

be ch

ecke

d ha

d to

be

view

ed b

efor

e th

e or

der c

ould

be

ente

red

Be

fore

-afte

r

X

X

Mag

nesi

um

55.2

a

Sant

os 2

012

Br

azil

Al

l res

iden

ts

and

phys

icia

ns

wor

king

at t

he

hosp

ital

Si

ngle

te

achi

ng

hosp

ital

A

mee

ting

was

hel

d to

dis

cuss

evi

denc

e fo

r cer

tain

test

s an

d th

e co

sts

of

the

test

s. P

atte

rns

of te

stin

g an

d ap

prop

riate

ness

of v

ario

us te

sts

was

di

scus

sed

Be

fore

-afte

r

X

CR

P

48.4

Segu

in 2

002

Fr

ance

Al

l phy

sici

ans

wor

king

in IC

U

Sing

le

teac

hing

ho

spita

l

Co

sts

for e

ach

test

was

sho

wn

on th

e or

der f

orm

Be

fore

-afte

r

X

El

ectro

lyte

s, C

BC, A

BG,

LFTs

, coa

gula

tion

pane

l, CX

R

18

.9

Shal

ev 2

009

Is

rael

Al

l prim

ary

care

ph

ysic

ians

in

Isra

el

Si

ngle

hea

lth

serv

ice

prov

ider

27

test

s w

ere

rem

oved

from

the

requ

isiti

on a

nd 2

wer

e ad

ded

Be

fore

-afte

r

X

Al

l blo

od te

st

4.3a

Sorit

a 20

11

USA

Re

side

nts

train

ing

in m

edic

ine

and

surg

ery

Si

ngle

te

achi

ng

hosp

ital

Th

ere

was

a 1

h te

achi

ng s

essi

on

rega

rdin

g w

hen

to o

rder

STA

T bl

ood

test

s an

d re

sults

of a

n au

dit w

ere

pres

ente

d.

The

high

est u

sers

of S

TAT

test

s w

ere

give

n in

divi

dual

feed

back

Be

fore

-afte

r

X

X

All s

tat l

abor

ator

y te

sts

21

.0a

Spra

y 20

12

USA

Ph

ysic

ians

taki

ng

care

of i

npat

ient

s

Sing

le

com

mun

ity

hosp

ital

An

ele

ctro

nic p

roto

col f

or e

lect

roly

te

repl

acem

ent a

nd re

test

ing

was

inst

itute

d

Befo

re-a

fter

X

Mag

nesi

um, p

hosp

horo

us

and

ioni

zed

calc

ium

32

.0a

Staf

ford

200

3

USA

11

7 pr

imar

y ca

re

prov

ider

s

Prim

ary

care

pr

actic

es

affil

iate

d w

ith a

sin

gle

teac

hing

ho

spita

l

Au

dits

of i

ndiv

idua

l and

gro

up E

CG

usag

es p

atte

rns

wer

e m

aile

d to

ph

ysic

ians

Be

fore

-afte

r

X

EC

G

28.3



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Suco

v 19

98

USA

Al

l phy

sici

ans

wor

king

in th

e ho

spita

l

Si

ngle

te

achi

ng

hosp

ital

Gu

idel

ines

wer

e cr

eate

d re

gard

ing

wha

t te

sts

to o

rder

for w

hich

pat

ient

gro

ups.

Th

e gu

idel

ine

was

then

rolle

d ou

t with

a

serie

s of

edu

catio

nal s

essi

ons.

Eac

h de

partm

ent r

ecei

ved

feed

back

on

its

orde

ring

prac

tices

but

ther

e w

as n

o in

divi

dual

feed

back

Be

fore

-afte

r

X

X

All b

lood

test

s

22.0

a

Suss

man

198

4

USA

Al

l phy

sici

ans

wor

king

in th

e ho

spita

l

Si

ngle

te

achi

ng

hosp

ital

Al

l rec

urrin

g or

ders

wer

e el

imin

ated

Be

fore

-afte

r

X

CB

C, e

lect

roly

tes,

CXR

, ch

emis

try p

anel

, FBG

, el

ectro

card

iogr

am, P

T

20

.8

Thom

pson

198

3

USA

Al

l phy

sici

ans

part

of th

e he

alth

pla

n

Sing

le p

re-

paid

hea

lth

plan

A

reco

mm

enda

tion

was

mad

e ag

ains

t sc

reen

ing

heal

thy

adul

ts w

ith b

lood

test

s an

d CX

R. A

n ed

ucat

iona

l cam

paig

n w

as

then

impl

emen

ted

Be

fore

-afte

r

X

Al

l blo

od ch

emis

try te

sts

and

CXR

60

.0a

Tier

ney

1987

US

A

Resi

dent

s an

d st

aff w

orki

ng in

th

e cl

inic

Si

ngle

un

iver

sity

as

soci

ated

ge

nera

l m

edic

ine

clin

ic

Al

l pre

viou

s re

sults

for a

test

wer

e sh

own

whe

n us

ing

the

com

pute

r ord

er e

ntry

sy

stem

. An

optio

n to

canc

el th

e cu

rren

t te

st w

as th

en a

vaila

ble

Ra

ndom

ized

co

ntro

lled

trial

X

All l

abor

ator

y an

d im

agin

g an

d EC

Gs

8.9a

Tier

ney

1990

US

A

Phys

icia

ns

wor

king

in th

e cl

inic

Si

ngle

un

iver

sity

as

soci

ated

ge

nera

l m

edic

ine

clin

ic

Co

st to

the

patie

nt o

f ord

erin

g ea

ch

test

s w

as d

ispl

ayed

bef

ore

the

orde

r is

conf

irmed

Ra

ndom

ized

co

ntro

lled

trial

X

All b

lood

test

s

17.6

a

Toub

ert 2

000

Fr

ance

Al

l phy

sici

ans

in

the

hosp

ital

Si

ngle

te

achi

ng

hosp

ital

Ed

ucat

iona

l mat

eria

l on

thyr

oid

dise

ase

was

dis

tribu

ted.

Phy

sici

ans

wer

e to

ld

that

any

test

oth

er th

an TS

H re

quire

d ju

stifi

catio

n. Te

sts

wer

e st

ill p

roce

ssed

ev

en if

no

just

ifica

tion

was

pro

vide

d

Be

fore

-afte

r

X

TS

H, T

3, T

4,

39.6

van

Wijk

200

1

Neth

erla

nds

GPs

wor

king

in

prac

tices

with

el

ectro

nic m

edic

al

reco

rds

Si

ngle

hea

lth

regi

on

Com

pute

rized

dec

isio

n su

ppor

t su

gges

ted

appr

opria

te te

sts

base

d on

ba

sic c

linic

al in

form

atio

n

Ra

ndom

ized

co

ntro

lled

trial

X

All b

lood

test

s

20.3

a



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

Vard

y 20

05

Isra

el

All p

hysi

cian

s or

derin

g te

sts

thro

ugh

the

HMO

Si

ngle

hea

lth

man

agem

ent

orga

niza

tion

Li

sts

of a

ppro

pria

te la

bora

tory

test

s fo

r ea

ch cl

inic

al co

nditi

on w

ere

crea

ted

and

phys

icia

ns w

ere

educ

ated

abo

ut th

em.

Each

list

was

then

put

into

the

com

pute

r or

der e

ntry

sys

tem

Be

fore

-afte

r

X

X

All b

lood

test

s

4.2

Vers

tapp

en 2

003

Ne

ther

land

s Ph

ysic

ians

– n

ot

spec

ified

5

heal

th

regi

ons

in th

e co

untry

Ea

ch p

hysi

cian

rece

ived

feed

back

on

thei

r ord

erin

g pr

actic

es re

gard

ing

uppe

r and

low

er a

bdom

inal

pai

n an

d ca

rdio

vasc

ular

cond

ition

s co

mpa

red

with

th

eir p

eers

. The

phy

sici

ans

then

met

to

geth

er to

dis

cuss

how

to ch

ange

thei

r or

derin

g pr

actic

es

Ra

ndom

ized

co

ntro

lled

trial

X

X

Bi

oche

mis

try a

nd

hem

atol

ogy

test

s, E

CGs,

si

gmoi

dosc

opy

and

imag

ing

12

.0a

Each

phy

sici

an re

ceiv

ed fe

edba

ck o

n th

eir o

rder

ing

prac

tices

rega

rdin

g CO

PD,

asth

ma,

deg

ener

ativ

e jo

int c

ompl

aint

s an

d no

n-sp

ecifi

c com

plai

nts

com

pare

d w

ith th

eir p

eers

. The

phy

sici

ans

then

met

to

geth

er to

dis

cuss

how

to ch

ange

thei

r or

derin

g pr

actic

es

Ra

ndom

ized

co

ntro

lled

trial

X

X

Bi

oche

mis

try a

nd

hem

atol

ogy

test

s, E

CGs,

si

gmoi

dosc

opy

and

imag

ing

8.

0

Wag

holik

ar 2

011

Au

stra

lia

All p

hysi

cian

s w

orki

ng in

the

emer

genc

y de

partm

ent

Si

ngle

te

achi

ng

hosp

ital

Ap

prov

al fr

om a

sen

ior p

hysi

cian

w

as re

quire

d to

ord

er te

sts

in ce

rtain

ci

rcum

stan

ces

Be

fore

-afte

r

X

CB

C an

d ES

R

11.8

a

Win

kens

199

5

Neth

erla

nds

Fam

ily p

hysi

cian

s

Sing

le

regi

onal

m

edic

al

labo

rato

ry

In

divi

dual

ized

feed

back

was

sen

t to

each

phy

sici

an fr

om a

resp

ecte

d in

tern

al

med

icin

e ph

ysic

ian.

The

feed

back

in

clud

ed a

ltern

ativ

e te

stin

g st

rate

gies

Be

fore

-afte

r

X

Ce

rvic

al s

mea

r, EC

G,

endo

scop

y, a

llerg

y te

sts,

radi

ogra

phic

and

ul

traso

und

test

s

5.

5a

Win

kens

199

2

Neth

erla

nds

Fam

ily p

hysi

cian

s

Sing

le

regi

onal

m

edic

al

labo

rato

ry

Bi

annu

al re

port

s w

ere

sent

on

the

num

ber a

nd ra

tiona

lity

of la

bora

tory

te

stin

g co

mpa

red

to p

eers

Be

fore

-afte

r

X

X

46 b

lood

test

s

25.0

a

Won

es 1

987

US

A

Firs

t yea

r int

erna

l m

edic

ine

resi

dent

s

Si

ngle

te

achi

ng

hosp

ital

Th

e gr

oup

was

sim

ply

info

rmed

that

la

bora

tory

util

izat

ion

was

bei

ng

mon

itore

d

Ra

ndom

ized

co

ntro

lled

trial

X

All l

abor

ator

y te

sts

–1

1.4



(Tab

le 1

Cont

inue

d)

Auth

or Y

ear

Co

untr

y

Heal

thca

re

wor

kers

targ

eted

Si

te

Inte

rven

tion

Co

mpa

rato

r

Cate

gory

of

Inte

rven

tion

Ta

rget

ed te

sts

Re

lativ

e re

duct

ion

of ta

rget

ed

test

s

E

S

A

I

This

gro

up re

ceiv

ed in

divi

dual

feed

back

on

labo

rato

ry u

sage

and

cost

s ev

ery

2 w

eeks

X

Al

l lab

orat

ory

test

s

–13.

15

This

gro

up re

ceiv

ed in

divi

dual

feed

back

on

labo

rato

ry u

sage

and

cost

s ev

ery

2 w

eeks

alo

ng w

ith g

roup

dat

a fo

r co

mpa

rison

X

Al

l lab

orat

ory

test

s

–7.2

7

Zaat

199

2

Neth

erla

nds

Gene

ral

Prac

titio

ners

Tw

o he

alth

re

gion

s

The

labo

rato

ry re

quis

ition

was

mod

ified

so

that

it o

nly

had

15 te

sts

on it

. All

othe

r te

sts

had

to b

e w

ritte

n in

Be

fore

-afte

r

X

Al

l hem

atol

ogic

al a

nd

chem

istry

test

s

18.2

a

Zaid

i 199

9

USA

Al

l phy

sici

ans

send

ing

sam

ples

to

the

hosp

ital l

ab

Si

ngle

te

achi

ng

hosp

ital

Gu

idel

ines

wer

e cr

eate

d fro

m lo

cal d

ata

and

shar

ed w

ith a

ll m

edic

al s

taff.

Afte

r th

is a

ll sa

mpl

es n

ot m

eetin

g th

e cr

iteria

w

ere

reje

cted

Be

fore

-afte

r

X

X

Stoo

l cul

ture

30

.9

Zia

2013

US

A

All P

hysi

cian

s at

th

e ho

spita

l

Sing

le

teac

hing

ho

spita

l

Th

e te

st w

as re

mov

ed fr

om th

e la

bora

tory

re

quis

ition

but

coul

d st

ill b

e or

dere

d if

writ

ten

in b

y ha

nd

Be

fore

-afte

r

X

Se

rum

pro

tein

el

ectro

phor

esis

and

im

mun

ofix

atio

n

29

.6

a Deno

tes

stud

ies

in w

hich

a te

st o

f sta

tistic

al s

igni

fican

ce w

as p

erfo

rmed

and

foun

d to

be

sign

ifica

nt b

y th

e st

udie

s’ a

utho

rs. A

, Aud

it an

d fe

edba

ck; E

, Edu

catio

nal;

I, In

cent

ive

or p

enal

ty;

S, S

yste

m-b

ased

.



Figure 2 Box plots of relative reductions in laboratory utilization stratified by intervention type.1) Interventions with an educational component; 2) Interventions classified as educational only; 3) Interventions classified as educational only that use distribution of a guideline to educate; 4) Interventions with a system-based component; 5) Interventions classified as system-based only; 6) Interventions classified as system-based only that used a clinical decision support tool; 7) Interventions with an audit and feedback component; 8) Interventions classified as audit and feedback only; 9) Interventions classified as audit and feedback only that use cost information display.

Table 2 Percent relative reductions in test utilization by intervention type.a

Number of interventions

Median RR (IQR)

Minimum RR

Maximum RR

Educational interventions 51 34.5 (16.5–49.0) –12.3 99.6 –Δ between intervention and control statistically significant 33 31.2 (18.1–47.5) 4.7 85.0 –Δ between intervention and control statistically insignificant 18 38.3 (8.0–53.0) –12.3 99.6Exclusively education intervention 22 30.6 (16.5–48.5) –12.3 75.0Exclusively guideline distribution 13 30.0 (20.8–53.0) 4.7 99.6

System-based interventions 54 22.2 (3.6–68.3) 0.1 99.7 –Δ between intervention and control statistically significant 36 19.6 (10.4–36.1) 3.6 99.7 –Δ between intervention and control statistically insignificant 18 24.8 (10.0–37.1) 0.1 77.8Exclusively system-based intervention 37 21.6 (10.0–32.7) 0.1 99.7Exclusively clinical decision support 10 22.8 (9.3–23.9) 4.7 85.0

Audit feedback interventions 41 22.0 (8.6–34.6) –13.1 99.6 –Δ between intervention and control statistically significant 24 23.2 (13.8–34.5) 5.5 63.4 –Δ between intervention and control statistically insignificant 17 18.9 (0–36.2) –13.1 99.6Exclusively audit and feedback intervention 20 18.4 (2.1–24.8) –13.1 36.2Exclusively cost display 12 18.4 (2.1–28.8) 5.5 99.6

Physicians targeted were involved in creating and implementing the intervention 17 23.0 (8.0–39.6) 3.6 99.6Physicians targeted were not involved in creating and implementing the intervention

103 22.2 (10.1–36.7) –27.7 99.7

aCategories of interventions are not mutually exclusive unless stated.



Figure 3 Box plots of relative reductions in laboratory utilization stratified by intervention type among interventions targeting four or more tests.1) Interventions with an educational component; 2) Interventions classified as educational only; 3) Interventions classified as educational only that use distribution of a guideline to educate; 4) Interventions with a system-based component; 5) Interventions classified as system-based only; 6) Interventions classified as system-based only that used a clinical decision support tool; 7) Interventions with an audit and feedback component; 8) Interventions classified as audit and feedback only; 9) Interventions classified as audit and feedback only that use cost information display.

Table 3 General descriptions of study quality (n = 109).

Number of studies

Median relative reduction (IQR)

Where patients similar between groups? Yes 37 19.0 (8.6–37.7) No 72 24.8 (14.0–41.5)Were those ordering the tests similar between groups? Yes 103 23.8 (11.9–40.2) No 6 11.7 (4.2–25.0)Was there a concurrent control group? Yes 27 16.5 (5.8–27.0) No 82 24.9 (14.0–47.5)Was the intervention described adequately enough to be replicated? Yes 88 23.5 (12.4–38.6) No 21 20.8 (4.7–36.7)Was there a risk of contamination between experimental and control groups? Yes 89 24.9 (13.5–40.8) No 20 15.8 (4.8–29.6)Were the results reported per patient instead of per institution or per physician? Yes 79 19.9 (9.1–35.4) No 30 33.4 (23.4–48.3)Was a time-series analysis conducted? Yes 11 26.1 (14.0–55.2) No 98 23.5 (10.1–37.1)

a 99.7% reduction in test use [53]. Lewandrowski et al. distributed guidelines stating when it was appropriate to order LD-isozymes with a laboratory physician notifying the ordering physician when the LD-isozyme request was not compliant with the guideline [69]. This combination of education and feedback led to a 99.6% reduction in LD-isozymes being ordered. The two largest relative reduc-tions among interventions that targeted four or more tests were by Chu et al. and Dixon et al. Chu et al. used a pro-tocol driven algorithm for admitting trauma patients and found a 68.3% reduction in the number of tests ordered per patient [33]. Dixon et al. limited the number of tests that interns could order to eight per patient per day, result-ing in a 66.7% reduction in the number of tests ordered per patient [38].

The most successful interventions targeting four or more tests that were purely system based were the study by Chu et al, discussed above, and a study by Neilson et al. Neilson et al. sequentially implemented a number of system-based interventions using a computerized order entry system including: the display of previous labora-tory results to physicians ordering tests; the prevention of recurring orders in various circumstances; and the unbun-dling of tests so that each component had to be ordered



individually [82]. This package of interventions resulted in a 64.2% relative reduction in the tests targeted.

DiscussionWe summarized 109 studies evaluating the effect of inter-ventions aimed at reducing test utilization by calculating relative reductions to compare interventions. We found that all intervention categories (including education, system based, audit feedback, and incentive penalty) can reduce test utilization. Even though educational interven-tions had the largest relative reduction in test utilization, the inter-quartile ranges were large and overlapping. The greatest difference in our stratified analysis was between interventions that targeted less than four tests, which had much larger relative reductions than interventions that targeted four or more tests. Consistent with past reviews, we found that interventions using multiple strategies were more effective [8]. Even though all intervention types seem to be effective, we found a large range of effects and much heterogeneity between studies. Many studies did not use appropriate statistical techniques and follow-up was often short. These shortcomings call into question the sustainability and generalizability of many interventions in our review.

Sustainability

Although interventions with an educational component had the greatest median relative reductions, their long-term sustainability is questionable. The resources needed to perpetuate educational strategies (i.e., lectures or indi-vidual teaching sessions) are significant and probably need to be continuous to maintain an effect, especially in teaching hospitals where ordering physicians are con-stantly changing. Even in studies where the educational strategy was low effort (i.e., the distribution of a guide-line), multiple reminders and repeated meetings were necessary to maintain the change in behavior [89, 97, 106, 111]. There was only one purely educational intervention that reported results from > 1 year after the intervention began but this study did not use time series analysis to determine if utilization returned to baseline during this time [70]. With all but one purely educational interven-tions having a time horizon of < 1 year there is no evidence in our review that educational strategies have long-term sustainability. In contrast there are several examples of system-based interventions which by definition require

few resources to maintain, that are shown to maintain their effect for > 1 year [14, 47, 53]. The ratio of resource input to test reduction is not measured in the included studies but is an important consideration when deciding to implement a particular intervention.

Generalizability

Many of the interventions have questionable generaliz-ability. There is a wide range of relative reductions among included studies due to heterogeneity at every level. There are differences in how an invention type is implemented, who is target by the intervention, and the pre-existing ordering practices among the targeted physicians. Due to the heterogeneity, similar results cannot be expected even if an intervention is perfectly replicated. Some of the large reductions ( > 70%) suggest that anomalous ordering is being targeted which further calls into question the gen-eralizability of results. Lastly many of the studies target physicians in training who would likely respond differ-ently than physicians in independent practice.

Quality

The quality of included studies was poor. As expected studies with lower quality had higher relative reductions suggesting that poor quality studies were over estimating the effects of interventions. Surprisingly, very few of the before and after studies used time-series analysis or auto-correlation to analyze results, bringing into question the validity of results.

Limitations

This review has several limitations. Our search strategy used only studies published in English ignoring poten-tially important studies published in other languages and only searched two databases. Another limitation is that the heterogeneity between interventions makes generalizations about effective strategies difficult. Despite this weakness the relative reduction does tell us which interventions were associated with the largest reductions. This allows these successes to be examined to see if they are applicable to other settings. This review is broad in scope and therefore does not allow detailed examination of particular intervention types to find the elements that lead to success. Reviews of particu-lar types of interventions have been done and explore



these factors closely [122–124]. Lastly, our use of rela-tive reductions leaves out absolute reduction and cost savings information that may be important to decision makers.

ConclusionsOur review found that research into interventions to reduce test utilization is ongoing and many strategies can be successful. There are many examples of low investment interventions that work, especially when they target tests that have a high rate of inappropriate use. Due to questionable validity and generalizability of these results, this review should be used as a refer-ence for finding high quality studies with large relative reductions that apply to a setting of interest. Future research needs to determine how low input strategies using computerized order entry can be used to imple-ment system changes, audit and feedback, and educa-tion. Database monitoring should be developed so that institutions know when anomalous test use is occurring and which physicians are deviating from the mean. This will allow for targeting of interventions to where they will have the greatest effect. Lastly, further studies in this field must monitor for adverse events caused by reduced testing.

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.Financial support: None declared.Employment or leadership: None declared.Honorarium: None declared.Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.

References1. National Health Expenditure trends, 1975 to 2013. Ottawa, ON:

Canadian Institute for Health Information, 2013.2. Health at a Glance 2013: OECD Indicators. Paris: OECD Publish-

ing, 2013.3. Qaseem A, Alguire P, Dallas P, Feinberg LE, Fitzgerald FT,

Horwitch C, et al. Appropriate use of screening and diagnostic tests to foster high-value, cost-conscious care. Ann Intern Med 2012;156:147–9.

4. van Walraven C, Naylor CD. Do we know what inappropriate laboratory utilization is? A systematic review of laboratory clini-cal audits. J Am Med Assoc 1998;280:9.

5. Grossman RM. A review of physician cost-containment strate-gies for laboratory testing. Med Care 1983;21:783–802.

6. Axt-Adam P, van der Wouden JC, van der Does E. Influencing behavior of physicians ordering laboratory tests: a literature study. Med Care 1993;31:784–94.

7. Young DW. Improving laboratory usage: a review. Postgrad Med 1988;64:283–9.

8. Solomon DH, Hashimoto H, Daltroy L, Liang MH. Techniques to improve physicians’ use of diagnostic tests: a new conceptual framework. J Am Med Assoc 1998;280:2020–7.

9. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Br Med J 2009;339:b2535.

10. Egger M, Smith GD, Altman DG. Principles of and procedures for systematic reviews. Systematic Reviews in Health Care: meta analysis in context, 2nd ed. London: BMJ Publishing Group, 2008:23–42.

11. Cochrane Effective Practice and Organization of Care Review Group. Data collection checklist. Ottawa: Institute of Population Health, University of Ottawa, 2002.

12. Al AM, Zidan A, Hamza AA, Al-Suwaidi R. Evaluation of laboratory investigations in accident and emergency depart-ment, Salmaniya Medical Complex. Bahrain Med Bull 2002;24:95–7.

13. Amukele TK, Baird GS, Chandler WL. Reducing the use of coagu-lation test panels. Blood Coagul Fibrinolysis 2011;22:688–95.

14. Archambault AJ, Arkin CF, Skelton T, Abel G, Gawoski J. Blood (ASH Annual Meeting Abstracts) 2011;118:Abstract 4774.

15. Attali M, Barel Y, Somin M, Beilinson N, Shankman M, Acker-man A, et al. A cost-effective method for reducing the volume of laboratory tests in a university-associated teaching hospital. Mt Sinai J Med 2006;73:787–94.

16. Barazzoni F, Grilli R, Amicosante AM, Brescianini S, Marca MA, Baggi M, et al. Impact of end user involvement in implementing guidelines on routine pre-operative tests. Int J Qual Health Care 2002;14:321–7.

17. Bareford D, Hayling A. Inappropriate use of laboratory services: long term combined approach to modify request patterns. Br Med J 1990;301:1305–7.

18. Baricchi R, Zini M, Nibali MG, Vezzosi W, Insegnante V, Manfuso C, et al. Using pathology-specific laboratory profiles in clinical pathology to reduce inappropriate test requesting: two com-pleted audit cycles. BMC Health Serv Res 2012;12:187.

19. Barie PS, Hydo LJ. Lessons learned: durability and progress of a program for ancillary cost reduction in surgical critical care. J Trauma 1997;43:590–6.

20. Bates DW, Kuperman GJ, Jha A, Teich JM, Orav EJ, Ma’luf N, et al. Does the computerized display of charges affect inpatient ancil-lary test utilization? Arch Intern Med 1997;157:2501–8.

21. Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma’luf N, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med 1999;106:144–50.

22. Berenholtz S, Pronovost P, Lipsett P, Dawson P, Dorman T. Assessing the effectiveness of critical pathways on reducing resource utilization in the surgical intensive care unit. Intens Care Med 2001;27:1029–36.



23. Berwick DM, Coltin KL. Feedback reduces test use in a health maintenance organization. J Am Med Assoc 1986;255:1450–4.

24. Boon-Falleur L, Sokal E, Peters M, Ketelslegers JM. A rule-based decision support application for laboratory investigations man-agement. Proc Annu Symp Comput Appl Med Care 1995;314–8.

25. Buckingham K, Russell I, Ross I, Gibson P, Paterson N. The effect of allowing clinical discretion in ordering biochemical tests. Evaluation by complementary methods. Int J Technol Assess 1994;10:695–700.

26. Bunting PS, Van Walraven C. Effect of a controlled feedback intervention on laboratory test ordering by community physi-cians. Clin Chem 2004;50:321–6.

27. Burnett L, Chesher D, Burnett JR. Optimizing the availability of ‘stat’ laboratory tests using Shewhart ‘C’ control charts. Ann Clin Biochem 2002;39:140–4.

28. Calderon-Margalit R, Mor-Yosef S, Mayer M, Adler B, Shapira SC. An administrative intervention to improve the utilization of laboratory tests within a university hospital. Int J Qual Health Care 2005;17:243–8.

29. Carson K, Murillo P, Runge A, Vogel DG, Baum LG. A test-ordering algorithm can reduce laboratory test use. Lab Med 1995;26:277–81.

30. Carter E, Bennett BD. Reference test review by pathology house staff: a cost-containment strategy for the clinical laboratory. Clin Leader Manag Rev 2002;16:3–6.

31. Chen P, Tanasijevic MJ, Schoenenberger RA, Fiskio J, Kuperman GJ, Bates DW. A computer-based intervention for improving the appropriateness of antiepileptic drug level monitoring. Am J Clin Pathol 2003;119:432–8.

32. Chu K, Wagholikar A, Greenslade J. Reducing laboratory test ordering by limiting the range of tests that may be requested by interns and residents on pathology request forms. Emerg Med Australas 2012;24:21.

33. Chu UB, Clevenger FW, Imami ER, Lampard SD, Frykberg ER, Tepas JJ. The impact of selective laboratory evaluation on utiliza-tion of laboratory resources and patient care in a level-I trauma center. Am J Surg 1996;172:558–3.

34. Cohen DI, Jones P, Littenberg B, Neuhauser D. Does cost infor-mation availability reduce physician test usage? A randomized clinical trial with unexpected findings. Med Care 1982;20: 286–92.

35. Davidoff F, Goodspeed R, Clive J. Changing test ordering behavior. A randomized controlled trial comparing probabil-istic reasoning with cost-containment education. Med Care 1989;27:45–58.

36. Detsky AS, McLaughlin JR, Abrams HB, L’Abbe K, Markel FM. Do interns and residents order more tests than attending staff? Results of a house staff strike. Med Care 1986;24: 526–34.

37. Dickinson JC. From process to policy: a generic prescription for test over-utilization in the emergency department. Fam Pract Res J 1987;7:12–21.

38. Dixon RH, Laszlo J. Utilization of clinical chemistry services by medical house-staff. Arch Intern Med 1974;134:1064–7.

39. Dowling PT, Alfonsi G, Brown MI, Culpepper L. An education program to reduce unnecessary laboratory tests by residents. Acad Med 1989;64:410–2.

40. Durand-Zaleski I, Rymer JC, Roudot-Thoraval F, Revuz J, Rosa J. Reducing unnecessary laboratory use with new test request form: example of tumour markers. Lancet 1993;342:150–3.

41. Durieux P, Ravaud P, Porcher R, Fulla Y, Manet C-S, Chaussade S. Long-term impact of a restrictive laboratory test ordering form on tumor marker prescriptions. Int J Technol Assess Health Care 2003;19:106–13.

42. Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet 2001;357:1406–9.

43. Eisenberg JM. An educational program to modify laboratory use by house staff. J Med Educ 1977;52:578–81.

44. Emerson JF, Emerson SS. The impact of requisition design on laboratory utilization. Am J Clin Pathol 2001;116:879–84.

45. Everett GD. Impact of supervision by medical teachers and in-patient test control programmes on the out-patient test utiliza-tion of residents. Med Educ 1985;19:138–42.

46. Everett GD, deBlois CS, Chang PF, Holets T. Effect of cost educa-tion, cost audits, and faculty chart review on the use of labora-tory services. Arch Intern Med 1983;143:942–4.

47. Feldkamp CS, Carey JL. An algorithmic approach to thyroid func-tion testing in a managed care setting: 3-Year experience. Am J Clin Pathol 1996;105:11–6.

48. Feldman LS, Shihab HM, Thiemann D, Yeh HC, Ardolino M, Mandell S, et al. Impact of providing fee data on laboratory test ordering: a controlled clinical trial. JAMA Intern Med 2013;173:903–8.

49. Finegan BA, Rashiq S, McAlister FA, O’Connor P. Selective order-ing of preoperative investigations by anesthesiologists reduces the number and cost of tests. Can J Anaesth 2005;52:575–80.

50. Fong BW. Common diagnostic tests: use and interpretation. Hawaii Med J 1989;48:7–9.

51. Fowkes FG, Hall R, Jones JH, Scanlon MF, Elder GH, Hobbs DR, et al. Trial of strategy for reducing the use of laboratory tests. Br Med J 1986;292:883–5.

52. Friedman KG, Rathod RH, Farias M, Graham D, Powell AJ, Fulton DR, et al. Resource utilization after introduction of a standard-ized clinical assessment and management plan. Congenit Heart Dis 2010;5:374–81.

53. Froom P, Barak M. Cessation of dipstick urinalysis reflex testing and physician ordering behavior. Am J Clin Pathol 2012;137:486–9.

54. Gama R, Nightingale PG, Broughton PM, Peters M, Bradby GV, Berg J, et al. Feedback of laboratory usage and cost data to clinicians: does it alter requesting behaviour? Ann Clin Biochem 1991;28(Pt 2):143–9.

55. Gama R, Nightingale PG, Broughton PM, Peters M, Ratcliffe JG, Bradby GV, et al. Modifying the request behaviour of clinicians. J Clin Pathol 1992;45:248–9.

56. Goddard K, Austin SJ. Appropriate regulation of routine labora-tory testing can reduce the costs associated with patient stay in intensive care. Crit Care 2011;15:S47–8.

57. Golden WE, Pappas AA, Lavender RC. Financial unbun-dling reduces outpatient laboratory use. Arch Intern Med 1987;147:1045–8.

58. Gortmaker SL, Bickford AF, Mathewson HO, Dumbaugh K, Tirrell PC. A successful experiment to reduce unneces-sary laboratory use in a community hospital. Med Care 1988;26:631–42.

59. Grivell AR, Forgie HJ, Fraser CG, Berry MN. Effect of feedback to clinical staff of information on clinical biochemistry requesting patterns. Clin Chem 1981;27:1717–20.



60. Groopman DS, Powers RD. Effect of ‘standard order’ deletion on emergency department coagulation profile use. Ann Emerg Med 1992;21:524–7.

61. Gross PA, Van Antwerpen CL, Hess WA, Reilly KA. Use and abuse of blood cultures: program to limit use. Am J Infect Control 1988;16:114–7.

62. Haschke-Becher E, Totzke U, Afazel S, Johansson T, Schwarz M, Ladurner G, et al. Clinical decision rules for the use of liquor diagnostics in hospitalized neurology patients reduced costs without affecting clinical outcomes. Int J Technol Assess Health Care 2009;25:208–13.

63. Hutton HD, Drummond HS, Fryer AA. The rise and fall of C-reac-tive protein: managing demand within clinical biochemistry. Ann Clin Biochem 2009;46:155–8.

64. Jelinek GA, Banham ND. Reducing the use of plain abdominal radiographs in an emergency department. Arch Emerg Med 1990;7:241–5.

65. Kelly AM. Rationalising the ordering of blood cultures. Aust Health Rev 1998;21:245–50.

66. Kumwilaisak K, Noto A, Schmidt UH, Beck CI, Crimi C, Lewand-rowski K, et al. Effect of laboratory testing guidelines on the utilization of tests and order entries in a surgical intensive care unit. Crit Care Med 2008;36:2993–9.

67. Larocque BJ, Maykut RJ. Implementation of guidelines for preoperative laboratory investigations in patients scheduled to undergo elective surgery. Can J Surg 1994;37:397–401.

68. Levick DL, Stern G, Meyerhoefer CD, Levick A, Pucklavage D. Reducing unnecessary testing in a CPOE system through implementation of a targeted CDS intervention. BMC Med Inform Decis 2013;13:43.

69. Lewandrowski K, Bailey E, Dhanak E, Laposata M, Flood J. Mandatory laboratory consultation: how one hospital’s program reduced overuse of cardiac tests. Lab Med 1994;25:460–3.

70. Mancuso CA. Impact of new guidelines on physicians’ ordering of preoperative tests. J Gen Intern Med 1999;14:166–72.

71. Martin AR, Wolf MA, Thibodeau LA, Dzau V, Braunwald E. A trial of two strategies to modify the test-ordering behavior of medi-cal residents. N Engl J Med 1980;303:1330–6.

72. Marton KI, Tul V, Sox HC. Modifying test-ordering behavior in the outpatient medical clinic. A controlled trial of two educational interventions. Arch Intern Med 1985;145:816–21.

73. May TA, Clancy M, Critchfield J, Ebeling F, Enriquez A, Gallagher C, et al. Reducing unnecessary inpatient laboratory testing in a teaching hospital. Am J Clin Pathol 2006;126:200–6.

74. Mehari SM, Havill JH, Montgomery C. A written guideline imple-mentation can lead to reductions in laboratory testing in an intensive care unit. Anaesth Intensive Care 1997;25:33–7.

75. Meng QH, Zhu S, Booth C, Stevens L, Bertsch B, Qureshi M, et al. Impact of the cardiac troponin testing algorithm on exces-sive and inappropriate troponin test requests. Am J Clin Pathol 2006;126:195–9.

76. Merlani P, Garnerin P, Diby M, Ferring M, Ricou B. Quality improvement report: linking guideline to regular feedback to increase appropriate requests for clinical tests: blood gas analysis in intensive care. Br Med J 2001;323:620–4.

77. Meyer L, Schmeler K, Wallbillich J, Urbauer D, Soliman P, Frumo-vitz M, et al. Clinical practice guidelines decrease unnecessary Pap tests in survivors of gynecologic malignancies. Gynecol Oncol 2011;120:S30–1.

78. Miyakis S, Karamanof G, Liontos M, Mountokalakis TD. Factors contributing to inappropriate ordering of tests in an academic

medical department and the effect of an educational feedback strategy. Postgrad Med 2006;82:823–9.

79. Morris AJ, Wilson ML, Reller LB. Application of rejection criteria for stool ovum and parasite examinations. J Clin Microbiol 1992;30:3213–6.

80. Mutimer D, McCauley B, Nightingale P, Ryan M, Peters M, Neuberger J. Computerised protocols for laboratory investiga-tion and their effect on use of medical time and resources. J Clin Pathol 1992;45:572–4.

81. Nardella A, Pechet L, Snyder LM. Continuous improvement, quality control, and cost containment in clinical laboratory testing: effects of establishing and implementing guidelines for preoperative tests. Arch Pathol Lab Med 1995;119:518–22.

82. Neilson EG, Johnson KB, Rosenbloom ST, Dupont WD, Talbert D, Giuse DA, et al. The impact of peer management on test-order-ing behavior. Ann Intern Med 2004;141:196–204.

83. Nelson III RB. Teaching technologic restraint. An evaluation of a single session. Eval Health Prof 1978;1:21–8.

84. Nightingale PG, Peters M, Mutimer D, Neuberger JM. Effects of a computerised protocol management system on ordering of clini-cal tests. Qual Health Care 1994;3:23–8.

85. Nirel N, Rosen B, Sharon A, Samuel H, Cohen AD. The impact of an integrated hospital-community medical information system on quality of care and medical service utilisation in primary-care clinics. Inform Health Soc Ca 2011;36:63–74.

86. Pageler N, Longhurst C, Shin A, Adams E, Widen E, Barth R, et al. Using computerized provider order entry (CPOE) to promote sus-tained optimization of diagnostic radiologic service utilization. Crit Care Med 2009;37:A363.

87. Pageler NM, Franzon D, Longhurst CA, Wood M, Shin AY, Adams ES, et al. Embedding time-limited laboratory orders within com-puterized provider order entry reduces laboratory utilization. Pediatr Crit Care Med 2013;14:413–9.

88. Patel HD, Livsey SA, Swann RA, Bukhari SS. Can urine dipstick testing for urinary tract infection at point of care reduce labora-tory workload? J Clin Pathol 2005;58:951–4.

89. Pilon CS, Leathley M, London R, McLean S, Phang PT, Priestley R, et al. Practice guideline for arterial blood gas measurement in the intensive care unit decreases num-bers and increases appropriateness of tests. Crit Care Med 1997;25:1308–13.

90. Pop P, Winkens RA. A diagnostic centre for general practitioners: results of individual feedback on diagnostic actions. J R Coll Gen Pract 1989;39:507–8.

91. Power LM, Thackray NM. Reduction of preoperative investigations with the introduction of an anaesthetist-led preoperative assessment clinic. Anaesth Intensive Care 1999;27:481–8.

92. Powles L, Green J, Taylor E. Can redesigning a laboratory request form reduce the number of inappropriate PSA requests without compromising clinical outcome. Urology 2009;74:S148.

93. Prat G, Lefevre M, Nowak E, Tonnelier J-M, Renault A, L’Her E, et al. Impact of clinical guidelines to improve appropriateness of laboratory tests and chest radiographs. Intens Care Med 2009;35:1047–53.

94. Pysher TJ, Bach PR, Lowichik A, Petersen MD, Shields LH. Chemistry test ordering patterns after elimination of predefined multitest chemistry panels in a children’s hospital. Pediatr Dev Pathol 1999;2:446–53.

95. Ramoska EA. Information sharing can reduce laboratory use by emergency physicians. Am J Emerg Med 1998;16:34–6.



96. Ratnaike S, Hunt D, Eilermann LJ, Hazen R, Deam D. The inves-tigation of chest pain: audit and intervention. Med J Australia 1993;159:666–71.

97. Rhyne RL, Gehlbach SH. Effects of an educational feedback strategy on physician utilization of thyroid function panels. J Fam Practice 1979;8:1003–7.

98. Roberts DE, Bell DD, Ostryzniuk T, Dobson K, Oppenheimer L, Martens D, et al. Eliminating needless testing in intensive care – an information-based team management approach. Crit Care Med 1993;21:1452–8.

99. Rosenbloom ST, Chiu K-W, Byrne DW, Talbert DA, Neilson EG, Miller RA. Interventions to regulate ordering of serum magne-sium levels: report of an unintended consequence of decision support. J Am Med Informat Assoc 2005;12:546–53.

100. Santos IS, Bensenor IM, Machado JB, Fedeli LM, Lotufo PA. Intervention to reduce C-reactive protein determination requests for acute infections at an emergency department. Emerg Med J 2012;29:965–8.

101. Seguin P, Bleichner J, Grolier J, Guillou Y, Malledant Y. Effects of price information on test ordering in an intensive care unit. Intens Care Med 2002;28:332–5.

102. Shalev V, Chodick G, Heymann AD. Format change of a labora-tory test order form affects physician behavior. Int J Med Inform 2009;78:639–44.

103. Sorita A, Steinberg D, Leitman M, Luhan P, Burger A, Sivaprasad L. Impact of targeted individual feedback on pro-vider patterns of stat laboratory test ordering frequency. J Hosp Med 2011;6:S135–6.

104. Spray J, Brackbill M, Baker P. Evaluation of effect of electronic decision tree electrolyte replacement protocol on number of electrolyte labs ordered for patients in critical care. Crit Care Med 2012;40:248.

105. Stafford RS. Feedback intervention to reduce routine electrocardiogram use in primary care. Am Heart J 2003;145:979–85.

106. Sucov A, Bazarian JJ, deLahunta EA, Spillane L. Test ordering guidelines can alter ordering patterns in an academic emer-gency department. J Emerg Med 1999;17:391–7.

107. Sussman E, Goodwin P, Rosen H. Administrative change and diagnostic test use: the effect of eliminating standing orders. Med Care 1984;22:569–72.

108. Thompson RS, Kirz HL, Gold RA. Changes in physician behavior and cost savings associated with organizational recommen-dations on the use of ‘routine’ chest X rays and multichannel blood tests. Prev Med 1983;12:385–96.

109. Tierney WM, McDonald CJ, Hui SL, Martin DK. Computer predic-tions of abnormal test results. Effects on outpatient testing. J Am Med Assoc 1988;259:1194–8.

110. Tierney WM, Miller ME, McDonald CJ. The effect on test order-ing of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med 1990;322:1499–504.

111. Toubert ME, Chevret S, Cassinat B, Schlageter MH, Beressi JP, Rain JD. From guidelines to hospital practice: reducing inap-

propriate ordering of thyroid hormone and antibody tests. Eur J Endocrinol 2000;142:605–10.

112. van Wijk MA, van dL, Mosseveld M, Bohnen AM, van Bemmel JH. Assessment of decision support for blood test ordering in pri-mary care. a randomized trial. Ann Intern Med 2001;134:274–81.

113. Vardy DA, Simon T, Limoni Y, Kuperman O, Rabzon I, Cohen A, et al. The impact of structured laboratory routines in computer-ized medical records in a primary care service setting. J Med Syst 2005;29:619–26.

114. Verstappen WH, van der Weijden T, Sijbrandij J, Smeele I, Hermsen J, Grimshaw J, et al. Effect of a practice-based strat-egy on test ordering performance of primary care physicians: a randomized trial. J Am Med Assoc 2003;289:2407–12.

115. Wagholikar A, O’Dwyer J, Hansen D, Chu K. Observing effective-ness of pathology ordering controls in emergency depart-ments. Stud Health Tech Informat 2011;168:172–8.

116. Winkens RA, Pop P, Bugter-Maessen A, Grol RP, Kester AD, Beusmans GH, et al. Randomised controlled trial of routine individual feedback to improve rationality and reduce numbers of test requests. Lancet 1995;345:498–502.

117. Winkens RA, Pop P, Grol RP, Kester AD, Knottnerus JA. Effect of feedback on test ordering behaviour of general practitioners. Br Med J 1992;304:1093–6.

118. Wones RG. Failure of low-cost audits with feedback to reduce laboratory test utilization. Med Care 1987;25:78–82.

119. Zaat JO. Attempts to change test ordering behavior in laborato-ries. Ann Clin Biochem 1992;17:8–14.

120. Zaidi AK, Macone A, Goldmann AD. Impact of simple screening criteria on utilization of low-yield bacterial stool cultures in a Children’s Hospital. Pediatrics 1999;103: 1189–92.

121. Zia HM, Singh G. Optimization of utilization of serum protein analysis: role of the electronic medical record in promoting consultation by pathology. Am J Clin Pathol 2013;139:793–7.

122. Roshanov PS, You JJ, Dhaliwal J, Koff D, Mackay JA, Weise-Kelly L, et al. Can computerized clinical decision support systems improve practitioners’ diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review. Implement Sci 2011;6:88.

123. Buntinx F, Winkens R, Grol R, Knottnerus JA. Influencing diagnostic and preventive performance in ambulatory care by feedback and reminders. A review. Fam Pract Res J 1993;10:219–28.

124. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012;6:CD000259.

Supplemental Material: The online version of this article (DOI: 10.1515/cclm-2014-0778) offers supplementary material, available to authorized users.



Daniel Kobewka MD, FRCPC is a Master’s of science student and general internist at the Ottawa Hospital. He received his MD from the University of Alberta and completed his training in General Internal Medicine at the University of Ottawa. His research interests include quality improvement and end-of-life care.

Paul Ronksley PhD is a post-doctoral fellow in the department of clinical epidemiology at the Ottawa Hospital Research Institute. He received his PhD in Epidemiology from the University of Calgary and is currently funded by a CIHR post-doctoral fellowship award. His research interests are in chronic disease epidemiology and the use of administrative and population-based survey data to study access to care and health care utilization for patients with multi-morbidity.

Jennifer McKay is a Master’s student in the Epidemiology program at the University of Ottawa. Jennifer’s research interests are quality of care, health economics and the funding dynamics of the health care system.

Alan J. Forster MD, FRCPC, MSc is a general internist and Chief Quality and Performance Officer at the Ottawa Hospital. He is a Professor of Medicine at the University of Ottawa and Senior Sci-entist in the Clinical Epidemiology Program at the Ottawa Hospital Research Institute. His research focuses on patient safety and quality improvement. He has received several prestigious awards recognizing his work within the field of health services research, including a Career Scientist Award with the Ontario Ministry of Health and Long-Term Care and an Early Research Award and Ontario Research Fund grant from the Ontario Government’s Min-istry of Research and Innovation. He has performed seminal work evaluating the incidence of adverse events following discharge from hospital.

Carl van Walraven MD, FRCPC, MSc is a Senior Scientist with the Ottawa Hospital Research Institute (OHRI) and is the Site Director for the Institute for Clinical Evaluative Sciences uOttawa Site. He is a general internist at the Ottawa Hospital and is a Professor in the Faculties of Medicine and Epidemiology and Community Medicine at the University of Ottawa. Carl’s research focuses on quality improvement, continuity of care, and administrative database research. He has over 150 publications in peer-reviewed journals. Carl has also been the recipient of several prestigious awards recognizing his work within the field of health services research, including the Ontario Ministry of Health Career Scientist Award and the Premier Research Excellence Award.


Influence of educational, audit and feedback, system based, and incentive and penalty interventions...

Documents

Transcript of Influence of educational, audit and feedback, system based, and incentive and penalty interventions...