Computer-generated Patient Education Materials: Do They Affect Professional Practice? : A Systematic...

346 TREWEEK ET AL., Patient Education Materials and Professional Practice

JAMIAThe Practice of Informatics

The traditional consultation between a health profes-sional and a patient suffers from the flawed memo-ries of both participants. A study by Kitching1 foundthat, on average, patients forget half of what theywere told by a doctor within 5 min of leaving the con-sultation room. In addition, health professionals caneasily forget to pass on important information to thepatient. There is, therefore, scope for improvement incommunication between professionals and patients.

Affiliations of the authors: Department of Health ServicesResearch, Oslo, Norway (SPT, CG, ADO); Wairau Hospital,Blenheim, New Zealand (AP).

Correspondence and reprints: Shaun P. Treweek, PhD,Department of Health Services Research, Norwegian Directoratefor Health and Social Welfare, P.O. Box 8054 Dep, N-0031 Oslo,Norway; e-mail: <[email protected]>.

Received for publication: 12/21/01; accepted for publication:3/11/02.

Review ■

Computer-generated PatientEducation Materials: Do They Affect ProfessionalPractice?

A Systematic Review

A b s t r a c t A systematic search of seven electronic databases was done to identify randomized controlled trials that assessed the effect of computer-generated patient educationmaterial (PEM) on professional practice. Three studies met the authors’ criteria.

All three studies involved preventive care. All used a complex intervention of which computer-generated PEM was a major component. Improvements in practice were seen in all studies,although these gains were generally modest. One study showed improvement in patient outcomes.Mann-Whitney statistics calculated for the studies’ outcome measures ranged from 0.48 to 0.66,equivalent to risk differences of �4 to 32 percent.

Computer-generated PEM seems to have a small, positive effect on professional practice. The smallnumber of included studies and the complex nature of the interventions makes it difficult to drawconclusions about the ability of computer-generated PEM to change professional practice. Futurework should involve well-defined interventions that can be clearly evaluated in terms of effect andcost.

■ J Am Med Inform Assoc. 2002;9:346–358. DOI 10.1197/jamia.M1070.

SHAUN P. TREWEEK, PHD, CLAIRE GLENTON, ANDREW D. OXMAN, MSC, MD,ALISTER PENROSE, B COMM

This is especially true if patients are to be more activeparticipants in their own care. One solution proposedby Coulter et al.2 is to ensure that patients have accessto written or audiovisual material that supplementsthe material discussed during the consultation,which can also be used to guide questioning duringlater consultations.

This material could be delivered in several ways. Pre-printed booklets or leaflets are the most widely usedmethod,3 but some authors4,5 suggest that computer-ized clinical information systems should routinelyinclude education materials. These materials couldalso be tailored to individual patients. Anothermethod would be to use interactive educational soft-ware packages that allow the patient to direct thedelivery of information. These packages have beenused to combine audio, video, text, and graphics toprovide information on a wide range of topics.6 TheInternet is also an increasingly important source ofhealth information, although the quality of this infor-mation is currently variable.7

Are health professionals willing to make use of educa-tion materials and packages in their practice? Otherreviews have focused on patient behavior and out-comes.8,9 The health professional is, however, often thepoint of access to these materials, and an educationpackage should, at the very least, be good enough fora professional to want to make it available to patients.

Two potential barriers to increased use of educationmaterials by professionals are storage and accessproblems (booklets and leaflets take up space and arenot always to hand) and the need to keep the materi-als up to date.10 Wilson10 suggests computer-generat-ed materials as a solution, because these materials arenot subject to storage limitations and can be easilyupdated. Moreover, these materials are immediatelyavailable to the health care professional during a con-sultation, which is often not the case for preprintedleaflets. The computer system can also be used toremind the clinician about the patient information.11

However, use of computer-based materials may belimited by financial, administrative, and attitudinalbarriers in a health care organization or among itshealth care professionals.

As Mayberry and Mayberry12 point out, it is importantthat the effect of patient education material (PEM)should be assessed with the same rigor as other inter-ventions. In this review, we aim to systematicallydetermine the effects of computer-generated PEM onprofessional practice (our primary outcome) and onpatient outcomes.

ObjectivesTo examine the effect of computer-generated PEM asa way of changing the practice of health care profes-sionals and patient outcomes. Three primary com-parisons will be considered:

■ Comparison 1—providing computer-generatedmaterial compared to alternative material(preprinted leaflets, booklets, etc.).

■ Comparison 2—providing computer-generatedmaterial compared to no material.

■ Comparison 3—providing patient-specific comput-er-generated material compared to generic com-puter-generated material.

MethodsOur review is based on searching the following data-bases, from their start date to the end of 2000:■ MEDLINE (1966–2000)■ EMBASE (1980–2000)■ CINAHL (1982–2000)■ Best Evidence (1991–2000)■ Cochrane Collaboration EPOC specialized register

(1966–2000)■ Cochrane Controlled Trials Register (1947–2000)■ Science Citation Index (1987–2000)Individual search strategies were developed for eachdatabase (Appendix A). The search strategies werevery liberal and designed to pick up as many potentialarticles as possible. Publications in any language wereconsidered. The reference lists of review articles iden-tified through the searches were also checked. The ini-tial search identified 1,147 articles. A further 14 articleswere identified by checking the reference lists of thereview articles, giving a total of 1,161 articles.

Study Selection

Studies were considered eligible for inclusion in thereview if they met inclusion criteria in four cate-gories—types of intervention, types of studies, typesof participants, and types of outcome measures.

Types of Intervention

The intervention must be PEM that involves the use ofa computer in its storage and delivery, including inter-active packages. The information in such material maybe general or specific to individual patients. Educationmaterials used with and without the health profession-al being present were considered eligible.

347Journal of the American Medical Informatics Association Volume 9 Number 4 Jul / Aug 2002

TREWEEK ET AL., Patient Education Materials and Professional Practice348

To be included in this review, education interven-tions must be delivered in one of three ways—beforea consultation (e.g., information available on com-puters in the waiting room), during a consultation(e.g., a leaflet printed out and handed to the patient),or after a consultation (e.g., a computer-generatedtailored leaflet, based on the consultation, that is sentto the patient) Interventions that were simple patient reminders(e.g., a simple computer-generated letter suggestingthat a patient come in for a health check) wereexcluded, since we considered these to have no edu-cation content. Moreover, a number of reviews havealready considered these interventions13–15 or are inprogress.16

Types of Studies

Studies must be randomized controlled trials or con-trolled trials. The unit of allocation must be the healthprofessional or practice, since we are primarily inter-ested in professional practice.17

Types of Participants

A study must involve health care professionals whoare responsible for patient care. Studies involvingonly research staff without clinical responsibility forpatients were excluded.

Types of Outcome Measures

All objective measures of health care professionalpractice and patient outcome were considered.The primary outcome measure for this review iseffect on professional practice, including delivery ofinformation, cost, time spent with patients, prescrib-ing, referrals, and other clinical activities. Studieswith no measure of effect on professional practicewere excluded.

Abstracts for the 1,161 articles were scanned for rele-vance by at least two reviewers. Full-text copies of allpotentially relevant studies were obtained. Thesearticles were then considered for inclusion in thereview by at least two reviewers. Any discrepanciesbetween reviewers arising from the inclusion assess-ment were resolved by discussion.

Analysis

The Mann-Whitney statistic was used to compareoutcomes for intervention and control groups. Thisstatistic estimates the probability that a subject cho-sen at random and given the intervention would

have a better outcome than a subject similarly chosenand given the control. The statistic ranges from 0 to 1,with 0.5 meaning that both groups perform orrespond equally well.

For dichotomous data, the Mann-Whitney statisticwas calculated using:

M �W = 0.5 �0.5 (pi�pc)

where pi and pc are the proportions of intervention“successes” (e.g., a screening test was done) and con-trol successes, respectively. The quantity pi�pc isalso called the risk difference.

For continuous data, a Z statistic was calculated, andnormal tables were then used to find the Mann-Whitney statistic. The Z statistic was calculated usingthe equation:

Z = (x�i�x�c)/�(SDi )2�(SDc )2

where x�i�x�c is the difference between the interven-tion and control means, and SDi and SDc are thestandard deviations for the intervention and controlmeans, respectively. More details on the use of theMann-Whitney statistic to compare the results ofintervention studies can be found in Colditz et al.18

and Moses et al.19

Results

General Information

Of the 1,161 articles identified in our search, onlythree met all our inclusion criteria—Williams et al.20

(referred to here as “Williams”), Lowensteyn et al., 21

(“Lowensteyn”), and McPhee et al.22 (“McPhee”).Most articles were excluded because they discussedstudies that were randomized by patient and not byprofessional, even if the study aimed to effect achange in professional practice. Articles thatappeared relevant but were rejected after considera-tion of the full article text are listed in Appendix B.

Table 1 gives an overview of the Williams,Lowensteyn, and McPhee studies. All three are basedin primary care, Williams and McPhee aiming toimprove use of cancer screening, Lowensteyn aimingto support primary prevention of coronary heart dis-ease. The Williams and McPhee studies were done inthe United States, the Lowensteyn study in Canada.

Physicians participating in the McPhee andLowensteyn studies were not paid for their participa-tion. Physicians in the Williams study who alreadysubscribed to the Virginia Insurance Reciprocal mal-practice insurance scheme received a one-time 6 per-


cent reduction in their annual premium in return fortheir participation. Physicians involved in theMcPhee study were all based at fee-for-service prac-tices. Reimbursement arrangements at practicesinvolved in the two other studies were not clear,although one of the five strata in the Williams studyinvolved federally funded practices. None of thestudies adequately described the randomizationprocess.

Interventions

No study used an intervention involving only com-puter-generated PEM. Williams asked patients to usea touch-sensitive computer system to complete aquestionnaire on personal and family medical histo-ry and lifestyle. The computer then produced PEM,chart organizers, order sheets, and patient-specificreminders for the physician, the PEM being given tothe patient during the consultation. A nurse was alsoavailable to help practices during implementation ofthe system. Control practices did not receive the com-puter system until the end of the 1-year study period.

Lowensteyn collected risk factor information fromthe patient, and this information was then mailed toa central collection point. This center returned tworisk profiles to the practice, one of which was given

to the patient at a return visit about 2 weeks after theinitial visit. Control practices did not receive the tworisk profiles unless the patient was clinically re-eval-uated at a follow-up visit following a minimum3-month delay.

McPhee combined computer-generated PEM withbooklet-based PEM and a physician reminder sys-tem. The physician reminder system generated infor-mation on appropriate screening, assessment, andcounseling and other test information. Two copies ofthis information were generated, one each for thephysician and patient. Control practices receivednothing.

All three studies considered Comparison 2 (comput-er-generated PEM vs. no material) and provided esti-mates of the effects of computer-generated PEM onprofessional practice. One study (Lowensteyn) alsoprovided estimates of effect on patient outcomes.

Williams reported a significant increase in the com-pletion of screening mammography (8.8 percent dif-ference between intervention and control) and clini-cal breast examinations (8.3 percent) in women50 years of age and older. These results were signifi-cant at the p �0.05 level. There were no significantchanges for the five remaining screening tests.

Table 1 ■

Overview of the Three Included Studies

Study No. in Length Study Design Age of Disease or Event Main Process Effect Study of Study Patients and Intervention/Control Measured

Lowensteyn 253 general 1 yr Randomized controlled 30–74 yr Prevention of coronary heart Ratio of high-risk to et al., 1998 practitioners trial by general practice, disease (CHD). Computer- low-risk patients seen

blocking for presence generated CHD risk profile at 3-mo follow-upor absence ofmedical PEM given to patient 2 wkschool in community after first consultation vs.

profile given at 3-mofollow-up.

McPhee 40 general 2 yr Randomized controlled 64% Prevention of cancer. Average change in et al., 1991 practitioners trial with two groups >50 yr Computer-generated the proportion of

screening reminder for eligible patients patient and doctor plus having cancer booklet-based PEM vs. screening testscurrent practice

Williams 58 primary 1 yr Randomized controlled >18 yr Prevention of cancers of the Average change in et al., 1998 care practices, trail by general practice, breast, cervix, colon and the proportion of

average 2.3 stratified by type (e.g., rectum, and oral cavity. PEM eligible patients physicians high or low link to and chart reminders having cancerper practice medical college); total generated by a touch- screening tests

of five groups. sensitive computer system plus nurse liaison vs. currentpractice

ABBREVIATION: PEM indicates patient education material.

Among patients who did not have a health mainte-nance examination (HME), use of the touch-screensystem showed a significant increase (p �0.05) in thenumber of patients who had fecal occult blood tests(3.9 percent increase). Patients who did have an HMEwere associated with higher (p �0.01) proportions ofscreening mammographies (30.3 percent increase)and clinical breast examinations (32.4 percent).Williams also reported that patients who used thetouch-screen system had a significantly higher rate ofcompletion of screening tests than did non-users forall seven screening tests. However, not all non-userswere in the control group, which may lead to a biasedresult in favor of the intervention (see Discussion).

Lowensteyn found that the coronary heart diseaserisk profile given to both patients and physicians ledto a significant increase in the proportion of high-riskpatients being reassessed at 3 months, compared

with low-risk patients. The difference in the likeli-hood ratio (ratio of high-risk patients to low-riskpatients returning for follow-up) between the riskprofile group and the control group was 0.46 (95%confidence interval, 0.08–0.87). Lowensteyn had adifference in the unit of allocation and analysis,which is a methodological weakness, although theanalysis of patient outcome does take this intoaccount. This was not done for the risk likelihood cal-culation, making it unclear whether statistical signif-icance was actually achieved for this result.

McPhee developed a percentage “performancescore” for completion of screening tests and assess-ments, and the intervention package led to increasesof between 8 and 34 percent in completion. Theseincreases were significant for five tests or assess-ments (rectal examination, Papanicolaou smear,smoking assessment, smoking counseling, and diet


Table 2 ■

Summary of the Mann-Whitney Statistics for Each of the Three Included Studies

Study Outcome Measures Mann-Whitney Statistic Published p Values

Lowensteyn et al., 1988 High-risk likelihood ratio Not possible 95%CI, 0.08–0.87Total C (mmol/l) 0.61 p=0.05HDL C 0.48 NSLDL C 0.61 p=0.05Total C/HDL C ratio 0.57 p=0.05Systolic BP 0.52 NS Diastolic BP 0.53 NS Body mass index 0.50 NS Smokers 0.50 NS 8-yr coronary risk 0.54 p<0.01 Cardiovascular age 0.51 p<0.01

McPhee et al., 1991 Stool occult blook test 0.56 p<0.01Digital rectal examination 0.55 p<0.05Flexible sigmoidoscopy 0.57 NSPapanicolaou smear 0.66 p<0.05Pelvic examination 0.56 p<0.01Breast examination 0.54 NSMammography 0.52 NSSmoking assessment 0.54 p<0.05Smoking counseling 0.61 p<0.05Diet assessment 0.56 p<0.05Diet counseling 0.57 p<0.01

Williams et al., 1998 Screening mammography 0.54 p�0.05Clinical breast examination 0.54 p�0.05Digital rectal examination 0.51 NSFecal occult blood test 0.51 NSFlexible sigmoidoscopy 0.51 NSPapanicolaou smear 0.51 NSOral cavity examination 0.51 NS

ABBREVIATIONS: 95%CI indicates 95% confidence interval; C, cholesterol; HDL C, high-density lipoprotein; LDL, low-density lipoprotein; BP,blood pressure; NS, not significant.

assessment) at the p �0.05 level and for three (stooloccult-blood test, pelvic examination, and diet coun-seling) at the p �0.01 level. The results for sigmoi-doscopy, breast examination, and mammographywere not significant.

The results were then adjusted to remove the effects ofpatient characteristics, using patients as the unit ofanalysis. As mentioned above, using patients as theunit of analysis when allocation was by physician is amethodological weakness. The adjusted results werebroadly similar to the unadjusted results, but withhigher levels of significance generally being claimed.

Neither Williams nor McPhee reported patient out-comes. Lowensteyn found that patients in the inter-vention group demonstrated greater reductions(p �0.05) in total cholesterol, low-density lipoproteincholesterol, and the total cholesterol/high-densitylipoprotein cholesterol ratio. This resulted in a signif-icant improvement (p �0.01) in both cardiovascularage and predicted 8-year coronary risk, comparedwith the control group. As mentioned earlier, theunits of allocation and analysis are different in thisstudy. The authors did, however, include physicianas a variable in an analysis of variance model whenconsidering patient outcomes.

Table 2 summarizes the Mann-Whitney statistics forthe three studies, together with the published p val-ues for each outcome. A Mann-Whitney statistic hasnot been calculated for the Lowensteyn “high-risklikelihood ratio,” because different units of allocationand analysis were used for this outcome. The spread-sheet used to calculate the Mann-Whitney statistics isavailable on request from the authors of this article.

Figure 1 shows a plot of the Mann-Whitney statisticsfor each study’s major outcome measures. The pointsize is proportional to the sample size (actually1/variance; see Figure 1). To make differentiationbetween the points easier, the x-axis has been startedat 0.4, not 0.

The Mann-Whitney statistics were largely in favor ofthe intervention (i.e., greater than 0.50), ranging from0.48 (equivalent to a risk difference [RD] of �4 per-cent) to 0.66 (RD, 32 percent). The mean Mann-Whitney statistics were all in favor of the interven-tion: Williams had a mean of 0.52 (RD, 4 percent),McPhee 0.57 (RD, 14 percent), and Lowensteyn 0.54(RD, 8 percent).

Discussion

This review has examined the effects of computer-generated PEM on the practice of health profession-als. The small number of studies included in thisreview is a clear indication that the effect of comput-er-generated PEM on professional practice is anaspect of PEM that is rarely evaluated. In addition,several studies that did assess the effect of computergenerated PEM on professional practice failed to ran-domize by professional or practice, a methodologicalerror that tends to overestimate any effect.17,23 Oneincluded study that was randomized by practiceused the patient as the unit of analysis, which canlead to over-narrow confidence intervals and spuri-ously significant findings.23

It is, therefore, difficult to draw conclusions about theeffects of computer-generated PEM on professionalpractice. The Williams, Lowensteyn, and McPhee


F i g u r e 1 Mann-Whitneystatistics for each of the threeincluded studies. The size ofeach point is proportional to1/variance, where variance isgiven by

(ni �nc �1)/12ninc

and where ni and nc are thesample sizes for the interven-tion and control groups,respectively. The crosshairrepresents the mean Mann-Whitney statistic for eachstudy.

TREWEEK ET AL., Patient Education Materials and Professional Practice352T

able

3■

Ove

rvie

w o

f th

e E

ight

Stu

die

s th

at D

id N

ot M

eet t

he R

and

omiz

atio

n C

rite

ria

but D

id M

eet t

he T

hree

Oth

er I

nclu

sion

Cri

teri

a

Stud

yN

o. in

L

engt

h St

udy

Des

ign

Age

of

Dis

ease

or

Eve

ntM

ain

Proc

ess

Eff

ect

Feat

ures

of

Syst

emSt

udy

of S

tud

yPa

tien

ts (

yr)

and

Int

erve

ntio

n/C

ontr

olM

easu

red

Cam

pbel

l57

9, s

elec

ted

6 m

oR

and

omiz

ed c

ontr

olle

d18

–70

Cer

vica

l can

cer.

Tou

ch

Upt

ake

of P

apan

icol

aou

Tou

chsc

reen

sys

tem

loca

ted

inet

al.,

199

7by

sta

fftr

ial w

ith

two

grou

pssc

reen

sur

vey

in w

aiti

ng

scre

enin

g te

sts

wai

ting

roo

m, p

atie

nts

give

n a

room

gen

erat

es P

EM

vs.

pr

inte

d s

umm

ary

of th

eir

resp

onse

s.us

e of

touc

h sc

reen

sy

stem

wit

hout

PE

MG

raha

m87

5, s

elf-

9 m

oR

and

omiz

ed c

ontr

olle

d

30 (

mea

n)Pr

enat

al te

sts.

Tou

chU

ptak

e of

pre

nata

l tes

tsT

ouch

scr

een

syst

em w

ith

vid

eo

et a

l., 2

000

sele

cted

tria

l wit

h tw

o gr

oups

scre

en in

form

atio

ncl

ips

and

voi

ce-o

vers

. sys

tem

sy

stem

plu

s in

form

atio

n lo

cate

d in

wai

ting

are

a. P

riva

cy

leaf

let v

s. le

afle

t onl

yen

hanc

ed b

y th

e av

aila

bilit

y of

m

icro

phon

e he

adse

ts.

Hom

er

137,

sel

f1

yrR

and

omiz

ed c

ontr

olle

d

3–12

Ast

hma.

Ed

ucat

iona

l E

mer

genc

y d

epar

tmen

t V

ideo

gam

e th

at in

corp

orat

ed

et a

l., 2

000

sele

cted

(by

tria

l wit

h tw

o gr

oups

asth

ma-

spec

ific

com

pute

r vi

sits

and

acu

te o

ffic

e in

form

atio

n on

the

man

agem

ent o

fpa

rent

s)ga

me

vs. p

rint

ed m

ater

ials

visi

ts f

or a

sthm

a.as

thm

a.Jo

nes

et a

l.,52

5, s

elf-

3 m

oR

and

omiz

ed c

ontr

olle

dR

ange

not

Can

cer.

Com

pute

r-C

ost o

f pr

ovid

ing

Tou

ch s

cree

n sy

stem

in tw

o 19

99se

lect

edtr

ial w

ith

thre

e gr

oups

give

nge

nera

ted

PE

M th

at w

as

info

rmat

ion

vari

ants

, per

sona

lized

and

gen

eral

. ge

nera

l, pe

rson

aliz

ed, o

r Pe

rson

aliz

ed s

yste

m li

nked

to

both

vs.

info

rmat

ion

elec

tron

ic m

edic

al r

ecor

d.

book

let

Nav

igat

ion

syst

em s

imila

r to

that

us

ed o

n W

eb p

ages

. Pat

ient

s w

ere

sent

pri

nted

sum

mar

ies

of

info

rmat

ion

they

had

vie

wed

.L

iaw

et a

l.,

364,

261

6

mo

Ran

dom

ized

con

trol

led

25–6

5G

ener

al h

ealt

h pr

omot

ion.

Att

end

ance

at a

hea

lth

Use

d h

ealt

h pr

omot

ion

tem

plat

e on

19

96re

crui

ted

tria

l wit

h fo

ur g

roup

sFo

ur in

terv

enti

ons—

HE

A

chec

ka

stan

dar

d e

lect

roni

c m

edic

al r

ecor

d

by m

ail,

pers

onal

hea

lth

reco

rd

syst

em. P

atie

nts

give

n a

prin

ted

10

3 by

(PH

R),

sum

mar

y of

su

mm

ary

of th

eir

med

ical

rec

ord

.op

port

unis

tic

com

pute

r-ba

sed

med

ical

co

ntac

t at

reco

rd (

CM

R),

both

PH

R

clin

ican

d C

MR

, or

neit

her.

Mor

gan

240,

sel

f-6

mo

Ran

dom

ized

con

trol

led

60 (

mea

n)Is

chem

ic h

eart

dis

ease

.U

ptak

e of

rev

ascu

lari

zati

on

Inte

ract

ive

vid

eo s

yste

m p

rovi

din

g et

al.,

200

0se

lect

edtr

ial w

ith

two

grou

psin

tera

ctiv

e vi

deo

dis

k pr

oced

ures

risk

and

ben

efit

info

rmat

ion

dec

isio

n ai

d p

lus

rega

rdin

g th

ree

trea

tmen

t br

ochu

re v

s. c

urre

nt

alte

rnat

ives

. Inf

orm

atio

n ta

ilore

d to

pr

acti

cepa

tien

t’s m

edic

al a

nd p

erso

nal

circ

umst

ance

s. I

nclu

ded

film

clip

s of

pat

ient

s w

ho to

ok th

e va

riou

s tr

eatm

ent a

lter

nati

ves.

Osm

an

801,

sel

ecte

d1

yrPr

agm

atic

ran

dom

ized

?16

Ast

hma.

Fou

r co

mpu

ter-

Num

ber

of h

ospi

tal

Use

d s

impl

e sp

read

shee

t and

mai

let

al.

1994

by s

taff

gene

rate

d P

EM

boo

klet

s ad

mis

sion

s, c

onsu

ltat

ions

m

erge

sof

twar

e to

gen

erat

e pl

us n

orm

al v

erba

l w

ith

gene

ral p

ract

itio

ner

pers

onal

ized

info

rmat

ion

for

advi

ce v

s. n

orm

al v

erba

l fo

r as

thm

a an

d p

resc

rip-

pati

ents

.ad

vice

alo

ne.

tion

of

bron

chod

ilato

rsan

d in

hale

d s

tero

ids

Rub

in65

, sel

f-1

yrR

and

omiz

ed c

ontr

olle

d7–

12A

sthm

a. E

duc

atio

nal

Num

ber

of a

cute

vis

its

Vid

eo g

ame

that

inco

rpor

ated

et

al.,

198

6se

lect

ed (

bytr

ial w

ith

two

grou

psas

thm

a-sp

ecif

ic c

ompu

ter

info

rmat

ion

on th

e m

anag

emen

tpa

rent

s)ga

me

vs. n

orm

al

of a

sthm

a.co

mpu

ter

gam

e.

AB

BR

EV

IAT

ION

S: P

EM

ind

icat

es p

atie

nt e

duc

atio

n m

ater

ials

; HE

A, H

ealt

h E

duc

atio

n A

utho

rity

.


studies report positive effects, and most of the Mann-Whitney statistics given in Table 2 favor the interven-tion. Table 2 and Figure 1 suggest that for these threeNorth American studies, involving similar groups ofpatients and health professionals and using broadlysimilar interventions, PEM can contribute to a smallimprovement in professional and patient outcomes.Taking the Mann-Whitney statistics together, the riskdifference between PEM intervention and control isaround 8 to 10 percent.

Williams found a significant increase (about 8 per-cent) in the proportion of women older than 50 yearswho received screening mammography and clinicalbreast examinations. These authors also comparedpatients who had used the touch-screen system withpatients who had not. Patients who used the systemshowed a significant increase in the completion of allseven cancer screening tests. However, this result islikely to be optimistic, since not all non-users were inthe control group.

At practices in which the touch-screen system wasplaced in the waiting area (20 of 29), patients who usedthe system were self-selected. At the remaining ninepractices, the system was used by a group of patientsselected by the practice. Both methods are likely toresult in bias regarding use of the system, and non-users at intervention practices should be consideredwhen assessing the intervention’s effect. The interven-tion also involved a physician reminder, and it is diffi-cult to disentangle the effect of the reminder from thecomputer-generated PEM. The positive effect of physi-cian reminders is well documented.13,14 This problem isperhaps more acute with the McPhee study.

The Lowensteyn study, which combined computer-generated PEM with a physician reminder system formanaging coronary risk, achieved a significantincrease in the ratio of high-risk to low-risk patientsreturning for follow-up. However, the unit of alloca-tion was different from the unit of analysis. It is like-ly that the result would have been less favorable hadthe data been analyzed (correctly) using the 129physicians who actively participated instead of the958 patients. An alternative for the original authorswould have been to use an intra-cluster correlationcoefficient, which would describe the extent to whichpatients within a cluster (a practice) are truly inde-pendent of each other.

We were interested primarily in the effect of PEM onprofessional practice. Eight studies were thereforeexcluded from our review because patients ratherthan professionals were randomized. For studies that

aim to measure this effect, randomization by profes-sional is appropriate because randomizing by patientwould lead to the same professionals being in boththe experimental and the control groups.

Randomizing by patient would be appropriate onlyif the PEM was delivered independent of the profes-sional and the aim of a study was to measure changesin patient rather than professional behavior. Internet-based PEM obtained without professional inputwould ensure delivery independent of professionalsand prevent contamination in that respect, but thesame professionals would still be exposed (via thepatients) to both the experimental and control inter-ventions. Thus, randomization at the level of the pro-fessional is the only way to reliably measure effectson professional practice.

To illustrate other computer-based PEM systems,however, we thought it would be interesting to con-sider these eight articles in the Discussion; they arelisted in Table 3. Of these, four consideredComparison 1,24–27 five considered Comparison 2,27–31

and two considered Comparison 3.25,27 Four studiesreported a significant positive change in an outcomerelated to professional practice,24,26–28 whereas the fourremaining studies reported no significant change inthese outcomes. None of the eight studies reportedobjective measures of patient outcome; patients’knowledge of their condition was the most commonmeasure reported.

It must be emphasized that these studies are notincluded to provide quantitative evidence to supportor refute the use of particular techniques. As men-tioned earlier, studies that assess the effect of an inter-vention on professional practice should be random-ized by professional or practice, not by patient.17 Evenif the focus is patient outcome, if patients are drawnfrom several clinical sites, then clustering effects mustbe considered. These eight studies do, however, pro-vide further (qualitative) food for thought.

None of the three studies meeting our full inclusioncriteria considered cost. It is therefore uncertainwhether the effects of the interventions are worth thecosts involved. One of the studies that did not meetour randomization criteria, that by Jones et al.,25 didmention cost. These authors found that, in theabsence of an electronic medical record system, theirpersonalized computer-based PEM system wouldcost more than nine times that of their general com-puter-based PEM. With an electronic record, the costof the personalized system was similar to that of thegeneral system. The cost of the general computer-


based PEM system was about 40 percent the cost offull access to booklets and would be less than the costof booklets within the first year. This suggests thatthe cost of computer-based PEM may be modest andthat even small positive effects may be worth theinvestment, particularly in the presence of an elec-tronic medical record.

Personalized systems were used in all three of thestudies that met our full inclusion criteria. In addition,four of the six studies that did not meet our random-ization inclusion criteria24,25,27,28 used personalizedsystems. Only two studies25,27 compared these withgeneral computer-based PEM, and both studiesreported benefits from use of the personalized system.As Jones et al.25 point out, in the presence of an elec-tronic medical record, such systems are of similar costto general systems, and a personalized approach maybe worth considering in future computer-based PEMsystems. The evidence supporting a positive effect onprofessional practice is, however, currently weak.

Making computer-based PEM part of an interventionpackage, rather than the sole intervention, may leadto a greater effect on practice. Combining PEM sys-tems with reminder systems may be especially use-ful, since reminders have been found to have a sig-nificant positive effect on professional practice.13–15

This approach was taken in all three studies that metour full inclusion criteria, although it is difficult towholeheartedly endorse the approach on the basis ofthe results of these three studies.

Conclusions

Implications for Practice

Computer-generated PEM appears to have a small,positive effect on professional practice. However,there is currently scant evidence to support this con-clusion. Combination with other interventions, par-ticularly patient and physician reminder systemsmay also be promising, but further work is requiredto ascertain which interventions should be combinedand in which circumstances. The cost effectiveness ofcomputer-generated PEM has been inadequatelyassessed.

Until further evidence becomes available, purchasersshould be cautious about using computer-generatedPEM as a vehicle for large-scale change in profes-sional practice. Other considerations, however, suchas improved patient participation in decision makingand increased patient satisfaction, may make suchinvestment worthwhile. In decisions about whether

investment in computer-generated PEM meets aprovider’s health care strategy, the current reviewshould be considered with reviews that take a morepatient-centered focus.8,9

Implications for Future Research

It is surprising that only three studies can be said toadequately address the effects of computer-generat-ed PEM on professional practice. More and betterPEM is a feature of many national health policies,and the attitudes of health professionals toward this,and the potential effect of PEM on their practices, isstill largely unknown. Future work should addressthese issues, should be of higher methodologicalquality, should have a well-defined interventionpackage that can be clearly evaluated, and shouldconsider cost. How such systems can be implement-ed into routine care should also be considered.

Internet-based services are already an importantsource of PEM, and the importance of the Internet asa source of PEM is likely to increase. The effect ofthese systems on professional practice and patientoutcomes should be evaluated. Since computer-gen-erated PEM is likely to have, at best, a modest effecton professional practice and patient outcome, devel-opers should aim to make their systems cheap,preferably linked to existing electronic medicalrecord systems.

References ■

1. Kitching JB. Patient information leaflets: the state of the art. J RSoc Med. 1990;83:298–300.

2. Coulter A, Entwistle V, Gilbert D. Sharing decisions withpatients: is the information good enough? BMJ. 1999;318:318–22.

3. Meredith C, Symonds P, Webster L, et al. Information needs ofcancer patients in west Scotland: cross-sectional survey ofpatients’ views. BMJ. 1996;313:724–6.

4. Kenny T, Wilson RG, Purves IN, et al. A PIL for every ill?Patient information leaflets (PILs): a review of past, presentand future use. Fam Pract. 1998; 5:471–9.

5. Hurtado MP, Swift EK, Corrigan JM (eds). Crossing theQuality Chasm: A New Health System for the 21st Century.Institute of Medicine, Committee on the National QualityReport on Health Care Delivery. Washington, DC: NationalAcademy Press, 2001.

6. Krishna S, Balas A, Spencer DC, Griffin JZ, Boren SA. Clinicaltrials of interactive computerized patient education: implica-tions for family practice. J Fam Pract. 1997;45:25–33.

7. Sheppard S, Charnock D, Gann B. Helping patients accesshigh-quality health information. BMJ. 1999;319:764–6.

8. Revere D, Dunbar, PJ. Review of computer-generated outpa-tient health behaviour interventions: clinical encounters “inabsentia.” J Am Med Inform Assoc. 2001;8:62–79.

9. Lewis D. Computer-based approaches to patient education: areview of the literature. J Am Med Inform Assoc. 1999;6:272–82.

10. Wilson R, Kenny T, Clark J, et al. PILs Project Summary


Report: Ensuring the Readability, Understandability andEfficacy of Patient Information Leaflets. Newcastle, UK:Sowerby Centre for Health Informatics, 1998.

11. Gordon RB, Grimshaw JM, Eccles M, Rowe RE, Wyatt JC. On-screen computer reminders: effects on professional practice andhealth care outcomes (protocol for a Cochrane Review). TheCochrane Library, Issue 3. Oxford, UK: Update Software, 1999.

12. Mayberry JF, Mayberry MK. Effective instructions forpatients. J R Coll Physicians Lond. 1996;30(3):205–8.

13. Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16randomized controlled trials to evaluate computer-based clin-ical reminder systems for preventive care in the ambulatorysetting. J Am Med Inform Assoc. 1996;3:399–409.

14. Balas EA, Austin SM, Mitchell JA, Ewigman BG, Bopp KD,Brown GD. The clinical value of computerised informationservices: a review of 98 randomized clinical trials. Arch FamMed. 1996;5:271–8.

15. Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects ofcomputer-based clinical decision support systems on clinicianperformance and patient outcome: a critical appraisal ofresearch. Ann Intern Med. 1994;120:135–42.

16. Gorman PN, Redfern C, Liaw T, et al. Reminders II: computergenerated paper reminders—their effectiveness in improvinghealth care professional practice and patient outcomes (proto-col for a Cochrane Review). The Cochrane Library, Issue 2.Oxford, UK: Update Software, 2001.

17. Wood J, Freemantle N. Choosing an appropriate unit of analy-sis in trials of interventions that attempt to influence practice.J Health Serv Res Policy. 1999;4:44–8.

18. Colditz GA, Miller JN, Mosteller F. Measuring gain in the eval-uation of medical technology. Intl J Technol Assess HealthCare. 1988;4:637–42.

19. Moses LE, Emerson JD, Hosseini H. Analyzing data fromordered categories. N Engl J Med. 1984;311:442–8.

20. Williams RB, Boles M, Johnson RE. A patient-initiated systemfor preventive health care: a randomized trial in community-based primary care practices. Arch Fam Med. 1998;7:338–45.

21. Lowensteyn I, Joseph L, Levinton C, Abrahamowicz M, Steinert

Y, Grover S. Can computerized risk profiles help patientsimprove their coronary risk? The results of the coronary healthassessment study (CHAS). Prev Med. 1998;27:730–7.

22 McPhee SJ, Bird JA, Fordham D, Rodnick JE, Osborn EH.Promoting cancer prevention activities by primary care physi-cians. JAMA. 1991;266:538–44.

23. Campbell MK, Grimshaw JM. Cluster randomized trials: timefor improvement. The implications of adopting a clusterdesign are still largely being ignored. BMJ. 1998;317:1171.

24. Osman LM, Abdalla MI, Beattie JAG, et al. Reducing hospitaladmission through computer supported education for asthmapatients. BMJ. 1994;308:568–71.

25. Jones R, Pearson J, McGregor S, et al. Randomized trial of per-sonalized computer based information for cancer patients.BMJ. 1999;319:1241–7.

26. Graham W, Smith P, Kamal A, Fitzmaurice A, Smith N,Hamilton N. Randomized controlled trial comparing effective-ness of touch screen system with leaflet for providing womenwith information on prenatal tests. BMJ. 2000;320:155–9.

27. Liaw T, Lawrence M, Rendell J. The effect of a computer-gener-ated patient-held medical record summary and/or written per-sonal health record on patients’ attitudes, knowledge and behav-iour concerning health promotion. Fam Pract. 1996;13:289–3.

28. Campbell E, Peterkin D, Abbott R, Rogers J. Encouragingunderscreened women to have cervical cancer screening: theeffectiveness of a computer strategy. Prev Med. 1997;26:801–7.

29. Rubin DH, Leventhal JM, Sadcock RT, et al. Educational inter-vention by computer in childhood asthma: a randomized clin-ical trial testing the use of a new teaching intervention inchildhood asthma. Pediatrics. 1986;77:1–10.

30. Homer C, Susskind O, Hillel RA, et al. An evaluation of aninnovative multimedia educational software program forasthma management: report of a randomized, controlled trial.Pediatrics. 2000;106:210–5.

31. Morgan M, Deber RB, Llewellyn-Thomas HA, et al.Randomized, controlled trial of an interactive videodisc deci-sion aid for patients with ischemic heart disease. J Gen InternMed. 2000;15:685–93.

Effective Practice and Organization of Care (EPOC)specialized register search strategy:

comput* [text term]

Cochrane Controlled Trials Register (CCTR) searchstrategy:1. comput* [text term]2. Patient-education [exploded MeSH term]3. 1 and 2

MeSH terms for medline search strategy (throughInternet Grateful Med):1. Patient Education

2. Health Education3. Delivery of Health Care4. Patient Participation 5. Health Promotion6. Preventive Medicine7. Professional-Patient relations8. Consumer Participation9. Computers

10. comput* [text term]11. Computer-Assisted Instruction 12. Therapy, Computer-Assisted

Appendix A

SEARCH STRATEGIES FOR THE REVIEW


Appendix B

STUDIES EXCLUDED FROM THE REVIEW

Aisen ML. Justifying neurorehabilitation: a few steps forward.Neurology. 1999;52:8–10.

Albisser AM, Harris RI, Sakkal S, Parson ID, Chao SC. Diabetesintervention in the information age. Med Inform (Lond).1996;21:297–316.

Arthur HM, Daniels C, McKelvie R, Hirsh J, Rush B. Effect of a pre-operative intervention on preoperative and postoperative out-comes in low-risk patients awaiting elective coronary arterybypass graft surgery: a randomized, controlled trial. AnnIntern Med. 2000;133(4):253–62.

Balas EA, Jaffrey F, Kuperman GJ, et al. Electronic communica-tions with patients: evaluation of distance medicine technolo-gy. JAMA. 1997;278:152–9.

Bartholomew LK, Gold RS, Parcel GS, et al. Watch, Discover,Think, and Act: evaluation of computer-assisted instruction toimprove asthma self-management in inner-city children.Patient Educ Couns. 2000;39(2–3):269–80

Beyth RJ, Quinn L, Landefeld CS. A multicomponent interventionto prevent major bleeding complications in older patientsreceiving warfarin: a randomized, controlled trial. Ann InternMed. 2000;133:687–95.

Bird JA, McPhee SJ, Jenkins C, Fordham D. Three strategies to pro-mote cancer screening. How feasible is wide-scale implemen-tation? Med Care. 1990;28:1005–12.

Brownbridge G, Evans A, Fitter M, Platts M. An interactive com-puterized protocol for the management of hypertension:effects on the general practitioner’s clinical behaviour. J R CollGen Pract. 1986;36:198–202.

Brownbridge G, Lilford RJ, Tindale-Biscoe S. Use of a computer totake booking histories in a hospital antenatal clinic: accept-

ability to midwives and patients and effects on the midwife-patient interaction. Med Care. 1988;26:474–87.

Calltorp J. Satsa pa experiment och oberoende organisationer. Nikan skapa varldens basta informationssystem. (Investing inexperiments and independent organizations. You can createthe world’s best information system). Lakartidningen.1988;85:2713–5.

Casebeer L, Roesener GH. Patient informatics: using a computer-ized system to monitor patient compliance in the treatment ofhypertension. Medinfo. 1995;8:1500–2.

Clark M, Ghandour G, Miller NH, Taylor CB, Bandura A, DeBuskRF. Development and evaluation of a computer-based systemfor dietary management of hyperlipidemia. J Am Diet Assoc.1997;97:146–50.

Coe FL, Norton E, Oparil S, Tatar A, Pullman TN. A prospectivecontrolled study of hypertension treatment by computer andphysician. In: McCormick EH (ed). Proceedings of the 3rdIllinois Conference on Medical Information Systems. Chicago,Ill.: University of Illinois, 1976:206–17.

Coe FL, Norton E, Oparil S, Tatar A, Pullman TN. Treatment ofhypertension by computer and physician: a prospective con-trolled study. J Chronic Dis. 1977;30:81–92.

Cramer JA, Rosenheck R. Enhancing medication compliance forpeople with serious mental illness. J Nerv Ment Dis.1999;187:53–5.

Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physi-cian performance: a systematic review of the effect of continu-ing medical education strategies. JAMA. 1995;274:700–5.

Davis NA, Lewis MJ, Rimer BK, Harvey CM, Koplan JP.Evaluation of a phone intervention to promote mammogra-

13. Medical Informatics Applications14. Medical Informatics15. Clinical Trial [exploded Publication Type]16. (1 or 2 or 3 or 4 or 5 or 6 or 7 or 8) and

(9 or 10 or 11 or 12 or 13 or 14) and 15

MeSH terms for EMBASE search strategy (throughOvid):1. exp Patient Education/2. exp Health Promotion/3. exp Patient counseling/ or Patient guidance/

or Patient information/ or Patient education4. exp ‘automation, computers and computer

applications’/ or Computer/5. exp Clinical trial/6. (1 or 2 or 3) and 4 and 5

Best Evidence search strategy (through Ovid):Comput$ [text terms]

MeSH terms for Cinahl search strategy (throughOvid):

1. exp Patient education/

2. exp ‘Computers and computerization’/

3. exp Experimental studies/

4. 1 and 2 and 3

Science Citation Index search strategy (throughBibSys):

1. computer (text word)

2. patient (text word)

3. education (text word)

4. 1 and 2 and 3

phy in a managed care plan. Am J Health Promot.1997;11:247–9.

Degner L, Goodwin C, Carriere KC, Davison J. Clinical trial of acomputer-assisted intervention for women with breast cancer:a study in progress. Can Oncol Nurs J, 1997;7:120.

Dickinson JC, Warshaw GA, Gehlbach SH, Bobula JA, MuhlbaierLH, Parkerson GR Jr. Improving hypertension control: impactof computer feedback and physician education. Med Care.1981;19:843–54.

Douglas TS, Mann NH, Hodge AL. Evaluation of preoperativepatient education and computer-assisted patient instruction. JSpinal Dis. 1998;11:29–35.

Edworthy SM, Devins GM. Improving medication adherencethrough patient education distinguishing between appropri-ate and inappropriate utilization. Patient Education StudyGroup. J Rheumatol. 1999;26:1793–801.

Egan GF, Liu ZQ. Computers and networks in medical and healthcare systems. Comput Biol Med. 1995;25:355–65.

Firby PA, Luker KA, Caress AL. Nurses opinions of the introduc-tion of computer-assisted learning for use in patient educa-tion. J Adv Nurs. 1991;16:987–95.

Fitzmaurice DA, Hobbs FD, Murray ET, Bradley CP. Evaluation ofcomputerized decision support for oral anticoagulation man-agement based in primary care. Br J Gen Pract. 1996;46:533–5.

Fos PJ. An alternative approach to prevention: computer-assistedpatient education. J Dent Child. 1988;55:43–6.

Friedman RH, Kazis LE, Jette A, et al. A telecommunication systemfor monitoring and counseling patients with hypertension:impact on medication adherence and blood pressure control.Am J Hypertens. 1996;9(4 pt 1):285–92.

Fries JF, Carey C, McShane DJ. Patient education in arthritis: ran-domized controlled trial of a mail-delivered program. JRheumatol. 1997;24:1378–83.

Garrett LE Jr, Hammond WE, Stead WW. The effects of computer-ized medical records on provider efficiency and quality ofcare. Methods Inf Med. 1986;25:151–7.

Glasgow RE, Toobert DJ, Hampson SE, Noell JW. A brief office-based intervention to facilitate diabetes dietary self-manage-ment. Health Educ Res. 1995;10:467–78.

Gorman R. Expert system for management of urinary incontinencein women. Proc Annu Symp Comput Appl Med Care.1995:27–31.

Gregory C, Cifaldi M, Tanner LA. Targeted intervention programs:creating a customized practice model to improve the treat-ment of allergic rhinitis in a managed care population. Am JManag Care. 1999;5:485–96.

Gustafson DH, Hawkins R, Boberg E, et al. Impact of a patient-cen-tered, computer-based health information/support system.Am J Prev Med. 1999;16:1–9.

Hadiyono JE, Suryawati S, Danu SS, Sunartono, Santoso B.Interactional group discussion: results of a controlled trialusing a behavioral intervention to reduce the use of injectionsin public health facilities. Soc Sci Med. 1996;42:1177–83.

Hannah KJ, Conleyprice P, Fenty D, et al. Computer applicationsfor staff development and patient education. Methods InfMed. 1989;28:261–6.

Hershey CO, Goldberg HI, Cohen DI. The effect of computerizedfeedback coupled with a newsletter upon outpatient prescrib-ing charges: a randomized controlled trial. Med Care. 1988;26:88–94.

Herzmark G, Brownbridge G, Fitter M, Evans A. Consultation useof a computer by general practitioners. J R Coll Gen Pract.1984;34:649–54.

Holleman DR Jr, Simel DL. Effectiveness of automatic diagnostic

test result feedback on outpatient laboratory and radiology test-ing in veterans: a controlled trial. Med Care. 1996;34:857–61.

Holleman DR Jr, Simel DL. Effectiveness of automatic diagnostictest result feedback on outpatient laboratory and radiology test-ing in veterans: a controlled trial. Med Care. 1996;34:857–61.

Holmes-Rovner M, Kroll J, Rovner DR, et al. Patient decision sup-port intervention: increased consistency with decision analyt-ic models. Med Care. 1999;37:270–84.

Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on phycisian perform-ance and patient outcomes: a systematic review. JAMA. 1998;280:1339–46.

Huss K, Huss RW, Squire EN, et al. Computer education for asth-matics: What effects? J Nurs Care Qual. 1992;6:57–66.

Huss K, Salarno M, Huss RW. Computer assisted reinforcement ofinstruction: effects on adherence in adult atopic asthmatics.Res Nurs Health. 1991;14:259–67.

Jimison HB, Sher PP. Consumer health informatics: health informa-tion technology for consumers. J Am Soc Inf Sci. 1995;46:783–90.

Jones R, Pearson J, Cawsey A, Barrett A. Information for patientswith cancer. Does personalization make a difference? Pilotstudy results and randomized trial in progress. Proc AMIAAnnu Fall Symp. 1996:423–7.

Jones R, Pearson J, McGregor S, et al. Randomized trial of person-alized computer based information for cancer patients. BMJ.1999;319:1241–7.

Kahn G. Computer-based patient education: a progress report. MDComput. 1993;10:93–9.

Kinzie MB, Schorling JB, Siegel M. Prenatal alcohol edcuation forlow income women with interactive multimedia. Patient EducCouns. 1993;21:51–60.

Kreuter MW, Strecher VJ. Do tailored behavior change messagesenhance the effectiveness of health risk appraisal? Resultsfrom a randomized trial. Health Educ Res. 1996;11:97–105.

Krishna S, Balas EA, Spencer DC, Griffin JZ, Boren SA. Clinical tri-als of interactive computerized patient education: implicationsfor family practice. J Fam Pract. 1997;45:25–33.

Lehmann ED, Deutsch T. Application of computers in diabetescare: a review, part 2—Computers for decision support andeducation. Med Inform. 1995;20:303–29.

Leon AC, Kelsey JE, Pleil A, Burgos TL, Portera L, Lowell KN. Anevaluation of a computer-assisted telephone interview forscreening for mental disorders among primary care patients. JNerv Ment Dis. 1999;187:308–11.

Lewis CE, Resnik BA. Nurse clinics and progressive ambulatorypatient care. N Engl J Med. 1967;277:1236–41.

Lewis D. Computer-based approaches to patient education: areview of the literature. J Am Med Inform Assoc.1999;6:272–82.

Liedholm H, Linne AB, Agelii L. The development of an interac-tive education program for heart failure patients: The KodakPhoto CD Portfolio concept. Patient Educ Couns.1996;29:199–206.

Lo R, Lo B, Wells E, Chard M, Hathaway J. The development andevaluation of a computer-aided diabetes education program.Aust J Adv Nurs. 1996;13:19–27.

Lobach DF, Hammond WE. Computerized decision support basedon a clinical practice guideline improves compliance with carestandards. Am J Med. 1997;102:89–98.

Madoff SA, Pristach CA, Smith CM, Pristach EA. Computerizedmedication instruction for psychiatric inpatients admitted foracute care. MD Comput. 1996;13:427–31, 441.

Maslin A, Baum M, Walker JS, A’Hern R, Prouse A. Using an inter-active video disk in breast cancer patient support. Nurs Times.


1998;94:52–5.McPhee SJ, Bird JA, Jenkins CN, Fordham D. Promoting cancer

screening: a randomized, controlled trial of three interven-tions. Arch Intern Med. 1989;149:1866–72.

Modai I, Walevski A, Mark M, Rabinowitz J, Munitz H. Successfulreplacement of traditional psychiatric records with a multi-user computer system. Med Care. 1991;29:1057–60.

Montgomery EB Jr, Lieberman A, Singh G, Fries JF. Patient educa-tion and health promotion can be effective in Parkinson’s dis-ease: a randomized controlled trial. PROPATH AdvisoryBoard. Am J Med. 1994;97:429–35.

Murphy KR. Computer-based patient education. Otolarygol ClinNorth Am. 1998;31:309.

Ockene JK, Adams A, Pbert L, et al. The Physician-deliveredSmoking Intervention Project: factors that determine howmuch the physician intervenes with smokers. J Gen InternMed. 1994;9:379–84.

O’Reilly M. Is Internet-based disease management on the way?CMAJ. 1999;160:1039.

Osman LM, Abdalla MI, Beattie JA, et al. Reducing hospital admis-sion through computer-supported education for asthmapatients. Grampian Asthma Study of Integrated Care (GRAS-SIC). BMJ. 1994;308:568–71.

Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets:a systematic review of 102 trials of interventions to improveprofessional practice. Can Med Assoc J. 1995;153:1423–31.

Petrucci K, Petrucci P, Canfield K, McCormick KA, Kjerulff K, ParksP. Evaluation of UNIS: Urological Nursing InformationSystems. Proc Annu Symp Comput Appl Med Care. 1991:43–7.

Ravdin PM. A computer-based program to assist in adjuvant ther-apy decisions for individual breast cancer patients. BullCancer. 1995;82:S561–4.

Rippey RM, Bill D, Abeles M, et al. Computer-based patient edu-cation for older persons with osteoarthritis. Arthritis Rheum.1987;30:932–5.

Rogers JL, Haring OM, Goetz JP. Changes in patient attitudes fol-lowing the implementation of a medical information system.QRB Qual Rev Bull. 1988;10:65–74.

Rogers JL, Haring OM, Wortman PM, Watson RA, Goetz JP. Med-ical information systems: assessing impact in the areas of hyper-tension, obesity and renal disease. Med Care. 1982;20:63–74.

Rogers JL, Haring OM. Effect of a computerized medical recordssummary system on patient hospitalization. In: McCormickEH (ed). Proceedings of the 3rd Illinois Conference on MedicalInformation Systems. Chicago, Ill.: University of Illinois, 1976.

Rolnick SJ, Owens B, Botta R, et al. Computerized information andsupport for patients with breast cancer or HIV infection. NursOutlook. 1999;47:78–83.

Sands DZ, Safran C. Closing the loop of patient care: a clinical trialof a computerized discharge medication program. Proc AnnuSymp Comput Appl Med Care. 1994:841–5.

Schoneberger A, Raschka C. Computer-based patient education in

cardiac rehabilitation. Herz Kreislauf. 1992;24:298–301.Sciamanna, CN, Ford DE, Flynn JA, Langford C. An evidence-

based interactive computer program to assist physicians incounseling smokers to quit. MD Comput. 1999;16:54–60.

Simon GE, VonKorff M, Rutter C, Wagner E. Randomized trial ofmonitoring, feedback, and management of care by telephoneto improve treatment of depresion in primary care. BMJ. 2000;320:550–4.

Skinner CS, Siegfried JC, Kegler MC, Strecher VJ. The potential ofcomputers in patient education. Patient Educ Couns. 1993;22:27–34.

Sullivan F, Mitchell E. Has general practitioner computing made adifference to patient care? A systematic review of publishedreports. BMJ. 1995;311:848–52.

Tape TG, Givner N, Wigton RS, Seelig CB, Patil K, Campbell JR.Process in ambulatory care: a controlled clinical trial of com-puterized records. Proc Annu Symp Comput Appl Med Care.1988;749–52.

Turner BJ. A controlled trial of strategies to improve delivery ofpreventive care. Proc Annu Conf Res Med Educ. 1987;26:9–13.

Van Cura LJ, Jensen NM, Greist JH, Lewis WR, Frey SR. Venerealdisease: interviewing and teaching by computer. Am J PublicHealth. 1975;65:1159–64.

Watson HE. A study of minimal interventions for problemdrinkers in acute care settings. Int J Nurs Stud 1999;36:425–34.

Weber BE, Reilly BM. Enhancing mammography use in the innercity: a randomized trial of intensive case management. ArchIntern Med. 1997;157:2345–9.

Wetstone SL, Sheehan TJ, Votaw RG, Peterson MG, Rothfield N.Evaluation of a computer-based education lesson for patientswith rheumatoid arthritis. J Rheumatol. 1985; 2:907–12.

Wilkinson WE, Gehlbach SH, Hammond WE, Clapp NE. Usingcomputer generated feedback to improve physician prescrib-ing. Proc Annu Symp Comput Appl Med Care. 1982;6:76–8.

Wirtschafter DD, Scalise M, Henke C, Gams RA. Do informationsystems improve the quality of clinical research? Results of arandomized trial in a cooperative multi-institutional cancergroup. Comput Biomed Res. 1981;14:78–90.

Wofford JL, Currin D, Wofford MM, Michielutte R. The multi-mediacomputer for low-literacy patient education: a pilot project ofcancer risk perceptions. J Gen Intern Med. 2000;15:S233.

Wones RG. Failure of low-cost audits with feedback to reduce lab-oratory test utilization. Med Care. 1987;25:78–82.

Woolf AD, Zeidler H, Berliner M, et al. An interactive multimediacomputer system for patient education and information inRheumatology. Arthritis Rheum. 1993;36:S142.

Yilmaz S, Ozlu Y, Ekuklu G. The effect of dental training on thereactions of mentally handicapped children’s behavior in thedental office. ASDC J Dent Child. 1999;66:188–91.

Zieve FJ, Ciesco E. Computer-focused modification of physicianprescribing behavior. Proc Annu Symp Comput Appl MedCare. 1992:617–20.


Computer-generated Patient Education Materials: Do They Affect Professional Practice? : A Systematic...

Documents

Transcript of Computer-generated Patient Education Materials: Do They Affect Professional Practice? : A Systematic...