i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 8 ( 2 0 0 9 ) 170–181

journa l homepage: www. int l .e lsev ierhea l th .com/ journa ls / i jmi

Same system, different outcomes: Comparing thetransitions from two paper-based systems to the samecomputerized physician order entry system

Zahra Niazkhania,∗, Heleen van der Sijsb, Habibollah Pirnejada,William K. Redekopc, Jos Aartsa

a Institute of Health Policy and Management (iBMG), Erasmus University Medical Center, Rotterdam, The Netherlandsb Department of Hospital Pharmacy, Erasmus University Medical Center, Rotterdam, The Netherlandsc Institute for Medical Technology Assessment (iMTA), Erasmus University Medical Center, Rotterdam, The Netherlands

a r t i c l e i n f o

Article history:

Received 20 November 2007

Received in revised form 5 June 2008

Accepted 28 June 2008

Keywords:

Evaluation Studies

Prescriptions, Drug

Medication systems, Hospital

Medical Order Entry Systems

Computer Communication

Networks

Questionnaires

a b s t r a c t

Objective: To compare how nurses in two different paper-based systems perceive the impact

of a computerized physician order entry (CPOE) system on their medication-related activi-

ties.

Setting: 13 non-surgical, adult inpatient wards in a Dutch academic hospital.

Methods: Questionnaire survey of 295 nurses before and 304 nurses after the implementation

of a CPOE system. These nurses worked with two different paper-based medication systems

before the implementation: ‘Kardex-system’ and ‘TIMED-system’. In the Kardex-system, the

structure of the nursing medication work was similar to that of after the CPOE implemen-

tation, while in the TIMED-system, it was different. ‘Adaptive Structuration Theory’ (AST)

was used to interpret the results.

Results: The response rates were 52.2% (154/295) before and 44.7% (136/304) after the imple-

mentation. Kardex-nurses reported more positive effects than TIMED-nurses. TIMED-nurses

reported that the computerized system was more inflexible, more difficult to work with, and

slower than the TIMED-system. In the TIMED group, the overall mean score of the computer-

ized process was not significantly different from that of the paper-based process. Moreover,

nurses in both groups were more satisfied with the post-implementation process than with

the pre-implementation process. Nevertheless, none of groups reported a better workflow

support in the computerized system when compared to that of the paper-based systems.

Conclusions: Our findings suggest that not only the technology but also large differences

between pre- and post-implementation work structure influence the perceptions of users,

and probably make the transition more difficult. This study also suggests that greater sat-

isfaction with a system may not necessarily be a reflection of better workflow support.

∗ Corresponding author at: Institute of Health Policy and Management,dam, The Netherlands. Tel.: +31 10 408 8811; fax: +31 10 408 9094.

E-mail address: z.niazkhani@erasmusmc.nl (Z. Niazkhani).1386-5056/$ – see front matter © 2008 Elsevier Ireland Ltd. All rights resdoi:10.1016/j.ijmedinf.2008.06.012

© 2008 Elsevier Ireland Ltd. All rights reserved.

Erasmus University Medical Center, P.O. Box 1738, 3000 DR Rotter-

erved.

a l i n

1

T(tAaatCnbsoan

an[fitflai[ctstr[twtbantmo

bmaatweohwcpeppsw

i n t e r n a t i o n a l j o u r n a l o f m e d i c

. Introduction

he implementation of a computerized physician order entryCPOE) system is considered as a pivotal transitional stepowards the more effective management of medications [1].

CPOE system is defined as a computer application wherephysician directly enters medical orders. Because nurses

re also involved in patient care, they inevitably interact withhese systems or their outputs. Studies have shown that aPOE system can eliminate a number of intermediate steps forurses. For example, they no longer have to deal with illegi-le and incomplete hand-written orders, which are a commonource of extra workload for nurses [2–4]. The system, more-ver, facilitates order communication to other parties suchs the pharmacy, which in turn saves considerable time forurses [5,6].

However, something which has recently received consider-ble attention is the extent to which these systems change theature of workflow for health professionals, including nurses

7–9]. In fact, in addition to the literature that reports bene-ts of CPOE systems, there is a growing number of studieshat focus on unintended changes in many aspects of work-ow following the implementation [8,10]. Beuscart-Zephir etl. described the role of nurses in distributed decision mak-ng in the medication ordering and administration process11]. Coleman observed that nurses normally interpret physi-ians’ intents in their orders [12]. Therefore, if nurses wereo be bypassed after implementation of a CPOE system, theystem would not be able to handle this interpretation effec-ively. Both studies criticized the fact that the organizationalole of nurse was ignored during the design of CPOE systems11,12]. Moreover, in a study of perceived impact of CPOE sys-ems, nurses reported a sense of loss of control over theirork [13]. Goorman and Berg argued that at least some of

he problems with these systems occur because of the clashetween the nursing workflow model embedded in the systemnd actual nursing practice [14]. This evidence indicates thaturse-related medication activities, and more importantlyheir organizational role in the medication process, deserve

ore attention in the design, implementation and evaluationf CPOE systems.

Depending on different work organizations, nurses maye assigned different roles and responsibilities. As the imple-entation of CPOE systems brings a new work organization

long with it, this unavoidably transforms their roles andctivities. The study of how nurses perceive this transforma-ion in the transition from a paper-based to a computerizedork structure can give insight into how this transition can

ffectively be managed. In 2001, a computerized medicationrder entry system was implemented in a Dutch academicospital. Several different paper-based medication systemsere in use before the implementation. To compare the per-

eived impact of this CPOE system on nursing medicationractice, we conducted a before-and-after study in two differ-nt paper-based medication systems. In particular, we com-

ared the perceived benefits and/or drawbacks of the com-uterized system with those of the two different paper-basedystems. We also examined nurse satisfaction and perceivedorkflow support before and after CPOE implementation.

f o r m a t i c s 7 8 ( 2 0 0 9 ) 170–181 171

2. Theoretical background

We used the ‘Adaptive Structuration Theory (AST)’ [15] as atheoretical framework to study the changes that occurredin two different work practices following the CPOE imple-mentation. AST is based on Anthony Giddens’ StructurationTheory [16]. This theory is formulated as “the production andreproduction of the social systems through members’ useof rules and resources in interaction”. DeSanctis and Pooleadapted Giddens’ theory to study the interaction of groups andorganizations with information technology (IT), and called it‘Adaptive Structuration Theory’ [15].

AST criticizes the technocentric view of technology use andemphasizes its social aspects. This theory focuses on “socialstructures, rules and resources provided by technologies andinstitutions as the basis for human activity” (page 125) [15].The social structures in this theory include the technologyitself, the content and constraints of a given work task, theorganizational environment, corporate information, historiesof task accomplishment, cultural beliefs, modes of conductand so on. These structures act as templates for planningand accomplishing tasks and may vary across groups. Design-ers incorporate some of the structures of institutions intothe technology; the structures may be reproduced so as toimitate their non-technology counterparts, or they may bemodified, thus creating new work structures within the tech-nology. The AST helps to explain the different outcomes afterthe implementation of one information system in differentwork structures.

3. Study context

3.1. Study environment and the CPOE system

This study was conducted at Erasmus University MedicalCenter (Erasmus MC) in Rotterdam, a 1237-bed academichospital in the Netherlands. We studied a commerciallyavailable computerized medication order entry systemnamed Medicatie/EVS®. To retrieve patient and drug data,Medicatie/EVS communicates with the existing hospital infor-mation system (HIS) and patient medical record (Patient 98).This system was first piloted in six wards of two specialtiesfrom December 2001 to December 2002. It was followed by sub-sequent implementation in 39 wards from September 2003 toMarch 2005.

3.1.1. Two paper-based medication ordering andadministration processesBefore the CPOE implementation, Erasmus MC had two differ-ent paper-based systems on adult wards: Kardex and TIMED.

In the Kardex-system, to prescribe medications, physicianswrote a drug’s name, dosage form, dosage regimen, adminis-tration route, start date, and exact administration time on aspecial tear-off order form with two additional carbon copies

(Fig. 1a). Nurses could add missing information (e.g., dosageform or strength), but no transcription took place. The originalorder was put on a Kardex-card for registration of drug admin-istration. This was registered by signing next to the order on

i c a l

172 i n t e r n a t i o n a l j o u r n a l o f m e d

the Kardex-card. This registration form had room for ten daysand after that a new form for the next ten days could be addedon the card. To request non-stock items, nurses had to manu-ally write drug requests and send them to the pharmacy. Forurgent medications unavailable in the ward stock, nurses hadto refer to the pharmacy personally with the hand-writtendrug requests. These requests then were entered into theHIS by pharmacy technicians. These processes in the Kardex-system are shown in Fig. 2.

In the TIMED-system, physicians wrote a medication’sname, dosage regimen, administration route, and start date ona pre-printed slip (Fig. 1b). A nurse had to transcribe a physi-cian’s orders, select the suitable dosage forms available in thehospital, and choose their administration times. The nurse, forinstance, translated the dosage regimen of an order of threetimes daily into the exact administration times during the day.The transcribed orders together with an administration regis-tration form then were put in the patient medication chart.Each day a new administration form was put next to the tran-scribed order form (Fig. 2). Once a drug was administered, thiswas registered by sticking the flag labels of the administereddrugs on the administration form. Whenever flag labels wereabsent, the nurse had to write the drug name and dosage onthe administration form. For urgent and also non-stock med-ications, the same procedures as in the Kardex-system werefollowed.

For the sake of clarity, we will refer to ‘Kardex units’ aswards which used the Kardex-system before the implemen-tation. Likewise, ‘TIMED units’ are those wards that had theTIMED-system.

3.1.2. The computerized medication ordering andadministration processThe CPOE system is available in all physician offices as well asin all workstations throughout the hospital. Only physicians

Fig. 1 – The medication orders in different medication systems. (

i n f o r m a t i c s 7 8 ( 2 0 0 9 ) 170–181

and midwives are authorized for electronic order entry in thishospital. Physicians must enter their medication orders intothe system; nurses may not accept any hand-written prescrip-tion. A physician enters a medication order by selecting a drugand its dosage form, strength, administration route, dosageregimen, start date and time. A detailed description of the pre-scription process with the Medicatie/EVS has been publishedelsewhere [17]. After electronic ordering, medication ordersare printed on special labels called Medication Order (MO)-labels (Fig. 1c). Nurses were trained in groups to work with thesystem.

The printed labels are affixed to a Kardex-card which isspecific for the medication administration record. Next tothe MO-labels on the Kardex-card, nurses are supposed toprovide a signature once they give medications to patients.Transcription of a prescription by the nurse is no longer nec-essary. Nurses are notified about the availability of drugs inthe ward stock by means of codes specified on the MO-labels(“J” = available, “N” = unavailable). Pharmacy technicians con-trol the supply of in-stock items by scanning them at wardstwo or three times a week. Whenever an MO-label containsa drug that is out of stock, nurses can select it in the systemand thereby sent an electronic drug request to the pharmacy.Technicians in the pharmacy check these non-stock drugrequests twice a day at 8 o’clock and 12 o’clock and providethe requested drugs later that day. The process after the CPOEimplementation is shown in Fig. 3.

Comparison of Figs. 2 and 3 shows that the medicationordering and administration process after the implementationresembles that of the Kardex-system, while it is completelydifferent from that of the TIMED-system. In both Kardex andTIMED units, we compared nurse attitudes towards the com-

puterized process in the post-implementation phase withtheir attitudes towards the paper-based process in the pre-implementation phase.

a) Kardex system, (b) TIMED system, and (c) CPOE system.

i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 8 ( 2 0 0 9 ) 170–181 173

Fig. 2 – The medication ordering and administration processes in Kardex-system and TIMED-system; MO (MedicationO r ret

4

4

Ontq

Fc

rder); HIS (Hospital Information System); NS (Non-Stock); fohe pharmacy with hand-written requests.

. Methods

.1. Study design and measurements

ur evaluation was based on questionnaire administered tourses before and after the CPOE implementation. Design ofhe questionnaire was based in part on previously publisheduestionnaires for the assessment of user satisfaction with

ig. 3 – The medication ordering and administration process afteomputers throughout the hospital; MO-label (Medication Order-

questing urgent NS drugs, nurses often directly referred to

CPOE – such as [18] – and was done in a close collaborationwith nursing staff. In addition to demographics, the originalquestionnaire contained 28–40 questions to measure attitudesregarding the paper-based systems (Kardex and TIMED) and

the CPOE system. In the present study, we report on the resultsof the questions that were similar in the questionnaires usedin the three systems. The list of these questions is available assupplementary material. These questions asked respondents

r the CPOE system; this system is available in all thelabel); NS (Non-Stock).

i c a l

174 i n t e r n a t i o n a l j o u r n a l o f m e d

about: overall reaction towards the medication process (1.1,1.2, and 1.3), the characteristics of medication orders (2.1, 2.2,and 2.3), registration of drug administration (3.1, 3.2, 3.3, 3.4,and 3.5), the learning and speed of the process (4.1 and 4.2),and managing the non-stock medication orders (5.1 and 5.2).These questions were designed to evaluate the attitudes basedon a 5-point Likert scale. We also included two other ques-tions which asked the respondents to give their impression ofworkflow support (6.1) and system preference (6.2).

The questionnaire was checked for the applicability andunderstandability of its wording by two nurses in each sys-tem. The questionnaire was considered ready for distributionafter modifications suggested by these nurses. In each phase,a packet containing the questionnaire and a cover letterexplaining the aim of the study were distributed by head

nurses among all nurses in the target wards. The completedquestionnaires were collected by the head nurses or directlysent to the researcher via the hospital’s internal mail ser-vice.

Table 1 – Characteristics of survey respondents

Characteristics Kardex units

Pre-implementationN (%)

Post-im

Number of questionnaires distributed 144Number of respondents 66

SpecialtyPsychiatry 30 (45.5)Hematology and oncology 36 (54.5)Internal medicine –Neurology –

Female 54 (81.8)

Age (years old)≤23 1 (1.5)24–33 20 (30.3)34–43 18 (27.3)44–53 21 (31.8)≥54 5 (7.6)

Professional statusPracticing nurse 52 (78.1)Head nurse 7 (10.6)Others 6 (9.1)

Home use of computerNever 7 (10.6)Sometimesb 11 (16.7)Regularlyc 14 (21.2)Oftend 33 (50.0)

Use of computer at workNever 0 (0)Sometimes 9 (13.9)Regularly 20 (30.3)Often 36 (54.5)

Prior experience with an electronicprescription system

6 (9.0)

a Nine nurses who had been forgotten in the first phase received the quesb Once a month or less.c Once in a week to few times per month.d Daily to few times per week.

i n f o r m a t i c s 7 8 ( 2 0 0 9 ) 170–181

4.2. Course of the study and participants

Nurses working in 13 non-surgical, adult inpatient wardswere chosen to participate in this study. Six wards used theKardex-system and consisted of Psychiatry (three wards), andHematology and Oncology (three wards). Seven wards used theTIMED-system and consisted of Internal Medicine (six wards)and Neurology (one ward). The CPOE system was implementedin these wards one after another. The questionnaires weresent two weeks before and approximately five months afterthe introduction of the CPOE system. Since the introduction ofthe system across the hospital was conducted in a step-wisebasis, the distribution of the questionnaire in both phases fol-lowed the implementation order (September 2003 to October2004).

All nurses who were working in the selected units dur-ing the course of this study were invited to participate. Inthe pre-implementation phase, 295 nurses received the ques-tionnaire, of whom 154 nurses responded (52.2%). In the

TIMED units

plementationN (%)

Pre-implementationN (%)

Post-implementationN (%)

144 151 160a

48 88 88

23 (47.9) – –25 (52.1) – –

– 76 (86.3) 73 (83.0)– 12 (13.7) 15 (17.0)

35 (72.9) 71 (80.7) 73 (83.0)

1 (2.1) 13 (14.8) 17 (19.3)14 (29.2) 25 (28.4) 27 (30.7)10 (20.8) 19 (21.6) 17 (19.3)18 (37.5) 22 (25.0) 22 (25.0)1 (2.1) 5 (5.7) 3 (3.4)

38 (79.2) 65 (73.9) 63 (71.6)7 (14.9) 6 (6.8) 8 (9.1)2 (4.2) 16 (18.2) 16 (18.2)

5 (10.4) 4 (4.5) 8 (9.1)5 (10.4) 5 (5.7) 11 (12.5)

12 (25.2) 31 (35.2) 27 (30.7)25 (52.1) 46 (52.3) 41 (46.6)

1 (2.1) 1 (1.1) 0 (0)0 (0) 5 (5.7) 4 (4.5)

14 (29.2) 17 (19.3) 21 (23.9)32 (66.7) 62 (70.5) 60 (68.2)5 (10.5) 7 (7.9) 10 (11.3)

tionnaire only in the second phase.

a l i n f o r m a t i c s 7 8 ( 2 0 0 9 ) 170–181 175

pwnp7ptiiwaw

4

Aroeuttfsa(tvccpatac1

5

TgupWdiuqts

5pu

5As(

Fig. 4 – Box plots showing median, interquartile range, andwhiskers for overall scores of the process in thepaper-based systems and the electronic system in differentunits; in Kardex units the overall score improved after the

i n t e r n a t i o n a l j o u r n a l o f m e d i c

ost-implementation phase, 304 nurses were contacted, ofhom 136 nurses responded (44.7%). Overall, 290 question-aires were returned. Two hundred and eleven nurses (70.56%)articipated in at least one phase of the study. In total, at least9 nurses were identified as nurses who participated in bothhases while 132 nurses completed only one questionnaire. Ashe recording of the identification number was not mandatory,t is possible that more nurses answered the questionnairesn both phases. One nurse in the post-implementation phase,

ho did not use computers at work, was excluded from thenalysis of one question requiring the use of computers atork.

.3. Data analysis

nalyses using the Kolmogorov-Smirnov (Lilliefors) testevealed that the scores provided by the respondents wereften not normally distributed. Therefore, we tested for differ-nce between scores for before and after the implementationsing the Mann–Whitney U test. Overall scores of the sys-ems, which were normally distributed, were compared usinghe t tests. The t tests were also performed to test for dif-erences between the change scores (mean differences andtandard errors in paper vs. electronic system) in Kardex unitsnd change scores in TIMED units. 95% confidence intervalsCIs) for means are reported. We used the Spearman correla-ion coefficient to measure the degree of association betweenariables and overall satisfaction with the computerized pro-ess. The Mann–Whitney U tests were performed to determinehanges in ratings of the preference of the systems and theerceived support of workflow (items 6.1 and 6.2) between pre-nd post-implementation. We measured the internal consis-ency of the questions (1.1–5.2) using Cronbach’s Alpha. Anlpha level of .05 was used for all statistical tests. All statisti-al analyses were performed using SPSS for Windows (version4).

. Results

able 1 provides the demographics of the different studyroups. Most nurses were women, practicing nurses, oftensed computers both at home and at work, and had norior experience with an electronic prescription system.ith regard to demographics, there were no important

ifferences neither between respondents of pre- and post-mplementation phases nor between respondents in Kardexnits and respondents in TIMED units. Cronbach’s Alpha foruestions 1.1–5.2 was 0.84 for the paper-based and 0.88 forhe computerized process, representing a high internal con-istency of the questionnaire.

.1. Comparison between pre- andost-implementation, and between Kardex and TIMEDnits

.1.1. Overall mean scoresn overall mean score for each nurse was calculated byumming the scores for the 15 items of the questionnaire1 = minimum, 5 = maximum). Afterwards, the overall mean

implementation, p < .001; however, in TIMED units, thisimprovement was not significant, p > .05.

score was calculated for pre- and post-implementation inKardex and TIMED units. Kardex-nurses, whose medicationprocess after the implementation was similar to that of beforethe implementation, gave a higher mean score for the com-puterized process than for the paper-based process (3.6 vs.3.2; p < .001). However, in TIMED units in which the elec-tronic system brought a completely new work structure fornurses, the overall mean score of the computerized processwas not significantly different from that of the paper-basedprocess (3.5 vs. 3.4, p > .05). Fig. 4 shows median, interquar-tile range, and whiskers for the overall scores of pre- andpost-implementation in these two units. Considering thesimilarity or differences of work structures in pre- and post-implementation phases, there was a significantly greaterimprovement in the overall score for Kardex units (i.e., from apaper-based to a computerized process) than for TIMED units(mean difference = −0.29; CI: −0.55, −0.02; p < .05).

Table 2 summarizes the mean scores for individual ques-tions and the significance of difference between pre- andpost-implementation within Kardex or TIMED units. It alsoprovides the significance of difference between these unitswith regard to the change scores.

5.1.2. Overall reactionsIn Kardex units, there was greater satisfaction with the com-puterized process than with the paper-based process (3.7 vs.2.9, p < .001) (item 1.3). This was also the case for TIMED units,where satisfaction was 0.3 points higher with the comput-erized process than with the paper-based process (3.5 vs.3.2, p < .05). The comparison between units showed that theimprovement in satisfaction (from paper-based to computer-

ized process) was significantly greater in Kardex units than inTIMED units (mean difference = −.5; CI: −.93, −.01; p < .05).

In contrast with Kardex units, nurses in TIMED unitsreported that their paper-based process was significantly

176in

te

rn

at

ion

al

jou

rn

al

of

me

dic

al

inf

or

ma

tic

s7

8(2

00

9)

170–181

Table 2 – Mean scores to the individual questions in the Kardex and TIMED units categorized based on the paper-based and computerized systems

Item# Questionnaire items Kardex units TIMED units p value*

Mean p value† Mean p value‡

Pre-implementation

(No.)

Post-implementation

(No.)

Pre-implementation

(No.)

Post-implementation

(No.)

1.1 Overall impression(difficult. . .easy)

3.7 (64) 3.8 (48) NS§ 4.2 (87) 3.9 (86) †† †

1.2 Overall impression(rigid. . .flexible)

3.2 (64) 3.3 (46) NS 3.5 (83) 3.0 (85) †† †

1.3 Overall impression(frustrating. . .satisfying)

2.9 (62) 3.7 (46) ††† 3.2 (84) 3.5 (84) † †

2.1 The layout of the medicationorders (confusing. . .clear)

3.3 (64) 3.4 (48) NS 3.7 (88) 3.6 (87) NS NS

2.2 The legibility of the medicationorders (bad. . .good)

2.1 (66) 3.9 (48) ††† 2.8 (88) 4.0 (88) ††† ††

2.3 The completeness of themedication orders (bad. . .good)

2.9 (66) 3.7 (48) ††† 3.4 (88) 3.9 (88) ††† NS

3.1 The arrangement of data(unhelpful. . .helpful)

3.5 (66) 3.7 (48) NS 3.7 (88) 3.8 (88) NS NS

3.2 The layout of the drug overviewform (confusing. . .clear)

3.4 (66) 3.6 (48) NS 3.5 (88) 3.8 (88) †† NS

3.3 The drug overview form(unreliable. . .reliable)

2.9 (66) 3.6 (47) ††† 3.2 (87) 3.7 (88) ††† NS

3.4 The administration record(confusing. . .clear)

2.9 (66) 3.5 (48) †† 3.1 (88) 3.3 (88) NS †

3.5 The administration record(unreliable. . .reliable)

3.0 (65) 3.4 (46) †† 3.2 (88) 3.1 (88) NS †

4.1 The learning of the system(difficult. . .easy)

4.0 (65) 3.9 (48) NS 4.2 (88) 3.8 (87) †† NS

4.2 The speed of prescribing byphysicians (too slow. . .fastenough)

3.2 (66) 3.5 (47) NS 3.3 (88) 2.8 (86) †† †††

5.1 Insight over available drugs inthe pharmacy(unsatisfactory. . .satisfactory)

2.9 (66) 3.2 (47) NS 2.0 (86) 2.8 (85) ††† †

5.2 The ordering of non-stock drugs(difficult. . .easy)

3.7 (64) 3.7 (47) NS 3.4 (88) 3.4 (83) NS NS

∗ p value by ‘t test’ for difference between the change scores in Kardex units and change scores in TIMED units.† p value by ‘Mann–Whitney U test’ between pre- and post-implementation in Kardex units.‡ p value by ‘Mann–Whitney U test’ between pre- and post-implementation in TIMED units.§NS: not significant; †: p<.05; † †: p<.01; † † †: p<.001.

i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 8 ( 2 0 0 9 ) 170–181 177

Table 3 – Relationship between overall score of the computerized process and ratings of its specific aspects

Item # Survey itemsa nb rc Significance

1.1 Overall impression (difficult. . .easy) 134 .628 <.0011.2 Overall impression (rigid. . .flexible) 131 .571 <.0011.3 Overall impression (frustrating. . .satisfying) 130 .748 <.0012.1 Layout of the medication orders (confusing. . .clear) 135 .609 <.0012.2 Prescription legibility (bad. . .good) 136 .493 <.0012.3 Completeness of medication orders (bad. . ..good) 136 .531 <.0013.1 Arrangement of data (unhelpful. . .helpful) 136 .608 <.0013.2 Layout of the drug overview form (confusing. . .clear) 136 .627 <.0013.3 Drug overview form (unreliable. . .reliable) 135 .568 <.0013.4 Administration record (confusing. . .clear) 136 .657 <.0013.5 Administration record (unreliable. . .reliable) 134 .585 <.0014.1 Learning of the system (difficult. . .easy) 135 .550 <.0014.2 Speed of prescribing by physicians (slow. . .fast enough) 133 .467 <.0015.1 Insight over available drugs in the pharmacy (unsatisfactory. . .satisfactory) 132 .486 <.0015.2 Ordering of non-stock drugs (difficult. . .easy) 131 .456 <.0017.2 Specialty 136 .204 <.057.3 Gender 134 .171 <.057.4 Age categories 130 .182 <.05

eacheach

eprtMuct(pwcidatcSoc

5Nop(tTTlt(

5Ia

a Items 1.1 to 5.2 are based on a 5-point likert scale.b Not all of the 136 respondents after the implementation answeredc Spearman correlation coefficient (rho) for the relationship between

asier to use than the computerized process that was com-letely new for them (4.2 vs. 3.9, p < .01) (item 1.1). They alsoated their paper-based process significantly more flexiblehan the computerized process (3.5 vs. 3.0, p < .01) (item 1.2).oreover, learning of the computerized process in TIMEDnits was more difficult than learning of the paper-based pro-ess (3.8 vs. 4.2, p < .01). Kardex-nurses however reported ito be as easy as their paper-based process (3.9 vs. 4, p > .05)item 4.1). TIMED nurses believed that the ordering process byhysicians was slower in the computerized system comparedith the paper-based system (2.8 vs. 3.3, p < .01) (item 4.2). The

hange in attitudes towards speed in TIMED units was signif-cantly greater than the change seen in Kardex units (meanifference = −0.8; CI: −1.3, −0.4; p < .001). This difference prob-bly arose because the electronic system completely changedhe structure of medication ordering and administration pro-ess not only for TIMED-nurses but also for TIMED-physicians.pecifically, it required these physicians to enter detailedrders into the system instead of concise, paper-based medi-ation orders (Fig. 1b).

.1.3. Characteristics of medication ordersurses in both groups rated the legibility and completenessf medication orders (items 2.2 and 2.3) higher in the com-uterized system than in the paper-based system (p < .001)

Table 2). Interestingly, the degree of this improvement relatedo legibility was perceived to be higher in Kardex units than inIMED units (mean difference = −0.64; CI: −1.1, −0.2; p < .01).he computerized system did not change the clarity of the

ayout in medication orders (item 2.1) compared with that ofhe paper-based orders, neither in Kardex nor in TIMED unitsp > .05).

.1.4. Registration of administrationn both units, the computerized system was associated with

greater reliability of the drug overview form (p < .001) (item

question.item and overall score.

3.3). In Kardex units, the clarity of administration record in thecomputerized system was significantly greater than that in thepaper-based system (3.5 vs. 2.9, p < .01) (item 3.4). Likewise, theadministration record in the computerized system was per-ceived to be more reliable than that in the paper-based system(3.4 vs. 3.0, p < .01) (item 3.5). However, after the implementa-tion, TIMED-nurses, who no longer used the flag labels of theadministered drugs for registration purposes, reported no anysignificant difference between pre- and post-implementationphases with regard to the clarity and reliability of the admin-istration record (p > .05) (items 3.4 and 3.5).

5.1.5. Drug delivery request from the pharmacyTIMED nurses reported a better insight regarding the availabledrugs in the pharmacy after the implementation comparedwith the situation before (2.8 vs. 2.0, p < .001) (item 5.1). How-ever, in Kardex units, this difference between two phases wasnot significant (p > .05). Moreover, the electronic system didnot facilitate the ordering of non-stock medications (item 5.2),neither in Kardex nor in TIMED units (p > .05) (Table 2).

5.1.6. System preference and workflow supportWhen nurses were asked whether they wanted to change thecurrent process (item 6.2), the majority of the respondentsin both Kardex (97.91%) and TIMED units (79.31%) preferredto continue using the computerized system. However, as theMann–Whitney U tests showed, nurses believed that the com-puterized system did not support their work processes morethan the paper-based systems did (item 6.1); this was the casefor both Kardex (p = .07) and TIMED units (p = .38).

5.2. Correlation of overall score of the computerized

process

Overall score of the computerized process was strongly cor-related with the user satisfaction (r = 0.75, p < .001), clarity

i c a l

178 i n t e r n a t i o n a l j o u r n a l o f m e d

of administration record (r = 0.66, p < .001, ease of the pro-cess (r = 0.63, p < .001), and clarity of layout of drug overviewform (r = 0.63, p < .001). Overall score was not correlated withprofessional status, experience with computers at home orwork, and the wards where nurses work. Table 3 summa-rizes the correlates of overall score in the computerizedprocess.

6. Discussion

Our study showed that although the system eliminatedthe workload of transcription and translation tasks forthe TIMED-nurses, they showed a less positive attitudecompared with the Kardex-nurses. Nurses in both groupsbenefited from improved legibility and completeness ofmedication orders and greater reliability of drug overviewafter the implementation. TIMED-nurses were sometimesless positive about the new medication process than thepre-implementation process, with the new process beingassociated with increased rigidity and difficulty, and aslower physician prescription speed. These increased prob-lems were not seen by the Kardex-nurses. Furthermore,in TIMED units, the computerized process did not resultin a significantly higher overall score compared with thepaper-based process. The overall score of the computerizedprocess was most strongly correlated with user satisfaction,clarity of the administration record, and easiness of theprocess.

Moreover, in our study, nurses in both groups found thatthe computerized system did not support workflow betterthan the paper-based systems. However, nurses in both groupsfavored the medication process after the implementation overthe situation before the implementation. It is often expectedthat the process following computerized physician order entrybecomes more efficient for nurses [19]. However, our casesshowed that in spite of a higher satisfaction and perceivedadvantages of the system, none of groups reported a greaterworkflow support after the implementation compared to thatof the paper-based systems. Similar to this finding, in onesurvey of ICU nurses, 56.7% of respondents commented thatthe post-CPOE workflow had become less efficient, althoughthey were also generally satisfied with the system [20]. Thesefindings therefore imply that workflow support with a CPOEsystem is a multi-layered issue and suggest that greater sat-isfaction with a system is not necessarily a reflection ofbetter support for workflow. Because medical workflow is com-plex and multidimensional, efficiency of the process in factdepends on the interplay between multiple factors in work-flow [21]. A health professional user may be satisfied withsome aspects of the workflow involving the system while dis-satisfied with other aspects. In a mixed method study, wefurther investigated this issue in TIMED units [22]. We foundthat while the CPOE system improved certain non-supportivefeatures of the TIMED-system, it lacked its main supportivefeatures.

Sociologists who study medical practice often assert thatthe construction of medical practice is intertwined with thecapabilities of tools used in daily practice [23,24]. Two dif-ferent paper-based orders in our case therefore were the

i n f o r m a t i c s 7 8 ( 2 0 0 9 ) 170–181

representatives of two different nursing organizational pro-cesses. In Kardex units, some of the social structures of themedication process incorporated into the system were sim-ilar in the paper-based and computerized processes. Basedon the principles of ‘Adaptive Structuration Theory’ describedin our theoretical background, we suggest that the familiar-ity of the Kardex-nurses with these structures led to morepositive perceived effects. In contrast, the implementation ofthe system imposed new social structures for the workflowof TIMED nurses resulting in less positive perceived effects.It therefore could be the case that the TIMED-nurses, as aconsequence of a different work organization, experiencedmore difficulty in appropriation of the system in the prac-tice.

The concise and semi-structured physician orders in theTIMED system were highly abstract representations of orders.Our results can be understood if we consider that the transla-tion and transcription tasks to process these orders compellednurses to take a more active role in the medication process.Nurses would here assume a more authoritative position inthe medication process than their counterparts in Kardexunits. While the roles and responsibilities of nurses in Kardexunits remained intact, the implementation of the CPOE sys-tem may have changed and challenged the position of nursesin TIMED units. This is in accordance with previous studiesthat emphasize the organizational context of clinical prac-tice in which the implementation and application of a CPOEsystem take place [25,26]. Aarts and Berg in a qualitativestudy of two hospitals described how changes in the exist-ing organizational contexts resulted in different outcomes ofthe implementation of the same CPOE system [27]. In ourcase, TIMED nurses judged the effectiveness of the new pro-cess in light of the effectiveness of the paper-based process(Fig. 4). This suggests that user perceptions can be influencedby the extent to which a technology changes established workpatterns.

6.1. Unexpected results and unanswered questions

Unexpected results were also seen in our study. In both ourstudy and the study by Lee et al. [4], nurses were pleasedwith improved legibility and completeness of medicationorders. However, the computerized system in our study did notimprove the clarity of the layout in medication orders, eventhough a relationship between the two had been expected.Clarity of the layout refers in general to the ability of nursesto clearly visualize a medication order or its concepts. Somestudies have suggested that the negotiation between nursesand physicians during medical rounds helps a nurse to under-stand an order better [28]. Although the limited negotiationbetween physicians and nurses with the implemented CPOEsystem may explain our finding, another reason might be thefact that the highly detailed medication orders are all printedwith black ink, making all orders look alike and prohibiting theuse of any of the visual cues that are available in paper-based

orders. These factors can reduce the clarity of an order andthereby make it difficult for busy nurses to correctly under-stand the orders. Hence, the clarity of an order entails morethan just legibility and completeness.

a l i n

sdannpasi

aectecwvdlw

6

UtifkiWitp[iloioa(rfi

Fntcspapwirois

questionnaires. We also thank Enrico Coiera, Antoinettede Bont, and Marc Berg for their insightful comments on

i n t e r n a t i o n a l j o u r n a l o f m e d i c

Furthermore, in theory, a computerized order entry processhould improve communication of nurses with the pharmacyepartment, since many elements in the order entry processre automated. The CPOE system examined in our study didot achieve this: it did not appear to facilitate ordering ofon-stock items from the pharmacy. This quantitative studyrovides no answers about how and why this was the case. Inqualitative study, we explored how the organizational and

ocial context in which the system was implemented affectedts intended behavior [29].

Moreover, to determine a possible carry over effect, we didsub-analysis on the wards that implemented the system

arly vs. those that implemented it late. No differences inhange in satisfaction and overall score were seen betweenhe wards in Kardex units that implemented the systemarly vs. those that implemented it later. While differences inhange in satisfaction and overall score were seen betweenards in TIMED units that implemented the system early

s. those that implemented it later, collinearity made itifficult to determine whether this was due to early vs.

ate implementation or due to other reasons (e.g., type ofard).

.2. Strengths and weaknesses of the study

ser satisfaction studies often tend to focus on an IT sys-em’s technical characteristics [18,30]. In our study, we werenterested in evaluating the perceived effects of CPOE on dif-erent activities in the nursing medication process. To ournowledge no other study has performed a before-after CPOE

mplementation study of two different paper-based systems.hile other studies have extended our understanding of nurs-

ng practice in paper-based and CPOE-based environments,hey have mostly described the CPOE environment and com-ared it with a paper-based environment (see, for example

11,28]). These studies did not use the same pool of personneln the two environments. We found that a study of simi-ar groups of respondents before and after implementationf a CPOE system can yield valuable insight into the true

mpact of the system on workflow. Two other strengths ofur study are the similarity of respondent groups and thebsence of any relevant differences between the two phasesbesides the implementation of a CPOE system). These factorseduce the chance that there are other explanations for ourndings.

However, our study has limitations that deserve discussion.irst, we focused on the perceived impact of the system onursing practice and did not measure the actual impact ofhe system on the quality of care (e.g., the quality of medi-ation orders). This was beyond the scope of this study andhould be examined more carefully in the future. Anotherossible limitation is non-response bias which may haverisen during our study. Our results therefore have to be inter-reted cautiously. Since the participation rate in this studyas fairly good (70.6% of nurses), this would suggest that the

mpact of non-response bias was limited, though a higher

ate would be more convincing for validity purposes. Third,ur findings mainly relate to the transitional phase of a CPOE

mplementation project. The long term impact of a CPOEystem would require taking measurements after a longer

f o r m a t i c s 7 8 ( 2 0 0 9 ) 170–181 179

period of time. By that time users would have become moreacquainted with the strengths and weaknesses of the sys-tem and thereby more capable of identifying and reportingthem.

7. Conclusion

Implementing CPOE systems reorganizes the medicationwork. These systems are referred to as transformational tech-nology and nursing practice is not an exception. The patternof nursing work in general and their roles and responsibilitiesin particular are all affected. Yet, the perceived impact of thesystem on nursing practice depends in part on the extent towhich the system changes existing work patterns. Use of ASTallowed us to explain why the outcome of transition from apaper-based to computerized process can vary widely acrossgroups that are using the same information technology. How-ever, our approach does not explain how these two groupsappropriated this technology in their practice. Observationalstudies are needed to describe and explain the appropriationprocess.

Our study suggests that not merely the CPOE system, thetechnology itself, influences the perceptions of its users. Thesize of the differences between pre- and post-implementationwork processes can also play an important role. Large dif-ferences can in fact make it much more difficult for healthprofessionals to switch from a paper to a computerized sys-tem. The strengths and limitations of a new system forexisting work organizations should therefore be carefullyconsidered beforehand. If a difficult transition is expected,activities such as extra support or training sessions can beplanned to alleviate that. This insight should inform strate-gies for change management when these systems are beingdesigned, implemented, and maintained.

Contributions of the authors

HS designed the questionnaire and collected data. ZN andHP designed the study and analyzed data. ZN wrote theearly draft of the manuscript. HS was also involved in thecomments on the manuscript. WR was involved in analysisof data and critical review and editing of the manuscript.ZN, HP, and JA participated in the development of the con-cept and critical review of the content. ZN revised differentversions of the manuscript. JA scientifically supervised thepaper.

Acknowledgements

The authors gratefully acknowledge the time and effortof all head nurses and all of those who completed the

an earlier version of this paper. The first and the thirdauthors were supported by a grant from the Iranian Min-istry of Health and Medical Education while conducting thisresearch.

i c a l

r

(2) (1997) 94–101.

180 i n t e r n a t i o n a l j o u r n a l o f m e d

Summary pointsWhat was known before the study

• The implementation of CPOE systems changes workpractice and the role and responsibilities of careproviders.

• Nurses’ roles, responsibilities, and activities in themedication process deserve more attention in thedesign, implementation and evaluation of CPOE sys-tems.

What the study has added to the body of knowledge

• One CPOE system may have different impact on differ-ent work structures.

• Depending on the previous work structure, the degreeof change in work structure and in roles and respon-sibilities of care providers after the implementation ofa CPOE system can greatly influence the system’s per-ceived effects. The strengths and limitations of a newsystem for existing work structure should therefore becarefully considered beforehand.

• Workflow support with a CPOE system is a multi-layered issue; hence, mere greater satisfaction with asystem may not necessarily be a reflection of greatersupport for workflow.

Appendix A. Supplementary data

Supplementary data associated with this article can be found,in the online version, at doi:10.1016/j.ijmedinf.2008.06.012.

e f e r e n c e s

[1] R. Briere (Ed.), Crossing the Quality Chasm, a New HealthSystem for the 21st Century, National Academy Press,Washington, DC, 2001.

[2] P. Francois, E. Chirpaz, H. Bontemps, J. Labarere, J.L. Bosson,J. Calop, Evaluation of prescription-writing quality in aFrench university hospital, Clin. Perform. Qual. Health Care5 (July–September (3)) (1997) 111–115.

[3] E.H. Winslow, V.A. Nestor, S.K. Davidoff, P.G. Thompson, J.C.Borum, Legibility and completeness of physicians’handwritten medication orders, Heart Lung 26 (March–April(2)) (1997) 158–164.

[4] F. Lee, J.M. Teich, C.D. Spurr, D.W. Bates, Implementation ofphysician order entry: user satisfaction and self-reportedusage patterns, J. Am. Med. Inform. Assoc. 3 (1) (1996)42–55.

[5] L. Cordero, L. Kuehn, R.R. Kumar, H.S. Mekhjian, Impact ofcomputerized physician order entry on clinical practice in anewborn intensive care unit, J. Perinatol. 24 (February (2))(2004) 88–93.

[6] H.S. Mekhjian, R.R. Kumar, L. Kuehn, T.D. Bentley, P. Teater,

A. Thomas, et al., Immediate benefits realized followingimplementation of physician order entry at an academicmedical center, J. Am. Med. Inform. Assoc. 9(September–October (5)) (2002) 529–539.

i n f o r m a t i c s 7 8 ( 2 0 0 9 ) 170–181

[7] P.N. Gorman, M.B. Lavelle, J.S. Ash, Order creation andcommunication in healthcare, Methods Inf. Med. 42 (4)(2003) 376–384.

[8] R. Dykstra, Computerized physician order entry andcommunication: reciprocal impacts, in: Proc. AMIA Symp.,2002, pp. 230–234.

[9] J.D. Carpenter, P.N. Gorman, What’s so special aboutmedications: a pharmacist’s observations from the POEstudy, in: Proc. AMIA Symp., 2001, pp. 95–99.

[10] J. Richards, Pandora’s box: physician order entry and nursingwork redesign, Can. J. Nurs. Leadersh. 13 (May–June (2))(2000) 15–19.

[11] M.C. Beuscart-Zephir, S. Pelayo, F. Anceaux, J.J. Meaux, M.Degroisse, P. Degoulet, Impact of CPOE on doctor-nursecooperation for the medication ordering and administrationprocess, Int. J. Med. Inform. 74 (August (7–8)) (2005)629–641.

[12] R.W. Coleman, Translation and interpretation: the hiddenprocesses and problems revealed by computerized physicianorder entry systems, J. Crit. Care 19 (December (4)) (2004)279–282.

[13] C. Weir, V. Johnsen, D. Roscoe, A. Cribbs, The impact ofphysician order entry on nursing roles, in: Proc. AMIA Annu.Fall Symp., 1996, pp. 714–717.

[14] E. Goorman, M. Berg, Modelling nursing activities: electronicpatient records and their discontents, Nurs. Inq. 7 (March(1)) (2000) 3–9.

[15] G. DeSanctis, M. Poole, Capturing the complexity inadvanced technology use: adaptive structuration theory,Organ. Sci. 5 (2) (1994) 121–147.

[16] A. Giddens, The Constitution of Society: Outline of theTheory of Structuration, University of California Press, 1986.

[17] M.D. Kalmeijer, W. Holtzer, R. van Dongen, H.J. Guchelaar,Implementation of a computerized physician medicationorder entry system at the Academic Medical Centre inAmsterdam, Pharm. World Sci. 25 (June (3)) (2003) 88–93.

[18] H.J. Murff, J. Kannry, Physician satisfaction with two orderentry systems, J. Am. Med. Inform. Assoc. 8(September–October (5)) (2001) 499–509.

[19] M.J. Ball, C. Weaver, P.A. Abbott, Enabling technologiespromise to revitalize the role of nursing in an era of patientsafety, Int. J. Med. Inform. 69 (January (1)) (2003) 29–38.

[20] M.L. Popernack, A critical change in a day in the life ofintensive care nurses: rising to the e-challenge of anintegrated clinical information system, Crit. Care Nurs. Q. 29(October–December (4)) (2006) 362–375.

[21] M. Berg, The search for synergy: interrelating medical workand patient care information systems, Methods Inf. Med. 42(4) (2003) 337–344.

[22] H. Pirnejad, et al., Impact of a computerized Physician orderentry system on nurse-physician collaboration in themedication process, Int. J. Med. Inform. (2008),doi:10.1016/j.ijmedinf.2008.04.001.

[23] M. Berg, G. Bowker, The multiple bodies of the medicalrecord: towards a sociology of an artifact, Sociol. Q. 38 (1997)511–535.

[24] M. Berg, Practices of reading and writing: the constitutiverole of the patient record in medical work, Sociol. HealthIllness 18 (4) (1996) 499–524.

[25] T.A. Massaro, Introducing physician order entry at a majoracademic medical center: I. Impact on organizational cultureand behavior, Acad. Med. 68 (January (1)) (1993) 20–25.

[26] B. Kaplan, Addressing organizational issues into theevaluation of medical systems, J. Am. Med. Inform. Assoc. 4

[27] J. Aarts, M. Berg, Same systems, differentoutcomes—comparing the implementation of computerizedphysician order entry in two Dutch hospitals, Methods Inf.Med. 45 (1) (2006) 53–61.

a l i n

i n t e r n a t i o n a l j o u r n a l o f m e d i c

[28] M.C. Beuscart-Zephir, S. Pelayo, F. Anceaux, D. Maxwell, S.

Guerlinger, Cognitive analysis of physicians and nursescooperation in the medication ordering and administrationprocess, Int. J. Med. Inform. 76S (June) (2007) S65–S77.

[29] Z. Niazkhani, H. Pirnejad, A. de Bont, J. Aarts, Evaluatinginter-professional work support by a computerized

f o r m a t i c s 7 8 ( 2 0 0 9 ) 170–181 181

physician order entry system, Stud. Health Technol. Inform.

136 (2008) 321–326.

[30] M. Weiner, T. Gress, D.R. Thiemann, M. Jenckes, S.L. Reel, S.F.Mandell, et al., Contrasting views of physicians and nursesabout an inpatient computer-based provider order-entrysystem, J. Am. Med. Inform. Assoc. 6 (3) (1999) 234–244.