A

TpcCRVa

c

TA

nachv

4

2

0d

The American Journal of Surgery (2009) 197, 113–118

ssociation for Surgical Education

eaching communication skills using the integratedrocedural performance instrument (IPPI): A randomizedontrolled trialarol-anne Moulton, M.D., B.S., M.Eda,b,*, Diana Tabakc,oger Kneebone, B.M., Ch.B., Ph.D.d, Debra Nestel, Ph.D.e, Helen MacRae, M.D.b,icki R. LeBlanc, Ph.D.a,f

Wilson Centre, Faculty of Medicine, University of Toronto; bDepartment of Surgery, University of Toronto;Standardized Patient Program, University of Toronto, Ontario, Canada; dDepartment of Biosurgery and Surgicalechnology, Imperial College London, London UK; eGippsland Medical School, Monash University, Churchill,

ustralia; fORNGE Transport Medicine, Toronto, Ontario, Canada

AbstractBACKGROUND: The Integrated Procedural Performance Instrument (IPPI) uses various bench-top

models positioned to standardized patients (SP) to recreate realistic clinical encounters. This studyassessed the effectiveness of using an IPPI format as a teaching tool for communication skills.

METHOD: Thirty-two participants underwent 2 videotaped IPPI scenarios before randomization into2 groups—experimental (SP-led feedback) or control (no feedback). Participants then completed 2further IPPI format scenarios. Videotapes were scored by 2 blinded independent raters using validatedassessment scales (communication and technical).

RESULTS: The experimental group performed significantly better on the communication scoresfollowing feedback compared with the control group (mean 77% vs 66%, P � .05). No difference inscores for technical skills post-intervention were demonstrated (checklist: experimental mean � 64%vs control � 59%, P � .40; global ratings: experimental mean � 66% vs no control � 62%, P � .37).

CONCLUSIONS: The IPPI is an effective tool for teaching communication skills in residents andmedical students and should be considered for incorporation into undergraduate and surgical curricula.© 2009 Elsevier Inc. All rights reserved.

KEYWORDS:Teaching;Formative assessment;Communication skill;Technical skill

aaeCsmdwroc

Surgeons are required to be good communicators.1 Theyeed to ensure that patients have all necessary informationbout their condition and identify and address their patients’oncerns. Surgeons also need to be good listeners and showigh levels of interpersonal skill.2 They have to read non-erbal cues of patients—the cues of anxiety, uncertainty, or

Presented at the ASE in Toronto, Ontario, Canada.* Corresponding author. Tel.: �1-416-340-4800, ext. 8819; fax: �1-

16-340-3792.E-mail address: Carol-anne.Moulton@uhn.on.caManuscript submitted June 6, 2008; revised manuscript August 16,

c008

002-9610/$ - see front matter © 2009 Elsevier Inc. All rights reserved.oi:10.1016/j.amjsurg.2008.09.006

nger—and what they are not saying and why. If these cuesre not read, the patient is left with an unsatisfying experi-nce and the surgeon has not achieved what was intended.ommunication and interpersonal challenges in the medical

etting are first introduced to future surgeons when they areedical students. This continues throughout surgical resi-

ency. Residents communicate daily with patients, oftenhile they are performing a procedure on them. Whether the

esident is on the surgical ward inserting a urinary catheterr in the emergency room suturing a wound, the technicalomponent of the procedure is combined with a communi-

ation and interpersonal challenge.

tacwfmnaratss

tmncct(tpvpite

ismamtpti

M

P

aprrar

E

m

mflfgwseta3vfsfaa

I

Pb

WwTSascd

UtsudTuw

114 The American Journal of Surgery, Vol 197, No 1, January 2009

In the core curriculum, many basic technical skills areaught in the skills laboratory, removed from patient inter-ction and, therefore, out of context.3 According to theognitive psychology literature, learning is most efficienthen done under contextually relevant situations.4 Aside

rom this, separating technical from communication skillsay be sending an erroneous message to trainees that tech-

ical skills are of higher value than communication skillsnd that these skills do not need to be integrated. There is aelatively small window of opportunity to teach these valu-ble skills before the resident is exposed to potential nega-ive learning experiences through poor role modeling in theo-called hidden curriculum. Therefore, residency programshould take a proactive approach to teaching these skills.5,6

Kneebone et al, through the development of an innova-ive tool—the Integrated Procedural Performance Instru-ent (IPPI)—have provided a means for combining tech-

ical challenge with communication challenge in variouslinical contexts using standardized patients.7–9 Each clini-al scenario uses a bench-top model (necessary for theechnical skill) placed on a fully briefed standardized patientSP) to simulate the realistic clinical setting. Skills requiredo deal with the anxious, confused, or angry patient can beracticed in “safe” settings with feedback obtained fromarious sources (the SP, supervisors, other residents). Im-ortantly, the encounter can be videotaped allowing formmediate debriefing and feedback.10 Prior to implementinghe IPPI in undergraduate medical or surgical curricula, itsffectiveness as a teaching tool needs evaluation.

The primary goal of this study was to determine whethermmediate feedback on communication and interpersonalkills in videotaped IPPI scenarios would lead to improve-ents of these skills in further IPPI scenarios. The second-

ry goal of this study was to assess whether any improve-ents in communication skills occurred at the expense of

he participants’ technical skills. Because of the need forarticipants to dual-task, we wanted to evaluate the effecthat this communication challenge and teaching had on partic-pants’ technical performance in the experimental group.11

ethods

articipants

Participants included 16 fourth-year medical studentsnd 16 junior surgical residents at the end of their firstostgraduate year at a large Canadian university. They wereecruited to participate in this study on a voluntary basis andeceived an honorarium for their time commitments. Ethicspproval was obtained from both university and hospitalesearch ethics boards.

xperimental design

Participants were randomized into 2 groups—experi-

ental (feedback) and control (no feedback)—before com-

encement of the IPPI scenarios on the day they presentedor the study. Participants were stratified according to theirevel of training (medical student or surgical resident) be-ore randomization to ensure an equal distribution to bothroups. Results of the differences between levels of trainingill be published in a follow-up study evaluating the con-

truct validity and psychometric properties of IPPI as anvaluation tool. All participants were involved in 2 video-aped IPPI scenarios (details below). Feedback was SP-lednd involved review of videotaped performances for up to0 minutes. All participants completed 2 post-interventionideotaped IPPI scenarios (details below). To capture per-ormance of their technical as well as their communicationkills, 2 videos were used for each scenario—1 close-up andocused on the technical component and the other wide-ngled and focused on the interaction between participantnd SP.

PPI scenarios

re-intervention scenarios. All participants completedoth scenarios.

ound closure. A simulated skin pad (used for bench top.ork) was attached to an SP’s right upper arm (Figure 1).he arm was then draped, exposing only the laceration. TheP was to behave as though she was mildly intoxicated andnxious about being within a hospital environment. Thecenario was outlined and the participants were instructed tolose the wound using interrupted 3/0 silk sutures and toeal with any patient concerns.

rinary catheterization. The urinary catheterization ana-omical model was placed on a clinical bed. The SP waseated at the head of the bed, leaning back with his legsnder the table. Both the SP and the bench-top model wereraped to create the appearance of a man lying on the bed.he SP was to behave as though he was anxious andncomfortable with the clinical procedure. The scenarioas outlined and the participant was instructed to insert a

Figure 1 IPPI wound suturing scenario.

un

P

Apiwpst

Sswwitcla

S

svbssFithtSivprrw

O

pibmpse

EEtea

F

tfnIitwfbt3t

D

sfsnirlrG

rawa

115C.-A. Moulton et al. Teaching communication skills using the IPPI

rinary catheter into this postoperative patient in acute uri-ary retention and deal with any patient concerns.

ost-intervention scenarios

All participants completed both scenarios.

pplication of a cast. A SP had a severe wrist sprain andresented to the emergency room. She was to behave in anrritated manner for being told to keep her hand immobilehile she waited for nearly 2 hours for splinting. Thearticipants were instructed to place the patient in a volarplint (immobilizing the wrist joint with a plaster slab) ando deal with any patient concerns.

kin lesion removal station. A skin pad containing aimulated mole was placed on an SP’s lower arm. The SPas to portray a patient who was congenitally blind (whichas not to be revealed if the participant did not recognize

t.) She wore dark glasses and had a white cane leaning onhe wall beside her. She was worried that the mole wasancerous. The participants were instructed to remove theesion on the skin under local anesthetic, to close the woundnd deal with any patient concerns.

tandardized patient training

The SPs were trained before the commencement of thetudy. They were told what to expect from a technicaliewpoint and notified of potential differences in skill leveletween participants. They were able to familiarize them-elves with the bench-top models and were shown videos ofimilar scenarios performed at Imperial College London.or each scenario, they were given a character description

ncluding name, age, behavior, affect, mannerisms, ques-ions and prompts, history of present illness, past medicalistory, social history, and family history. For consistency,he same SP was used for each scenario in all study sessions.Ps who delivered feedback were not involved in the post-

ntervention IPPI scenarios. All SPs in this study had pre-iously been trained by the Toronto SP Program (SPP) inroviding feedback for communication and interpersonalelated issues they were experienced in using the globalating form as a framework for the feedback process andere selected based on their ability.

utcome measures

The participants’ technical skills were assessed usingreviously validated assessment scales—a task specific 26-tem checklist and a 5-item (each item anchored by 5-pointehavioral descriptors) global rating scale (GRS).12,13 Com-unication and interpersonal skills were assessed using a

reviously validated global 5-item scale that divides thekills into overall communication ability and 4 components:

mpathy, coherence, verbal, and nonverbal expression.14,15 b

ach category is anchored by 5-point behavioral descriptors.ach scenario was videotaped to enable assessment of both

echnical and communication skills at a later date by 2xpert surgeons blinded to the participants’ level of trainingnd group assignment.

eedback sessions

Immediately after each interaction with the participant,he SP completed the validated communication assessmentorm (same as that used by the blinded raters) and wroteotes relevant to issues of communication. After the first 2PPI scenarios, those participants randomized to the exper-mental group met with the involved SP to receive struc-ured feedback. The SP and participant met in a private areaith a television monitor and discussed the interaction re-

erring to the tape when necessary (Figure 2). Each feed-ack session lasted 15 minutes. Across the 2 scenarios,herefore, the total duration of feedback and discussion was0 minutes. Participants then completed the 2 post-interven-ion scenarios.

ata analysis

Communication scores were submitted to a mixed-de-ign analysis of variance (ANOVA), with feedback (noeedback, feedback) and level of training (residents, medicaltudents) as between-subjects variables and scenario (uri-ary catheterization, wound closure, cast, skin lesion) andtem (overall, empathy, coherence, verbal, nonverbal) as aepeated measure. To determine the level of inter-rater re-iability of the IPPI format examination, the interclass cor-elations coefficient were calculated for the checklist, theRS, and the communications scales.Expert scores on the technical (checklists and global

atings) scales were submitted to separate mixed-designnalyses of variance with task (urinary catheterization,ound closure, cast, skin lesion) as the repeated measures,

nd level (residents, students) and group (no feedback, feed-

Figure 2 Standardized patient feedback session.

ack) as the between-subject variables.

R

C

pnpifctstogwbpoetwdpf

s(

Ep

ptt(

C

StcbvtasviIf

Fp

Fp

116 The American Journal of Surgery, Vol 197, No 1, January 2009

esults

ommunication scores

Overall ANOVA results with 4 scenarios (2 pre- and 2ost-interventions) are shown in Table 1. There was a sig-ificant interaction between group and scenarios (pre- andost-feedback; P � .05) demonstrating that in the post-ntervention scenarios, the experimental group that receivedeedback outperformed the control group that did not re-eive feedback. Figure 3 demonstrates that, before interven-ion, there were no differences between communicationcores of the 2 groups (experimental vs control) indicatinghat the randomization process was effective. Figure 4 dem-nstrates that communication scores within each subcate-ory of the communication on post-intervention scenariosere significantly higher for participants who received feed-ack than for those who did not receive feedback. Partici-ants in the feedback group outperformed the control groupn every item of the communication scale—overall score,mpathy, coherence, verbal, and nonverbal expression—inhe 2 IPPI scenarios following intervention. These resultsere confirmed by separate mixed-design ANOVAs con-ucted on pre- and post-intervention scenarios. For there-intervention scenarios, there was no significant effect ofeedback (P � .75, Table 2). For the 2 post-intervention

igure 3 Average scores (%) for communication subscales on

Table 1 Overall ANOVA results in communication scoreswith 4 IPPI scenarios

df F MSE P

Scenario 3,84 2.33 .010 .08Level 1,28 .326 .058 .573Feedback 1,28 2.23 .058 .147S � L 3,84 .47 .010 .98S � F 3,84 2.86 .010 .042L � F 1,28 .094 .058 .761S � L � F 3,84 1.61 .010 .193

S � scenario, pre- and post-intervention scenarios; L � level,medical student or surgical resident; F � feedback, experimental orcontrol group.

re-intervention scenarios.

cenarios, there was a significant main effect of feedbackP � .05, Table 3).

ffect of communication feedback on technicalerformance

The results of the anova on the participants’ technicalerformance (GRS and the checklist scores) indicate thathere was no significant difference between the experimen-al group and the control group for any of the scenariosFigures 5 and 6).

omments

In this study, it was demonstrated that a brief exposure toP feedback on communication and interpersonal skills led

o a significantly better performance in all subscales inommunication performance than those receiving no feed-ack. There were no other significant interactions when allariables were entered into an ANOVA (Table 2). Impor-antly, the enhanced communication performance did notppear to be at the expense of technical performance. Thistudy is encouraging as it demonstrates that even after aery brief interaction with a trained SP, communication andnterpersonal skills can likely be enhanced. Given that thePPI scenarios were not standardized in terms of difficultyor either communication or technical skill, it is not possible

igure 4 Average scores (%) for communication subscales onost-intervention scenarios.

Table 2 Pre-intervention ANOVA results

df F MSE P

Scenario 1,28 4.91 .012 .035Level 1,28 .313 .035 .580Feedback 1,28 .102 .035 .752S � L 1,28 .008 .012 .929S � F 1,28 .248 .012 .623L � F 1,28 .001 .035 .979S � L � F 1,28 2.95 .012 .097

S � scenario, 2 pre-intervention scenarios; L � level, medicalstudent or surgical resident; F � feedback, experimental or controlgroup.

tvcmgbowhuasomtomp“tpr

ddwme

OaftpOua

taccppgccpetgomnlivtitpcistmHctwpF

t

Fi

117C.-A. Moulton et al. Teaching communication skills using the IPPI

o discuss improvements in performance from the pre-inter-ention to the post-intervention IPPI scenarios. What wean say is that after the feedback intervention, the experi-ental group performed significantly better than the control

roup. We did not assess whether these differences lastedeyond the immediate period of the study, and this questionf retention should be addressed with further research. Weould anticipate that communication skills need to be re-earsed like any other learned skill, and should be practicedsing the principles of “deliberate practice,” with reflectionnd feedback an essential component.16 How frequent theseessions should be is unclear though weekly training as partf the normal surgical skills curricula would be a reasonableodel to start with.17 Unlike technical skills that may be

aught in the laboratory and not practiced for a long periodf time, communication and interpersonal skills are de-anded of the resident daily in their interactions with real

atients. It is therefore possible that this may provide thedistributed” practice necessary for ongoing improvements,hough incorporation of teaching communication and inter-ersonal skills into a more formal curriculum will enable theequirements of “deliberate practice” to be satisfied.16

Dual-tasking is a term used in the attention literature toescribe the need to split cognitive resources between 2ifferent tasks.18 Each of us has a limited capacity withhich we can attend to stimuli or tasks in our environ-ent.19 We are free to split our attention between numerous

vents or stimuli if we remain within our capacity threshold.

igure 5 Average technical checklist scores (%) for experimen-

Table 3 Post-intervention ANOVA results

df F MSE P

Scenario 1,28 .076 .008 .785Level 1,28 .242 .034 .627Feedback 1,28 6.042 .034 .020S � L 1,28 .149 .008 .702S � F 1,28 1.10 .008 .303L � F 1,28 .298 .034 .589S � L � F 1,28 1.10 .008 .303

S � scenario, 2 post-intervention scenarios; L � level, medicalstudent or surgical resident; F � feedback, experimental or controlgroup.

dal and control groups for each scenario.

nce we reach that threshold, we are no longer able tottend to additional stimuli, unless attention is taken awayrom other activities. Performance in these other activitieshen necessarily falters. The amount of attention needed toerform a particular task or attend to certain stimuli varies.ne of the variables is how “expert” you are at that partic-lar task. A surgeon, for example, would use much lessttention to suture a wound than a medical student.

Medical students and junior residents are at the start ofheir learning curves for many minor clinical proceduresnd are therefore using a larger proportion of their attentionapacity to perform that activity than experienced clini-ians.20 The additional demands of communicating whileerforming a procedure require dual-tasking and may affecterformance in either activity. If the technical component isiven higher priority, attention may be taken away fromommunicating. If communication is given priority, techni-al performance may falter. This study found that bettererformance on the communication scales occurred in thexperimental group compared with the control group afterhe intervention with no significant differences betweenroups in technical performance for either medical studentsr residents. In other words, enhanced performance in com-unication did not coexist with a deterioration in the tech-

ical performance as might be expected by the dual-taskingiterature. In fact, both groups scored lower in the post-ntervention scenarios when compared with the pre-inter-ention scenarios. A possible explanation for this may behat the IPPI scenarios postintervention were more challeng-ng than pre-intervention, although further testing is neededo explore this hypothesis. It could also be explained if allarticipants found the communication and interpersonalhallenges of the post-intervention scenarios more challeng-ng (dual-tasking effect). Importantly, the lower technicalcores occurred in both groups and so the effect of dual-asking was not found to be more prominent in the experi-ental (feedback) group compared with the control group.ad this been the case, the higher communication scores

ould be explained by attention that was diverted from theechnical component. What was demonstrated in this studyas significantly higher communication scores in the ex-erimental group compared with the control group with no

igure 6 Average technical global rating scores (%) for exper-mental and control groups for each scenario.

ifferences between the groups in technical performance.

bssrtnCmdtbapstaa

A

g

R

1

1

1

1

1

11

1

1

1

2

118 The American Journal of Surgery, Vol 197, No 1, January 2009

This study provides preliminary evidence that IPPI cane used to provide effective formative feedback. Furthertudies are needed to see how best to incorporate this intourgical curricula and whether these skills transfer to theeal clinical setting. Formal surgical curricula that teachechnical skills, while ignoring communication are at risk ofot fulfilling the requirements for a fully trained surgeon.ommunication skills are taught in a formal way in manyedical schools but this teaching appears to be discontinued

uring residency. However, with increased exposure to pa-ients and communication challenges, residency may be theest time to focus on teaching these skills and providingvenues for structured feedback. Communication and inter-ersonal skills training should be formally incorporated intourgical curricula—both in terms of teaching and evalua-ion. The integration of SPs and bench-top models opens up

realm of possibilities for both formative and summativessessment and should be the focus of further research.

cknowledgment

This study was graciously funded by a METI researchrant from the American College of Surgeons.

eferences

1. Frank JR, Jabbour M, Tugwell P, et al. Skills for the new millennium:report of the Societal Needs Working Group, CanMEDS; 2000 Project.Ann R Coll Physicians Surg Can 1996;206–16.

2. Outcome Project. ACGME 1999. Available from: http://www.acgme.org/outcome/comp/compMin.asp.

3. Undergraduate education. University of Toronto, Department of Sur-gery. 2003. Available from: http://www.surg.med.utoronto.ca/AnnRep/

AR01_02/Undergrad.html.

4. Custers EJ, Regehr G, Norman GR. Mental representations of medicaldiagnostic knowledge: a review. Acad Med 1996;71:S55–61.

5. Ginsburg SR, Regehr G, Mylopoulos M. Reasoning when it counts:students’ rationales for action on a professionalism exam. Acad Med2007;82:S40–3.

6. Epstein RM. Mindful practice. JAMA 1999;282:833–9.7. Kneebone R, Nestel D, Yadollahi F, et al. Assessing procedural skills

in context: exploring the feasibility of an integrated procedural per-formance instrument (IPPI). Med Educ 2006;40:1105–14.

8. Nestel D, Kneebone R, Black S. Simulated patients and the develop-ment of procedural and operative skills. Med Teach 2006;28:390–1.

9. Kneebone R, Bello F, Nestel D, et al. Training and assessment ofprocedural skills in context using an integrated procedural perfor-mance instrument (IPPI). Stud Health Technol Inform 2007;125:229 –31.

0. Barrows HS, Norman GR, Neufeld VR, et al. The clinical reasoning ofrandomly selected physicians in general medical practice. Clin InvestMed 1982;5:49–55.

1. Treisman A. How the deployment of attention determines what we see.Vis Cognit 2006;14:411–43.

2. Martin JA, Regehr G, Reznick R, et al. Objective structured assess-ment of technical skill (OSATS) for surgical residents. Br J Surg1997;84:273–8.

3. Regehr G, Macrae H, Reznick RK, et al. Comparing the psychometricproperties of checklists and global rating scales for assessing perfor-mance on an OSCE-format examination. Acad Med 1998;73:993–7.

4. Hodges B, McIlroy JH. Analytic global OSCE ratings are sensitive tolevel of training. Med Educ 2003;37:1012–6.

5. Hodges B. Validity and the OSCE. Med Teach 2003;25:250–4.6. Ericsson KA. Deliberate practice and the acquisition and maintenance

of expert performance in medicine and related domains. Acad Med2004;10:S1–12.

7. Moulton CA, Dubrowski A, Macrae H, et al. Teaching surgical skills:what kind of practice makes perfect?: A randomized, controlled trial.Ann Surg 2006;244:400–9.

8. Kahneman D. Attention and Effort. Englewood Cliffs, NJ: Prentice-Hall; 1973.

9. Moray N. Where is capacity limited? A survey and a model. ActaPsychol 2007;27:84–92.

0. Moulton CA, Regehr G, Mylopoulos M, et al. Slowing down whenyou should: a new model of expert judgment. Acad Med 2007;82:

S109 –16.