Augmented Reality Based Skill Training for Robot-Assisted Urethro-Vesical Anastamosis: A...

Augmented-reality-based skills training forrobot-assisted urethrovesical anastomosis:a multi-institutional randomised controlled trialAshirwad Chowriappa*†, Syed Johar Raza*, Anees Fazili‡, Erinn Field*, Chelsea Malito*,Dinesh Samarasekera§, Yi Shi*, Kamran Ahmed¶, Gregory Wilding*, Jihad Kaouk§,Daniel D. Eun**, Ahmed Ghazi‡, James O. Peabody††, Thenkurussi Kesavadas†,James L. Mohler* and Khurshid A. Guru**Roswell Park Cancer Institute, †State University of New York, Buffalo, ‡University of Rochester Medical Center, Rochester,NY, §Cleveland Clinic Foundation, Cleveland, OH, **Temple University Hospital, Philadelphia, PA, ††Henry Ford HealthSystem, Detroit, MI, USA, and ¶King's College, London, UK

ObjectiveTo validate robot-assisted surgery skills acquisition using anaugmented reality (AR)-based module for urethrovesicalanastomosis (UVA).

MethodsParticipants at three institutions were randomised to aHands-on Surgical Training (HoST) technology group or acontrol group. The HoST group was given procedure-basedtraining for UVA within the haptic-enabled AR-based HoSTenvironment. The control group did not receive any training.After completing the task, the control group was offeredto cross over to the HoST group (cross-over group). Aquestionnaire administered after HoST determined thefeasibility and acceptability of the technology. Performance ofUVA using an inanimate model on the daVinci SurgicalSystem (Intuitive Surgical Inc., Sunnyvale, CA, USA) wasassessed using a UVA evaluation score and a Global EvaluativeAssessment of Robotic Skills (GEARS) score. Participantscompleted the National Aeronautics and Space AdministrationTask Load Index (NASA TLX) questionnaire for cognitiveassessment, as outcome measures. A Wilcoxon rank-sum testwas used to compare outcomes among the groups (HoST

group vs control group and control group vs cross-overgroup).

ResultsA total of 52 individuals participated in the study. UVAevaluation scores showed significant differences in needledriving (3.0 vs 2.3; P = 0.042), needle positioning (3.0 vs 2.4;P = 0.033) and suture placement (3.4 vs 2.6; P = 0.014) in theHoST vs the control group. The HoST group obtainedsignificantly higher scores (14.4 vs 11.9; P 0.012) on theGEARS. The NASA TLX indicated lower temporal demandand effort in the HoST group (5.9 vs 9.3; P = 0.001 and 5.8 vs11.9; P = 0.035, respectively). In all, 70% of participants foundthat HoST was similar to the real surgical procedure, and 75%believed that HoST could improve confidence for carrying outthe real intervention.

ConclusionTraining in UVA in an AR environment improves technicalskill acquisition with minimal cognitive demand.

Keywordsanastomosis, augmented reality, robot-assisted, robotic, skillstraining, urethrovesical anastomosis

IntroductionSurgical simulation-training platforms such as virtual reality(VR) have emerged as useful adjuncts to training in andoutside the operating theatre [1]. Simulation-based trainingcan enhance safety profile by bridging the gap between thesafe acquisition of technical skills and effective operativeperformance [2,3].

Procedure-based training allows the trainee to put intopractice fundamental surgical skills to perform more complex

procedural tasks. To master robot-assisted surgery, a surgeonneeds to develop an understanding of three-dimensionalspatial orientation, depth perception and the role ofmagnification, with coordinated hand manoeuvres [4]. Arecent review on laparoscopic training concluded thatsimulators offering procedure-based training were moreefficient at training [5,6]. Although VR training devices havebeen shown to be valid and effective for training, their abilityremains limited in terms of their representation of a realisticprocedure-specific environment [7]. Augmented reality (AR) is

© 2014 The AuthorsBJU International © 2014 BJU International | doi:10.1111/bju.12704 BJU Int 2014Published by John Wiley & Sons Ltd. www.bjui.org wileyonlinelibrary.com

an effective alternative that enhances the face validity oftraining modules [8–10]. AR offers a real view of an operatingenvironment in which the events on the field are enhancedand supplemented [11].

Difficulty in mastering intracorporeal suturing andanastomosis may prevent practising surgeons fromperforming advanced procedures. Despite the presence ofEndo-wrist® technology in robot-assisted surgery, developingthis skill has limitations in terms of complexity of steps, timerestraints and related complications [12–14]. AR environmentscould promote trainees’ cognitive skills and conceptualunderstanding and correct their misconceptions [15,16].

The aim of the present randomised controlled trial was toevaluate the effectiveness of Hand-on Surgical Training(HoST) technology-based (Fig. 1) urethrovesical anastomosis(UVA) by comparing this technology with the existingstandards of training.

Materials and MethodsStudy Design and Participants

After obtaining institutional review board approval (I228012), we enrolled participants from the Roswell ParkCancer Institute, the Cleveland Clinic Foundation and theUniversity of Rochester Medical Center in a multicentre,randomised controlled trial. In phase I, all eligibleparticipants enrolled were randomised to receiveHoST-based UVA training (HoST group) or to a controlgroup that did not receive HoST. In phase II (Fig. 2), thecontrol group participants were offered the opportunity tocross over and complete HoST-based training (cross-overgroup). Participants in the study had minimal (≤25 h onrobotic console) or no previous robotic experience; thosewho had previous simulation and robotic console experience(>25 h) were excluded from the study. Before randomisationall participants completed a basic demographic pre-studyquestionnaire. All participants were randomised based on a1:1 stratified permuted block scheme.

Development of Hands-on Surgical Training

The HoST-AR-based procedures used in the present studywere developed by Roswell Park Cancer Institute and the StateUniversity of New York at Buffalo, Virtual Reality Laboratoryin 2009. HoST technology provides a novel simulation-basedenvironment that augments a real surgical procedure within aVR framework. It analyses the steps of a procedure performedby expert surgeons and converts them to a sequence ofhaptic-enabled AR-based procedures.

The HoST environment is enhanced with didactic audio andvisual explanations along with anatomically relevantannotations/illustrations of the critical steps of the procedure(Fig. 3). Once a trainee is immersed in the HoST environmenton the Robot-assisted Surgical Simulator (RoSS), the RoSSmanipulator takes over and guides the trainee throughhaptic-enabled prompts during the procedure.

Robot-assisted Surgical Simulator and FundamentalSkills of Robotic Surgery Curriculum

We used a RoSS console [17,18] with its Fundamental Skills ofRobotic Surgery (FSRS) curriculum [19,20] and HoST-basedUVA in the study. The FSRS curriculum is a sequential,modular criterion-based structured curriculum for acquiringthe basic skills of robot-assisted surgery. The FSRS curriculumhelps develop a safe human–machine interface, allowing thenovice surgeon to master the basic functions of the daVinciSurgical System (dVSS™; Intuitive Surgical Inc., Sunnyvale,CA, USA) console.

Training and Testing for Urethrovesical AnastomosisPerformance on the daVinci Surgical System

Participants in the HoST group were introduced to the RoSSand completed four basic simulation tasks once from the FSRScurriculum; ball placement, spatial control, tissue retractionand knot-tying, before moving on to HoST training.Participants in the HoST group were given four HoSTsessions, with each session lasting for no more than 20 min.

Fig. 1 Hands-on Surgical Training is a

framework that provides haptic-enabled

augmented-reality-based training for

robot-assisted surgery. This exercise divides the

entire operation into various steps and

provides the trainees with simultaneous

proctoring throughout the procedure.

Chowriappa et al.

© 2014 The Authors2 BJU International © 2014 BJU International

HoST and RoSS training were completed within 2 weeks ofrecruitment to the study. Assessment of UVA performancewas carried out within 24 h of the last HoST session. Beforethe assessment, the participants were given a short didacticsession to introduce them to the features of the dVSS. Thissession included a one-on-one familiarisation by an

experienced robot operator. After a 5-min period ofacclimation, participants completed the ball placement, suturepass, fourth arm manipulation and suturing tasks on the dVSS.These tasks were selected because they encompass theimportant robotic skills required to successfully perform UVAon an inanimate model using the dVSS.

HoST Study Consort Diagram

Enro

lmen

tA

lloca

tion

Ana

lysis

Cros

s-ov

er

Excluded (n=0)

Assessed for Eligibility (n=52)

Randomisation

Lost to Follow-Up (n=4)Lost to Follow-Up (n=2)

HoST Group (n=22)Performed tasks on the RoSS Simulator

& daVinci Surgical SystemTM

Test: Urethrovesical Anastomosison an Inanimate Model

NASA TLX

Phas

e IPh

ase I

ILost to Follow-Up (n=5)

Cross-over to HoST (n=17)

Video-recordings of the Test assessed by Expert Reviewers using1. GEARS global rating scale 2. Objective Urethrovesical Anastamosis Evaluation

Control Group (n=24)Performed tasks on the daVinci

Surgical SystemTM

Fig. 2 Algorithm for randomisation process

used in the study. HoST, Hands-on Surgical

Training; RoSS, Robot-assisted Surgical

Simulator; NASA TLX, National Aeronautics and

Space Administration Task Load Index; GEARS,

Global Evaluative Assessment of Robotic Skills.

UVA test on an in-animate Model

Description: Participants performed UVAusing V-LocTM (Covidien) suture on the daVinci Surgical SystemTM (IntuitiveSurgical Inc.).

Fig. 3 Final test performed on daVinci Surgical

system™.

Augmented-reality-based procedure specific training

© 2014 The AuthorsBJU International © 2014 BJU International 3

Participants in the control group were introduced to the RoSSsystem in the same way as the HoST group. The control groupdid not undergo HoST, instead they watched the four videosshowing UVA being performed. They did this for the samelength of time as those who underwent HoST. Within 24 h oflast viewing of the videos the control group were assessed ontheir performance of UVA. The control group were given thesame short didactic session as the HoST group to introducethem to robotic surgery and the features of the dVSS. After a5-min period of acclimation on the console, participantscompleted the four tasks on the dVSS console.

After completion of the four tasks using the robot, bothgroups performed UVA on an inanimate model and theirperformance was video-recorded for expert analysis. Oncompletion of this session, participants in the control groupwere offered the opportunity to cross over and completeHoST, and form the cross-over group. All participantscompleted the National Aeronautics and Space Administrationtask load index (NASA TLX) assessment questionnaire aftercompleting UVA on the dVSS console.

Outcome Measures

Feasibility and acceptability

Information was collected, using the post-HoST quantitativequestionnaire, about the realism of the HoST-basedaugmented environment, technical issues related to theHoST, and its overall efficacy pertinent to UVA training(Table 1).

Concurrent validity

Concurrent validity was established by comparing theperformances of the different participants in the HoST-basedtraining on the ‘gold standard’ dVSS, using an inanimate UVAmodel. Performance was recorded from a direct video feedfrom the dVSS. The performance was assessed by expertrobotic surgeons using the Global Evaluative Assessment ofRobotic Skills (GEARS) score and a UVA assessment tool(Fig. 4).

Cognitive assessment

The NASA TLX was used to evaluate mental fatigue (Table 2).The participants completed the NASA TLX questionnaireimmediately after completing UVA on the dVSS.

Tools for Evaluation

GEARS assessment score

Basic robotic skills deemed essential for performing UVA wereassessed using the GEARS assessment score [21]. GEARS iscomposed of six domains, including depth perception,bimanual dexterity, efficiency, autonomy, force sensitivity androbotic control. Proficiency is measured on a five-point Likertscale, with specific performance anchors at 1, 3 and 5.Urethrovesical anastomosis evaluation score

A UVA evaluation score was developed by deconstructingthe fundamental elements of UVA (needle positioning,needle driving, suture placement and tissue manipulation)in consultation with an expert robotic surgeon (>1000procedures with >50% console time). A rating of 1corresponded to the lowest quality of performance (Fig. 4).NASA Task Load Index assessment

The level of cognitive workload while performing UVA wasevaluated using the NASA TLX scale [22]. The NASA TLX isa subjective workload assessment technique that relies on amultidimensional construct to derive an overall workloadscore based on six weighted subscales: mental demand,physical demand, temporal demand, performance, effort andfrustration level (Table 2).

In summary, participants were assessed using the following:GEARS score; objective UVA evaluation score; and cognitivetask load using the NASA TLX. Videos were blinded andanalysed by four expert independent surgeons. Inter-raterreliability for both GEARS and the object UVA evaluationscores was evaluated.

Statistical Analyses

Subjects were randomised to the HoST group or control groupusing a permuted block randomisation scheme with a block of

Table 1 Results of the questionnaire administered after Hands-on Surgical Training.

Question type Description Participants who agreedwith description, %

Realism HoST simulation was as realistic as the surgical procedure 70The rehearsal environment and simulated surgical steps were appropriate and a learning experience 76

Technical issues The simulation can possibly increase the overall efficiency of procedure 76The simulation can possibly alter preconceived choice of surgical steps 54The simulation is useful to practise the real case before performing it on the patient 70

Flow and overall efficacy The simulation can possibly aid in patient safety 76The HoST can possibly enhance confidence for the real intervention 75

HoST, Hands-on Surgical Training.

Chowriappa et al.


size 2. The measured outcomes were summarised both overalland according to relevant demographic and baseline variables.Descriptive statistics such as frequencies and relativefrequencies were computed for all categorical variables.Numeric variables were summarised using simple descriptive

statistics such as mean, SD and range values. Wilcoxonrank-sum tests were used to compare outcomes statisticallybetween the independent groups (HoST group vs controlgroup) as well as the dependent groups (control group vscross-over group). A sample size of 15 participants in each

Needle Position *Level (1-5) Critical Error (CE) CE Check

Held needle at 1/3 to 2/3 from tip

Not able to see tip due to inappropriate position

Angles entering tissue are appropriate

Needle leading to tearing of adjacent tissue

Use wrist for rotation Tear tissue and not using therotation of the endo-wrist

Less than two attempts at proper positioning of needle Trauma to adjacent area

Needle Driving

Tissue torn while manipulating tissue If yes then critical error

Remove needle along curve of the needle Touch adjacent tissue

Appropriate tension and pull around the suture

Disrupt suture from needle in-vivo

Suture Placement & Tissue Manipulation

Appropriate distance between each suture Wide gaps

Proper tissue approximation Unable to align opening

Proper alignment of posterior urethral plate

Wide opening/easily will lead to false passage

Appropriate knot placement Air-knot

*Level: 1=poor; 2=below average; 3=average; 4=above average; 5=excellent

Fig. 4 Objective urethrovesical anastomosis

evaluation.

Table 2 National Aeronautics and Space Administration Task Load Index assessment.

NASA TLX rating scale description

Endpoint Level Rating scale definitions

Mental Demand Low/High How much mental and perceptual activitywas required (e.g. thinking, deciding, etc.)?

Physical Demand Low/High How much physical activity was required?Temporal Demand Low/High How much time pressure did you feel due to the rate or pace at which

the tasks or task elements occurred?Performance Low/High How hard did you have to work (mentally and physically) to

accomplish your level of performance?Effort Low/High How successful do you think you were inaccomplishing your

responsibilities?Frustration Level Low/High How discouraged, irritated, stressed and annoyed versus gratified,

content, relaxed and complacent did you feel?

NASA TLX, National Aeronautics and Space Administration Task Load Index.



group was calculated to be sufficient to detect a difference of1.1 SD points with 80% power. Fisher’s exact test was used toassess the statistical association between questionnaireoutcome variables and group variables. All tests weretwo-sided and tested at a 0.05 nominal significance level.

ResultsDemographic Characteristics

Out of 52 participants, 22 were residents and 30 were fellows.There were no significant differences in the baselineparticipant characteristics between the HoST and controlgroups with respect to demographics, education or previoussurgical experience (Table 3).

Feasibility and Acceptability

In all, 70% of participants believed that the HoST AR-basedenvironment was as realistic as the actual surgical procedure(Fig. 4). A total of 76% of participants felt that the ARenvironment and surgical steps were appropriate for learningand 70% believed that HoST would prove useful for practicebefore performing the real case procedure.

Concurrent Validity

GEARS assessment

The HoST group scored significantly better than the controlgroup in overall GEARS score (P = 0.012). Differences werepredominantly seen in bimanual dexterity (P = 0.016) andforce sensitivity (P < 0.001). The cross-over group also had asignificantly higher overall GEARS score (P = 0.008) than thecontrol group. Cross-over group performance was significantlyhigher in the bimanual dexterity (P = 0.048), efficiency (0.046),force sensitivity (P = 0.011) and autonomy (P = 0.004)domains (Table 4).

Objective urethrovesical anastomosis assessment

The HoST group demonstrated significantly betterperformance with regard to needle positioning, needle driving,suture placement and tissue manipulation compared with thecontrol group (P = 0.008, 0.042 and 0.014, respectively). Thecross-over group similarly outperformed the control group inthese three domains (P = 0.018, 0.037 and 0.013, respectively[Table 5]).

Table 3 Demographic characteristics of the participants.

Participant characteristic HoST group, N = 26 Control group, N = 26 P

DemographicsAge, n 0.27

<40 years 25 23>40 years 1 3

Gender, n 0.24Female 10 7Male 16 19

Dominant hand, n 0.91Left 2 3Right 24 23

EducationLevel of training, n 0.87

Fellow 14 16Resident 12 10

PGY level, n 0.94<5 12 10>5 14 16

Formal laparoscopic training, n 0.91No 13 11Yes 13 15

dvSS experiencePrevious robotic console experience, n 1.0

No 24 23Yes 2 3

Console experience, n 0.470 h 24 23<25 h 2 3>25 h 0 0

Robotic simulator experiencePrevious simulator training for the dVSS 1.0

No 26 26Yes 0 0

HoST, Hands-on Surgical Training; dVSS, daVinci Surgical System; PGY, Post Graduate Year.

Chowriappa et al.


Inter-rater reliability for the GEARS score and theUVA evaluation score were found to be 0.72 and 0.68respectively.

Cognitive Load Assessment

The participants in the HoST group had significantly lowerNASA TLX scores for temporal demand (9.3 vs 5.9; P =0.001) and effort (11.9 vs 5.8; P = 0.035) compared with thecontrol group, and were found to have less mental fatigue(Table 6). The cross-over group had significantly lower scoresin most domains and had a lower overall score comparedwith the control group (6.2 vs 9.4; P = 0.012). The cross-overgroup also had significantly lower mental demand (11.8 vs6.8; P < 0.001), physical demand (9.2 vs 6.1; P = 0.013), effort(11.0 vs 7.8; P = 0.037) and frustration scores (7.6 vs 4.2;P = 0.006).

Discussion

This randomised controlled trial is the first to observe theeffects of any AR-based UVA training in robot-assistedsurgery. We aimed to determine the feasibility, acceptabilityand concurrent validity of a novel HoST framework onUVA in robot-assisted surgery. The study showed thatprocedure-based training in an AR environment can improveboth operating performance and cognitive workload whenusing the dVSS.

We assessed performance using the GEARS assessment tool,which is a validated scoring system used to assess surgicalskills during robot-assisted surgery. GEARS assessment isgeneric and not pertinent to a specific task performed [21].To address task-specific skills in performing UVA, anobjective UVA evaluation score was developed. Themajority of the participants confirmed that UVA

Table 4 Global Evaluative Assessment of Robotic Skills assessment scores for phase I (control groupvs HoST group) and phase II (control group vs cross-over group).

Skill Control group, n = 24Mean (SD) GEARS score

HoST group, n = 22Mean (SD) GEARS score

P

Control group vs HoST groupBimanual dexterity 2.4 (1.0) 2.9 (0.2) 0.016Efficiency 2.5 (0.9) 2.8 (0.3) 0.915Force sensitivity 2.0 (0.8) 2.5 (0.2) <0.001Autonomy 2.0 (1.1) 2.7 (0.4) 0.099Robotic control 3.0 (0.7) 3.5 (0.3) 0.101Overall score 11.9 (4.1) 14.4 (1.2) 0.012

Control group vs cross-over groupBimanual dexterity 2.4 (1.0) 3.0 (0.7) 0.048Efficiency 2.5 (0.9) 3.0 (0.8) 0.046Force sensitivity 2.0 (0.8) 2.7 (0.7) 0.011Autonomy 2.0 (1.1) 3.1 (1.0) 0.004Robotic control 3.0 (0.7) 3.4 (1.1) 0.259Overall score 11.9 (4.1) 15.3 (3.2) 0.008

GEARS, Global Evaluative Assessment of Robotic Skills.

Table 5 Objective urethrovesical anastomosis evaluation for phase I (control group vs HoST group)and phase II (control group vs cross-over group).

Control group vs HoST group

Skill type Control group, n = 24Mean (SD) UVA score

HoST group, n = 22Mean (SD) UVA score

P

Needle position 2.4 (0.8) 3.0 (0.7) 0.008Needle driving 2.3 (1.0) 3.0 (0.9) 0.042Suture placement and tissue manipulation 2.6 (0.9) 3.4 (0.9) 0.014

Control group vs cross-over group

Skill type Control group, n = 24Mean (SD) UVA score

Cross-over group, n = 17Mean (SD) UVA score

P

Needle position 2.4 (0.8) 3.5 (0.3) 0.018Needle driving 2.3 (1.0) 3.3 (0.6) 0.037Suture placement and tissue manipulation 2.6 (0.9) 3.9 (0.5) 0.013



performed on the dVSS resembled the HoST-based UVAmodule.

Training modules based on AR may specifically help toaddress anatomic variability, different operating scenarios anddidactic education, along with a true hands-on surgical skillsacquisition. The participants who underwent AR-based UVAtraining had significantly better GEARS scores in terms ofbimanual dexterity and force sensitivity. Similar results wereobserved using all objective UVA evaluation metrics.Additionally, the effect of AR-based training was substantiatedby the observed improvement in performance once thecontrol group crossed over to the HoST arm.

Cognitive workload was also assessed using the NASA TLXscore. The HoST group had a consistently lower task load inall aspects of the NASA TLX score but scored significantlylower in terms of temporal demand and effort in comparisonwith the control group. A similar trend of a statisticallysignificant reduction in the overall NASA TLX score and inmost of the sub-domain scores was observed when the controlgroup crossed over to undergo AR-based UVA training. Theexistence of a relationship between cognitive workload andskills acquisition in the early phase of learning new surgicalskills has been debated, but these correlations seem to declinewhen the procedure becomes routine [23]. This could beanother explanation for the lower scores in both traininggroups.

Robot-assisted surgical training has focused mainly onaddressing the concerns related to the human–machine

interface; namely, mastering the robotic console, and varioussimulators have been assessed in validation studies [1]. Toobtain the full benefit of training, robotic skills simulatorsmust also provide training for procedure-specific skills. Thepresent study is the first to address the concerns of acquiringskills pertinent to a specific procedure in robot-assistedsurgery. Various studies have evaluated box trainers withanimal tissue models and VR-based training modules tosimulate UVA and knot-tying [24–27]. All these moduleshave failed to immerse the trainee in a true surgical field.The unique aspect of guiding the trainee’s hand (like that ofan expert) allows the trainee to master these skills throughrepetitive practice. To our knowledge, the literature shows alack of procedure-based simulators to train surgeons inrobot-assisted surgery [1]. The VR-based simulation systemscurrently available have a limited ability to incorporate arealistic environment for teaching procedure-specific skills,especially in robot-assisted surgery training [9]. AR canovercome these limitations and has compelling features fortraining purposes, such as learning content in an actualsurgical perspective, situation learning, realistic senses ofpresence, immediacy and immersion, and it also bridgesformal and informal training. Matsumoto et al. [28], using aglobal rating system, found bench model trainees performedsignificantly better than those receiving didactic training.

In surgery, multiple studies have validated the effectiveness ofAR in training and assessing laparoscopic skills [9,24,29].Botden et al. [9] compared AR with VR laparoscopic training,and found their AR simulator was preferred to the VR

Table 6 National Aeronautics and Space Administration Task Load Index assessment for phase I(HoST group vs control group) and phase II (cross-over group vs control group).

HoST vs control group

Load type HoST group, n = 22Mean (SD) NASA TLX score

Control group, n = 24Mean (SD) NASA TLX score

P

Mental demand 10.6 (3.0) 11.8 (4.0) 0.153Physical demand 8.5 (3.9) 9.2 (3.7) 0.502Temporal demand 5.9 (3.4) 9.3 (3.0) 0.001Performance 7.7 (4.2) 8.5 (4.2) 0.527Effort 5.8 (2.5) 11.9 (3.7) 0.035Frustration 6.3 (3.7) 7.6 (4.2) 0.373Overall score 7.6 (2.1) 9.4 (2.9) 0.344

Cross-over group vs control group

Load type Cross-over group, n = 17Mean (SD) NASA TLX score

Control group, n = 24Mean (SD) NASA TLX score

P

Mental demand 6.8 (4.3) 11.8 (4.0) <0.001Physical demand 6.1 (3.8) 9.2 (3.7) 0.013Temporal demand 5.6 (3.9) 5.9 (3.4) 0.830Performance 6.1 (4.3) 7.7 (4.2) 0.113Effort 7.8 (5.1) 11.0 (4.5) 0.037Frustration 4.2 (3.5) 7.6 (4.2) 0.006Overall score 6.2 (3.6) 9.4 (2.9) 0.012

NASA TLX, National Aeronautics and Space Administration Task Load Index.

Chowriappa et al.


simulator. AR has been shown to be an effective tool forgaining laparoscopic suturing skills [24]. Yudkowsky et al. [8]reported an AR simulator, which incorporates tomographicimages of real patients, to train residents in performingventriculostomy. HoST takes this technology further byincorporating real surgical experience into teaching more of atrue hands-on experience in a safe environment. Task-specificskills acquisition is critical in learning to perform any surgicalprocedure [30].

The present study has some limitations. Firstly, the objectiveUVA evaluation score is not a validated system for assessingparticipant performance. The NASA TLX score is a subjectivemeasure, so scores used may not completely representparticipant cognitive workload. Nevertheless, the overall NASATLX scores provide some indication of participants’ familiaritywith the procedure. Secondly, the cross-over group performedsignificantly better in nearly all aspects of GEARS assessmentand on the objective UVA evaluation metrics. This findingmay be attributed to the additional amount of time spent onthe robot after entering the cross-over group; however, theHoST group achieved similar performance scores incomparison with the cross-over group on the GEARSassessment.

This study was aimed at determining the impact ofprocedure-based training without learning on a patient inrobot-assisted surgery by using AR. Future work with theHoST framework will focus on the development andrefinement of AR modules for other robot-assisted surgeryprocedures. Such procedure-based training may improveprocedure-specific skills by identifying specific areas in needof improvement and tailoring training to the individuals’needs.

In conclusion, UVA training in an AR environmentimproves technical skill acquisition with minimal cognitivedemand.

Conflict of InterestK.A.G. reports support from Simulated Surgical Systems LLC,outside the submitted work. J.L.M. reports support from theGenomic Health Scientific Advisory Board and the NationalComprehensive Cancer Network, and is a member of theProstate Cancer Early Detection Panel and the Prostate CancerGuidelines Panel Chair, outside the submitted work. Theremaining authors have no conflicts to declare.

AcknowledgementsThis work was funded, in part, by the Roswell Park AllianceFoundation and the John R. Osheii Foundation. The authorswould like to thank Covidien for providing us with V-Loc™sutures during the study.

References1 Abboudi H, Khan MS, Aboumarzouk O et al. Current status of

validation for robotic surgery simulators – a systematic review. BJU Int2013; 111: 194–205

2 McCulloch P, Mishra A, Handa A, Dale T, Hirst G, Catchpole K. Theeffects of aviation-style non-technical skills training on technicalperformance and outcome in the operating theatre. Qual Saf Health Care2009; 18: 109–15

3 Editorial comment. Guru KA. J Urol 2011; 186: 10244 Botden SM, Jakimowicz JJ. What is going on in augmented reality

simulation in laparoscopic surgery? Surg Endosc 2009; 23: 1693–7005 Lee JY, Mucksavage P, Sundaram CP, McDougall EM. Best practices

for robotic surgery training and credentialing. J Urol 2011; 185:1191–7

6 Larsen CR, Oestergaard J, Ottesen BS, Sørensen JL. The efficacy ofvirtual reality simulation training in laparoscopy: a systematic review ofrandomized trials. Acta Obstet Gynecol Scand 2012; 91: 1015–28

7 Lallas CD. Davis, Members of the Society of Urologic Robotic SurgeonsJW. Robotic surgery training with commercially available simulationsystems in 2011: a current review and practice pattern survey from thesociety of urologic robotic surgeons. J Endourol 2012; 26: 283–93

8 Yudkowsky R, Luciano C, Banerjee P et al. Practice on an augmentedreality/haptic simulator and library of virtual brains improves residents’ability to perform a ventriculostomy. Simul Healthc 2013; 8: 25–31

9 Botden SM, Buzink SN, Schijven MP, Jakimowicz JJ. Augmented versusvirtual reality laparoscopic simulation: what is the difference? World JSurg 2007; 31: 764–72

10 El Sayed NA, Zayed HH, Sharawy MI. ARSC: augmented reality studentcard. Comput Educ 2011; 56: 1045–61

11 Mitha AP, Almekhlafi MA, Janjua MJ, Albuquerque FC, McDougallCG. Simulation and augmented reality in endovascular neurosurgery:lessons from aviation. Neurosurgery 2013; 72 (Suppl. 1): 107–14

12 Sabbagh R, Chatterjee S, Chawla A, Hoogenes J, Kapoor A, MatsumotoED. Transfer of laparoscopic radical prostatectomy skills from benchmodel to animal model: a prospective, single-blind, randomized,controlled study. J Urol 2012; 187: 1861–6

13 Korndorffer JR Jr, Dunne JB, Sierra R, Stefanidis D, Touchard CL, ScottDJ. Simulator training for laparoscopic suturing using performance goalstranslates to the operating room. J Am Coll Surg 2005; 201: 23–9

14 Novara G, Ficarra V, Rosen RC, Artibani W, Costello A etal. Systematicreview and meta-analysis of perioperative outcomes and complicationsafter robot-assisted radical prostatectomy. Eur Urol 2012; 62: 431–52

15 Hwang G-J, Yang T-C, Tsai C-C, Yang SJ. A context-aware ubiquitouslearning environment for conducting complex science experiments.Comput Educ 2009; 53: 402–13

16 Sotiriou S, Bogner FX. Visualizing the invisible: augmented reality as aninnovative science education scheme. Adv Sci Lett 2008; 1: 114–22

17 Seixas-Mikelus SA, Stegemann AP, Kesavadas T et al. Content validationof a novel roboticsurgicalsimulator. BJU Int 2011; 107: 1130–5

18 Seixas-Mikelus SA, Adal A, Kesavadas T et al. Can image-based virtualreality help teach anatomy? J Endourol 2010; 24: 629–34

19 Stegemann AP, Ahmed K, Syed JR et al. Fundamental skills of roboticsurgery: a multi-institutional randomized controlled trial for validation ofa simulation-based curriculum. Urology 2013; 81: 767–74

20 Chowriappa AJ, Shi Y, Raza SJ et al. Development and validation of acomposite scoring system for robot-assisted surgical training-the RoboticSkills Assessment Score. J Surg Res 2013; 185: 561–9

21 Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ. Globalevaluative assessment of robotic skills: validation of a clinical assessmenttool to measure robotic surgical skills. J Urol 2012; 187: 247–52

22 Hart SG, Staveland LE. Development of NASA-TLX (Task Load Index):results of empirical and theoretical research. Hum Ment Workload 1988; 1:139–83



23 Keehner M, Lippa Y, Montello DR, Tendick F, Hegarty M. Learning aspatial skill for surgery: how the contributions of abilities change withpractice. Appl Cogn Psychol 2006; 20: 487–503

24 McDougall EM, Kolla SB, Santos RT, Gan JM, Box GN et al.Preliminarystudy of virtual reality and model simulation for learninglaparoscopic suturing skills. J Urol 2009; 182: 1018–25

25 Seymour NE. VR to OR: a review of the evidence that virtual realitysimulation improves operating room performance. World J Surg 2008; 32:182

26 Laguna MP, Arce-Alcazar A, Mochtar CA et al. Construct validity of thechicken model in the simulation of radical prostatectomy suture.J Endourol 2006; 20: 69

27 Nadu A, Olsson LE, Abbou CC. Simple model for training in thelaparoscopic vessicouretheral running anastamosis. J Endourol 2003; 17:481

28 Matsumoto ED, Hamstra SJ, Radomski SB et al. The effect of benchmodel fidelity on endourologic skills: a randomized controlled study.J Urol 2002; 167: 1243

29 Ritter E, Kindelan T, Michael C, Pimentel E, Bowyer M. Concurrentvalidity of augmented reality metrics applied to the fundamentals oflaparoscopic surgery (FLS). Surg Endosc 2007; 21: 1441–5

30 Sidhu RS, Park J, Brydges R et al. Laboratory based vascular anastomosistraining: a randomized controlled trial evaluating the effects of benchmodel fidelity and level of training on skill acquisition. J Vasc Surg 2007;45: 343

Correspondence: Khurshid A. Guru, Applied TechnologyLaboratory for Advanced Surgery (ATLAS) Program,Department of Urology, Roswell Park Cancer Institute, Elmand Carlton Streets, Buffalo, NY 14263, USA.

e-mail: [email protected]

Abbreviations: UVA, urethrovesical anastomosis; AR,augmented reality; HoST, Hands-on Surgical Training;GEARS, Global Evaluative Assessment of Robotic Skills; NASATLX, National Aeronautics and Space Administration TaskLoad Index; VR, virtual reality; RoSS, Robot-assisted SurgicalSimulator; FSRS, Fundamental Skills of Robotic Surgery; dVSS,daVinci Surgical System.

Chowriappa et al.


Augmented Reality Based Skill Training for Robot-Assisted Urethro-Vesical Anastamosis: A...

Documents

Transcript of Augmented Reality Based Skill Training for Robot-Assisted Urethro-Vesical Anastamosis: A...