Laparoscopic virtual reality and box trainers

Is one superior to the other?

Y. Munz, B. D. Kumar, K. Moorthy, S. Bann, A. Darzi

Department of Surgical Oncology and Technology, Imperial College of Science, Technology, and Medicine, St. Mary’s Hospital,QEQM Wing, London W2 1NY, England, United Kingdom

Received: 13 February 2003/Accepted: 2 September 2003/Online publication: 2 February 2004

AbstractBackground: Virtual reality (VR) simulators now havethe potential to replace traditional methods of laparo-scopic training. The aim of this study was to comparethe VR simulator with the classical box trainer and de-termine whether one has advantages over the other.Methods: Twenty four novices were tested to determinetheir baseline laparoscopic skills and then randomizedinto the following three group: LapSim, box trainer, andno training (control). After 3 weekly training sessionslasting 30-min each, all subjects were reassessed. As-sessment included motion analysis and error scores.Nonparametric tests were applied, and p < 0.05 wasdeemed significant.Results: Both trained groups made significant improve-ments in all parameters measured (p< 0.05). Comparedto the controls, the box trainer group performed sig-nificantly better on most of the parameters, whereas theLapSim group performed significantly better on someparameters. There were no significant differences be-tween the LapSim and box trainer groups.Conclusions: LapSim is effective in teaching skills thatare transferable to a real laparoscopic task. However,there appear to be no substantial advantages of onesystem over the other.

Key words: Laparoscopy — Surgical training — Sim-ulator — LapSim — Virtual reality — Box trainer

Over the last 2 decades, minimally invasive surgery hasbecome established as an acceptable technique, if not themethod of choice, for a wide variety of operations.Furthermore, the minimal-access approach has broughtconsiderable benefits to patients—most notably, re-duced morbidity, better cosmetic results, and an earlierreturn to a normal lifestyle [4].

However, the competent performance of laparo-scopic surgery requires novel and unique psychomotorskills that are quite different from those needed to per-form open surgical procedures (Table 1). The acquisi-tion of these skills demands considerable time anddedication on the part of trainees and also requires ap-propriate teaching facilities.The operating theater has always been relied on as

the setting for the teaching and assessment of juniorsurgeons in technical skills, and the situation is no dif-ferent in laparoscopy. However, due to issues such asquality control and patient safety, which have gained theattention of the public as well as health authorities,combined with increasing financial constraints that haveforced surgeons to become more efficient and cost-ef-fective in the operating theater, a need has emerged formore skills training to be undertaken outside the oper-ating theater [2, 10, 13, 17].Surgical simulators, devices that recreate operative

conditions as a substitute for real-life performance, playan increasingly important role in complementing surgi-cal training. They enable trainees to progress partwayalong the learning curve before they perform an opera-tion on a patient [12, 14]. They also enable trainees topractice their skills in a pressure-free environmentwithout supervision or time constraints. Among theseveral types of simulators now available, some useabstract graphics whereas others create a more realisticenvironment (higher-fidelity) (Table 2). Simulator tech-nology was initially proven to be a useful teaching toolin the aviation industry; later on, it was applied to thepractice of anesthesiology and recently has been recog-nized as a valuable means of teaching the psychomotorskills necessary to perform basic laparoscopic surgery[8].The box (video) trainer, which incorporates conven-

tional laparoscopic equipment, is a relatively inexpensiveand highly versatile device that enables the training onanimal parts as well as synthetic inanimatemodels [3, 7, 9,16, 18].Correspondence to: Y. Munz

Surg Endosc (2004) 18: 485–494

DOI: 10.1007/s00464-003-9043-7

� Springer-Verlag New York Inc. 2004

The newer computer-based virtual reality (VR) sys-tems, such as the well-known MIST-VR and the newerLapSim (Surgical Science, Gothenburg Sweden), show agreat potential for both training and assessment pur-poses. These systems are easy to operate and need nomaintenance, making them attractive to instructors aswell as trainees. This new technology enables the user tointeract efficiently with three-dimensional compute-generated images by means of an interface, which rep-licates laparoscopic instruments, as they navigatethrough and interact with the environment using theirnatural senses and skills. Currently available laparo-scopic VR software replicate tasks such as cutting,grasping, and suturing, thereby enabling the operator togradually acquire the psychomotor skills necessary toperform real-time procedures. Instantaneous objectivefeedback is available, enabling the user as well as theteacher to monitor and assess progress [10, 15, 19, 20].To date, there is not enough evidence of the transfer

efficiency ratio (TER) of any type of medical/surgicalsimulator to make a conclusive judgment about theircost-effectiveness. In the aviation industry, it has beenestablished that 1 h of training on the simulator equals½ h of training in an airplane (TER = 2:1); however,

an equivalent TER ratio has not yet been proved insurgery. No consensus exists as yet as to what type oramount of training is necessary to make a true impacton operative performance [1, 11, 21].The aim of this study was threefold: first, to deter-

mine whether the LapSim VR simulator is a useful toolfor the acquisition of psychomotor skills; second, toevaluate whether the skills acquired are transferable to areal task; and third, to determine whether the LapSim orthe box trainer, if either, is more effective for the pur-pose of teaching basic laparoscopic psychomotor skills.

Materials and methods

Study design

The study was conducted as a randomized controlled trial. All of theparticipants were novices (medical students), thus assuring standardi-zation for any effects of previous experience of laparoscopic surgery,whether simulated or real. They all gave their informed consent toparticipate in this study. All subjects were asked to perform a preas-sessment task on the box trainer. Upon its completion, they wererandomized into three groups of eight participants each—box trainer,LapSim, and control. The training was distributed over a period of 3

Table 1. Psychomotor skills necessary to perform laparoscopic surgery

Shift from three-dimensional operating environment to two-dimensional monitor displayJudgment of depth perception and spatial relationshipsVideo–eye–hand coordinationAdaptation to the fulcrum effect (pivoting action of body wall on instrument handling)Manipulation of long surgical instruments while adjusting for amplified tremor, minimal haptics, and fewer degrees of freedomAdaptation to reduced view of operative field

Table 2. Advantages and disadvantages of the simulators available for laparoscopic skills training

Type of simulator Advantages Disadvantages

OrganicHumanLive Best modality: provides exact anatomy

and bleeding and real operating theaterenvironment

Ethical considerationsRequires consentPressure of training vs provision of a serviceInability to standardize assessments due toindividual variations

Cadaver High fidelity CostlySame anatomy Limited availabilityNo issues of consent Noncompliant bloodless tissue makes operating difficultFewer ethical concerns Inability to standardize assessments due to

individual variationsNo time pressure

AnimalLive Live animal operating banned

in UK (Cruelty to Animals Act [1876])Cadaver Cheap Anatomical differences from humans

Good availability Ethical concernsGood tissue handlingwhen tissue is fresh

Inherent cost of the need to provide appropriatefacilities and personnelPotential risk of infection

Inorganic Unproven transferability from inanimate modelsto human operating

Synthetic Reproducible and standardizedCan be used for training and assessingisolated skills

High-fidelity models are expensive

Virtual reality Ability to perform operation/individual tasksin real timeInstant objective feedback

Expensive equipmentQuestionable reliability of software and interface

486

consecutive weeks, with each weekly session limited to 30 min, so as toenable the subjects to maintain maximal concentration and thus op-timize skill acquisition. Weekly intervals were allowed in order toprovide a significant assimilation period.

Experimental setup

The box trainer (Simulations Trainer; Germany) is covered with anopaque fabric through which two ports penetrate at 30� on each side ofthe 10-mm 0� laparoscope (Surgiview Laparoscope; United StatesSurgical Corporation , USA), which passes through the midline (Fig.1). A laparoscopic grasper (Autosuture, United States Surgical Cor-poration) was passed through the left port, and laparoscopic scissors(Autosuture, United States Surgical Corporation) were passed throughthe right. The laparoscope was held at a predetermined set angle andfocal length by means of a clamp, enabling the operator to work in-dependently. The laparoscope was connected to a camera box (Tele-cam Pal; Karl Storz ) and a xenon 300 light source (Karl Storz). Theimage was visualized on a 14" monitor (Sony Trinitron) placed at afixed height directly in front of the operator.The VR simulator used was the LapSim (Fig. 2). This computer-

based simulator consists of a generic platform on which variousmodules can be added. Each module contains specific tasks that can beconducted in a relatively realistic anatomical environment using aframe (Immersion, San Jose, CA, USA) that consists of two laparo-scopic instruments. The degree of difficulty of the tasks can be varied,and performance measures can be stored and retrieved from the da-tabase for further feedback and analysis. In this study, the easy levelwas selected after the system was tested by a number of experiencedlaparoscopic surgeons, all of whom came to the conclusion that theeasy level (default) was the best for the purpose of training. For thepurpose of standardization we attempted to match the box trainertasks with those taught by LapSim (Fig. 3).The model consisted of a medium-sized yellow surgical glove filled

with a fixed volume of water and tied at the end (Fig. 4). A 6" piece ofplastic drip tubing was filled with red dye, and the ends were sealed byheat; this tube was intended to represent a blood vessel. Two thin lines5 mm apart were marked in the middle of the tube. The tube wassecured on top of the filled glove and positioned along the mediansagittal plane of the palm of the glove. A series of concentric circleswas marked on a large clear surgical glove and shaded with blue ink toindicate safe and danger zones for cutting (Fig. 5). This second glove

was then placed and tied securely over the water-filled glove, so thatthe markings on the red tube were visible within the middle of theconcentric blue circles. The model was placed in the box on an angledplastic board at a fixed position and held fast to the board with elasticbands.

The laparoscopic task

Before carrying out the task, each subject was asked to watch a 5-minvideo, previously recorded in our laboratory, of the task being done byan experienced laparoscopic surgeon. While watching the video, thesubjects were briefed on the goal of the task, the handling of theinstrumentation, the system for positional data recording, and thescoring system. The goal was to cut out a circle within the 2-mm–wideblue zone, then apply a clip directly over the markings on the tube bymeans of a clip applier (Endoclip II; Autosuture , United States Sur-gical Corporation), which was exchanged with the scissors by the in-structor. After the clip gun was exchanged for the scissors, the task wascompleted by cutting the tube halfway between the two clips (Fig. 6).There was no time limit for completion of the task. Subjects were thenasked to perform the task themselves in order to establish a baselinemeasure of their laparoscopic skills for the pretest. No instructionswere given during the subject’s performance of the task.

Assessment of performance

Performance was assessed by means of quantitative and qualitativemeasures. Quantitative data were recorded by the Imperial CollegeSurgical Assessment Device (ICSAD) [5]. The device is comprised of acommercially available motion-tracking system (Polhemus Isotrack II ;Virtual Presence, UK) and a software package for recording the data(Track 3D) and analyzing it (Track). Motion data are retrieved viaelectromagnetic motion-tracking sensors worn over the dorsum onboth hands of the subject. Results are generated in terms of totalnumber of hand movements made, total distance traveled for eachhand, and total time taken to perform the task. Economy of handmovement (EOM = total number of hand movements [left + right] ‚the time taken) was then calculated. Filters were applied as necessaryto minimize noise interference; they were set at 10 kHz for filter widthand 1.5 cm/sec for velocity threshold.

Fig. 1. Box (video) trainer. Fig. 2. The LapSim virtual reality (VR) simulator.

487

Qualitative data were obtained bymeans of an error scoring systemthat discriminated between major and minor errors. Each pre- andposttest performance was analyzed and scored accordingly (Table 3).

Training

All 16 participants allocated to the two training groups were trainedunder the direct supervision of an instructor over 3 weekly sessionslasting 30 min each.

Statistical analysis

Because the data were nonparametric, the Kruskal-Wallis test was usedfor analysis of performance across thewhole range.TheWilcoxon signedrank test for relateddatawasused toassessdifferences inperformance forall of theparametersmeasured for eachgroup.TheMann-WhitneyU testwas applied for comparisons between eachof the training groups and thecontrol group; the same test was used for comparisons between the two

training groups for all pre- and posttest results, as well as for all theimprovement scores. Significance was set at p< 0.05.

Results

All 24 participants assigned to take part in this studycompleted it (Table 4). A blinded observer conductederror scoring for both pre- and posttest performances.Analysis of the positional data showed no significant

differences among the three groups in all the pretestparameters measured; however, significant differenceswere found in the posttest data (Figs. 7, 8, and 9) (fortotal number of movements made by both hands,ppre = 0.912 and ppost = 0.002; for total distancetraveled by both hands, ppre = 0.935 and ppost = 0.025;for total time taken to complete the task, ppre = 0.50and ppost = 0.105; for economy of hand movement,

Fig. 3. Training tasks. (Continued onnext page.)

488

ppre = 0.282 and ppost = 0.001; and for number of er-rors, ppre = 0.715 and ppost = 0.141). In comparisonwith the control group, the box trainer group performedsignificantly (p < 0.05) better on all parameters exceptfor time taken. No significant differences were foundbetween the box trainer and LapSim groups or betweenthe LapSim and control groups for all posttest scores.

Analysis of the data on improvements made frompre- to postassessment (posttest minus pretest perform-ance) showed that, unlike the control group, bothtrained groups had become significantly more econom-

Fig. 3. Continued.

Fig. 4. Model used for pre- and posttests of laparoscopic skills.

Fig. 5. Safe and danger zones for cutting on the model. Participantswere asked to cut out a circle within the thick (2-mm) or blue zone.

489

ical in their hand movements (p = 0.036 and p = 0.05for box trainer and LapSim, respectively). Moreover,both groups had a significantly shorter right-hand pathlength (p = 0.012 and p = 0.05) after receiving thetraining (Figs. 10, 11, and 12). No significant differencesin these improvements were found between the twotraining groups.For error scores across the range, there were no

significant differences between pre- and posttest scores;however, there were significant differences within eachgroup between the pre- and posttest scores (Fig. 13).In summary, median scores for all three groups

dropped in the posttest, with the single exception ofdistance traveled by the left hand of the control group,which increased from the pre- to posttest, thus causing asignificant increase in the EOM value (reduced economyof movement).

Discussion

In view of the current emphasis on increasing patientsafety and quality control in surgery, there is a growingneed for improvements in surgical training. It is clearthat, to achieve these goals, a greater proportion of the

teaching, training, and assessment of laparoscopic skillsshould take place outside the operating theater. Both theconventional box (video) trainer and VR simulatorshave previously been shown to fasten the acquisition ofpsychomotor skills. The laparoscopic task used for thepre- and posttests involved manipulation of the model,grasping, cutting, and clip-applying, thus testing themain psychomotor skills required for basic laparoscopicsurgery. The water-filled glove realistically simulated thedangers of heavy-handedness. A perforated glove couldtranslate into a perforated gall bladder or bowel or evendamage to the bile duct, liver, or a blood vessel—allerrors that can have dire consequences. The scoringsystem reflected the potential danger of being imprecise;deviation from the cutting zone by more than a fewmillimeters was considered a serious miscalculation,equivalent to a misapplied clip to the cystic duct orartery.The results suggest that both modalities are effective

means of teaching and improving basic psychomotorskills, and that these skills are transferable to a rela-tively complicated real task. Surgical dexterity is re-flected by measures such as efficiency and accuracy.The significant improvements in economy of movement(efficiency) and total distance traveled by both hands

Fig. 6. Performance of the task.

Table 3. Error scoring system

Type Error Points

Major Cutting beyond inner/outer rings 4Model perforation 4Misapplied clips 4

Minor Cutting between assignedline and inner/outer lines

1

Inaccurately applied clips 1

Table 4. Group demographics

LapSim training Box training No training

No. of subjects 8 8 8Male 0 5 5Female 8 3 3Age (yr) 21.5 21.6 21.1Corrected vision 5 6 6Videogame players 0 5 1

490

(accuracy) achieved by both trained groups vs the con-trol group suggest that the subjects felt more comfort-able with instrument handling and navigation after theirtraining. This indicates that they had acquired suchbasic laparoscopic psychomotor skills as video–eye–hand coordination, adjusting for the fulcrum effect,judgment of spatial relationships, and depth perception.Time taken for task completion is a measure of

dexterity, and one would expect a reduction in time withincreasing proficiency. The time taken to complete the

task decreased, but not significantly, from the pre- to theposttest (p = 0.105). There are two possible explana-tions for this ‘‘lack of improvement.’’ It may be that asthe trained subjects acquired the skills to perform better,they became more accurate by not rushing to completethe task. Or it may be that if they had been allowed morepractice, the decrease in time from pre- to posttest wouldhave become significant. It is also possible that the rea-son the LapSim group improved as compared to thecontrol, although not significantly, was that the trainees

Fig. 7. Total number of movements made byparticipants in the pre- and posttest. Kruskal-Wallis: ppre = 0.912, ppost = 0.002.Wilcoxon signed rank: pre /posttests werep = 0.012, p = 0.836, and p = 0.012 for thebox trainer, LapSim, and control groups,respectively.

Fig. 8. Total distance traveled by bothhands (cm). Kruskal-Wallis: ppre = 0.935,ppost = 0.025. Wilcoxon signed rank:p = 0.012, p = 0.088, and p = 0.674 forthe box trainer, LapSim, and control groups,respectively.

491

had not yet surpassed the learning curve (which tends toplateau after five or six sessions). If this is the case, theirimprovement vs the control group would have becomestatistically significant after a few more sessions.It is well recognized that with increasing experience,

laparoscopic surgeons tend to become equally dextrousand confident with both hands. However, in this study,the path length of the nondominant hand did not im-prove significantly, reflecting the inexperience of the

participants. Once the trainees acquired the basic skills,they became more dextrous with both hands, althoughless improvement was seen with the nondominant hand.The box trainer group, but not the LapSim group,

had significantly lower posttest scores for most test pa-rameters in comparison to the control group and theirperformance became much more consistent. This sug-gests—but by no means proves—that the box trainermay be more effective in the teaching of basic psycho-

Fig. 9. Total time needed for taskcompletion (sec). Kruskal-Wallis:ppre = 0.50, ppost = 0.105. Wilcoxon signedrank: p = 0.123, p = 0.123, and p = 0.161for the box trainer, LapSim, and controlgroups, respectively.

Fig. 10. Economy of movement (EOM).Kruskal-Wallis: ppre = 0.282, ppost = 0.001.Wilcoxon signed rank: p = 0.036, p = 0.05,and p = 0.012 for the box trainer, LapSim,and control groups, respectively. Note thatthe EOM for the control group wassignificantly reduced.

492

motor skills than the LapSim. However, the fact that thebox trainer group was using genuine laparoscopic in-struments for training as well as for the pre- and post-tests might have given them an advantage over theLapSim group. Unlike the box trainer instruments, theImmersion frame used by LapSim has no haptics, whichin any case are much diminished in laparoscopic sur-gery; however, they still exist in the handle of the realinstrument when it is used to grasp solid objects.

Despite some of the differences found between thetwo simulators, the study did not demonstrate any sig-nificant advantages of one modality over the other.Therefore, we can only conclude that they are equallyeffective in teaching psychomotor skills—at least tonovices. An expansion of this pilot study—providinga greater total period of training, maintaining thedistributive method of training, and using a largercohort—may establish conclusively that there are no

Fig. 11. Number of movements made byeach hand. Kruskal-Wallis: ppre = 0.282,ppost = 0.001. Wilcoxon signed rank:prt = 0.012, prt = 0.05, and prt = 0.036 forthe box trainer, LapSim, and control groups,respectively.

Fig. 12. Distance traveled by each hand(cm). Kruskal-Wallis: ppre = 0.282,ppost = 0.001. Wilcoxon signed rank:prt = 0.012, prt = 0.05, and prt = 0.093 forthe box trainer, LapSim, and control groups,respectively.

493

significant differences between the two modalities.LapSim provides what appears to be a useful means ofteaching basic psychomotor skills, demanding a con-siderable level of expertise to complete the tasks accu-rately and efficiently. However, before it can be used forthe assessment of laparoscopic skills, it would have to beable to differentiate effectively between different levels oflaparoscopic experience.At present, it seems that the effectiveness of these

computer-based systems is questionable, which isprobably why proposals to integrate VR into the sur-gical training curriculum have been met with someskepticism. VR is still, however, a relatively new tech-nology in surgery, and we believe that it will play agreater role in the future, particularly given the expectedadvances in VR software and improved compatibilitybetween software and hardware.

References

1. Anastakis DJ, Regehr G, Reznick RK, Cusimano M, MurnaghanJ, Brown M, Hutchison C (1999) Assessment of technical skillstransfer from the bench-training model to the human model. Am JSurg 117: 167–170

2. Bridges M, Diamond DL (1999) The financial impact of teachingsurgical residents in the operating room. Am J Surg 177: 28–32

3. Chung J, Sackier J (1998) A method of objectively evaluatingimprovements in laparoscopic skills. Surg Endosc 12: 1117–1120

4. Darzi A, Mackay S (2002) Recent advances in minimal accesssurgery. BMJ 324: 31–34

5. Datta V, Chang A, Mackay S, Darzi A (2002) The relationshipbetween motion analysis and surgical technical assessments. Am JSurg 184: 70–73

6. Datta V, Mandalia M, Mackay S, Chang A, Cheshire N, Darzi A(2002) Relationship between skill and outcome in the laboratorybased model. Surgery 131: 318–323

7. Derossis A, Bothwell J, Sigman H, Fried G (1998) The effect ofpractice on performance in a laparoscopic simulator. Surg Endosc12: 1111–1116

8. Derossis AM, Fried GM, Abrahamowicz M, Sigman HH, BarkunJS, Meakins JL (1998) Development of a model for training andevaluation of laparoscopic skills. Am J Surg 175: 482–487

9. Fried GM, Derossis AM, Bothwell J, Sigman HH (1999) Com-parison of laparoscopic performance in vivo with performancemeasured in a laparoscopic simulator. Surg Endosc 13: 1077–1081

10. Gallagher A, Satava RM (2002) Virtual reality as a metric for theassessment of laparoscopic psychomotor skills: learning curvesand reliability measures. Surg Endosc 16: 1746–1752

11. Hamilton EC, Scott DJ, Fleming JB, Rege RV, Laycock R, Ber-gen PC, Tesfay ST, et al. (2001) Comparison of video trainer andvirtual reality training systems on acquisition of laparoscopicskills. Surg Endosc 16: 406–411

12. Hunter J (1997) The learning curve in laparoscopic cholecystec-tomy. Min Invas Ther Allied Technol 6: 102–104

13. Kirk R (1998) Surgical excellence—threats and opportunities. AnnR Coll Surg Engl 80(Suppl): 256–259

14. McCloy R, Stone R (2001) Virtual reality in surgery. BMJ 323:912–915

15. Reznick R, Regehr G, MacRae H, Martin J, McCulloch W (1996)Testing technical skill via an innovative ‘‘bench station’’ exami-nation. Am J Surg 173: 226–230

16. Rosser J, Rosser L, Savalgi R (1997) Skill acquisition and as-sessment for laparoscopic surgery. Arch Surg 132: 200–204

17. Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, ValentineRJ, Euhus DM, et al. (2000) Laparoscopic training on benchmodels: better and more cost effective than operating room ex-perience? J Am Coll Surg 191: 272–283

18. Scott DJ, Young WN, Tesfay ST, Frawley WH, Rege RV, JonesDB (2001) Laparoscopic skills training. Am J Surg 182: 137–142

19. Taffinder N, Smith SG, Huber J, Russell RC, Darzi A (1999) Theeffect of a second generation 3D endoscope on the laparoscopicprecision of novices and experienced surgeons. Surg Endosc 13:1087–1092

20. Torkington J, Smith SG, Rees BI, Darzi A (2000) The role ofsimulation in surgical training. Ann R Coll Surg Engl 82: 88–94

21. Torkington J, Smith SG, Rees BI, Darzi A (2001) Skill transferfrom virtual reality to a real laparoscopic task. Surg Endosc 15:1076–1079

Fig. 13. Total number of errors made.Kruskal-Wallis: ppre = 0.715, ppost 0.141.Wilcoxon signed rank: p = 0.012, p = 0.017,and p = 0.017 for the box trainer, LapSim,and control groups, respectively.

494