Development of Laparoscopic Cholecystectomy Simulator ...

Development of Laparoscopic Cholecystectomy Simulator Basedon Unity Game Engine

Jinglu ZhangNational Center for Computer

AnimationBournemouth University

zhangjbournemouthacuk

Yao LyuNational Center for Computer

AnimationBournemouth Universityylyubournemouthacuk

Yukun WangNational Center for Computer


i7254611bournemouthacuk

Yinyu NieNational Center for Computer

AnimationBournemouth Universityyniebournemouthacuk

Xiaosong YangNational Center for Computer


xyangbournemouthacuk

Jianjun ZhangNational Center for Computer


jzhangbournemouthacuk

Jian ChanglowastNational Center for Computer


jchangbournemouthacuk

ABSTRACTFast development of the minimally invasive surgery (MIS) tech-nology demands extra surgical skills training to meet advancedtechnological challenges However massive capital expendituresand ethical issues with safety considerations exist in traditionalsurgical training methods (eg using cadavers or animals) Thoselimitations turn Virtual Reality (VR) surgery simulation into a plau-sible alternative to provide a safe and repeatable virtual trainingenvironment In this paper we design and develop a game enginebased laparoscopic cholecystectomy training simulator for surgeonsto understand the surgery procedure and practice their surgicalskills as well as decision making skills Our design leverages physi-cal simulation and haptic force feedback to offer trainees a realisticvisual and tactile experience respectively We explore the possi-bility of using game engine rather than developing from scratchto build the surgical simulator Based on the results and user feed-backs from a pilot experiment we conclude that game engine is aviable option for creating a cost-effective flexible and highly inter-active virtual surgery training platform for pedagogical purposewhich can shorten the development time with some compromisein functionality

lowastCorresponding Author

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page Copyrights for components of this work owned by others than ACMmust be honored Abstracting with credit is permitted To copy otherwise or republishto post on servers or to redistribute to lists requires prior specific permission andor afee Request permissions from permissionsacmorgCVMP rsquo18 December 13ndash14 2018 London United Kingdomcopy 2018 Association for Computing MachineryACM ISBN 978-1-4503-6058-61812 $1500httpsdoiorg10114532784713278474

CCS CONCEPTSbull Human-centered computing rarr Virtual reality Haptic de-vices bull Computing methodologiesrarr Physical simulation

KEYWORDSSurgery simulation cholecystectomy game-based training virtualsurgery modelingACM Reference FormatJinglu Zhang Yao Lyu Yukun Wang Yinyu Nie Xiaosong Yang JianjunZhang and Jian Chang 2018 Development of Laparoscopic Cholecystec-tomy Simulator Based on Unity Game Engine In Proceedings of CVMP rsquo18European Conference on Visual Media Production (CVMP rsquo18) ACM NewYork NY USA 9 pages httpsdoiorg10114532784713278474

1 INTRODUCTIONCholecystectomy a surgical procedure to remove the gallbladderis one of the most common operating room surgeries With theexplosion of related minimally invasive surgery (MIS) techniquescholecystectomy can be operated with the assistance of a video cam-era and several laparoscopic instruments Benefits of laparoscopiccholecystectomy include few post-operative complications smallincisions on the skin and a relative shorter recovery period [Suzukiet al 2000] However the limited viewing angle and restricted op-erating space during laparoscopic cholecystectomy often causeundesirable complications (such as bile duct leak or injury) There-fore it is indispensable for surgeons to acquire pertinent surgicalskills prior to a real surgery

Traditionally using cadavers or animals for training faces manyethical issues Training by supervised experienced surgeons is afeasible solution but it is at the cost of the huge expenditure anda long training period Fortunately virtual reality (VR) simulatorhas gradually become an effective approach [Alaker et al 2016] toteach and assess surgical skills outside operation rooms Medical

CVMP rsquo18 December 13ndash14 2018 London United Kingdom Jinglu Zhang et al

students and surgeons can practice their professional skills alongwith the transferable skills such as team-working and emergencyreaction using this advanced technique Many research works haveproven the correlation between game-based training and the im-provement of surgical skills [Andreatta et al 2010 Creutzfeldt et al2010 Knight et al 2010 Kurenov et al 2009 Rosser et al 2007]Specifically advantages of the surgical simulator include

(1) offering a secure and efficient environment for surgeons todeeply understand the whole surgery process

(2) enabling surgeons to practice two essential skills for MISnamely eye-hand coordination and the ability to execute 3Dactions using a 2D screen as a guide

(3) allowing trainees to practice the challenging and significantprocedures repeatedly and save the training cost

It is complicated to model and simulate a typical laporoscopiccholecystectomy scene The simulation of multiple organs andtissues the interaction between tools and tissues and feedbacksfrom both visual and tactile are all necessary components Devel-oping and integrating all these functional components indepen-dently is time and resources consuming Unity [Goldstone 2009] isa cross-platform modern game engine which has a great supportof advanced audio and visual effects Its integrated developmentenvironment easy profiler on-shelf tools and modules make Unitystable and highly productive

In this study we design and develop a laparoscopic cholecystec-tomy simulator using Unity Game Engine Our surgical simulatorprovides trainees an effective platform to practice surgery proce-dures with realistic visual and haptic feedbacks (shown in Fig 1 andFig 2) In our design the organ and soft tissue simulation are basedon uflex a unity plugin originated from NVidia Flex[Korzeniowski2016] and the haptic feedback is in reference to Unity 5 HapticPlugin for Geomagic OpenHaptics 33 [Poyade et al 014 ] Thereare mainly two contributions of our work

(1) Developing an interactive laparoscopic cholecystectomy sim-ulator for surgeons to enhance their surgical skills

(2) Exploring the use of game engine for novel development ofa surgical simulator to achieve efficient and cost-effectivesolutions

The remaining of this paper is organized as follows Section 2reviews previous works in medical education and training laparo-scopic simulator and soft body deformation modeling Detailedgame objective architecture and target users are given in section3 Section 4 explains our real time physics simulation algorithmsin depth and section 5 gives the system evaluation and feedbacksFinally section 6 concludes the paper

2 RELATEDWORKGame like medical education and surgical skills training with dif-ferent objectives have been proven to be valuable where users canimprove their related skills and experience various circumstancesthat are impractical in the real world due to the safety cost or otherreasons [Susi et al 2007] James et al presented that surgeons whohad experienced selected representative video games (Super Mon-key Ball 2 Star Wars Racer Revenge and Silent Scope) performedbetter in laparoscopic handling and made fewer errors [Rosser et al

Figure 1 Laparoscopic cholecystectomy simulator workingenvironment

Figure 2 Screenshot from cholecystectomy simulator

2007] Another interesting experiment [Knight et al 2010] com-pared the tagging accuracy between participants who had under-taken the game Triage Trainer (Blitz Games Studios LeamingtonSpa Uk) training and who had taken traditional card-sort exerciseThe Triage Trainer group was more likely to triage precisely in ca-sualties than the card-sort group Game like training was also usedfor team training in emergency crisis management [Andreatta et al

Laparoscopic Cholecystectomy Simulator CVMP rsquo18 December 13ndash14 2018 London United Kingdom

2010 Kurenov et al 2009] daily clinical tasks for junior doctorsand cardiopulmonary resuscitation [Creutzfeldt et al 2010] etc Inthis research we focus on the laparoscopic surgical skills trainedvia the surgical simulator

Simulators for surgical skills training can be categorized into twotypes low fidelity and high fidelity Evaluation criteria of fidelityare decided by the extent of visual and tactile realism and systeminteractivity Synthetic models [Hammoud et al 2008] and VideoBox Trainer belong to low-fidelity simulators Synthetic modelsare static bench models such as tissue models for fascia closureand knot-tying training Video Box Trainer [Fried et al 2004] takesadvantage of real laparoscopic instruments as well as the cameraand video monitors to simulate surgery procedures Although lowfidelity simulators are cost-efficient easy to make and portablethey usually sacrifice the realism and only offer a single surgicalskill training rather than a whole process

Virtual reality surgery training is a type of high fidelity surgicalsimulation This type of technology satisfies the requirements oftaking 2D screen as a guide to performing 3D surgery with realisticvisual and tactile feedback while also has the ability to monitorand record the training progress Most of the existing surgicalsimulators either reinvent all components independently [Pan et al2015 Qian et al 2015] or build from an extendable frameworksuch as SOFA [Kim et al 2015] and GiPSi [Cavusoglu et al 2006]Whereas building medical educational or training systems upongame engines is not fully explored Few research works [Markset al 2007ab] have discussed the possibilities and advantages ofgame engine based surgery simulators In order to bring moreunderstanding in this direction we use a robust multipurpose gameengine Unity to build our simulation architecture We also evaluateand compare the pros and cons between the game engine and othersimulation methods for surgical training

Immersive surgery simulation requires accurate physical behav-iors the precision of soft tissue deformation enormously affectsthe sense of reality of the whole framework [Gallagher et al 2005Zhang et al 2017] Three types of simulation approaches have beenwidely applied nowadays

bull Force-based approaches Mass-spring [Baraff and Witkin1998 Bouaziz et al 2014 Liu et al 2013] system consists ofsets of point masses connected by spring dampers It is asimple and efficient scheme which takes Hookersquos Law as thetheoretical basis But it is hard to tune the spring constants toget a desired behavior and usually causes the overshootingproblem

bull Structural-based approaches Unlike the mass-spring systemwhich discretize an object into finite number of point massesfinite element method (FEM) [Zienkiewicz and Taylor 2005]is based on continuum mechanics theory It is capable ofhandling accurate physical behaviors for different types ofelastic and non-elastic material However the model com-plexity associates the technique with difficult initial settingsand high computational cost [Sifakis and Barbic 2012]

bull Position Based Dynamics (PBD) [Muumlller et al 2007] PBD is amethod which works on positions directly in each simulationstep to resolve constraints It is fast stable and controllablewhich makes the simulation process highly efficient and

functionally suitable for the interactive environment [Ben-der et al 2014] Despite the fact that PBD is not physicallyaccurate it achieves real time surgical simulation which isstill visually plausible We applied the PBD solver with differ-ent constraints for our physics simulation as a test prototype

3 SYSTEM INFRASTRUCTURE31 Procedures and Challenges in Laparoscopic

CholecystectomyLaparoscopic Cholecystectomy has gradually replaced the opensurgery to become the major modality of treating gallstones gall-bladder carcinoma trauma and porcelain gallbladder Four primarysteps are involved in the surgery as shown in Fig 3 a) Position thepatient correctly and insert the trocars and camera through threelabeled minimal ports b) Identify the position of the gallbladderand then dissect the tissue to find the cystic duct and cystic arteryc) Clip and ligate the cystic duct and cystic artery structure and d)Tease away adhesions between the gallbladder and the liver put thegallbladder into a pre-prepared bag and drag it out In our trainingsystem we focused on the core steps b) c) and d) as a) can beconducted in a separate training session

(a) Initial view (b) Dissect Calotrsquos Triangle

(c) Clipping and cut (d) Teasing away the gallbladder

Figure 3 Surgery process Image by School of Surgery [Jones2014]

During the intra-operative process some challenges and difficul-ties require a particular attention especially for a novice

bull Identifying the Calotrsquos triangle properly is the prerequisite tosafely remove the gallbladder Calotrsquos triangle shown in Fig4 is the most important anatomy structure in laparoscopiccholecystectomy It helps the surgeon clearly identify therelative position among the cystic duct common hepaticduct and inferior border of the liver

bull Always clipping the cystic duct and cystic artery first andthen cutting them to prevent uncontrollable bleeding or bileleaking

bull Gallbladder closely connects with the liver Correctly sep-arating adhesions between them prevents the harm to theliver

bull In laparoscopic surgery a surgeon operates 36-39cm long in-struments through a tiny entry point such that the unskilled


and careless manipulation is possible to damage intestinesor main blood vessels

Figure 4 Calotrsquos Triangle [Suzuki et al 2000]

32 Objectives and System DesignThis laparoscopic cholecystectomy simulator is designed and devel-oped for medical students and junior doctors to train their surgicalskills in a safe repeatable and cost-effective manner Medical stu-dents can thoroughly understand the surgery process and practiceessential skills before a hand-on operation on a real patient Asfor junior doctors lacking practical experiences they are able todraw lessons from their weakness and practice repeatedly to fur-ther enhance operation skills Laparoscopic experts are involvedto monitor the progress of students and juniors to give valuablefeedbacks and guidances during the training

Typically a surgery training simulator includes a renderingmodule for displaying 3D models and user interface a physicssimulation module for collision detection rigid body soft tissueand smogfluids simulation and an event handling module for in-putoutput In our development to exclude unnecessary complexitywe only prototype the anatomical structure for a normal patientwithout lesions The patient specific mode will be the future workwhich would be of better use to benefit surgical planning Con-cretely we design the game module as below (Fig 5 shows fourcrucial steps)

(1) A video [Jones 2014] illustrates the process of labeling thepatient and inserting trocars through abdomen

(2) Viewpoint selection is important on account of the limitedvision field of minimally invasive surgery Users are able toadjust the camera to get the optimal viewpoint by keyboard

(3) After identifying correlated anatomical structures traineesneed to clip three staples in correct positions on the cysticduct and ligate the gallbladder

(4) With the help of two haptic devices trainees are required toseparate the adhesion between the gallbladder and the liver

(5) The remaining task is rather simple so we use another videoclip to present the procedure of putting the gallbladder intoa pre-prepared bag and dragging it out

4 PHYSICAL SIMULATIONNVIDIA Flex is a particle-based simulation library for real timevisual effects In Flex everything is modeled as a system of particlesconnected by different constraints which is broadly based on PBDand Unified Particle Physics [Macklin et al 2014] NVIDIA FleXdoes not support the Unity Game Engine directly Instead uFlexa Unity asset integrated low-level Flex native library is appliedin our development The core idea is to solve a non-linear systemsequentially with equality and inequality constraints in differenttime steps

Ci (p + ∆p) = 0 i = 1 n (1)

Cj (p + ∆p) gt= 0 j = 1 n (2)

where p = [p1p2 pN ]T is the vector of particle positions Theunified representation allows modeling different materials and in-teractions between various types of models namely rigid body softbody cloth and fluids in a fully efficient and flexible manner Thereare mainly two types of the soft tissue associated with laparoscopiccholecystectomy volumetric soft tissue and surface deformabletissue We explain their modeling methods as follows

41 Volumetric Soft Body SimulationLiver gallbladder and fat tissues are regarded as 3D volumetric de-formable objects in cholecystectomy since they keep their intrinsicshape most of the time When instruments contact or manipulatethe organ only the contacted part deformed Shape matching con-straints of clusters are used to simulate soft bodies The core ideaof shape matching [Muumlller et al 2005] can be described as Attract-ing initial point sets x0i towards the goal positions pi by findingthe global optimal transformation and rotation matrix against thecurrent state That is to minimizesum

iwi (R(x0i minus t0) + t minus pi )2 (3)

whereweights wi |i = 1 n in this case aremassR is the rotationmatrix and t is the translation vector The optimal translationvectors turn out to be the displacement of the center of mass for theshape and the global rotation matrix is extracted from the globaltransformation matrix by polar decomposition Finally the goalpotion can be calculated as

pi = R(x0i minus x0cm ) + xcm (4)

where xcm is the center of massThroughout the simulation a soft body consists of several clus-

ters is shown in Fig 6 Each particle belongs to one or more clusterswith a weight between 0 and 1 standing for the extent of a parti-cle influenced by the corresponding cluster The final deformationresult is the overlapping and averaging of the constraints whichdefines in each cluster independently as described by the shapematching model The more clusters in a soft body the more elasticit will become Shape matching constraint does not need the con-nectivity information and the result is easy to compute thus it isefficient for an interactive environment which compromises somephysical accuracy


(a) Port insertion (b) Anatomical structure identification

(c) Clipping and ligating the cystic duct (d) Teasing away the gallbladder

Figure 5 Operation procedures

Figure 6 Clustering in shape matching

42 Surface Mesh SimulationWe need to simulate the fascia tissue between the gallbladder andthe liver in the dissection procedure Given the extensible and thin

features of the fascia it is modeled using cloth simulation The fasciais displayed as spring networks in which one triangle handles thestretch and two adjacent triangles handle the bending constraintFor each edge the stretching constraint function is

Cstr etch (p1p2) = |p1 minus p2 | minus l0 (5)

where l0 is the rest length of the edge As for two neighbor triangles(p1 p3 p2) and (p1 p2 p4) the bending constraint is generated as

Cbend (p1p2p3p4) = arccos(n1 middot n2) minus φ0 (6)

where n1 and n2 are the normal vector of two triangles and φ0is the dihedral angle between two triangles in the rest pose Theoverall goal is to minimize total energy defined as

E =sum

(kstr etchC2str etch + kbendC

2bend ) (7)

where kstr etch and kbend are global stiffness parameters providedby the user

However the fascia simulation is not visually plausible in our testbecause fascia is not a single layer of smooth structure as the clothWhile splitting the gallbladder from the liver in our simulation thefascia just looks like a piece of cloth with explicit triangle meshin the torn partition as shown in Fig 7(a) We design an adhesiveconstraint to simulate the breakable fascia to enhance the visualeffects Because uFlex can only access to the basic-level library ofNVIDIA Flex it cannot simulate the sticky effects We tested thealgorithm in Houdini and we will integrate the adhesive constraintinto system in the future work As the result the breakable fasciais visually satisfying as shown in Fig 7(b)


(a) Artifact in fascia simulation

(b) Adhesive constraint for the fascia

Figure 7 Comparison between original and improved re-sults

In cloth simulation fascia tissue is represented as connectivetriangle mesh While in our new algorithm we simply use lines andthe distance constraint to model the fascia Initially we randomlygenerate several points on the surface of the gallbladder and theliver and connect them with lines In each connection few pointsare randomly scattered and marked as breakable group At the sametime we assign a rest length and a very large stiffness parameterto each line Finally when we drag the gallbladder if the distancebetween two organs exceeds a given break threshold the stiffnessparameter of random breakable points will be reset to a very smallnumber such that any force can break the connections In such waythe connection between the gallbladder and the liver no longerlooks like a piece of papery cloth with apparent triangle meshInstead it presents a sticky behavior of the fascia when we dragthe gallbladder away

43 Haptic RenderingA system requires at least 500-1000 HZ for smooth haptic render-ing [Booth et al 2003] in which 25-30 Hz for visual rendering inorder to provide users realistic feedbacks Haptic feedback is ableto enhance both cognitive ability by offering continuous movementsensory and kinesthetic sense by differentiating different innerstructures Phantom Omni is a feasible and affordable haptic devicefor virtual-real interaction It takes the 6 DOF input and generates

forces to constrain the 3 DOF output in a high sensory frequencyHaptic rendering in Unity enables the user to control haptic ef-fectors as laparoscopic instruments to interact with organs andtissues

Figure 8 shows the interactive work flow of the haptic device Atthe beginning we predefine the haptic workspace dimensions[Poyadeet al 014 ] Within the workspace meshes and transformation ma-trices of haptic geometries (organs and tissues) are set into hapticframe During the training process the position of the haptic proxy(ie the graphic representation of the haptic device in the screenin our case is the position and orientation of the laparoscopic in-strument model) keeps updating If no collisions happen betweeninstruments and the organ the proxy position and the device posi-tion will stay the same But if the proxy get contact with the hapticobjects the proxy position will be assigned to the device positionAccording to the position and the collided particle ID informationthe simulation algorithm calculates the deformation and gives vi-sual feedbacks to the screen and force feedbacks to the user hand

Figure 8 Haptic device workflow

5 EVALUATION AND FEEDBACKA PC with an Intel Xeon 5 CPU a GeForceGTX 1080 graphics cardand two Phantom Omni devices is used to built this laparoscopiccholecystectomy simulator The criterion-based quantitative as-sessment [Jackson et al 2011] is employed to evaluate the overallcapacity of the simulator from system performance and usabilityperspectives Ten volunteer PhD students and five volunteers fromgeneral surgery department (including students and laparoscopicexperts) participated in the test and gave their suggestions accord-ingly

51 System EvaluationThere are five criteria chosen to evaluate the system performance1) ease of use 2) interactivity 3) visual realism 4) freedom of move-ment and effectiveness and 5) system stability Ten volunteer PhDstudents are asked to grade each criterion from 1 to 5 represent-ing Poor to Excellent Figure 9 presents the average result ofeach criterion about the performance of our simulator The sys-tem interactivity freedom of movement and effectiveness and ease


of use receive the positive feedback with the score around fourAll participants mentioned that haptic device with force feedbackhugely improve the user experience However the visual realismand system stability need to be improved due to the fact that therelated FLEX solver is not physically accurate

Figure 9 System performance evaluation (1-Poor 2-Fair 3-Average 4-Good 5-Excellent)

52 Simulator Usability EvaluationFive criteria namely 1) ease of use 2) anatomy precision 3) vi-sual realism 4) integrity of procedure and 5) utility are appliedto assess the usability of the simulator Evaluators from generalsurgery department are asked to grade each criterion from oneto five representing Poor to Excellent Figure 10 presents theaverage score of each criterion about the usability of our simulatorDue to the small sample numbers such evaluation provided someguidance and further validation will be carried in the future workThe utility and ease of use reach the highest score around four thusdemonstrating the usefulness of VR surgical simulation Neverthe-less there is still a space for improvement in anatomy precisionvisual realism and integrity of procedure Surgeons and medicalstudents claim that as there are no professionals involved in oursimulator design some minimal details are neglected For examplethe inner structure of calotrsquos triangle is more complex than the onewe built This defective anatomical representation has the potentialto mislead trainers In the future improvement we will improvethem with the guide from medical experts

53 Game Engine based SimulatorMost of the existing simulators are either developed independentlyor based on some open-source framework whereas we built thewhole architecture upon Unity Game Engine We have discussedour experimental results in a focus group of eight attendances fromboth computer scientists and laparoscopical surgeons Comparingwith other two types of simulators we summarized advantagesas well as limitations of using Game Engine for the surgical training

Figure 10 Simulator usability evaluation (1-Poor 2-Fair 3-Average 4-Good 5-Excellent)

531 Advantages

bull For the self-developed simulators reinventing essential func-tional blocks (rendering physic and event handling compo-nents) for a simulator is time-consuming and leads to hugedevelopment expenditure In our case we spent about fourmonths for the whole development cycle but it took oneyear or more for a simulator like [Qian et al 2015] to achievethe similar function

bull Robust game engine architectures allow developers to con-centrate more on the content rather than the implementationthus could improve the user experience

bull Most of the simulator systems only focus on the technicaltraining of the surgery process However other skills such asteam collaborative and patient care are also very importantfor a qualified surgeon The self-development simulators arenot able to network with other simulators for collaborativetasks None of them are audio supportive [Marks et al 2007b]Game engine has better support for network and audio

532 Limitations

bull Game engines are primarily designed for game developmentThe physical simulation capability of virtual surgery still hasa room for improvement Unlike the ordinary soft body thestructure of human organs and soft tissues are very complexThe physics engine in Unity is not able to provide highfidelity laparoscopic features eg cutting and multi-layerheterogeneous soft tissue simulation

bull It is a trade-off for a game engine to either sacrifice theprecise animation for speed or emphasize on pleasant visualwithout real time interactive ability

54 Improvement SuggestionsAccording to the evaluation results and suggestions from open-ended questionnaires further improvements need to be achieved

(1) Although the game engine has integrated a system develop-ment architecture for audio rendering event handling andnetworking the visual realism of soft tissue deformation and


the precision of collision detection needs to be improved Wewill first integrate adhesive constraint simulation for fasciainto our simulator

(2) Several participants from medical institution indicate thatwe need more precise anatomy structures for calotrsquos triangleThe success of laparoscopic cholecystectomy largely relieson whether surgeon could identify the anatomy structureproperly

(3) We will also add more interactive modules to improve theintegrity and efficiency of the simulator Currently only onetype of laparoscopic instrument is modelled in the simulatorSeveral instruments (retractor electrodes and anvil graspers)will be integrated and users will be allowed to change theinstruments for different tasks

6 CONCLUSIONIn this study we have designed and developed a novel game en-gine based surgical simulator for laparoscopic cholecystectomySurgeons are able to train their surgical and decision making skillsin this virtual environment before they encountering real patientsThe VR surgical training allows trainees to operate the immersivecholecystectomy by using haptic devices with force feedbacks anda computer screen as the guide The training environment is safecost-efficient and repeatable The user evaluation results demon-strate that the simulator is easy to use and interactive

Currently we only receive feedbacks from few laparoscopic sur-geons and medical students Before it can be validated as a surgerytraining tool we need to further improve the evaluation strategyControlled experiments will be taken by dividing surgeons intotwo groups with one trained by the simulator and one withouttraining to determine the benefits and drawbacks of our simula-tor Further improving works are desired including modeling theanatomic structures of Calotrsquos triangle accurately solving technicalchallenges related to realistic soft tissue simulation and integratingother training tasks for a better user experience

ACKNOWLEDGMENTSThe authors appreciate the volunteers fromNational Center for Com-puter Animation (NCCA) and surgeons for their valuable commentsand helpful suggestions This research has been partly supportedby Bournemouth University PhD scholarship and HengdaoruyiCompany

REFERENCESMedhat Alaker Greg R Wynn and Tan Arulampalam 2016 Virtual reality training in

laparoscopic surgery a systematic review amp meta-analysis International Journal ofSurgery 29 (2016) 85ndash94

Pamela B Andreatta Eric Maslowski Sean Petty Woojin Shim Michael MarshTheodore Hall Susan Stern and Jen Frankel 2010 Virtual reality triage train-ing provides a viable solution for disaster-preparedness Academic emergencymedicine 17 8 (2010) 870ndash876

David Baraff and Andrew Witkin 1998 Large steps in cloth simulation In Proceedingsof the 25th annual conference on Computer graphics and interactive techniques ACM43ndash54

Jan Bender Dan Koschier Patrick Charrier and Daniel Weber 2014 Position-basedsimulation of continuous materials Computers amp Graphics 44 (2014) 1ndash10

Stuart Booth FDAngelis and Thore Schmidt-Tjarksen 2003 The influence of changinghaptic refresh-rate on subjective user experiences-lessons for effective touch-basedapplications In Proceedings of eurohaptics Citeseer 374ndash383

Sofien Bouaziz Sebastian Martin Tiantian Liu Ladislav Kavan and Mark Pauly 2014Projective dynamics fusing constraint projections for fast simulation ACM Trans-actions on Graphics (TOG) 33 4 (2014) 154

Murat Cenk Cavusoglu Tolga GGoktekin and Frank Tendick 2006 GiPSi a frameworkfor open sourceopen architecture software development for organ-level surgicalsimulation IEEE Transactions on Information Technology in Biomedicine 10 2 (2006)312ndash322

Johan Creutzfeldt Leif Hedman Christopher Medin Wm LeRoy Heinrichs and LiFellaumlnder-Tsai 2010 Exploring virtual worlds for scenario-based repeated teamtraining of cardiopulmonary resuscitation in medical students Journal of medicalInternet research 12 3 (2010)

Gerald M Fried Liane S Feldman Melina C Vassiliou Shannon A Fraser DonnaStanbridge Gabriela Ghitulescu and Christopher G Andrew 2004 Proving thevalue of simulation in laparoscopic surgery Annals of surgery 240 3 (2004) 518

Anthony G Gallagher E Matt Ritter Howard Champion Gerald Higgins Marvin PFried Gerald Moses C Daniel Smith and Richard M Satava 2005 Virtual realitysimulation for the operating room proficiency-based training as a paradigm shiftin surgical skills training Annals of surgery 241 2 (2005) 364ndash372

Will Goldstone 2009 Unity game development essentials Packt Publishing LtdMaya M Hammoud Francis S Nuthalapaty Alice R Goepfert Petra M Casey Sandra

Emmons Eve L Espey Joseph M Kaczmarczyk Nadine T Katz James J Neutensand Edward G Peskin 2008 To the point medical education review of the role ofsimulators in surgical training American Journal of Obstetrics amp Gynecology 199 4(2008) 338ndash343

Mike Jackson Steve Crouch and Rob Baxter 2011 Software evaluation criteria-basedassessment Software Sustainability Institute (2011)

Keaton Jones 2014 Laparoscopic cholecystectomy explained httpswwwyoutubecomwatchv=a7rIFlvZM0Iampt=139s

Youngjun Kim Laehyun Kim Deukhee Lee Sangkyun Shin Hyunchul Cho FreacutedeacuterickRoy and Sehyung Park 2015 Deformable mesh simulation for virtual laparoscopiccholecystectomy training The Visual Computer 31 4 (2015) 485ndash495

James F Knight Simon Carley Bryan Tregunna Steve Jarvis Richard Smithies Sara deFreitas Ian Dunwell and Kevin Mackway-Jones 2010 Serious gaming technologyin major incident triage training a pragmatic controlled trial Resuscitation 81 9(2010) 1175ndash1179

Przemyslaw Korzeniowski 2016 uFLex integrates NVidia Flex with Unity3DSergei N Kurenov William W Cance Ben Noel and David W Mozingo 2009 Game-

based mass casualty burn training Studies in health technology and informatics 142(2009) 142ndash144

Tiantian Liu Adam W Bargteil James F OrsquoBrien and Ladislav Kavan 2013 Fastsimulation of mass-spring systems ACM Transactions on Graphics (TOG) 32 6(2013) 214

Miles Macklin Matthias Muumlller Nuttapong Chentanez and Tae-Yong Kim 2014 Uni-fied particle physics for real-time applications ACM Transactions on Graphics (TOG)33 4 (2014) 153

Stefan Marks JA Windsor and Burkhard Wuumlnsche 2007a Collaborative soft objectmanipulation for game engine-based virtual reality surgery simulators (2007)

Stefan Marks John Windsor and Burkhard Wuumlnsche 2007b Evaluation of gameengines for simulated surgical training In Proceedings of the 5th internationalconference on Computer graphics and interactive techniques in Australia and SoutheastAsia ACM 273ndash280

Matthias Muumlller Bruno Heidelberger Marcus Hennix and John Ratcliff 2007 Positionbased dynamics Journal of Visual Communication and Image Representation 18 2(2007) 109ndash118

Matthias Muumlller Bruno Heidelberger Matthias Teschner and Markus Gross 2005Meshless deformations based on shape matching In ACM transactions on graphics(TOG) Vol 24 ACM 471ndash478

Jun J Pan Jian Chang Xiaosong Yang Hui Liang Jian J Zhang Tahseen QureshiRobert Howell and Tamas Hickish 2015 Virtual reality training and assessmentin laparoscopic rectum surgery The International Journal of Medical Robotics andComputer Assisted Surgery 11 2 (2015) 194ndash209

M Poyade M Kargas and V Portela (2014)- Haptic Plug-In for UnityKun Qian Junxuan Bai Xiaosong Yang Junjun Pan and Jianjun Zhang 2015 Vir-

tual reality based laparoscopic surgery simulation In Proceedings of the 21st ACMSymposium on Virtual Reality Software and Technology ACM 69ndash78

James C Rosser Paul J Lynch Laurie Cuddihy Douglas A Gentile Jonathan Klonskyand Ronald Merrell 2007 The impact of video games on training surgeons in the21st century Archives of surgery 142 2 (2007) 181ndash186

Eftychios Sifakis and Jernej Barbic 2012 FEM simulation of 3D deformable solidsa practitionerrsquos guide to theory discretization and model reduction In ACM SIG-GRAPH 2012 Courses ACM 20

Tarja Susi Mikael Johannesson and Per Backlund 2007 Serious games An overviewM Suzuki S Akaishi T Rikiyama T Naitoh MM Rahman and S Matsuno 2000 Laparo-

scopic cholecystectomy Calotrsquos triangle and variations in cystic arterial supplySurgical endoscopy 14 2 (2000) 141ndash144

Jinglu Zhang Jian Chang Xiaosong Yang and Jian J Zhang 2017 Virtual RealitySurgery Simulation A Survey on Patient Specific Solution In International Work-shop on Next Generation Computer Animation Techniques Springer 220ndash233


Olek C Zienkiewicz and Robert L Taylor 2005 The finite element method for solid andstructural mechanics Butterworth-heinemann

Abstract

1 Introduction

2 Related Work

3 System Infrastructure

31 Procedures and Challenges in Laparoscopic Cholecystectomy

32 Objectives and System Design

4 Physical Simulation

41 Volumetric Soft Body Simulation

42 Surface Mesh Simulation

43 Haptic Rendering

5 Evaluation and Feedback

51 System Evaluation

52 Simulator Usability Evaluation

53 Game Engine based Simulator

54 Improvement Suggestions

6 Conclusion

Acknowledgments

References


students and surgeons can practice their professional skills alongwith the transferable skills such as team-working and emergencyreaction using this advanced technique Many research works haveproven the correlation between game-based training and the im-provement of surgical skills [Andreatta et al 2010 Creutzfeldt et al2010 Knight et al 2010 Kurenov et al 2009 Rosser et al 2007]Specifically advantages of the surgical simulator include

(1) offering a secure and efficient environment for surgeons todeeply understand the whole surgery process

(2) enabling surgeons to practice two essential skills for MISnamely eye-hand coordination and the ability to execute 3Dactions using a 2D screen as a guide

(3) allowing trainees to practice the challenging and significantprocedures repeatedly and save the training cost

It is complicated to model and simulate a typical laporoscopiccholecystectomy scene The simulation of multiple organs andtissues the interaction between tools and tissues and feedbacksfrom both visual and tactile are all necessary components Devel-oping and integrating all these functional components indepen-dently is time and resources consuming Unity [Goldstone 2009] isa cross-platform modern game engine which has a great supportof advanced audio and visual effects Its integrated developmentenvironment easy profiler on-shelf tools and modules make Unitystable and highly productive

In this study we design and develop a laparoscopic cholecystec-tomy simulator using Unity Game Engine Our surgical simulatorprovides trainees an effective platform to practice surgery proce-dures with realistic visual and haptic feedbacks (shown in Fig 1 andFig 2) In our design the organ and soft tissue simulation are basedon uflex a unity plugin originated from NVidia Flex[Korzeniowski2016] and the haptic feedback is in reference to Unity 5 HapticPlugin for Geomagic OpenHaptics 33 [Poyade et al 014 ] Thereare mainly two contributions of our work

(1) Developing an interactive laparoscopic cholecystectomy sim-ulator for surgeons to enhance their surgical skills

(2) Exploring the use of game engine for novel development ofa surgical simulator to achieve efficient and cost-effectivesolutions

The remaining of this paper is organized as follows Section 2reviews previous works in medical education and training laparo-scopic simulator and soft body deformation modeling Detailedgame objective architecture and target users are given in section3 Section 4 explains our real time physics simulation algorithmsin depth and section 5 gives the system evaluation and feedbacksFinally section 6 concludes the paper

2 RELATEDWORKGame like medical education and surgical skills training with dif-ferent objectives have been proven to be valuable where users canimprove their related skills and experience various circumstancesthat are impractical in the real world due to the safety cost or otherreasons [Susi et al 2007] James et al presented that surgeons whohad experienced selected representative video games (Super Mon-key Ball 2 Star Wars Racer Revenge and Silent Scope) performedbetter in laparoscopic handling and made fewer errors [Rosser et al

Figure 1 Laparoscopic cholecystectomy simulator workingenvironment

Figure 2 Screenshot from cholecystectomy simulator

2007] Another interesting experiment [Knight et al 2010] com-pared the tagging accuracy between participants who had under-taken the game Triage Trainer (Blitz Games Studios LeamingtonSpa Uk) training and who had taken traditional card-sort exerciseThe Triage Trainer group was more likely to triage precisely in ca-sualties than the card-sort group Game like training was also usedfor team training in emergency crisis management [Andreatta et al































Ci (p + ∆p) = 0 i = 1 n (1)

Cj (p + ∆p) gt= 0 j = 1 n (2)



















E =sum


2bend ) (7)




















531 Advantages




532 Limitations















































Abstract

1 Introduction

2 Related Work







43 Haptic Rendering






6 Conclusion

Acknowledgments

References































Ci (p + ∆p) = 0 i = 1 n (1)

Cj (p + ∆p) gt= 0 j = 1 n (2)



















E =sum


2bend ) (7)




















531 Advantages




532 Limitations















































Abstract

1 Introduction

2 Related Work







43 Haptic Rendering






6 Conclusion

Acknowledgments

References












E =sum


2bend ) (7)




















531 Advantages




532 Limitations















































Abstract

1 Introduction

2 Related Work







43 Haptic Rendering






6 Conclusion

Acknowledgments

References


















531 Advantages




532 Limitations















































Abstract

1 Introduction

2 Related Work







43 Haptic Rendering






6 Conclusion

Acknowledgments

References











































Abstract

1 Introduction

2 Related Work







43 Haptic Rendering






6 Conclusion

Acknowledgments

References

Development of Laparoscopic Cholecystectomy Simulator ...

Documents

Transcript of Development of Laparoscopic Cholecystectomy Simulator ...