Devices and systems targeted towards augmented robotic radical prostatectomy

9
Our reference: IRBM 252 P-authorquery-v9 AUTHOR QUERY FORM Journal: IRBM Please e-mail or fax your responses and any corrections to: E-mail: [email protected] Article Number: 252 Fax: +33 (0) 1 71 16 51 88 Dear Author, Please check your proof carefully and mark all corrections at the appropriate place in the proof (e.g., by using on-screen annotation in the PDF file) or compile them in a separate list. Note: if you opt to annotate the file with software other than Adobe Reader then please also highlight the appropriate place in the PDF file. To ensure fast publication of your paper please return your corrections within 48 hours. For correction or revision of any artwork, please consult http://www.elsevier.com/artworkinstructions. Any queries or remarks that have arisen during the processing of your manuscript are listed below and highlighted by flags in the proof. Click on the ‘Q ’ link to go to the location in the proof. Location in Query / Remark: click on the Q link to go article Please insert your reply or correction at the corresponding line in the proof Q1 Please confirm that given names and surnames have been identified correctly. Q2 Please supply the names of the cities and countries for affiliations “e” and “f”. Q3 Please provide the volume numbers and page range for the bibliography in references [1, 9]. Please check this box or indicate your approval if you have no corrections to make to the PDF file Thank you for your assistance.

Transcript of Devices and systems targeted towards augmented robotic radical prostatectomy

Our reference IRBM 252 P-authorquery-v9

AUTHOR QUERY FORM

Journal IRBM Please e-mail or fax your responses and any corrections to

E-mail correctionsesmeelsevierthomsondigitalcom

Article Number 252 Fax +33 (0) 1 71 16 51 88

Dear Author

Please check your proof carefully and mark all corrections at the appropriate place in the proof (eg by using on-screenannotation in the PDF file) or compile them in a separate list Note if you opt to annotate the file with software other thanAdobe Reader then please also highlight the appropriate place in the PDF file To ensure fast publication of your paper pleasereturn your corrections within 48 hours

For correction or revision of any artwork please consulthttpwwwelseviercomartworkinstructions

Any queries or remarks that have arisen during the processing of your manuscript are listed below and highlighted by flags inthe proof Click on the lsquoQrsquo link to go to the location in the proof

Location in Query Remark click on the Q link to go

article Please insert your reply or correction at the corresponding line in the proof

Q1 Please confirm that given names and surnames have been identified correctlyQ2 Please supply the names of the cities and countries for affiliations ldquoerdquo and ldquofrdquoQ3 Please provide the volume numbers and page range for the bibliography in references [1 9]

Please check this box or indicate your approval ifyou have no corrections to make to the PDF file

Thank you for your assistance

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

Disponible en ligne sur

wwwsciencedirectcom

IRBM xxx (2013) xxxndashxxx

Original article

Devices and systems targeted towards augmented robotic radicalprostatectomy

1

2

S Voros alowast A Moreau-Gaudry b B Tamadazte a G Custillon b R Heus a M-P Montmasson aQ1

F Giroud a O Gaiffe c C Pieralli c G Fiard d J-A Long d J-L Descotes d C Vidal eA Nguyen-Dinh f P Cinquin g

3

4

5

a UJF-Grenoble 1 CNRS INSERM TIMC-IMAG UMR 5525 38041 Grenoble France6

b Centre drsquoInvestigation Clinique Innovation Technologique INSERM CHU de Grenoble UJF-Grenoble 1 CIT803 38041 Grenoble France7

c FEMTO-ST UMR CNRS 6174 Universiteacute de Franche-Comteacute 25030 Besancon cedex France8

d Urology Department Grenoble University Hospital Grenoble France9

e Endocontrol-Medical SA10

f VERMON SAQ211

g UJF-Grenoble 1 CNRS TIMC-IMAG UMR 5525 Centre drsquoInvestigation Clinique Innovation Technologique INSERM CHU de Grenoble CIT803 38041

Grenoble France

12

13

Received 11 January 2013 received in revised form 11 January 2013 accepted 15 January 2013

14

Abstract15

Prostate cancer is the most frequent male cancer and the second cause of male cancer mortality in developed countries Therefore it represents

a major public health issue Health problem and the development of new therapeutic strategies to address this issue is essential During a prosta-

tectomy the surgeon looks for a compromise between an exhaustive removal of pathologic tissue (to achieve the best carcinogenic prognosis) and

the functional consequences linked to a wide excision (ie avoid as much as possible urinary incontinence and sexual dysfunction) In this con-

text the ANR TecSan DEPORRA project regroups French research laboratories (TIMC-IMAG FEMTO-ST) companies (Endocontrol-Medical

VERMON) and hospital departments (CIC-IT Urologyamp pathology Department of the Grenoble University Hospital) to bring innovative tools

for radical prostatectomy These tools will provide to the surgeon new information from several imaging modalities (video fluorescence and US

imaging) and combine them in an augmented environment We believe that this augmented environment will ultimately help the surgeon to perform

his surgical gesture ldquooptimallyrdquo and will improve the patientrsquos carcinogenic and functional prognosis

16

17

18

19

20

21

22

23

24

copy 2013 Published by Elsevier Masson SAS25

26

1 Introduction27

Prostate cancer is the most frequent male cancer and the sec-28

ond cause of male cancer mortality in developed countries with29

643000 cases and 20 new cases in 2008 according to the 200830

World Cancer Report of the International Agency for Research31

Cancer Its incidence has never stopped increasing in the past 2532

years because of the population ageing and individual screening33

In 75 of the cases it is diagnosed at a localized stage within the34

prostate (T1 or T2) At this localized stage different treatments35

lowast Corresponding author

E-mail address SandrineVorosimagfr (S Voros)

are available among which surgery radical prostatectomy (or 36

surgical ablation of the prostate) is often considered as the 37

ldquogold standardrdquo to treat prostate cancer in precise indications 38

Several approaches are in competition open surgery laparo- 39

scopic surgery or robotic surgery The laparoscopic approach 40

will probably replace open surgery in the future years notably 41

because it reduces perioperative morbidity Robotic surgery (the 42

reference robot for this approach being Intuitive Surgicalrsquos da 43

Vincireg robot) offers the surgeons a comfort close to open surgery 44

in a mini-invasive environment This comfort allows for the 45

reduction of the ldquolearning curverdquo However concerning the ben- 46

efits for the patient according to the European Association of 47

Urology (EAU) guidelines [1] ldquoit is not clear which technique 48

1959-0318$ ndash see front matter copy 2013 Published by Elsevier Masson SAS

httpdxdoiorg101016jirbm201301014

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

2 S Voros et al IRBM xxx (2013) xxxndashxxx

is superior in terms of oncological and functional results and49

cost-effectiveness Prospective trials are urgently neededrdquo an50

argument also shared in two recent Health Technology Assess-51

ments on robot assisted surgery (Belgian HTA 2009 Irish HTA52

2012)53

In the current surgical practice the surgeon must determine54

the best compromise between a complete removal of the prostate55

gland and morbidity The ideal removal consists in removing56

completely the prostate within its capsule a non complete57

removal may lead to positive surgical margins which are associ-58

ated to a higher risk of biochemical recurrence [2] (15ndash20 of59

the patients) Lesions of the urethral sphincter situated under the60

prostate can lead to incontinence (7 of the patients) Lesions61

of the neurovascular bundles (immediately in contact with the62

posterior side of the prostate) may lead to erectile problems of63

varying intensity The surgeon can try to spare more or less these64

organs depending on the preoperative staging completed by65

magnetic resonance imaging (MRI) The quality of the prostate66

removal (absence of positive surgical margins) is determined67

postoperatively thanks to a histopathology exam at the anato-68

mopathology laboratory Thus the surgeon cannot adapt his69

surgical strategy peroperatively70

2 Objectives71

In this controversial context regarding robotically assisted72

prostatectomy we propose a new robotic navigation concept73

guided by some of the principal complications of a radical pros-74

tatectomy (cancer relapse incontinence and impotence) based75

on peroperative multimodal imaging Surgical navigation sys-76

tems allow for the combination of several imaging modalities in77

a common environment thus providing more useful information78

to the surgeon during a surgery than a single imaging modality79

In most of the existing systems information from preoperative80

imaging modalities is combined with the peroperative imaging81

modality However in the case of laparoscopic surgery (soft tis-82

sue surgery) the registration of preoperative images with the83

laparoscopic images is very challenging because the organs can84

move and deform To our knowledge there are only a few navi-85

gation systems for laparoscopic navigation [3] none of which86

are used clinically87

Our proposed navigation concept is based on three coupled88

components that will ultimately ldquoaugmentrdquo the laparoscopic89

images and allow the surgeon to see beyond the visible and90

adapt his surgical strategy peroperatively91

bull an augmented laparoscope allowing the surgeon to navigate92

more harmoniously inside the abdominal cavity as he would93

do in open surgery and potentially to provide three dimen-94

sional information95

bull peroperative visualization technologies allowing the surgeon96

to visualize the important anatomical structures for radical97

prostatectomy98

visualization of the prostate and surrounding organs thanks99

to an innovative transurethral ultrasound probe100

identification and visualization of the tumor and prostatic 101

cells thanks to bimodal fluorescence probe 102

bull integration of the multimodal information to navigation 103

systems thanks to the registration of the different imaging 104

modalities when required 105

3 Material amp methods 106

31 Augmented laparoscopy thanks to an innovative video 107

device 108

Laparoscopic surgery in general can be challenging for a sur- 109

geon for several reasons among which the limited field of view 110

of the endoscope (ldquokeyholerdquo surgery) In consequence whether 111

it is displaced by a human assistant or a robotic endoscope holder 112

the endoscope is mobilized a lot since it is used to see precisely 113

the operating field but also to monitor the instruments introduc- 114

tion and displacement inside the abdominal cavity Movements 115

of the endoscope increase the risk of staining of the lens which 116

leads to a removal of the endoscope from the patient to clean the 117

lens increasing the surgery duration and discomfort 118

To provide a solution to this difficulty we have developed 119

an innovative vision system that can be used in combination 120

with a traditional endoscope thanks to an encapsulation in a 121

traditional trocar Our system is composed of two miniature 122

cameras that are positioned like a pair of glasses around the 123

endoscope (Fig 1) and provide a panoramic view of the abdom- 124

inal cavity These cameras are similar to those present on cell 125

phones and cost only a few US$ in large scale diffusion The 126

system has been designed for an easy insertion deployment and 127

removal of the cameras and has been patented [4] Since it is 128

positioned around the endoscope it provides roughly the same 129

view direction as the endoscope and a registration between the 130

laparoscopic image and the vision system images is unneces- 131

sary The cameras of the innovative vision system are mounted 132

in stereoscopic conditions which could allow for a local 3D 133

reconstruction of the scene Our preclinical evaluation of the 134

potential expected medical service of this device is presented in 135

section 4 136

32 Augmented laparoscopy thanks to an innovative 137

ultrasound system 138

Previous work [5] has already shown that the use of tran- 139

srectal ultrasound during laparoscopic radical prostatectomies 140

could assist the surgeons in the visualization of specific prostate 141

contour anatomy and of the neurovascular bundles and in 142

the bladder neck dissection According to the authors the use 143

of transrectal ultrasound in laparoscopic prostatectomy com- 144

pensated for the lack of tactile feedback compared to open 145

surgery achieved significant decrease in the incidence of posi- 146

tive surgical margins and achieved quicker and superior potency 147

recovery In this work the ultrasound images were not fused 148

with the laparoscopic images and the surgeon had to perform 149

a challenging mental registration of the two imaging modali- 150

ties Long et al [6] also showed the potential of a guidance of 151

a transrectal ultrasound probe with a robotic endoscope holder 152

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 3

Fig 1 The proposed global vision system a) the mini-cameras used for the vision system b) CAD drawing of the augmented endoscope concept the mini-cameras

are inserted into a modified trocar When the endoscope is inserted the vision system is deployed like glasses around the endoscope c) and d) rapid prototyping

realization

(Endocontrolrsquos ViKY system) Holmes et al [7] have developed153

a 2D transurethral ultrasound probe for brachytherapy that is154

manually rotated around its axis to provide 3D imaging They155

affirm that the endourethral approach permits a better resolu-156

tion than the transrectal approach for the visualization of the157

prostate The transurethral approach also allows avoiding some158

of the intrinsic limitations of transrectal ultrasound imaging (a159

layer of air forms between the prostate and the rectum during160

the prostate dissection making the ultrasound visualization of161

the prostate impossible after this surgical phase)162

In light of these previous works we have developed two163

transurethral ultrasound probe prototypes (one for 2D imaging164

and a motorized version where the probe rotates around its axis165

to provide 3D images) (Fig 2) The 2D probe is composed of166

64 piezoelectric elements with a frequency of 10 MHz and a167

3 mm diameter semi-rigid catheter for an easy introduction The168

3D probe has the same central frequency but it is composed of169

128 piezoelements and has a 6 mm diameter catheter They are170

Fig 2 The innovative intraurethral ultrasound probe Top the 3D transurethral

probe able to rotate around its axis Bottom the 2D transurethral probe

connected to an Ultrasonix ultrasound machine allowing us to 171

control all the piezoelectric elements 172

In parallel we have also developed an endoscopicultrasound 173

fusion demonstrator The registration is based on the POSIT 174

algorithm [8] and is based on the manual localization in both 175

imaging modalities of artificial landmarks In the frame of this 176

project two complementary approaches have been investigated 177

passive landmarks (laparoscopic needles planted in the prostate) 178

and innovative active ultrasound landmarks that emit an ultra- 179

sound signal that can be detected by the transurethral probe 180

allowing for their precise localization in the ultrasound referen- 181

tial [9] (Fig 3) 182

Our preliminary results for the registration of ultrasound and 183

laparoscopic images using passive landmarks and for the precise 184

3D localization of the ultrasound ldquoactiverdquo markers are described 185

in section 4 186

33 Augmented laparoscopy thanks to fluorescence 187

imaging 188

During a radical prostatectomy the ability to visualize bio- 189

logical characteristics of tissue (prostatic vs non-prostatic on 190

the one hand normal vs malignant on the other hand) could help 191

the surgeon to respectively determine precisely the location of 192

the prostate capsule and assess the extent of the cancer and thus 193

allow him or her to adapt his surgical strategy peroperatively 194

Based on these observations we have developed a bimodal 195

fluorescence fibered probe for the peroperative characterization 196

of tissue for radical prostatectomy 197

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

4 S Voros et al IRBM xxx (2013) xxxndashxxx

Fig 3 The proposed markers for the registration of ultrasound and endoscopic

images Top ldquoactiverdquo landmark (laparoscopic ultrasound emitter) Bottom ldquopas-

siverdquo landmark (laparoscopic needle) used for the registration of ultrasound and

laparoscopic images

bull the normalmalignant characterization of the tissues is based198

on the detection of the autofluorescence of the Protopor-199

phyrin IX (PpIX) protein which accumulates in malignant200

cells Indeed the elimination cycle of Protorphyrin is per-201

turbed in case of malignant cells It causes thus an increase202

of Protoporphyrin concentration A first prototype (already203

available see Fig 4 left) comes in the form of a testbench204

equipped with a laser source emitting at 405 nm (excitation 205

wavelength for the PpIX protein) and with optical fibers and 206

a spectrometer allowing the collection of a fluorescence spec- 207

trum of 3648 points in the range 345ndash1040 mm (the testbench 208

can image a 25 times 25 mm2 area with a spatial resolution of 209

100 m and a penetration depth of roughly 300 m) Using 210

this spectrum an intensity image can be created (Fig 4 211

right) The analysis of prostate tissue samples gathered in 212

the frame of a biomedical protocol (section 4) will allow us 213

to determine the optimal parameters for the acquisition of 214

the fluorescence spectrum and build discrimination criteria 215

from the measurements between malignanthealthy tissue and 216

prostaticnon-prostatic tissue 217

bull the prostaticnon-prostatic characterization of the tissues is 218

based on the detection of the prostate-specific membrane anti- 219

gen (PSMA) which is specific of the prostatic membrane 220

This detection is made possible by the immunofluorescence 221

tagging of the PSMA using a specific anti-PSMA antibody 222

labelled with a fluorescent tracer When excited by light at 223

a characteristic absorption wavelength the fluorescent tracer 224

emits light at a characteristic emission wavelength Two flu- 225

orescent tracers are investigated in the DEPORRA project 226

Fluorescein IsoThioCyanate (FITC) and Cyanine 5 Both 227

FITC and Cyanine 5 have a characteristic emission wave- 228

length different from the autofluorescence wavelength of the 229

Protoporpphyrin IX protein allowing for the characterization 230

of the tissue type (prostatic vs non-prostatic) and the tissue 231

status (healthy vs malignant) with the same testbench (Fig 5) 232

The testbench presented on Fig 4 has also been equipped 233

with two laser sources emitting at 488 nm and 642 nm exci- 234

tation wavelengths respectively for FITC and Cyanine 5 for 235

recognizing prostate tissue from those environing 236

A bimodal laparoscopic-compliant fluorescence probe based 237

on the miniaturization of the fluorescence testbench will be avail- 238

able in 2013 (Fig 6 top) Preliminary results based on the 239

analysis of histopathologic slices will be presented in section 240

4 In order to overlay the probersquos fluorescence measurement on 241

the laparoscopic images we plan on detecting automatically the 242

position of the tip of the laparoscopic probe using a real-time 243

Fig 4 Experimental fluorescence design and results Left experimental optical device Right typical intensity image obtained

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 5

Perc

ent

of

maxim

a

Wavelength (nm)

PpIX emission

FITC excitation

FITC emission

PpIX excitation

350 400 450 500 550 600 650 700 750

Perc

ent

of

maxim

a

10

00

08

06

04

02

Wavelength (nm)

PpIX emission

Cy5 excitation

Cy5 emission

PpIX excitation

BA

350 400 450 500 550 600 650 700 750

10

00

08

06

04

02

Fig 5 Excitation and emission spectra of Protoporphyrin IX Fluorescein IsoThioCyanate and Cyanine 5 (Cy5) fluorochromes A Overlay of the excitation (blue

dashed line) and emission (orange solid line) spectra of Protoporphyrin IX with excitation (green dashed line) and emission (emission spectrum) of Fluorescein

IsoThioCyanate B Overlay of the excitation (blue dashed line) and emission (orange solid line) spectra of Protoporphyrin IX with excitation (red dashed line) and

emission (red solid line) of Cy5

Fig 6 The bimodal fluorescence laparoscopic probe and the image-based anal-

ysis approach to localize it in laparoscopic images Top the CAD drawing

of the bimodal fluorescence laparoscopic probe Bottom automatic detection

of a laparoscopic instrument which will be used to overlay the probersquos tissue

characterization measurements on the laparoscopic images

image analysis approach that we have developed [10] (Fig 6244

bottom)245

34 Preclinical and clinical evaluation246

Preliminary evaluation of the devices and the software devel-247

oped in the frame of this project has already been performed on248

laboratory testbenchs and during three cadaver experiments at249

the anatomy laboratory (section 4) However in order to acquire250

the qualitative and quantitative proofs necessary to perform the251

fist clinical evaluations of our innovative medical devices on 252

patients we have submitted two biomedical research protocols 253

bull the first one called ldquofusion of echographic (ultrasound) and 254

endoscopic imagesrdquo (FEE) is a pilot monocentric study on a 255

cohort of 15 patients and is currently running It aims at val- 256

idating our ultrasound approach and evaluating its expected 257

medical benefit during radical prostatectomies the surgeons 258

will insert surgical needles in the prostate and endorectal 259

ultrasound images and laparoscopic images will be recorded 260

The data will be processed offline in the laboratory to assess 261

quantitatively the feasibility of the registration of ultrasound 262

and laparoscopic images using passive markers This proto- 263

col already accepted by the ethical instances will be a pilot 264

monocentric study with 15 patients 265

bull the second protocol called COPROST will allow us to obtain 266

fresh prostate chips during transurethral resections in the 267

frame of a pilot monocentric prospective non-randomized 268

open and controlled clinical trial The protocol with an 269

inclusion period of 24 months is already defined and writ- 270

ten following the French Regulation on Biomedical Research 271

and biological tissue collection It is being submitted to the 272

ethical committee ldquocomiteacute de protection des personnesrdquo and 273

the ldquoagence nationale de seacutecuriteacute du meacutedicament et des pro- 274

duits de santeacuterdquo (ANSM) The prostate chips will be on one 275

hand characterized by anatomopathologists to determine their 276

nature (pathologic vs healthy prostatic vs non-prostatic) and 277

will allow us to validate our immunofluorescence protocol and 278

fine-tune our auto and immunofluorescence measurements 279

protocols on fresh tissue samples 280

4 Results 281

In this section we present our preclinical evaluation of the 282

different prototypes and software presented in the previous sec- 283

tion 284

41 Augmented laparoscopy thanks to an innovative video 285

device 286

In order to evaluate the potential benefits of the pro- 287

posed vision system we asked a surgeon to perform a simple 288

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

6 S Voros et al IRBM xxx (2013) xxxndashxxx

Fig 7 Ultrasound image of a prostate acquired with our innovative transurethral

ultrasound probe during a cadaver experiment The image was acquired with our

first prototype (64 piezoelectric elements)

surgical task once with the traditional endoscope alone and289

once with the proposed system alone (although in practice they290

can be combined to associate local and global views) The tasks291

consisted in localizing a suture needle and bringing it to a fixed292

target point The experiment was performed on porcine organs293

placed in a training box and repeated six times by the surgeon294

At each realization the needlersquos initial position was repositioned295

randomly at a distance equivalent to its initial position and the296

surgeon started randomly with the traditional endoscope or the297

proposed system In both cases a robotic endoscope holder was298

used to mobilize the endoscope and the vision system allowing299

us to record the displacements of the vision systems300

The full methodology and results are currently under sub-301

mission but to summarize the mean time required to perform302

the six experiments with the traditional endoscope was of 190 s303

compared to 245 s with the proposed system Moreover the sur-304

geon needed to give an average of 232 commands to the robotic305

scope holder to perform one stitch with the endoscope alone306

compared to 46 commands with the proposed system These307

preliminary results suggest that the proposed system could sig-308

nificantly reduce the laparoscopic surgery time and the cognitive309

load required for the control of the endoscope We now plan on310

performing cadaver experiments to evaluate the deployment of311

the vision system in conditions close to the clinical reality and312

to evaluate the system with several surgeons313

42 Augmented laparoscopy thanks to an innovative314

ultrasound system315

We performed two cadaver experiments which allowed us316

to determine the optimal characteristics for the realization of317

the intraurethral ultrasound probe During these experiments318

we evaluated the (difficult) insertion of the catheter through319

the urethra and the prostate visualization It must be noted that320

our clinicians partners stressed that the rigor mortis made this321

insertion harder These experiments allowed us to find the best322

rigidity for the catheter of the probe it must be flexible enough323

Fig 8 Demonstration of the fusion of ultrasound and laparoscopic images using

passive landmarks on chicken breasts Top-left the ultrasound image top-right

the laparoscopic image The passive landmarks are indicated with the green

arrows in the laparoscopic image They are selected manually in both modalities

to perform the registration Bottom the fusion of the two imaging modalities

to be introduced in the urethra but rigid enough to avoid the 324

distortions of the catheter 325

The first ultrasound probe (2D) was designed with 64 326

piezoelements It appeared that it was not enough On the one 327

hand the probe imaged a too small part of the prostate so that 328

it was difficult to identify what was represented on the ultra- 329

sound image (Fig 7) Furthermore it was impossible to know 330

in which direction the probe was oriented On the other hand 331

we demonstrated [9] that a longer probe increases the precision 332

of the localization of the active ultrasound transducers That is 333

why the second probe designed for the project (3D) has 128 334

piezoelements This second probe has graduations located on 335

its catheter visible on Fig 2 They allow the surgeon to know 336

the probersquos insertion depth in the urethra and a line along the 337

catheter allows appreciating the torsion applied to the catheter 338

The localization of the active transducers is currently under 339

submission It is based on a global positioning system (GPS) 340

method We made some experiments on a testbench making 341

localization in water to have ideal homogeneous conditions and 342

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 7

Fig 9 Preliminary experiments of the autofluorescence detection with our testbench The autofluorescence of chlorophyll which has optical properties very similar

to Protoporphyrin IX was observed on our testbench Left direct image of a leaf right fluorescence image of the leaf at 680 nm As expected chlorophyll seems

absent in the veins

Fig 10 Specific fluorescence signal of anti-prostate-specific membrane antigen marker A Nuclear counter-staining of prostatic tissue with Hoechst (blue DNA

marker) We observe the nuclei of prostate cells regularly organized in the glandular epithelium Prostatic glands are separated by stroma B Fluorescence emission

of anti-prostate-specific membrane antigen-Fluorescein IsoThioCyanate on cellular membranes of prostatic cells C Overlay of the A and B images This figure

demonstrates the specificity on fixed prostatic tissue of a specific anti-PSMA antibody labelled with Fluorescein IsoThioCyanate We can also detect the contribution

of non-specific autofluorescence of prostatic concretions (Fig 11) (times20 magnification)

localization on a chicken breast to have more realistic condi-343

tions In both cases we were able to localize the active transducer344

with a precision less than 625 m Moreover the computation345

time is a few hundred milliseconds for a 2D localization These346

results demonstrate that the localization is fast enough to be347

implemented for a prostatectomy For the moment only a 2D 348

localization has been made with the active landmarks and the 349

3D localization is in progress Then we will be able to per- 350

form a registration with laparoscopic images using ldquoactiverdquo 351

markers 352

Fig 11 Non-specific autofluorescence observed in the prostate fixed tissues This figure shows the benefits of a near infrared marker like Cyanine 5 (excited at

633 nm) compared to Fluorescein IsoThioCyanate (excited at 488 nm) A Auto fluorescence under laser excitation at 488 nm We observe that prostate concretions

(calcified material) respond to this excitation by a strong fluorescence signal B Auto fluorescence under laser excitation at 633 nm The use of Cyanine 5 reduces

widely autofluorescence of some structures such as prostate concretions (times20 magnification)

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

8 S Voros et al IRBM xxx (2013) xxxndashxxx

Concerning the registration between laparoscopic and ultra-353

sound images we performed several testbench evaluations of354

the registration algorithm on chicken breasts The result of such355

a registration obtained by manually pairing passive markers356

visible on both the laparoscopic and ultrasound imaging modal-357

ities are presented on Fig 8 In this example we obtained a358

registration root mean square error (RMS) of 038 mm359

43 Augmented laparoscopy thanks to fluorescence360

imaging361

First experiments were performed on leaves to demonstrate362

the feasibility of the detection of chlorophyll autofluorescence363

which has optical properties very similar to PpIX (absorption at364

405 nm and emission at 680 nm) as described on Fig 9 Then365

experiments were conducted on histology slices of the human366

prostate with as we anticipated inconclusive results because367

of the expected low signal-to-noise ratio caused by the insuffi-368

cient thickness of a histology slice (3 m thickness) compared to369

the optimal use of the device (300 m thickness) These results370

motivated our biomedical research COPROST on macroscopic371

fresh prostate samples372

Concerning the immunofluorescence protocol we were able373

to demonstrate the specificity of the anti-PSMA antibody on374

fixed tissue samples labelled with FITC as illustrated by Fig 10375

We observed higher autofluorescence problems when excit-376

ing the sample at the FITC specific wavelength compared to377

an excitation at the Cyanine 5 specific wavelength (Fig 11)378

These results motivated the inclusion of both fluorophores in our379

biomedical research COPROST on macroscopic fresh prostate380

samples FITC because it was already used to demonstrate speci-381

ficity of anti-PSMA antibody and Cyanine 5 because its near382

infra-red characteristics make it a better candidate on fresh mate-383

rial (higher penetration in tissue) and will improve signalnoise384

ratio as previously demonstrated385

To our knowledge most of the optical biopsy approaches as386

previously defined have been validated either on histologic slices387

or on ratmice prostate models However these approaches have388

limitations prostatic tissue is impaired by the chemical treat-389

ment required for histologic preparation and animal models do390

not directly mimic all aspects of human prostate cancer [11] In391

short it has not been proved yet that a successful detection with392

the investigated devices on histopathology slices or on animal393

models is sufficient to guarantee a successful detection on fresh394

human tissues The experiments that will be performed in the395

frame of the COPROST protocol in 2013 will be determinant to396

confirm the feasibility of our approach on fresh tissue samples397

5 DiscussionConclusion398

The DEPORRA project allowed for the development of inno-399

vative devices and navigation prototypes in the objective of400

allowing the surgeon to see ldquobeyond the visiblerdquo during a radi-401

cal prostatectomy First evaluations of the devices and methods402

have been performed preclinically and are very encouraging for403

the development of innovative approaches to assist the surgeon

during such a complex surgery Our preliminary results have also 404

shown the limits of preclinical validation and have convinced us 405

of the necessity to launch biomedical researches that will allow 406

us to validate further the devices This process required a con- 407

sequent amount of effort and time but is mandatory for the 408

fine-tuning of our tissue characterization tools and to obtain the 409

first clinical proofs of the relevance of the developed medical 410

devices which are mandatory to perform clinical evaluations of 411

the complete navigation systems 412

We now need to push further the integration of the devices 413

into such navigation systems in order to exploit at best the infor- 414

mation provided by each modality and to determine the optimal 415

approach for displaying the relevant information to the surgeon 416

in a clinical environment This will imply a conception and 417

development effort a definition of qualitative and quantitative 418

parameters for the first assessment of the delivered medical ben- 419

efit of our developments risks analysis and biomedical research 420

preparations in order to meet the ethical instances requirements 421

Acknowledgments 422

This work has been supported by French National Research 423

Agency (ANR) through TecSan program (project DEPORRA 424

no ANR-09-TECS-006) 425

References 426

[1] Heidenreich A Bolla M Joniau S Mason MD Matveev V Mottet N et al 427

Guidelines on prostate cancer Eur Assoc Urol 2011 [www uroweb org] 428

[2] Pfitzenmaier J Pahernik S Tremmel T Haferkamp A Buse S Hohen- 429

fellner M Positive surgical margins after radical prostatectomy do 430

they have an impact on biochemical or clinical progression BJU Int 431

2008102(10)1413ndash8 432

[3] Soler L Nicolau S Schmid J Koehl C Marescaux J et al Virtual reality 433

and augmented reality in digestive surgery In Third IEEE and ACM Inter- 434

national Symposium on Mixed and Augmented Reality (ISMAR) 2004 435

p 278ndash9 436

[4] S Voros B Tamadazte P Cinquin C Fouard Systegraveme drsquoimagerie multi- 437

vision pour chirurgie laparoscopique demande de deacutepocirct de brevet FR 438

1259489 102012 439

[5] Ukimura O Ahlering TE Gill IS Transrectal ultrasound-guided 440

energy-free nerve-sparing laparoscopic radical prostatectomy J Endourol 441

200822(9)1993ndash5 442

[6] Long JA Lee BH Guillotreau J Autorino R Laydner H Yakoubi R et al 443

Real-time robotic transrectal ultrasound navigation during robotic radical 444

prostatectomy initial clinical experience Urology 201280(3)608ndash13 445

[7] Holmes III DR Davis BJ Bruce CJ Robb RA 3D visualization analysis 446

and treatment of the prostate using trans-urethral ultrasound Comput Med 447

Imaging Graph 200327339ndash49 [2003] 448

[8] Dementhon D Model-based object pose in 25 lines of code Int J Comput 449

Vis 199515(1ndash2)123ndash41 450

[9] Custillon G Voros S Cinquin P Nguyen-Dinh A Moreau-Gaudry A 451

Bidimensional localization of active ultrasound transducers for use in 452

laparoscopic prostate surgery IEEE Trans Med Imaging 2012 Q3 453

[10] Wolf R Duchateau J Cinquin P Voros S 3D tracking of laparoscopic 454

instruments using statistical and geometric modeling Med Image Com- 455

put Comput Assist Interv 20116891203ndash10 [Lecture notes in computer 456

science] 457

[11] Valkenburg KC Williams BO ldquoMouse models of prostate cancerrdquo Prostate 458

Cancer 2011201122 httpdxdoiorg1011552011895238 [Article ID 459

895238] 460

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

Disponible en ligne sur

wwwsciencedirectcom

IRBM xxx (2013) xxxndashxxx

Original article

Devices and systems targeted towards augmented robotic radicalprostatectomy

1

2

S Voros alowast A Moreau-Gaudry b B Tamadazte a G Custillon b R Heus a M-P Montmasson aQ1

F Giroud a O Gaiffe c C Pieralli c G Fiard d J-A Long d J-L Descotes d C Vidal eA Nguyen-Dinh f P Cinquin g

3

4

5

a UJF-Grenoble 1 CNRS INSERM TIMC-IMAG UMR 5525 38041 Grenoble France6

b Centre drsquoInvestigation Clinique Innovation Technologique INSERM CHU de Grenoble UJF-Grenoble 1 CIT803 38041 Grenoble France7

c FEMTO-ST UMR CNRS 6174 Universiteacute de Franche-Comteacute 25030 Besancon cedex France8

d Urology Department Grenoble University Hospital Grenoble France9

e Endocontrol-Medical SA10

f VERMON SAQ211

g UJF-Grenoble 1 CNRS TIMC-IMAG UMR 5525 Centre drsquoInvestigation Clinique Innovation Technologique INSERM CHU de Grenoble CIT803 38041

Grenoble France

12

13

Received 11 January 2013 received in revised form 11 January 2013 accepted 15 January 2013

14

Abstract15

Prostate cancer is the most frequent male cancer and the second cause of male cancer mortality in developed countries Therefore it represents

a major public health issue Health problem and the development of new therapeutic strategies to address this issue is essential During a prosta-

tectomy the surgeon looks for a compromise between an exhaustive removal of pathologic tissue (to achieve the best carcinogenic prognosis) and

the functional consequences linked to a wide excision (ie avoid as much as possible urinary incontinence and sexual dysfunction) In this con-

text the ANR TecSan DEPORRA project regroups French research laboratories (TIMC-IMAG FEMTO-ST) companies (Endocontrol-Medical

VERMON) and hospital departments (CIC-IT Urologyamp pathology Department of the Grenoble University Hospital) to bring innovative tools

for radical prostatectomy These tools will provide to the surgeon new information from several imaging modalities (video fluorescence and US

imaging) and combine them in an augmented environment We believe that this augmented environment will ultimately help the surgeon to perform

his surgical gesture ldquooptimallyrdquo and will improve the patientrsquos carcinogenic and functional prognosis

16

17

18

19

20

21

22

23

24

copy 2013 Published by Elsevier Masson SAS25

26

1 Introduction27

Prostate cancer is the most frequent male cancer and the sec-28

ond cause of male cancer mortality in developed countries with29

643000 cases and 20 new cases in 2008 according to the 200830

World Cancer Report of the International Agency for Research31

Cancer Its incidence has never stopped increasing in the past 2532

years because of the population ageing and individual screening33

In 75 of the cases it is diagnosed at a localized stage within the34

prostate (T1 or T2) At this localized stage different treatments35

lowast Corresponding author

E-mail address SandrineVorosimagfr (S Voros)

are available among which surgery radical prostatectomy (or 36

surgical ablation of the prostate) is often considered as the 37

ldquogold standardrdquo to treat prostate cancer in precise indications 38

Several approaches are in competition open surgery laparo- 39

scopic surgery or robotic surgery The laparoscopic approach 40

will probably replace open surgery in the future years notably 41

because it reduces perioperative morbidity Robotic surgery (the 42

reference robot for this approach being Intuitive Surgicalrsquos da 43

Vincireg robot) offers the surgeons a comfort close to open surgery 44

in a mini-invasive environment This comfort allows for the 45

reduction of the ldquolearning curverdquo However concerning the ben- 46

efits for the patient according to the European Association of 47

Urology (EAU) guidelines [1] ldquoit is not clear which technique 48

1959-0318$ ndash see front matter copy 2013 Published by Elsevier Masson SAS

httpdxdoiorg101016jirbm201301014

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

2 S Voros et al IRBM xxx (2013) xxxndashxxx

is superior in terms of oncological and functional results and49

cost-effectiveness Prospective trials are urgently neededrdquo an50

argument also shared in two recent Health Technology Assess-51

ments on robot assisted surgery (Belgian HTA 2009 Irish HTA52

2012)53

In the current surgical practice the surgeon must determine54

the best compromise between a complete removal of the prostate55

gland and morbidity The ideal removal consists in removing56

completely the prostate within its capsule a non complete57

removal may lead to positive surgical margins which are associ-58

ated to a higher risk of biochemical recurrence [2] (15ndash20 of59

the patients) Lesions of the urethral sphincter situated under the60

prostate can lead to incontinence (7 of the patients) Lesions61

of the neurovascular bundles (immediately in contact with the62

posterior side of the prostate) may lead to erectile problems of63

varying intensity The surgeon can try to spare more or less these64

organs depending on the preoperative staging completed by65

magnetic resonance imaging (MRI) The quality of the prostate66

removal (absence of positive surgical margins) is determined67

postoperatively thanks to a histopathology exam at the anato-68

mopathology laboratory Thus the surgeon cannot adapt his69

surgical strategy peroperatively70

2 Objectives71

In this controversial context regarding robotically assisted72

prostatectomy we propose a new robotic navigation concept73

guided by some of the principal complications of a radical pros-74

tatectomy (cancer relapse incontinence and impotence) based75

on peroperative multimodal imaging Surgical navigation sys-76

tems allow for the combination of several imaging modalities in77

a common environment thus providing more useful information78

to the surgeon during a surgery than a single imaging modality79

In most of the existing systems information from preoperative80

imaging modalities is combined with the peroperative imaging81

modality However in the case of laparoscopic surgery (soft tis-82

sue surgery) the registration of preoperative images with the83

laparoscopic images is very challenging because the organs can84

move and deform To our knowledge there are only a few navi-85

gation systems for laparoscopic navigation [3] none of which86

are used clinically87

Our proposed navigation concept is based on three coupled88

components that will ultimately ldquoaugmentrdquo the laparoscopic89

images and allow the surgeon to see beyond the visible and90

adapt his surgical strategy peroperatively91

bull an augmented laparoscope allowing the surgeon to navigate92

more harmoniously inside the abdominal cavity as he would93

do in open surgery and potentially to provide three dimen-94

sional information95

bull peroperative visualization technologies allowing the surgeon96

to visualize the important anatomical structures for radical97

prostatectomy98

visualization of the prostate and surrounding organs thanks99

to an innovative transurethral ultrasound probe100

identification and visualization of the tumor and prostatic 101

cells thanks to bimodal fluorescence probe 102

bull integration of the multimodal information to navigation 103

systems thanks to the registration of the different imaging 104

modalities when required 105

3 Material amp methods 106

31 Augmented laparoscopy thanks to an innovative video 107

device 108

Laparoscopic surgery in general can be challenging for a sur- 109

geon for several reasons among which the limited field of view 110

of the endoscope (ldquokeyholerdquo surgery) In consequence whether 111

it is displaced by a human assistant or a robotic endoscope holder 112

the endoscope is mobilized a lot since it is used to see precisely 113

the operating field but also to monitor the instruments introduc- 114

tion and displacement inside the abdominal cavity Movements 115

of the endoscope increase the risk of staining of the lens which 116

leads to a removal of the endoscope from the patient to clean the 117

lens increasing the surgery duration and discomfort 118

To provide a solution to this difficulty we have developed 119

an innovative vision system that can be used in combination 120

with a traditional endoscope thanks to an encapsulation in a 121

traditional trocar Our system is composed of two miniature 122

cameras that are positioned like a pair of glasses around the 123

endoscope (Fig 1) and provide a panoramic view of the abdom- 124

inal cavity These cameras are similar to those present on cell 125

phones and cost only a few US$ in large scale diffusion The 126

system has been designed for an easy insertion deployment and 127

removal of the cameras and has been patented [4] Since it is 128

positioned around the endoscope it provides roughly the same 129

view direction as the endoscope and a registration between the 130

laparoscopic image and the vision system images is unneces- 131

sary The cameras of the innovative vision system are mounted 132

in stereoscopic conditions which could allow for a local 3D 133

reconstruction of the scene Our preclinical evaluation of the 134

potential expected medical service of this device is presented in 135

section 4 136

32 Augmented laparoscopy thanks to an innovative 137

ultrasound system 138

Previous work [5] has already shown that the use of tran- 139

srectal ultrasound during laparoscopic radical prostatectomies 140

could assist the surgeons in the visualization of specific prostate 141

contour anatomy and of the neurovascular bundles and in 142

the bladder neck dissection According to the authors the use 143

of transrectal ultrasound in laparoscopic prostatectomy com- 144

pensated for the lack of tactile feedback compared to open 145

surgery achieved significant decrease in the incidence of posi- 146

tive surgical margins and achieved quicker and superior potency 147

recovery In this work the ultrasound images were not fused 148

with the laparoscopic images and the surgeon had to perform 149

a challenging mental registration of the two imaging modali- 150

ties Long et al [6] also showed the potential of a guidance of 151

a transrectal ultrasound probe with a robotic endoscope holder 152

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 3

Fig 1 The proposed global vision system a) the mini-cameras used for the vision system b) CAD drawing of the augmented endoscope concept the mini-cameras

are inserted into a modified trocar When the endoscope is inserted the vision system is deployed like glasses around the endoscope c) and d) rapid prototyping

realization

(Endocontrolrsquos ViKY system) Holmes et al [7] have developed153

a 2D transurethral ultrasound probe for brachytherapy that is154

manually rotated around its axis to provide 3D imaging They155

affirm that the endourethral approach permits a better resolu-156

tion than the transrectal approach for the visualization of the157

prostate The transurethral approach also allows avoiding some158

of the intrinsic limitations of transrectal ultrasound imaging (a159

layer of air forms between the prostate and the rectum during160

the prostate dissection making the ultrasound visualization of161

the prostate impossible after this surgical phase)162

In light of these previous works we have developed two163

transurethral ultrasound probe prototypes (one for 2D imaging164

and a motorized version where the probe rotates around its axis165

to provide 3D images) (Fig 2) The 2D probe is composed of166

64 piezoelectric elements with a frequency of 10 MHz and a167

3 mm diameter semi-rigid catheter for an easy introduction The168

3D probe has the same central frequency but it is composed of169

128 piezoelements and has a 6 mm diameter catheter They are170

Fig 2 The innovative intraurethral ultrasound probe Top the 3D transurethral

probe able to rotate around its axis Bottom the 2D transurethral probe

connected to an Ultrasonix ultrasound machine allowing us to 171

control all the piezoelectric elements 172

In parallel we have also developed an endoscopicultrasound 173

fusion demonstrator The registration is based on the POSIT 174

algorithm [8] and is based on the manual localization in both 175

imaging modalities of artificial landmarks In the frame of this 176

project two complementary approaches have been investigated 177

passive landmarks (laparoscopic needles planted in the prostate) 178

and innovative active ultrasound landmarks that emit an ultra- 179

sound signal that can be detected by the transurethral probe 180

allowing for their precise localization in the ultrasound referen- 181

tial [9] (Fig 3) 182

Our preliminary results for the registration of ultrasound and 183

laparoscopic images using passive landmarks and for the precise 184

3D localization of the ultrasound ldquoactiverdquo markers are described 185

in section 4 186

33 Augmented laparoscopy thanks to fluorescence 187

imaging 188

During a radical prostatectomy the ability to visualize bio- 189

logical characteristics of tissue (prostatic vs non-prostatic on 190

the one hand normal vs malignant on the other hand) could help 191

the surgeon to respectively determine precisely the location of 192

the prostate capsule and assess the extent of the cancer and thus 193

allow him or her to adapt his surgical strategy peroperatively 194

Based on these observations we have developed a bimodal 195

fluorescence fibered probe for the peroperative characterization 196

of tissue for radical prostatectomy 197

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

4 S Voros et al IRBM xxx (2013) xxxndashxxx

Fig 3 The proposed markers for the registration of ultrasound and endoscopic

images Top ldquoactiverdquo landmark (laparoscopic ultrasound emitter) Bottom ldquopas-

siverdquo landmark (laparoscopic needle) used for the registration of ultrasound and

laparoscopic images

bull the normalmalignant characterization of the tissues is based198

on the detection of the autofluorescence of the Protopor-199

phyrin IX (PpIX) protein which accumulates in malignant200

cells Indeed the elimination cycle of Protorphyrin is per-201

turbed in case of malignant cells It causes thus an increase202

of Protoporphyrin concentration A first prototype (already203

available see Fig 4 left) comes in the form of a testbench204

equipped with a laser source emitting at 405 nm (excitation 205

wavelength for the PpIX protein) and with optical fibers and 206

a spectrometer allowing the collection of a fluorescence spec- 207

trum of 3648 points in the range 345ndash1040 mm (the testbench 208

can image a 25 times 25 mm2 area with a spatial resolution of 209

100 m and a penetration depth of roughly 300 m) Using 210

this spectrum an intensity image can be created (Fig 4 211

right) The analysis of prostate tissue samples gathered in 212

the frame of a biomedical protocol (section 4) will allow us 213

to determine the optimal parameters for the acquisition of 214

the fluorescence spectrum and build discrimination criteria 215

from the measurements between malignanthealthy tissue and 216

prostaticnon-prostatic tissue 217

bull the prostaticnon-prostatic characterization of the tissues is 218

based on the detection of the prostate-specific membrane anti- 219

gen (PSMA) which is specific of the prostatic membrane 220

This detection is made possible by the immunofluorescence 221

tagging of the PSMA using a specific anti-PSMA antibody 222

labelled with a fluorescent tracer When excited by light at 223

a characteristic absorption wavelength the fluorescent tracer 224

emits light at a characteristic emission wavelength Two flu- 225

orescent tracers are investigated in the DEPORRA project 226

Fluorescein IsoThioCyanate (FITC) and Cyanine 5 Both 227

FITC and Cyanine 5 have a characteristic emission wave- 228

length different from the autofluorescence wavelength of the 229

Protoporpphyrin IX protein allowing for the characterization 230

of the tissue type (prostatic vs non-prostatic) and the tissue 231

status (healthy vs malignant) with the same testbench (Fig 5) 232

The testbench presented on Fig 4 has also been equipped 233

with two laser sources emitting at 488 nm and 642 nm exci- 234

tation wavelengths respectively for FITC and Cyanine 5 for 235

recognizing prostate tissue from those environing 236

A bimodal laparoscopic-compliant fluorescence probe based 237

on the miniaturization of the fluorescence testbench will be avail- 238

able in 2013 (Fig 6 top) Preliminary results based on the 239

analysis of histopathologic slices will be presented in section 240

4 In order to overlay the probersquos fluorescence measurement on 241

the laparoscopic images we plan on detecting automatically the 242

position of the tip of the laparoscopic probe using a real-time 243

Fig 4 Experimental fluorescence design and results Left experimental optical device Right typical intensity image obtained

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 5

Perc

ent

of

maxim

a

Wavelength (nm)

PpIX emission

FITC excitation

FITC emission

PpIX excitation

350 400 450 500 550 600 650 700 750

Perc

ent

of

maxim

a

10

00

08

06

04

02

Wavelength (nm)

PpIX emission

Cy5 excitation

Cy5 emission

PpIX excitation

BA

350 400 450 500 550 600 650 700 750

10

00

08

06

04

02

Fig 5 Excitation and emission spectra of Protoporphyrin IX Fluorescein IsoThioCyanate and Cyanine 5 (Cy5) fluorochromes A Overlay of the excitation (blue

dashed line) and emission (orange solid line) spectra of Protoporphyrin IX with excitation (green dashed line) and emission (emission spectrum) of Fluorescein

IsoThioCyanate B Overlay of the excitation (blue dashed line) and emission (orange solid line) spectra of Protoporphyrin IX with excitation (red dashed line) and

emission (red solid line) of Cy5

Fig 6 The bimodal fluorescence laparoscopic probe and the image-based anal-

ysis approach to localize it in laparoscopic images Top the CAD drawing

of the bimodal fluorescence laparoscopic probe Bottom automatic detection

of a laparoscopic instrument which will be used to overlay the probersquos tissue

characterization measurements on the laparoscopic images

image analysis approach that we have developed [10] (Fig 6244

bottom)245

34 Preclinical and clinical evaluation246

Preliminary evaluation of the devices and the software devel-247

oped in the frame of this project has already been performed on248

laboratory testbenchs and during three cadaver experiments at249

the anatomy laboratory (section 4) However in order to acquire250

the qualitative and quantitative proofs necessary to perform the251

fist clinical evaluations of our innovative medical devices on 252

patients we have submitted two biomedical research protocols 253

bull the first one called ldquofusion of echographic (ultrasound) and 254

endoscopic imagesrdquo (FEE) is a pilot monocentric study on a 255

cohort of 15 patients and is currently running It aims at val- 256

idating our ultrasound approach and evaluating its expected 257

medical benefit during radical prostatectomies the surgeons 258

will insert surgical needles in the prostate and endorectal 259

ultrasound images and laparoscopic images will be recorded 260

The data will be processed offline in the laboratory to assess 261

quantitatively the feasibility of the registration of ultrasound 262

and laparoscopic images using passive markers This proto- 263

col already accepted by the ethical instances will be a pilot 264

monocentric study with 15 patients 265

bull the second protocol called COPROST will allow us to obtain 266

fresh prostate chips during transurethral resections in the 267

frame of a pilot monocentric prospective non-randomized 268

open and controlled clinical trial The protocol with an 269

inclusion period of 24 months is already defined and writ- 270

ten following the French Regulation on Biomedical Research 271

and biological tissue collection It is being submitted to the 272

ethical committee ldquocomiteacute de protection des personnesrdquo and 273

the ldquoagence nationale de seacutecuriteacute du meacutedicament et des pro- 274

duits de santeacuterdquo (ANSM) The prostate chips will be on one 275

hand characterized by anatomopathologists to determine their 276

nature (pathologic vs healthy prostatic vs non-prostatic) and 277

will allow us to validate our immunofluorescence protocol and 278

fine-tune our auto and immunofluorescence measurements 279

protocols on fresh tissue samples 280

4 Results 281

In this section we present our preclinical evaluation of the 282

different prototypes and software presented in the previous sec- 283

tion 284

41 Augmented laparoscopy thanks to an innovative video 285

device 286

In order to evaluate the potential benefits of the pro- 287

posed vision system we asked a surgeon to perform a simple 288

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

6 S Voros et al IRBM xxx (2013) xxxndashxxx

Fig 7 Ultrasound image of a prostate acquired with our innovative transurethral

ultrasound probe during a cadaver experiment The image was acquired with our

first prototype (64 piezoelectric elements)

surgical task once with the traditional endoscope alone and289

once with the proposed system alone (although in practice they290

can be combined to associate local and global views) The tasks291

consisted in localizing a suture needle and bringing it to a fixed292

target point The experiment was performed on porcine organs293

placed in a training box and repeated six times by the surgeon294

At each realization the needlersquos initial position was repositioned295

randomly at a distance equivalent to its initial position and the296

surgeon started randomly with the traditional endoscope or the297

proposed system In both cases a robotic endoscope holder was298

used to mobilize the endoscope and the vision system allowing299

us to record the displacements of the vision systems300

The full methodology and results are currently under sub-301

mission but to summarize the mean time required to perform302

the six experiments with the traditional endoscope was of 190 s303

compared to 245 s with the proposed system Moreover the sur-304

geon needed to give an average of 232 commands to the robotic305

scope holder to perform one stitch with the endoscope alone306

compared to 46 commands with the proposed system These307

preliminary results suggest that the proposed system could sig-308

nificantly reduce the laparoscopic surgery time and the cognitive309

load required for the control of the endoscope We now plan on310

performing cadaver experiments to evaluate the deployment of311

the vision system in conditions close to the clinical reality and312

to evaluate the system with several surgeons313

42 Augmented laparoscopy thanks to an innovative314

ultrasound system315

We performed two cadaver experiments which allowed us316

to determine the optimal characteristics for the realization of317

the intraurethral ultrasound probe During these experiments318

we evaluated the (difficult) insertion of the catheter through319

the urethra and the prostate visualization It must be noted that320

our clinicians partners stressed that the rigor mortis made this321

insertion harder These experiments allowed us to find the best322

rigidity for the catheter of the probe it must be flexible enough323

Fig 8 Demonstration of the fusion of ultrasound and laparoscopic images using

passive landmarks on chicken breasts Top-left the ultrasound image top-right

the laparoscopic image The passive landmarks are indicated with the green

arrows in the laparoscopic image They are selected manually in both modalities

to perform the registration Bottom the fusion of the two imaging modalities

to be introduced in the urethra but rigid enough to avoid the 324

distortions of the catheter 325

The first ultrasound probe (2D) was designed with 64 326

piezoelements It appeared that it was not enough On the one 327

hand the probe imaged a too small part of the prostate so that 328

it was difficult to identify what was represented on the ultra- 329

sound image (Fig 7) Furthermore it was impossible to know 330

in which direction the probe was oriented On the other hand 331

we demonstrated [9] that a longer probe increases the precision 332

of the localization of the active ultrasound transducers That is 333

why the second probe designed for the project (3D) has 128 334

piezoelements This second probe has graduations located on 335

its catheter visible on Fig 2 They allow the surgeon to know 336

the probersquos insertion depth in the urethra and a line along the 337

catheter allows appreciating the torsion applied to the catheter 338

The localization of the active transducers is currently under 339

submission It is based on a global positioning system (GPS) 340

method We made some experiments on a testbench making 341

localization in water to have ideal homogeneous conditions and 342

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 7

Fig 9 Preliminary experiments of the autofluorescence detection with our testbench The autofluorescence of chlorophyll which has optical properties very similar

to Protoporphyrin IX was observed on our testbench Left direct image of a leaf right fluorescence image of the leaf at 680 nm As expected chlorophyll seems

absent in the veins

Fig 10 Specific fluorescence signal of anti-prostate-specific membrane antigen marker A Nuclear counter-staining of prostatic tissue with Hoechst (blue DNA

marker) We observe the nuclei of prostate cells regularly organized in the glandular epithelium Prostatic glands are separated by stroma B Fluorescence emission

of anti-prostate-specific membrane antigen-Fluorescein IsoThioCyanate on cellular membranes of prostatic cells C Overlay of the A and B images This figure

demonstrates the specificity on fixed prostatic tissue of a specific anti-PSMA antibody labelled with Fluorescein IsoThioCyanate We can also detect the contribution

of non-specific autofluorescence of prostatic concretions (Fig 11) (times20 magnification)

localization on a chicken breast to have more realistic condi-343

tions In both cases we were able to localize the active transducer344

with a precision less than 625 m Moreover the computation345

time is a few hundred milliseconds for a 2D localization These346

results demonstrate that the localization is fast enough to be347

implemented for a prostatectomy For the moment only a 2D 348

localization has been made with the active landmarks and the 349

3D localization is in progress Then we will be able to per- 350

form a registration with laparoscopic images using ldquoactiverdquo 351

markers 352

Fig 11 Non-specific autofluorescence observed in the prostate fixed tissues This figure shows the benefits of a near infrared marker like Cyanine 5 (excited at

633 nm) compared to Fluorescein IsoThioCyanate (excited at 488 nm) A Auto fluorescence under laser excitation at 488 nm We observe that prostate concretions

(calcified material) respond to this excitation by a strong fluorescence signal B Auto fluorescence under laser excitation at 633 nm The use of Cyanine 5 reduces

widely autofluorescence of some structures such as prostate concretions (times20 magnification)

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

8 S Voros et al IRBM xxx (2013) xxxndashxxx

Concerning the registration between laparoscopic and ultra-353

sound images we performed several testbench evaluations of354

the registration algorithm on chicken breasts The result of such355

a registration obtained by manually pairing passive markers356

visible on both the laparoscopic and ultrasound imaging modal-357

ities are presented on Fig 8 In this example we obtained a358

registration root mean square error (RMS) of 038 mm359

43 Augmented laparoscopy thanks to fluorescence360

imaging361

First experiments were performed on leaves to demonstrate362

the feasibility of the detection of chlorophyll autofluorescence363

which has optical properties very similar to PpIX (absorption at364

405 nm and emission at 680 nm) as described on Fig 9 Then365

experiments were conducted on histology slices of the human366

prostate with as we anticipated inconclusive results because367

of the expected low signal-to-noise ratio caused by the insuffi-368

cient thickness of a histology slice (3 m thickness) compared to369

the optimal use of the device (300 m thickness) These results370

motivated our biomedical research COPROST on macroscopic371

fresh prostate samples372

Concerning the immunofluorescence protocol we were able373

to demonstrate the specificity of the anti-PSMA antibody on374

fixed tissue samples labelled with FITC as illustrated by Fig 10375

We observed higher autofluorescence problems when excit-376

ing the sample at the FITC specific wavelength compared to377

an excitation at the Cyanine 5 specific wavelength (Fig 11)378

These results motivated the inclusion of both fluorophores in our379

biomedical research COPROST on macroscopic fresh prostate380

samples FITC because it was already used to demonstrate speci-381

ficity of anti-PSMA antibody and Cyanine 5 because its near382

infra-red characteristics make it a better candidate on fresh mate-383

rial (higher penetration in tissue) and will improve signalnoise384

ratio as previously demonstrated385

To our knowledge most of the optical biopsy approaches as386

previously defined have been validated either on histologic slices387

or on ratmice prostate models However these approaches have388

limitations prostatic tissue is impaired by the chemical treat-389

ment required for histologic preparation and animal models do390

not directly mimic all aspects of human prostate cancer [11] In391

short it has not been proved yet that a successful detection with392

the investigated devices on histopathology slices or on animal393

models is sufficient to guarantee a successful detection on fresh394

human tissues The experiments that will be performed in the395

frame of the COPROST protocol in 2013 will be determinant to396

confirm the feasibility of our approach on fresh tissue samples397

5 DiscussionConclusion398

The DEPORRA project allowed for the development of inno-399

vative devices and navigation prototypes in the objective of400

allowing the surgeon to see ldquobeyond the visiblerdquo during a radi-401

cal prostatectomy First evaluations of the devices and methods402

have been performed preclinically and are very encouraging for403

the development of innovative approaches to assist the surgeon

during such a complex surgery Our preliminary results have also 404

shown the limits of preclinical validation and have convinced us 405

of the necessity to launch biomedical researches that will allow 406

us to validate further the devices This process required a con- 407

sequent amount of effort and time but is mandatory for the 408

fine-tuning of our tissue characterization tools and to obtain the 409

first clinical proofs of the relevance of the developed medical 410

devices which are mandatory to perform clinical evaluations of 411

the complete navigation systems 412

We now need to push further the integration of the devices 413

into such navigation systems in order to exploit at best the infor- 414

mation provided by each modality and to determine the optimal 415

approach for displaying the relevant information to the surgeon 416

in a clinical environment This will imply a conception and 417

development effort a definition of qualitative and quantitative 418

parameters for the first assessment of the delivered medical ben- 419

efit of our developments risks analysis and biomedical research 420

preparations in order to meet the ethical instances requirements 421

Acknowledgments 422

This work has been supported by French National Research 423

Agency (ANR) through TecSan program (project DEPORRA 424

no ANR-09-TECS-006) 425

References 426

[1] Heidenreich A Bolla M Joniau S Mason MD Matveev V Mottet N et al 427

Guidelines on prostate cancer Eur Assoc Urol 2011 [www uroweb org] 428

[2] Pfitzenmaier J Pahernik S Tremmel T Haferkamp A Buse S Hohen- 429

fellner M Positive surgical margins after radical prostatectomy do 430

they have an impact on biochemical or clinical progression BJU Int 431

2008102(10)1413ndash8 432

[3] Soler L Nicolau S Schmid J Koehl C Marescaux J et al Virtual reality 433

and augmented reality in digestive surgery In Third IEEE and ACM Inter- 434

national Symposium on Mixed and Augmented Reality (ISMAR) 2004 435

p 278ndash9 436

[4] S Voros B Tamadazte P Cinquin C Fouard Systegraveme drsquoimagerie multi- 437

vision pour chirurgie laparoscopique demande de deacutepocirct de brevet FR 438

1259489 102012 439

[5] Ukimura O Ahlering TE Gill IS Transrectal ultrasound-guided 440

energy-free nerve-sparing laparoscopic radical prostatectomy J Endourol 441

200822(9)1993ndash5 442

[6] Long JA Lee BH Guillotreau J Autorino R Laydner H Yakoubi R et al 443

Real-time robotic transrectal ultrasound navigation during robotic radical 444

prostatectomy initial clinical experience Urology 201280(3)608ndash13 445

[7] Holmes III DR Davis BJ Bruce CJ Robb RA 3D visualization analysis 446

and treatment of the prostate using trans-urethral ultrasound Comput Med 447

Imaging Graph 200327339ndash49 [2003] 448

[8] Dementhon D Model-based object pose in 25 lines of code Int J Comput 449

Vis 199515(1ndash2)123ndash41 450

[9] Custillon G Voros S Cinquin P Nguyen-Dinh A Moreau-Gaudry A 451

Bidimensional localization of active ultrasound transducers for use in 452

laparoscopic prostate surgery IEEE Trans Med Imaging 2012 Q3 453

[10] Wolf R Duchateau J Cinquin P Voros S 3D tracking of laparoscopic 454

instruments using statistical and geometric modeling Med Image Com- 455

put Comput Assist Interv 20116891203ndash10 [Lecture notes in computer 456

science] 457

[11] Valkenburg KC Williams BO ldquoMouse models of prostate cancerrdquo Prostate 458

Cancer 2011201122 httpdxdoiorg1011552011895238 [Article ID 459

895238] 460

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

2 S Voros et al IRBM xxx (2013) xxxndashxxx

is superior in terms of oncological and functional results and49

cost-effectiveness Prospective trials are urgently neededrdquo an50

argument also shared in two recent Health Technology Assess-51

ments on robot assisted surgery (Belgian HTA 2009 Irish HTA52

2012)53

In the current surgical practice the surgeon must determine54

the best compromise between a complete removal of the prostate55

gland and morbidity The ideal removal consists in removing56

completely the prostate within its capsule a non complete57

removal may lead to positive surgical margins which are associ-58

ated to a higher risk of biochemical recurrence [2] (15ndash20 of59

the patients) Lesions of the urethral sphincter situated under the60

prostate can lead to incontinence (7 of the patients) Lesions61

of the neurovascular bundles (immediately in contact with the62

posterior side of the prostate) may lead to erectile problems of63

varying intensity The surgeon can try to spare more or less these64

organs depending on the preoperative staging completed by65

magnetic resonance imaging (MRI) The quality of the prostate66

removal (absence of positive surgical margins) is determined67

postoperatively thanks to a histopathology exam at the anato-68

mopathology laboratory Thus the surgeon cannot adapt his69

surgical strategy peroperatively70

2 Objectives71

In this controversial context regarding robotically assisted72

prostatectomy we propose a new robotic navigation concept73

guided by some of the principal complications of a radical pros-74

tatectomy (cancer relapse incontinence and impotence) based75

on peroperative multimodal imaging Surgical navigation sys-76

tems allow for the combination of several imaging modalities in77

a common environment thus providing more useful information78

to the surgeon during a surgery than a single imaging modality79

In most of the existing systems information from preoperative80

imaging modalities is combined with the peroperative imaging81

modality However in the case of laparoscopic surgery (soft tis-82

sue surgery) the registration of preoperative images with the83

laparoscopic images is very challenging because the organs can84

move and deform To our knowledge there are only a few navi-85

gation systems for laparoscopic navigation [3] none of which86

are used clinically87

Our proposed navigation concept is based on three coupled88

components that will ultimately ldquoaugmentrdquo the laparoscopic89

images and allow the surgeon to see beyond the visible and90

adapt his surgical strategy peroperatively91

bull an augmented laparoscope allowing the surgeon to navigate92

more harmoniously inside the abdominal cavity as he would93

do in open surgery and potentially to provide three dimen-94

sional information95

bull peroperative visualization technologies allowing the surgeon96

to visualize the important anatomical structures for radical97

prostatectomy98

visualization of the prostate and surrounding organs thanks99

to an innovative transurethral ultrasound probe100

identification and visualization of the tumor and prostatic 101

cells thanks to bimodal fluorescence probe 102

bull integration of the multimodal information to navigation 103

systems thanks to the registration of the different imaging 104

modalities when required 105

3 Material amp methods 106

31 Augmented laparoscopy thanks to an innovative video 107

device 108

Laparoscopic surgery in general can be challenging for a sur- 109

geon for several reasons among which the limited field of view 110

of the endoscope (ldquokeyholerdquo surgery) In consequence whether 111

it is displaced by a human assistant or a robotic endoscope holder 112

the endoscope is mobilized a lot since it is used to see precisely 113

the operating field but also to monitor the instruments introduc- 114

tion and displacement inside the abdominal cavity Movements 115

of the endoscope increase the risk of staining of the lens which 116

leads to a removal of the endoscope from the patient to clean the 117

lens increasing the surgery duration and discomfort 118

To provide a solution to this difficulty we have developed 119

an innovative vision system that can be used in combination 120

with a traditional endoscope thanks to an encapsulation in a 121

traditional trocar Our system is composed of two miniature 122

cameras that are positioned like a pair of glasses around the 123

endoscope (Fig 1) and provide a panoramic view of the abdom- 124

inal cavity These cameras are similar to those present on cell 125

phones and cost only a few US$ in large scale diffusion The 126

system has been designed for an easy insertion deployment and 127

removal of the cameras and has been patented [4] Since it is 128

positioned around the endoscope it provides roughly the same 129

view direction as the endoscope and a registration between the 130

laparoscopic image and the vision system images is unneces- 131

sary The cameras of the innovative vision system are mounted 132

in stereoscopic conditions which could allow for a local 3D 133

reconstruction of the scene Our preclinical evaluation of the 134

potential expected medical service of this device is presented in 135

section 4 136

32 Augmented laparoscopy thanks to an innovative 137

ultrasound system 138

Previous work [5] has already shown that the use of tran- 139

srectal ultrasound during laparoscopic radical prostatectomies 140

could assist the surgeons in the visualization of specific prostate 141

contour anatomy and of the neurovascular bundles and in 142

the bladder neck dissection According to the authors the use 143

of transrectal ultrasound in laparoscopic prostatectomy com- 144

pensated for the lack of tactile feedback compared to open 145

surgery achieved significant decrease in the incidence of posi- 146

tive surgical margins and achieved quicker and superior potency 147

recovery In this work the ultrasound images were not fused 148

with the laparoscopic images and the surgeon had to perform 149

a challenging mental registration of the two imaging modali- 150

ties Long et al [6] also showed the potential of a guidance of 151

a transrectal ultrasound probe with a robotic endoscope holder 152

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 3

Fig 1 The proposed global vision system a) the mini-cameras used for the vision system b) CAD drawing of the augmented endoscope concept the mini-cameras

are inserted into a modified trocar When the endoscope is inserted the vision system is deployed like glasses around the endoscope c) and d) rapid prototyping

realization

(Endocontrolrsquos ViKY system) Holmes et al [7] have developed153

a 2D transurethral ultrasound probe for brachytherapy that is154

manually rotated around its axis to provide 3D imaging They155

affirm that the endourethral approach permits a better resolu-156

tion than the transrectal approach for the visualization of the157

prostate The transurethral approach also allows avoiding some158

of the intrinsic limitations of transrectal ultrasound imaging (a159

layer of air forms between the prostate and the rectum during160

the prostate dissection making the ultrasound visualization of161

the prostate impossible after this surgical phase)162

In light of these previous works we have developed two163

transurethral ultrasound probe prototypes (one for 2D imaging164

and a motorized version where the probe rotates around its axis165

to provide 3D images) (Fig 2) The 2D probe is composed of166

64 piezoelectric elements with a frequency of 10 MHz and a167

3 mm diameter semi-rigid catheter for an easy introduction The168

3D probe has the same central frequency but it is composed of169

128 piezoelements and has a 6 mm diameter catheter They are170

Fig 2 The innovative intraurethral ultrasound probe Top the 3D transurethral

probe able to rotate around its axis Bottom the 2D transurethral probe

connected to an Ultrasonix ultrasound machine allowing us to 171

control all the piezoelectric elements 172

In parallel we have also developed an endoscopicultrasound 173

fusion demonstrator The registration is based on the POSIT 174

algorithm [8] and is based on the manual localization in both 175

imaging modalities of artificial landmarks In the frame of this 176

project two complementary approaches have been investigated 177

passive landmarks (laparoscopic needles planted in the prostate) 178

and innovative active ultrasound landmarks that emit an ultra- 179

sound signal that can be detected by the transurethral probe 180

allowing for their precise localization in the ultrasound referen- 181

tial [9] (Fig 3) 182

Our preliminary results for the registration of ultrasound and 183

laparoscopic images using passive landmarks and for the precise 184

3D localization of the ultrasound ldquoactiverdquo markers are described 185

in section 4 186

33 Augmented laparoscopy thanks to fluorescence 187

imaging 188

During a radical prostatectomy the ability to visualize bio- 189

logical characteristics of tissue (prostatic vs non-prostatic on 190

the one hand normal vs malignant on the other hand) could help 191

the surgeon to respectively determine precisely the location of 192

the prostate capsule and assess the extent of the cancer and thus 193

allow him or her to adapt his surgical strategy peroperatively 194

Based on these observations we have developed a bimodal 195

fluorescence fibered probe for the peroperative characterization 196

of tissue for radical prostatectomy 197

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

4 S Voros et al IRBM xxx (2013) xxxndashxxx

Fig 3 The proposed markers for the registration of ultrasound and endoscopic

images Top ldquoactiverdquo landmark (laparoscopic ultrasound emitter) Bottom ldquopas-

siverdquo landmark (laparoscopic needle) used for the registration of ultrasound and

laparoscopic images

bull the normalmalignant characterization of the tissues is based198

on the detection of the autofluorescence of the Protopor-199

phyrin IX (PpIX) protein which accumulates in malignant200

cells Indeed the elimination cycle of Protorphyrin is per-201

turbed in case of malignant cells It causes thus an increase202

of Protoporphyrin concentration A first prototype (already203

available see Fig 4 left) comes in the form of a testbench204

equipped with a laser source emitting at 405 nm (excitation 205

wavelength for the PpIX protein) and with optical fibers and 206

a spectrometer allowing the collection of a fluorescence spec- 207

trum of 3648 points in the range 345ndash1040 mm (the testbench 208

can image a 25 times 25 mm2 area with a spatial resolution of 209

100 m and a penetration depth of roughly 300 m) Using 210

this spectrum an intensity image can be created (Fig 4 211

right) The analysis of prostate tissue samples gathered in 212

the frame of a biomedical protocol (section 4) will allow us 213

to determine the optimal parameters for the acquisition of 214

the fluorescence spectrum and build discrimination criteria 215

from the measurements between malignanthealthy tissue and 216

prostaticnon-prostatic tissue 217

bull the prostaticnon-prostatic characterization of the tissues is 218

based on the detection of the prostate-specific membrane anti- 219

gen (PSMA) which is specific of the prostatic membrane 220

This detection is made possible by the immunofluorescence 221

tagging of the PSMA using a specific anti-PSMA antibody 222

labelled with a fluorescent tracer When excited by light at 223

a characteristic absorption wavelength the fluorescent tracer 224

emits light at a characteristic emission wavelength Two flu- 225

orescent tracers are investigated in the DEPORRA project 226

Fluorescein IsoThioCyanate (FITC) and Cyanine 5 Both 227

FITC and Cyanine 5 have a characteristic emission wave- 228

length different from the autofluorescence wavelength of the 229

Protoporpphyrin IX protein allowing for the characterization 230

of the tissue type (prostatic vs non-prostatic) and the tissue 231

status (healthy vs malignant) with the same testbench (Fig 5) 232

The testbench presented on Fig 4 has also been equipped 233

with two laser sources emitting at 488 nm and 642 nm exci- 234

tation wavelengths respectively for FITC and Cyanine 5 for 235

recognizing prostate tissue from those environing 236

A bimodal laparoscopic-compliant fluorescence probe based 237

on the miniaturization of the fluorescence testbench will be avail- 238

able in 2013 (Fig 6 top) Preliminary results based on the 239

analysis of histopathologic slices will be presented in section 240

4 In order to overlay the probersquos fluorescence measurement on 241

the laparoscopic images we plan on detecting automatically the 242

position of the tip of the laparoscopic probe using a real-time 243

Fig 4 Experimental fluorescence design and results Left experimental optical device Right typical intensity image obtained

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 5

Perc

ent

of

maxim

a

Wavelength (nm)

PpIX emission

FITC excitation

FITC emission

PpIX excitation

350 400 450 500 550 600 650 700 750

Perc

ent

of

maxim

a

10

00

08

06

04

02

Wavelength (nm)

PpIX emission

Cy5 excitation

Cy5 emission

PpIX excitation

BA

350 400 450 500 550 600 650 700 750

10

00

08

06

04

02

Fig 5 Excitation and emission spectra of Protoporphyrin IX Fluorescein IsoThioCyanate and Cyanine 5 (Cy5) fluorochromes A Overlay of the excitation (blue

dashed line) and emission (orange solid line) spectra of Protoporphyrin IX with excitation (green dashed line) and emission (emission spectrum) of Fluorescein

IsoThioCyanate B Overlay of the excitation (blue dashed line) and emission (orange solid line) spectra of Protoporphyrin IX with excitation (red dashed line) and

emission (red solid line) of Cy5

Fig 6 The bimodal fluorescence laparoscopic probe and the image-based anal-

ysis approach to localize it in laparoscopic images Top the CAD drawing

of the bimodal fluorescence laparoscopic probe Bottom automatic detection

of a laparoscopic instrument which will be used to overlay the probersquos tissue

characterization measurements on the laparoscopic images

image analysis approach that we have developed [10] (Fig 6244

bottom)245

34 Preclinical and clinical evaluation246

Preliminary evaluation of the devices and the software devel-247

oped in the frame of this project has already been performed on248

laboratory testbenchs and during three cadaver experiments at249

the anatomy laboratory (section 4) However in order to acquire250

the qualitative and quantitative proofs necessary to perform the251

fist clinical evaluations of our innovative medical devices on 252

patients we have submitted two biomedical research protocols 253

bull the first one called ldquofusion of echographic (ultrasound) and 254

endoscopic imagesrdquo (FEE) is a pilot monocentric study on a 255

cohort of 15 patients and is currently running It aims at val- 256

idating our ultrasound approach and evaluating its expected 257

medical benefit during radical prostatectomies the surgeons 258

will insert surgical needles in the prostate and endorectal 259

ultrasound images and laparoscopic images will be recorded 260

The data will be processed offline in the laboratory to assess 261

quantitatively the feasibility of the registration of ultrasound 262

and laparoscopic images using passive markers This proto- 263

col already accepted by the ethical instances will be a pilot 264

monocentric study with 15 patients 265

bull the second protocol called COPROST will allow us to obtain 266

fresh prostate chips during transurethral resections in the 267

frame of a pilot monocentric prospective non-randomized 268

open and controlled clinical trial The protocol with an 269

inclusion period of 24 months is already defined and writ- 270

ten following the French Regulation on Biomedical Research 271

and biological tissue collection It is being submitted to the 272

ethical committee ldquocomiteacute de protection des personnesrdquo and 273

the ldquoagence nationale de seacutecuriteacute du meacutedicament et des pro- 274

duits de santeacuterdquo (ANSM) The prostate chips will be on one 275

hand characterized by anatomopathologists to determine their 276

nature (pathologic vs healthy prostatic vs non-prostatic) and 277

will allow us to validate our immunofluorescence protocol and 278

fine-tune our auto and immunofluorescence measurements 279

protocols on fresh tissue samples 280

4 Results 281

In this section we present our preclinical evaluation of the 282

different prototypes and software presented in the previous sec- 283

tion 284

41 Augmented laparoscopy thanks to an innovative video 285

device 286

In order to evaluate the potential benefits of the pro- 287

posed vision system we asked a surgeon to perform a simple 288

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

6 S Voros et al IRBM xxx (2013) xxxndashxxx

Fig 7 Ultrasound image of a prostate acquired with our innovative transurethral

ultrasound probe during a cadaver experiment The image was acquired with our

first prototype (64 piezoelectric elements)

surgical task once with the traditional endoscope alone and289

once with the proposed system alone (although in practice they290

can be combined to associate local and global views) The tasks291

consisted in localizing a suture needle and bringing it to a fixed292

target point The experiment was performed on porcine organs293

placed in a training box and repeated six times by the surgeon294

At each realization the needlersquos initial position was repositioned295

randomly at a distance equivalent to its initial position and the296

surgeon started randomly with the traditional endoscope or the297

proposed system In both cases a robotic endoscope holder was298

used to mobilize the endoscope and the vision system allowing299

us to record the displacements of the vision systems300

The full methodology and results are currently under sub-301

mission but to summarize the mean time required to perform302

the six experiments with the traditional endoscope was of 190 s303

compared to 245 s with the proposed system Moreover the sur-304

geon needed to give an average of 232 commands to the robotic305

scope holder to perform one stitch with the endoscope alone306

compared to 46 commands with the proposed system These307

preliminary results suggest that the proposed system could sig-308

nificantly reduce the laparoscopic surgery time and the cognitive309

load required for the control of the endoscope We now plan on310

performing cadaver experiments to evaluate the deployment of311

the vision system in conditions close to the clinical reality and312

to evaluate the system with several surgeons313

42 Augmented laparoscopy thanks to an innovative314

ultrasound system315

We performed two cadaver experiments which allowed us316

to determine the optimal characteristics for the realization of317

the intraurethral ultrasound probe During these experiments318

we evaluated the (difficult) insertion of the catheter through319

the urethra and the prostate visualization It must be noted that320

our clinicians partners stressed that the rigor mortis made this321

insertion harder These experiments allowed us to find the best322

rigidity for the catheter of the probe it must be flexible enough323

Fig 8 Demonstration of the fusion of ultrasound and laparoscopic images using

passive landmarks on chicken breasts Top-left the ultrasound image top-right

the laparoscopic image The passive landmarks are indicated with the green

arrows in the laparoscopic image They are selected manually in both modalities

to perform the registration Bottom the fusion of the two imaging modalities

to be introduced in the urethra but rigid enough to avoid the 324

distortions of the catheter 325

The first ultrasound probe (2D) was designed with 64 326

piezoelements It appeared that it was not enough On the one 327

hand the probe imaged a too small part of the prostate so that 328

it was difficult to identify what was represented on the ultra- 329

sound image (Fig 7) Furthermore it was impossible to know 330

in which direction the probe was oriented On the other hand 331

we demonstrated [9] that a longer probe increases the precision 332

of the localization of the active ultrasound transducers That is 333

why the second probe designed for the project (3D) has 128 334

piezoelements This second probe has graduations located on 335

its catheter visible on Fig 2 They allow the surgeon to know 336

the probersquos insertion depth in the urethra and a line along the 337

catheter allows appreciating the torsion applied to the catheter 338

The localization of the active transducers is currently under 339

submission It is based on a global positioning system (GPS) 340

method We made some experiments on a testbench making 341

localization in water to have ideal homogeneous conditions and 342

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 7

Fig 9 Preliminary experiments of the autofluorescence detection with our testbench The autofluorescence of chlorophyll which has optical properties very similar

to Protoporphyrin IX was observed on our testbench Left direct image of a leaf right fluorescence image of the leaf at 680 nm As expected chlorophyll seems

absent in the veins

Fig 10 Specific fluorescence signal of anti-prostate-specific membrane antigen marker A Nuclear counter-staining of prostatic tissue with Hoechst (blue DNA

marker) We observe the nuclei of prostate cells regularly organized in the glandular epithelium Prostatic glands are separated by stroma B Fluorescence emission

of anti-prostate-specific membrane antigen-Fluorescein IsoThioCyanate on cellular membranes of prostatic cells C Overlay of the A and B images This figure

demonstrates the specificity on fixed prostatic tissue of a specific anti-PSMA antibody labelled with Fluorescein IsoThioCyanate We can also detect the contribution

of non-specific autofluorescence of prostatic concretions (Fig 11) (times20 magnification)

localization on a chicken breast to have more realistic condi-343

tions In both cases we were able to localize the active transducer344

with a precision less than 625 m Moreover the computation345

time is a few hundred milliseconds for a 2D localization These346

results demonstrate that the localization is fast enough to be347

implemented for a prostatectomy For the moment only a 2D 348

localization has been made with the active landmarks and the 349

3D localization is in progress Then we will be able to per- 350

form a registration with laparoscopic images using ldquoactiverdquo 351

markers 352

Fig 11 Non-specific autofluorescence observed in the prostate fixed tissues This figure shows the benefits of a near infrared marker like Cyanine 5 (excited at

633 nm) compared to Fluorescein IsoThioCyanate (excited at 488 nm) A Auto fluorescence under laser excitation at 488 nm We observe that prostate concretions

(calcified material) respond to this excitation by a strong fluorescence signal B Auto fluorescence under laser excitation at 633 nm The use of Cyanine 5 reduces

widely autofluorescence of some structures such as prostate concretions (times20 magnification)

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

8 S Voros et al IRBM xxx (2013) xxxndashxxx

Concerning the registration between laparoscopic and ultra-353

sound images we performed several testbench evaluations of354

the registration algorithm on chicken breasts The result of such355

a registration obtained by manually pairing passive markers356

visible on both the laparoscopic and ultrasound imaging modal-357

ities are presented on Fig 8 In this example we obtained a358

registration root mean square error (RMS) of 038 mm359

43 Augmented laparoscopy thanks to fluorescence360

imaging361

First experiments were performed on leaves to demonstrate362

the feasibility of the detection of chlorophyll autofluorescence363

which has optical properties very similar to PpIX (absorption at364

405 nm and emission at 680 nm) as described on Fig 9 Then365

experiments were conducted on histology slices of the human366

prostate with as we anticipated inconclusive results because367

of the expected low signal-to-noise ratio caused by the insuffi-368

cient thickness of a histology slice (3 m thickness) compared to369

the optimal use of the device (300 m thickness) These results370

motivated our biomedical research COPROST on macroscopic371

fresh prostate samples372

Concerning the immunofluorescence protocol we were able373

to demonstrate the specificity of the anti-PSMA antibody on374

fixed tissue samples labelled with FITC as illustrated by Fig 10375

We observed higher autofluorescence problems when excit-376

ing the sample at the FITC specific wavelength compared to377

an excitation at the Cyanine 5 specific wavelength (Fig 11)378

These results motivated the inclusion of both fluorophores in our379

biomedical research COPROST on macroscopic fresh prostate380

samples FITC because it was already used to demonstrate speci-381

ficity of anti-PSMA antibody and Cyanine 5 because its near382

infra-red characteristics make it a better candidate on fresh mate-383

rial (higher penetration in tissue) and will improve signalnoise384

ratio as previously demonstrated385

To our knowledge most of the optical biopsy approaches as386

previously defined have been validated either on histologic slices387

or on ratmice prostate models However these approaches have388

limitations prostatic tissue is impaired by the chemical treat-389

ment required for histologic preparation and animal models do390

not directly mimic all aspects of human prostate cancer [11] In391

short it has not been proved yet that a successful detection with392

the investigated devices on histopathology slices or on animal393

models is sufficient to guarantee a successful detection on fresh394

human tissues The experiments that will be performed in the395

frame of the COPROST protocol in 2013 will be determinant to396

confirm the feasibility of our approach on fresh tissue samples397

5 DiscussionConclusion398

The DEPORRA project allowed for the development of inno-399

vative devices and navigation prototypes in the objective of400

allowing the surgeon to see ldquobeyond the visiblerdquo during a radi-401

cal prostatectomy First evaluations of the devices and methods402

have been performed preclinically and are very encouraging for403

the development of innovative approaches to assist the surgeon

during such a complex surgery Our preliminary results have also 404

shown the limits of preclinical validation and have convinced us 405

of the necessity to launch biomedical researches that will allow 406

us to validate further the devices This process required a con- 407

sequent amount of effort and time but is mandatory for the 408

fine-tuning of our tissue characterization tools and to obtain the 409

first clinical proofs of the relevance of the developed medical 410

devices which are mandatory to perform clinical evaluations of 411

the complete navigation systems 412

We now need to push further the integration of the devices 413

into such navigation systems in order to exploit at best the infor- 414

mation provided by each modality and to determine the optimal 415

approach for displaying the relevant information to the surgeon 416

in a clinical environment This will imply a conception and 417

development effort a definition of qualitative and quantitative 418

parameters for the first assessment of the delivered medical ben- 419

efit of our developments risks analysis and biomedical research 420

preparations in order to meet the ethical instances requirements 421

Acknowledgments 422

This work has been supported by French National Research 423

Agency (ANR) through TecSan program (project DEPORRA 424

no ANR-09-TECS-006) 425

References 426

[1] Heidenreich A Bolla M Joniau S Mason MD Matveev V Mottet N et al 427

Guidelines on prostate cancer Eur Assoc Urol 2011 [www uroweb org] 428

[2] Pfitzenmaier J Pahernik S Tremmel T Haferkamp A Buse S Hohen- 429

fellner M Positive surgical margins after radical prostatectomy do 430

they have an impact on biochemical or clinical progression BJU Int 431

2008102(10)1413ndash8 432

[3] Soler L Nicolau S Schmid J Koehl C Marescaux J et al Virtual reality 433

and augmented reality in digestive surgery In Third IEEE and ACM Inter- 434

national Symposium on Mixed and Augmented Reality (ISMAR) 2004 435

p 278ndash9 436

[4] S Voros B Tamadazte P Cinquin C Fouard Systegraveme drsquoimagerie multi- 437

vision pour chirurgie laparoscopique demande de deacutepocirct de brevet FR 438

1259489 102012 439

[5] Ukimura O Ahlering TE Gill IS Transrectal ultrasound-guided 440

energy-free nerve-sparing laparoscopic radical prostatectomy J Endourol 441

200822(9)1993ndash5 442

[6] Long JA Lee BH Guillotreau J Autorino R Laydner H Yakoubi R et al 443

Real-time robotic transrectal ultrasound navigation during robotic radical 444

prostatectomy initial clinical experience Urology 201280(3)608ndash13 445

[7] Holmes III DR Davis BJ Bruce CJ Robb RA 3D visualization analysis 446

and treatment of the prostate using trans-urethral ultrasound Comput Med 447

Imaging Graph 200327339ndash49 [2003] 448

[8] Dementhon D Model-based object pose in 25 lines of code Int J Comput 449

Vis 199515(1ndash2)123ndash41 450

[9] Custillon G Voros S Cinquin P Nguyen-Dinh A Moreau-Gaudry A 451

Bidimensional localization of active ultrasound transducers for use in 452

laparoscopic prostate surgery IEEE Trans Med Imaging 2012 Q3 453

[10] Wolf R Duchateau J Cinquin P Voros S 3D tracking of laparoscopic 454

instruments using statistical and geometric modeling Med Image Com- 455

put Comput Assist Interv 20116891203ndash10 [Lecture notes in computer 456

science] 457

[11] Valkenburg KC Williams BO ldquoMouse models of prostate cancerrdquo Prostate 458

Cancer 2011201122 httpdxdoiorg1011552011895238 [Article ID 459

895238] 460

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 3

Fig 1 The proposed global vision system a) the mini-cameras used for the vision system b) CAD drawing of the augmented endoscope concept the mini-cameras

are inserted into a modified trocar When the endoscope is inserted the vision system is deployed like glasses around the endoscope c) and d) rapid prototyping

realization

(Endocontrolrsquos ViKY system) Holmes et al [7] have developed153

a 2D transurethral ultrasound probe for brachytherapy that is154

manually rotated around its axis to provide 3D imaging They155

affirm that the endourethral approach permits a better resolu-156

tion than the transrectal approach for the visualization of the157

prostate The transurethral approach also allows avoiding some158

of the intrinsic limitations of transrectal ultrasound imaging (a159

layer of air forms between the prostate and the rectum during160

the prostate dissection making the ultrasound visualization of161

the prostate impossible after this surgical phase)162

In light of these previous works we have developed two163

transurethral ultrasound probe prototypes (one for 2D imaging164

and a motorized version where the probe rotates around its axis165

to provide 3D images) (Fig 2) The 2D probe is composed of166

64 piezoelectric elements with a frequency of 10 MHz and a167

3 mm diameter semi-rigid catheter for an easy introduction The168

3D probe has the same central frequency but it is composed of169

128 piezoelements and has a 6 mm diameter catheter They are170

Fig 2 The innovative intraurethral ultrasound probe Top the 3D transurethral

probe able to rotate around its axis Bottom the 2D transurethral probe

connected to an Ultrasonix ultrasound machine allowing us to 171

control all the piezoelectric elements 172

In parallel we have also developed an endoscopicultrasound 173

fusion demonstrator The registration is based on the POSIT 174

algorithm [8] and is based on the manual localization in both 175

imaging modalities of artificial landmarks In the frame of this 176

project two complementary approaches have been investigated 177

passive landmarks (laparoscopic needles planted in the prostate) 178

and innovative active ultrasound landmarks that emit an ultra- 179

sound signal that can be detected by the transurethral probe 180

allowing for their precise localization in the ultrasound referen- 181

tial [9] (Fig 3) 182

Our preliminary results for the registration of ultrasound and 183

laparoscopic images using passive landmarks and for the precise 184

3D localization of the ultrasound ldquoactiverdquo markers are described 185

in section 4 186

33 Augmented laparoscopy thanks to fluorescence 187

imaging 188

During a radical prostatectomy the ability to visualize bio- 189

logical characteristics of tissue (prostatic vs non-prostatic on 190

the one hand normal vs malignant on the other hand) could help 191

the surgeon to respectively determine precisely the location of 192

the prostate capsule and assess the extent of the cancer and thus 193

allow him or her to adapt his surgical strategy peroperatively 194

Based on these observations we have developed a bimodal 195

fluorescence fibered probe for the peroperative characterization 196

of tissue for radical prostatectomy 197

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

4 S Voros et al IRBM xxx (2013) xxxndashxxx

Fig 3 The proposed markers for the registration of ultrasound and endoscopic

images Top ldquoactiverdquo landmark (laparoscopic ultrasound emitter) Bottom ldquopas-

siverdquo landmark (laparoscopic needle) used for the registration of ultrasound and

laparoscopic images

bull the normalmalignant characterization of the tissues is based198

on the detection of the autofluorescence of the Protopor-199

phyrin IX (PpIX) protein which accumulates in malignant200

cells Indeed the elimination cycle of Protorphyrin is per-201

turbed in case of malignant cells It causes thus an increase202

of Protoporphyrin concentration A first prototype (already203

available see Fig 4 left) comes in the form of a testbench204

equipped with a laser source emitting at 405 nm (excitation 205

wavelength for the PpIX protein) and with optical fibers and 206

a spectrometer allowing the collection of a fluorescence spec- 207

trum of 3648 points in the range 345ndash1040 mm (the testbench 208

can image a 25 times 25 mm2 area with a spatial resolution of 209

100 m and a penetration depth of roughly 300 m) Using 210

this spectrum an intensity image can be created (Fig 4 211

right) The analysis of prostate tissue samples gathered in 212

the frame of a biomedical protocol (section 4) will allow us 213

to determine the optimal parameters for the acquisition of 214

the fluorescence spectrum and build discrimination criteria 215

from the measurements between malignanthealthy tissue and 216

prostaticnon-prostatic tissue 217

bull the prostaticnon-prostatic characterization of the tissues is 218

based on the detection of the prostate-specific membrane anti- 219

gen (PSMA) which is specific of the prostatic membrane 220

This detection is made possible by the immunofluorescence 221

tagging of the PSMA using a specific anti-PSMA antibody 222

labelled with a fluorescent tracer When excited by light at 223

a characteristic absorption wavelength the fluorescent tracer 224

emits light at a characteristic emission wavelength Two flu- 225

orescent tracers are investigated in the DEPORRA project 226

Fluorescein IsoThioCyanate (FITC) and Cyanine 5 Both 227

FITC and Cyanine 5 have a characteristic emission wave- 228

length different from the autofluorescence wavelength of the 229

Protoporpphyrin IX protein allowing for the characterization 230

of the tissue type (prostatic vs non-prostatic) and the tissue 231

status (healthy vs malignant) with the same testbench (Fig 5) 232

The testbench presented on Fig 4 has also been equipped 233

with two laser sources emitting at 488 nm and 642 nm exci- 234

tation wavelengths respectively for FITC and Cyanine 5 for 235

recognizing prostate tissue from those environing 236

A bimodal laparoscopic-compliant fluorescence probe based 237

on the miniaturization of the fluorescence testbench will be avail- 238

able in 2013 (Fig 6 top) Preliminary results based on the 239

analysis of histopathologic slices will be presented in section 240

4 In order to overlay the probersquos fluorescence measurement on 241

the laparoscopic images we plan on detecting automatically the 242

position of the tip of the laparoscopic probe using a real-time 243

Fig 4 Experimental fluorescence design and results Left experimental optical device Right typical intensity image obtained

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 5

Perc

ent

of

maxim

a

Wavelength (nm)

PpIX emission

FITC excitation

FITC emission

PpIX excitation

350 400 450 500 550 600 650 700 750

Perc

ent

of

maxim

a

10

00

08

06

04

02

Wavelength (nm)

PpIX emission

Cy5 excitation

Cy5 emission

PpIX excitation

BA

350 400 450 500 550 600 650 700 750

10

00

08

06

04

02

Fig 5 Excitation and emission spectra of Protoporphyrin IX Fluorescein IsoThioCyanate and Cyanine 5 (Cy5) fluorochromes A Overlay of the excitation (blue

dashed line) and emission (orange solid line) spectra of Protoporphyrin IX with excitation (green dashed line) and emission (emission spectrum) of Fluorescein

IsoThioCyanate B Overlay of the excitation (blue dashed line) and emission (orange solid line) spectra of Protoporphyrin IX with excitation (red dashed line) and

emission (red solid line) of Cy5

Fig 6 The bimodal fluorescence laparoscopic probe and the image-based anal-

ysis approach to localize it in laparoscopic images Top the CAD drawing

of the bimodal fluorescence laparoscopic probe Bottom automatic detection

of a laparoscopic instrument which will be used to overlay the probersquos tissue

characterization measurements on the laparoscopic images

image analysis approach that we have developed [10] (Fig 6244

bottom)245

34 Preclinical and clinical evaluation246

Preliminary evaluation of the devices and the software devel-247

oped in the frame of this project has already been performed on248

laboratory testbenchs and during three cadaver experiments at249

the anatomy laboratory (section 4) However in order to acquire250

the qualitative and quantitative proofs necessary to perform the251

fist clinical evaluations of our innovative medical devices on 252

patients we have submitted two biomedical research protocols 253

bull the first one called ldquofusion of echographic (ultrasound) and 254

endoscopic imagesrdquo (FEE) is a pilot monocentric study on a 255

cohort of 15 patients and is currently running It aims at val- 256

idating our ultrasound approach and evaluating its expected 257

medical benefit during radical prostatectomies the surgeons 258

will insert surgical needles in the prostate and endorectal 259

ultrasound images and laparoscopic images will be recorded 260

The data will be processed offline in the laboratory to assess 261

quantitatively the feasibility of the registration of ultrasound 262

and laparoscopic images using passive markers This proto- 263

col already accepted by the ethical instances will be a pilot 264

monocentric study with 15 patients 265

bull the second protocol called COPROST will allow us to obtain 266

fresh prostate chips during transurethral resections in the 267

frame of a pilot monocentric prospective non-randomized 268

open and controlled clinical trial The protocol with an 269

inclusion period of 24 months is already defined and writ- 270

ten following the French Regulation on Biomedical Research 271

and biological tissue collection It is being submitted to the 272

ethical committee ldquocomiteacute de protection des personnesrdquo and 273

the ldquoagence nationale de seacutecuriteacute du meacutedicament et des pro- 274

duits de santeacuterdquo (ANSM) The prostate chips will be on one 275

hand characterized by anatomopathologists to determine their 276

nature (pathologic vs healthy prostatic vs non-prostatic) and 277

will allow us to validate our immunofluorescence protocol and 278

fine-tune our auto and immunofluorescence measurements 279

protocols on fresh tissue samples 280

4 Results 281

In this section we present our preclinical evaluation of the 282

different prototypes and software presented in the previous sec- 283

tion 284

41 Augmented laparoscopy thanks to an innovative video 285

device 286

In order to evaluate the potential benefits of the pro- 287

posed vision system we asked a surgeon to perform a simple 288

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

6 S Voros et al IRBM xxx (2013) xxxndashxxx

Fig 7 Ultrasound image of a prostate acquired with our innovative transurethral

ultrasound probe during a cadaver experiment The image was acquired with our

first prototype (64 piezoelectric elements)

surgical task once with the traditional endoscope alone and289

once with the proposed system alone (although in practice they290

can be combined to associate local and global views) The tasks291

consisted in localizing a suture needle and bringing it to a fixed292

target point The experiment was performed on porcine organs293

placed in a training box and repeated six times by the surgeon294

At each realization the needlersquos initial position was repositioned295

randomly at a distance equivalent to its initial position and the296

surgeon started randomly with the traditional endoscope or the297

proposed system In both cases a robotic endoscope holder was298

used to mobilize the endoscope and the vision system allowing299

us to record the displacements of the vision systems300

The full methodology and results are currently under sub-301

mission but to summarize the mean time required to perform302

the six experiments with the traditional endoscope was of 190 s303

compared to 245 s with the proposed system Moreover the sur-304

geon needed to give an average of 232 commands to the robotic305

scope holder to perform one stitch with the endoscope alone306

compared to 46 commands with the proposed system These307

preliminary results suggest that the proposed system could sig-308

nificantly reduce the laparoscopic surgery time and the cognitive309

load required for the control of the endoscope We now plan on310

performing cadaver experiments to evaluate the deployment of311

the vision system in conditions close to the clinical reality and312

to evaluate the system with several surgeons313

42 Augmented laparoscopy thanks to an innovative314

ultrasound system315

We performed two cadaver experiments which allowed us316

to determine the optimal characteristics for the realization of317

the intraurethral ultrasound probe During these experiments318

we evaluated the (difficult) insertion of the catheter through319

the urethra and the prostate visualization It must be noted that320

our clinicians partners stressed that the rigor mortis made this321

insertion harder These experiments allowed us to find the best322

rigidity for the catheter of the probe it must be flexible enough323

Fig 8 Demonstration of the fusion of ultrasound and laparoscopic images using

passive landmarks on chicken breasts Top-left the ultrasound image top-right

the laparoscopic image The passive landmarks are indicated with the green

arrows in the laparoscopic image They are selected manually in both modalities

to perform the registration Bottom the fusion of the two imaging modalities

to be introduced in the urethra but rigid enough to avoid the 324

distortions of the catheter 325

The first ultrasound probe (2D) was designed with 64 326

piezoelements It appeared that it was not enough On the one 327

hand the probe imaged a too small part of the prostate so that 328

it was difficult to identify what was represented on the ultra- 329

sound image (Fig 7) Furthermore it was impossible to know 330

in which direction the probe was oriented On the other hand 331

we demonstrated [9] that a longer probe increases the precision 332

of the localization of the active ultrasound transducers That is 333

why the second probe designed for the project (3D) has 128 334

piezoelements This second probe has graduations located on 335

its catheter visible on Fig 2 They allow the surgeon to know 336

the probersquos insertion depth in the urethra and a line along the 337

catheter allows appreciating the torsion applied to the catheter 338

The localization of the active transducers is currently under 339

submission It is based on a global positioning system (GPS) 340

method We made some experiments on a testbench making 341

localization in water to have ideal homogeneous conditions and 342

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 7

Fig 9 Preliminary experiments of the autofluorescence detection with our testbench The autofluorescence of chlorophyll which has optical properties very similar

to Protoporphyrin IX was observed on our testbench Left direct image of a leaf right fluorescence image of the leaf at 680 nm As expected chlorophyll seems

absent in the veins

Fig 10 Specific fluorescence signal of anti-prostate-specific membrane antigen marker A Nuclear counter-staining of prostatic tissue with Hoechst (blue DNA

marker) We observe the nuclei of prostate cells regularly organized in the glandular epithelium Prostatic glands are separated by stroma B Fluorescence emission

of anti-prostate-specific membrane antigen-Fluorescein IsoThioCyanate on cellular membranes of prostatic cells C Overlay of the A and B images This figure

demonstrates the specificity on fixed prostatic tissue of a specific anti-PSMA antibody labelled with Fluorescein IsoThioCyanate We can also detect the contribution

of non-specific autofluorescence of prostatic concretions (Fig 11) (times20 magnification)

localization on a chicken breast to have more realistic condi-343

tions In both cases we were able to localize the active transducer344

with a precision less than 625 m Moreover the computation345

time is a few hundred milliseconds for a 2D localization These346

results demonstrate that the localization is fast enough to be347

implemented for a prostatectomy For the moment only a 2D 348

localization has been made with the active landmarks and the 349

3D localization is in progress Then we will be able to per- 350

form a registration with laparoscopic images using ldquoactiverdquo 351

markers 352

Fig 11 Non-specific autofluorescence observed in the prostate fixed tissues This figure shows the benefits of a near infrared marker like Cyanine 5 (excited at

633 nm) compared to Fluorescein IsoThioCyanate (excited at 488 nm) A Auto fluorescence under laser excitation at 488 nm We observe that prostate concretions

(calcified material) respond to this excitation by a strong fluorescence signal B Auto fluorescence under laser excitation at 633 nm The use of Cyanine 5 reduces

widely autofluorescence of some structures such as prostate concretions (times20 magnification)

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

8 S Voros et al IRBM xxx (2013) xxxndashxxx

Concerning the registration between laparoscopic and ultra-353

sound images we performed several testbench evaluations of354

the registration algorithm on chicken breasts The result of such355

a registration obtained by manually pairing passive markers356

visible on both the laparoscopic and ultrasound imaging modal-357

ities are presented on Fig 8 In this example we obtained a358

registration root mean square error (RMS) of 038 mm359

43 Augmented laparoscopy thanks to fluorescence360

imaging361

First experiments were performed on leaves to demonstrate362

the feasibility of the detection of chlorophyll autofluorescence363

which has optical properties very similar to PpIX (absorption at364

405 nm and emission at 680 nm) as described on Fig 9 Then365

experiments were conducted on histology slices of the human366

prostate with as we anticipated inconclusive results because367

of the expected low signal-to-noise ratio caused by the insuffi-368

cient thickness of a histology slice (3 m thickness) compared to369

the optimal use of the device (300 m thickness) These results370

motivated our biomedical research COPROST on macroscopic371

fresh prostate samples372

Concerning the immunofluorescence protocol we were able373

to demonstrate the specificity of the anti-PSMA antibody on374

fixed tissue samples labelled with FITC as illustrated by Fig 10375

We observed higher autofluorescence problems when excit-376

ing the sample at the FITC specific wavelength compared to377

an excitation at the Cyanine 5 specific wavelength (Fig 11)378

These results motivated the inclusion of both fluorophores in our379

biomedical research COPROST on macroscopic fresh prostate380

samples FITC because it was already used to demonstrate speci-381

ficity of anti-PSMA antibody and Cyanine 5 because its near382

infra-red characteristics make it a better candidate on fresh mate-383

rial (higher penetration in tissue) and will improve signalnoise384

ratio as previously demonstrated385

To our knowledge most of the optical biopsy approaches as386

previously defined have been validated either on histologic slices387

or on ratmice prostate models However these approaches have388

limitations prostatic tissue is impaired by the chemical treat-389

ment required for histologic preparation and animal models do390

not directly mimic all aspects of human prostate cancer [11] In391

short it has not been proved yet that a successful detection with392

the investigated devices on histopathology slices or on animal393

models is sufficient to guarantee a successful detection on fresh394

human tissues The experiments that will be performed in the395

frame of the COPROST protocol in 2013 will be determinant to396

confirm the feasibility of our approach on fresh tissue samples397

5 DiscussionConclusion398

The DEPORRA project allowed for the development of inno-399

vative devices and navigation prototypes in the objective of400

allowing the surgeon to see ldquobeyond the visiblerdquo during a radi-401

cal prostatectomy First evaluations of the devices and methods402

have been performed preclinically and are very encouraging for403

the development of innovative approaches to assist the surgeon

during such a complex surgery Our preliminary results have also 404

shown the limits of preclinical validation and have convinced us 405

of the necessity to launch biomedical researches that will allow 406

us to validate further the devices This process required a con- 407

sequent amount of effort and time but is mandatory for the 408

fine-tuning of our tissue characterization tools and to obtain the 409

first clinical proofs of the relevance of the developed medical 410

devices which are mandatory to perform clinical evaluations of 411

the complete navigation systems 412

We now need to push further the integration of the devices 413

into such navigation systems in order to exploit at best the infor- 414

mation provided by each modality and to determine the optimal 415

approach for displaying the relevant information to the surgeon 416

in a clinical environment This will imply a conception and 417

development effort a definition of qualitative and quantitative 418

parameters for the first assessment of the delivered medical ben- 419

efit of our developments risks analysis and biomedical research 420

preparations in order to meet the ethical instances requirements 421

Acknowledgments 422

This work has been supported by French National Research 423

Agency (ANR) through TecSan program (project DEPORRA 424

no ANR-09-TECS-006) 425

References 426

[1] Heidenreich A Bolla M Joniau S Mason MD Matveev V Mottet N et al 427

Guidelines on prostate cancer Eur Assoc Urol 2011 [www uroweb org] 428

[2] Pfitzenmaier J Pahernik S Tremmel T Haferkamp A Buse S Hohen- 429

fellner M Positive surgical margins after radical prostatectomy do 430

they have an impact on biochemical or clinical progression BJU Int 431

2008102(10)1413ndash8 432

[3] Soler L Nicolau S Schmid J Koehl C Marescaux J et al Virtual reality 433

and augmented reality in digestive surgery In Third IEEE and ACM Inter- 434

national Symposium on Mixed and Augmented Reality (ISMAR) 2004 435

p 278ndash9 436

[4] S Voros B Tamadazte P Cinquin C Fouard Systegraveme drsquoimagerie multi- 437

vision pour chirurgie laparoscopique demande de deacutepocirct de brevet FR 438

1259489 102012 439

[5] Ukimura O Ahlering TE Gill IS Transrectal ultrasound-guided 440

energy-free nerve-sparing laparoscopic radical prostatectomy J Endourol 441

200822(9)1993ndash5 442

[6] Long JA Lee BH Guillotreau J Autorino R Laydner H Yakoubi R et al 443

Real-time robotic transrectal ultrasound navigation during robotic radical 444

prostatectomy initial clinical experience Urology 201280(3)608ndash13 445

[7] Holmes III DR Davis BJ Bruce CJ Robb RA 3D visualization analysis 446

and treatment of the prostate using trans-urethral ultrasound Comput Med 447

Imaging Graph 200327339ndash49 [2003] 448

[8] Dementhon D Model-based object pose in 25 lines of code Int J Comput 449

Vis 199515(1ndash2)123ndash41 450

[9] Custillon G Voros S Cinquin P Nguyen-Dinh A Moreau-Gaudry A 451

Bidimensional localization of active ultrasound transducers for use in 452

laparoscopic prostate surgery IEEE Trans Med Imaging 2012 Q3 453

[10] Wolf R Duchateau J Cinquin P Voros S 3D tracking of laparoscopic 454

instruments using statistical and geometric modeling Med Image Com- 455

put Comput Assist Interv 20116891203ndash10 [Lecture notes in computer 456

science] 457

[11] Valkenburg KC Williams BO ldquoMouse models of prostate cancerrdquo Prostate 458

Cancer 2011201122 httpdxdoiorg1011552011895238 [Article ID 459

895238] 460

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

4 S Voros et al IRBM xxx (2013) xxxndashxxx

Fig 3 The proposed markers for the registration of ultrasound and endoscopic

images Top ldquoactiverdquo landmark (laparoscopic ultrasound emitter) Bottom ldquopas-

siverdquo landmark (laparoscopic needle) used for the registration of ultrasound and

laparoscopic images

bull the normalmalignant characterization of the tissues is based198

on the detection of the autofluorescence of the Protopor-199

phyrin IX (PpIX) protein which accumulates in malignant200

cells Indeed the elimination cycle of Protorphyrin is per-201

turbed in case of malignant cells It causes thus an increase202

of Protoporphyrin concentration A first prototype (already203

available see Fig 4 left) comes in the form of a testbench204

equipped with a laser source emitting at 405 nm (excitation 205

wavelength for the PpIX protein) and with optical fibers and 206

a spectrometer allowing the collection of a fluorescence spec- 207

trum of 3648 points in the range 345ndash1040 mm (the testbench 208

can image a 25 times 25 mm2 area with a spatial resolution of 209

100 m and a penetration depth of roughly 300 m) Using 210

this spectrum an intensity image can be created (Fig 4 211

right) The analysis of prostate tissue samples gathered in 212

the frame of a biomedical protocol (section 4) will allow us 213

to determine the optimal parameters for the acquisition of 214

the fluorescence spectrum and build discrimination criteria 215

from the measurements between malignanthealthy tissue and 216

prostaticnon-prostatic tissue 217

bull the prostaticnon-prostatic characterization of the tissues is 218

based on the detection of the prostate-specific membrane anti- 219

gen (PSMA) which is specific of the prostatic membrane 220

This detection is made possible by the immunofluorescence 221

tagging of the PSMA using a specific anti-PSMA antibody 222

labelled with a fluorescent tracer When excited by light at 223

a characteristic absorption wavelength the fluorescent tracer 224

emits light at a characteristic emission wavelength Two flu- 225

orescent tracers are investigated in the DEPORRA project 226

Fluorescein IsoThioCyanate (FITC) and Cyanine 5 Both 227

FITC and Cyanine 5 have a characteristic emission wave- 228

length different from the autofluorescence wavelength of the 229

Protoporpphyrin IX protein allowing for the characterization 230

of the tissue type (prostatic vs non-prostatic) and the tissue 231

status (healthy vs malignant) with the same testbench (Fig 5) 232

The testbench presented on Fig 4 has also been equipped 233

with two laser sources emitting at 488 nm and 642 nm exci- 234

tation wavelengths respectively for FITC and Cyanine 5 for 235

recognizing prostate tissue from those environing 236

A bimodal laparoscopic-compliant fluorescence probe based 237

on the miniaturization of the fluorescence testbench will be avail- 238

able in 2013 (Fig 6 top) Preliminary results based on the 239

analysis of histopathologic slices will be presented in section 240

4 In order to overlay the probersquos fluorescence measurement on 241

the laparoscopic images we plan on detecting automatically the 242

position of the tip of the laparoscopic probe using a real-time 243

Fig 4 Experimental fluorescence design and results Left experimental optical device Right typical intensity image obtained

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 5

Perc

ent

of

maxim

a

Wavelength (nm)

PpIX emission

FITC excitation

FITC emission

PpIX excitation

350 400 450 500 550 600 650 700 750

Perc

ent

of

maxim

a

10

00

08

06

04

02

Wavelength (nm)

PpIX emission

Cy5 excitation

Cy5 emission

PpIX excitation

BA

350 400 450 500 550 600 650 700 750

10

00

08

06

04

02

Fig 5 Excitation and emission spectra of Protoporphyrin IX Fluorescein IsoThioCyanate and Cyanine 5 (Cy5) fluorochromes A Overlay of the excitation (blue

dashed line) and emission (orange solid line) spectra of Protoporphyrin IX with excitation (green dashed line) and emission (emission spectrum) of Fluorescein

IsoThioCyanate B Overlay of the excitation (blue dashed line) and emission (orange solid line) spectra of Protoporphyrin IX with excitation (red dashed line) and

emission (red solid line) of Cy5

Fig 6 The bimodal fluorescence laparoscopic probe and the image-based anal-

ysis approach to localize it in laparoscopic images Top the CAD drawing

of the bimodal fluorescence laparoscopic probe Bottom automatic detection

of a laparoscopic instrument which will be used to overlay the probersquos tissue

characterization measurements on the laparoscopic images

image analysis approach that we have developed [10] (Fig 6244

bottom)245

34 Preclinical and clinical evaluation246

Preliminary evaluation of the devices and the software devel-247

oped in the frame of this project has already been performed on248

laboratory testbenchs and during three cadaver experiments at249

the anatomy laboratory (section 4) However in order to acquire250

the qualitative and quantitative proofs necessary to perform the251

fist clinical evaluations of our innovative medical devices on 252

patients we have submitted two biomedical research protocols 253

bull the first one called ldquofusion of echographic (ultrasound) and 254

endoscopic imagesrdquo (FEE) is a pilot monocentric study on a 255

cohort of 15 patients and is currently running It aims at val- 256

idating our ultrasound approach and evaluating its expected 257

medical benefit during radical prostatectomies the surgeons 258

will insert surgical needles in the prostate and endorectal 259

ultrasound images and laparoscopic images will be recorded 260

The data will be processed offline in the laboratory to assess 261

quantitatively the feasibility of the registration of ultrasound 262

and laparoscopic images using passive markers This proto- 263

col already accepted by the ethical instances will be a pilot 264

monocentric study with 15 patients 265

bull the second protocol called COPROST will allow us to obtain 266

fresh prostate chips during transurethral resections in the 267

frame of a pilot monocentric prospective non-randomized 268

open and controlled clinical trial The protocol with an 269

inclusion period of 24 months is already defined and writ- 270

ten following the French Regulation on Biomedical Research 271

and biological tissue collection It is being submitted to the 272

ethical committee ldquocomiteacute de protection des personnesrdquo and 273

the ldquoagence nationale de seacutecuriteacute du meacutedicament et des pro- 274

duits de santeacuterdquo (ANSM) The prostate chips will be on one 275

hand characterized by anatomopathologists to determine their 276

nature (pathologic vs healthy prostatic vs non-prostatic) and 277

will allow us to validate our immunofluorescence protocol and 278

fine-tune our auto and immunofluorescence measurements 279

protocols on fresh tissue samples 280

4 Results 281

In this section we present our preclinical evaluation of the 282

different prototypes and software presented in the previous sec- 283

tion 284

41 Augmented laparoscopy thanks to an innovative video 285

device 286

In order to evaluate the potential benefits of the pro- 287

posed vision system we asked a surgeon to perform a simple 288

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

6 S Voros et al IRBM xxx (2013) xxxndashxxx

Fig 7 Ultrasound image of a prostate acquired with our innovative transurethral

ultrasound probe during a cadaver experiment The image was acquired with our

first prototype (64 piezoelectric elements)

surgical task once with the traditional endoscope alone and289

once with the proposed system alone (although in practice they290

can be combined to associate local and global views) The tasks291

consisted in localizing a suture needle and bringing it to a fixed292

target point The experiment was performed on porcine organs293

placed in a training box and repeated six times by the surgeon294

At each realization the needlersquos initial position was repositioned295

randomly at a distance equivalent to its initial position and the296

surgeon started randomly with the traditional endoscope or the297

proposed system In both cases a robotic endoscope holder was298

used to mobilize the endoscope and the vision system allowing299

us to record the displacements of the vision systems300

The full methodology and results are currently under sub-301

mission but to summarize the mean time required to perform302

the six experiments with the traditional endoscope was of 190 s303

compared to 245 s with the proposed system Moreover the sur-304

geon needed to give an average of 232 commands to the robotic305

scope holder to perform one stitch with the endoscope alone306

compared to 46 commands with the proposed system These307

preliminary results suggest that the proposed system could sig-308

nificantly reduce the laparoscopic surgery time and the cognitive309

load required for the control of the endoscope We now plan on310

performing cadaver experiments to evaluate the deployment of311

the vision system in conditions close to the clinical reality and312

to evaluate the system with several surgeons313

42 Augmented laparoscopy thanks to an innovative314

ultrasound system315

We performed two cadaver experiments which allowed us316

to determine the optimal characteristics for the realization of317

the intraurethral ultrasound probe During these experiments318

we evaluated the (difficult) insertion of the catheter through319

the urethra and the prostate visualization It must be noted that320

our clinicians partners stressed that the rigor mortis made this321

insertion harder These experiments allowed us to find the best322

rigidity for the catheter of the probe it must be flexible enough323

Fig 8 Demonstration of the fusion of ultrasound and laparoscopic images using

passive landmarks on chicken breasts Top-left the ultrasound image top-right

the laparoscopic image The passive landmarks are indicated with the green

arrows in the laparoscopic image They are selected manually in both modalities

to perform the registration Bottom the fusion of the two imaging modalities

to be introduced in the urethra but rigid enough to avoid the 324

distortions of the catheter 325

The first ultrasound probe (2D) was designed with 64 326

piezoelements It appeared that it was not enough On the one 327

hand the probe imaged a too small part of the prostate so that 328

it was difficult to identify what was represented on the ultra- 329

sound image (Fig 7) Furthermore it was impossible to know 330

in which direction the probe was oriented On the other hand 331

we demonstrated [9] that a longer probe increases the precision 332

of the localization of the active ultrasound transducers That is 333

why the second probe designed for the project (3D) has 128 334

piezoelements This second probe has graduations located on 335

its catheter visible on Fig 2 They allow the surgeon to know 336

the probersquos insertion depth in the urethra and a line along the 337

catheter allows appreciating the torsion applied to the catheter 338

The localization of the active transducers is currently under 339

submission It is based on a global positioning system (GPS) 340

method We made some experiments on a testbench making 341

localization in water to have ideal homogeneous conditions and 342

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 7

Fig 9 Preliminary experiments of the autofluorescence detection with our testbench The autofluorescence of chlorophyll which has optical properties very similar

to Protoporphyrin IX was observed on our testbench Left direct image of a leaf right fluorescence image of the leaf at 680 nm As expected chlorophyll seems

absent in the veins

Fig 10 Specific fluorescence signal of anti-prostate-specific membrane antigen marker A Nuclear counter-staining of prostatic tissue with Hoechst (blue DNA

marker) We observe the nuclei of prostate cells regularly organized in the glandular epithelium Prostatic glands are separated by stroma B Fluorescence emission

of anti-prostate-specific membrane antigen-Fluorescein IsoThioCyanate on cellular membranes of prostatic cells C Overlay of the A and B images This figure

demonstrates the specificity on fixed prostatic tissue of a specific anti-PSMA antibody labelled with Fluorescein IsoThioCyanate We can also detect the contribution

of non-specific autofluorescence of prostatic concretions (Fig 11) (times20 magnification)

localization on a chicken breast to have more realistic condi-343

tions In both cases we were able to localize the active transducer344

with a precision less than 625 m Moreover the computation345

time is a few hundred milliseconds for a 2D localization These346

results demonstrate that the localization is fast enough to be347

implemented for a prostatectomy For the moment only a 2D 348

localization has been made with the active landmarks and the 349

3D localization is in progress Then we will be able to per- 350

form a registration with laparoscopic images using ldquoactiverdquo 351

markers 352

Fig 11 Non-specific autofluorescence observed in the prostate fixed tissues This figure shows the benefits of a near infrared marker like Cyanine 5 (excited at

633 nm) compared to Fluorescein IsoThioCyanate (excited at 488 nm) A Auto fluorescence under laser excitation at 488 nm We observe that prostate concretions

(calcified material) respond to this excitation by a strong fluorescence signal B Auto fluorescence under laser excitation at 633 nm The use of Cyanine 5 reduces

widely autofluorescence of some structures such as prostate concretions (times20 magnification)

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

8 S Voros et al IRBM xxx (2013) xxxndashxxx

Concerning the registration between laparoscopic and ultra-353

sound images we performed several testbench evaluations of354

the registration algorithm on chicken breasts The result of such355

a registration obtained by manually pairing passive markers356

visible on both the laparoscopic and ultrasound imaging modal-357

ities are presented on Fig 8 In this example we obtained a358

registration root mean square error (RMS) of 038 mm359

43 Augmented laparoscopy thanks to fluorescence360

imaging361

First experiments were performed on leaves to demonstrate362

the feasibility of the detection of chlorophyll autofluorescence363

which has optical properties very similar to PpIX (absorption at364

405 nm and emission at 680 nm) as described on Fig 9 Then365

experiments were conducted on histology slices of the human366

prostate with as we anticipated inconclusive results because367

of the expected low signal-to-noise ratio caused by the insuffi-368

cient thickness of a histology slice (3 m thickness) compared to369

the optimal use of the device (300 m thickness) These results370

motivated our biomedical research COPROST on macroscopic371

fresh prostate samples372

Concerning the immunofluorescence protocol we were able373

to demonstrate the specificity of the anti-PSMA antibody on374

fixed tissue samples labelled with FITC as illustrated by Fig 10375

We observed higher autofluorescence problems when excit-376

ing the sample at the FITC specific wavelength compared to377

an excitation at the Cyanine 5 specific wavelength (Fig 11)378

These results motivated the inclusion of both fluorophores in our379

biomedical research COPROST on macroscopic fresh prostate380

samples FITC because it was already used to demonstrate speci-381

ficity of anti-PSMA antibody and Cyanine 5 because its near382

infra-red characteristics make it a better candidate on fresh mate-383

rial (higher penetration in tissue) and will improve signalnoise384

ratio as previously demonstrated385

To our knowledge most of the optical biopsy approaches as386

previously defined have been validated either on histologic slices387

or on ratmice prostate models However these approaches have388

limitations prostatic tissue is impaired by the chemical treat-389

ment required for histologic preparation and animal models do390

not directly mimic all aspects of human prostate cancer [11] In391

short it has not been proved yet that a successful detection with392

the investigated devices on histopathology slices or on animal393

models is sufficient to guarantee a successful detection on fresh394

human tissues The experiments that will be performed in the395

frame of the COPROST protocol in 2013 will be determinant to396

confirm the feasibility of our approach on fresh tissue samples397

5 DiscussionConclusion398

The DEPORRA project allowed for the development of inno-399

vative devices and navigation prototypes in the objective of400

allowing the surgeon to see ldquobeyond the visiblerdquo during a radi-401

cal prostatectomy First evaluations of the devices and methods402

have been performed preclinically and are very encouraging for403

the development of innovative approaches to assist the surgeon

during such a complex surgery Our preliminary results have also 404

shown the limits of preclinical validation and have convinced us 405

of the necessity to launch biomedical researches that will allow 406

us to validate further the devices This process required a con- 407

sequent amount of effort and time but is mandatory for the 408

fine-tuning of our tissue characterization tools and to obtain the 409

first clinical proofs of the relevance of the developed medical 410

devices which are mandatory to perform clinical evaluations of 411

the complete navigation systems 412

We now need to push further the integration of the devices 413

into such navigation systems in order to exploit at best the infor- 414

mation provided by each modality and to determine the optimal 415

approach for displaying the relevant information to the surgeon 416

in a clinical environment This will imply a conception and 417

development effort a definition of qualitative and quantitative 418

parameters for the first assessment of the delivered medical ben- 419

efit of our developments risks analysis and biomedical research 420

preparations in order to meet the ethical instances requirements 421

Acknowledgments 422

This work has been supported by French National Research 423

Agency (ANR) through TecSan program (project DEPORRA 424

no ANR-09-TECS-006) 425

References 426

[1] Heidenreich A Bolla M Joniau S Mason MD Matveev V Mottet N et al 427

Guidelines on prostate cancer Eur Assoc Urol 2011 [www uroweb org] 428

[2] Pfitzenmaier J Pahernik S Tremmel T Haferkamp A Buse S Hohen- 429

fellner M Positive surgical margins after radical prostatectomy do 430

they have an impact on biochemical or clinical progression BJU Int 431

2008102(10)1413ndash8 432

[3] Soler L Nicolau S Schmid J Koehl C Marescaux J et al Virtual reality 433

and augmented reality in digestive surgery In Third IEEE and ACM Inter- 434

national Symposium on Mixed and Augmented Reality (ISMAR) 2004 435

p 278ndash9 436

[4] S Voros B Tamadazte P Cinquin C Fouard Systegraveme drsquoimagerie multi- 437

vision pour chirurgie laparoscopique demande de deacutepocirct de brevet FR 438

1259489 102012 439

[5] Ukimura O Ahlering TE Gill IS Transrectal ultrasound-guided 440

energy-free nerve-sparing laparoscopic radical prostatectomy J Endourol 441

200822(9)1993ndash5 442

[6] Long JA Lee BH Guillotreau J Autorino R Laydner H Yakoubi R et al 443

Real-time robotic transrectal ultrasound navigation during robotic radical 444

prostatectomy initial clinical experience Urology 201280(3)608ndash13 445

[7] Holmes III DR Davis BJ Bruce CJ Robb RA 3D visualization analysis 446

and treatment of the prostate using trans-urethral ultrasound Comput Med 447

Imaging Graph 200327339ndash49 [2003] 448

[8] Dementhon D Model-based object pose in 25 lines of code Int J Comput 449

Vis 199515(1ndash2)123ndash41 450

[9] Custillon G Voros S Cinquin P Nguyen-Dinh A Moreau-Gaudry A 451

Bidimensional localization of active ultrasound transducers for use in 452

laparoscopic prostate surgery IEEE Trans Med Imaging 2012 Q3 453

[10] Wolf R Duchateau J Cinquin P Voros S 3D tracking of laparoscopic 454

instruments using statistical and geometric modeling Med Image Com- 455

put Comput Assist Interv 20116891203ndash10 [Lecture notes in computer 456

science] 457

[11] Valkenburg KC Williams BO ldquoMouse models of prostate cancerrdquo Prostate 458

Cancer 2011201122 httpdxdoiorg1011552011895238 [Article ID 459

895238] 460

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 5

Perc

ent

of

maxim

a

Wavelength (nm)

PpIX emission

FITC excitation

FITC emission

PpIX excitation

350 400 450 500 550 600 650 700 750

Perc

ent

of

maxim

a

10

00

08

06

04

02

Wavelength (nm)

PpIX emission

Cy5 excitation

Cy5 emission

PpIX excitation

BA

350 400 450 500 550 600 650 700 750

10

00

08

06

04

02

Fig 5 Excitation and emission spectra of Protoporphyrin IX Fluorescein IsoThioCyanate and Cyanine 5 (Cy5) fluorochromes A Overlay of the excitation (blue

dashed line) and emission (orange solid line) spectra of Protoporphyrin IX with excitation (green dashed line) and emission (emission spectrum) of Fluorescein

IsoThioCyanate B Overlay of the excitation (blue dashed line) and emission (orange solid line) spectra of Protoporphyrin IX with excitation (red dashed line) and

emission (red solid line) of Cy5

Fig 6 The bimodal fluorescence laparoscopic probe and the image-based anal-

ysis approach to localize it in laparoscopic images Top the CAD drawing

of the bimodal fluorescence laparoscopic probe Bottom automatic detection

of a laparoscopic instrument which will be used to overlay the probersquos tissue

characterization measurements on the laparoscopic images

image analysis approach that we have developed [10] (Fig 6244

bottom)245

34 Preclinical and clinical evaluation246

Preliminary evaluation of the devices and the software devel-247

oped in the frame of this project has already been performed on248

laboratory testbenchs and during three cadaver experiments at249

the anatomy laboratory (section 4) However in order to acquire250

the qualitative and quantitative proofs necessary to perform the251

fist clinical evaluations of our innovative medical devices on 252

patients we have submitted two biomedical research protocols 253

bull the first one called ldquofusion of echographic (ultrasound) and 254

endoscopic imagesrdquo (FEE) is a pilot monocentric study on a 255

cohort of 15 patients and is currently running It aims at val- 256

idating our ultrasound approach and evaluating its expected 257

medical benefit during radical prostatectomies the surgeons 258

will insert surgical needles in the prostate and endorectal 259

ultrasound images and laparoscopic images will be recorded 260

The data will be processed offline in the laboratory to assess 261

quantitatively the feasibility of the registration of ultrasound 262

and laparoscopic images using passive markers This proto- 263

col already accepted by the ethical instances will be a pilot 264

monocentric study with 15 patients 265

bull the second protocol called COPROST will allow us to obtain 266

fresh prostate chips during transurethral resections in the 267

frame of a pilot monocentric prospective non-randomized 268

open and controlled clinical trial The protocol with an 269

inclusion period of 24 months is already defined and writ- 270

ten following the French Regulation on Biomedical Research 271

and biological tissue collection It is being submitted to the 272

ethical committee ldquocomiteacute de protection des personnesrdquo and 273

the ldquoagence nationale de seacutecuriteacute du meacutedicament et des pro- 274

duits de santeacuterdquo (ANSM) The prostate chips will be on one 275

hand characterized by anatomopathologists to determine their 276

nature (pathologic vs healthy prostatic vs non-prostatic) and 277

will allow us to validate our immunofluorescence protocol and 278

fine-tune our auto and immunofluorescence measurements 279

protocols on fresh tissue samples 280

4 Results 281

In this section we present our preclinical evaluation of the 282

different prototypes and software presented in the previous sec- 283

tion 284

41 Augmented laparoscopy thanks to an innovative video 285

device 286

In order to evaluate the potential benefits of the pro- 287

posed vision system we asked a surgeon to perform a simple 288

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

6 S Voros et al IRBM xxx (2013) xxxndashxxx

Fig 7 Ultrasound image of a prostate acquired with our innovative transurethral

ultrasound probe during a cadaver experiment The image was acquired with our

first prototype (64 piezoelectric elements)

surgical task once with the traditional endoscope alone and289

once with the proposed system alone (although in practice they290

can be combined to associate local and global views) The tasks291

consisted in localizing a suture needle and bringing it to a fixed292

target point The experiment was performed on porcine organs293

placed in a training box and repeated six times by the surgeon294

At each realization the needlersquos initial position was repositioned295

randomly at a distance equivalent to its initial position and the296

surgeon started randomly with the traditional endoscope or the297

proposed system In both cases a robotic endoscope holder was298

used to mobilize the endoscope and the vision system allowing299

us to record the displacements of the vision systems300

The full methodology and results are currently under sub-301

mission but to summarize the mean time required to perform302

the six experiments with the traditional endoscope was of 190 s303

compared to 245 s with the proposed system Moreover the sur-304

geon needed to give an average of 232 commands to the robotic305

scope holder to perform one stitch with the endoscope alone306

compared to 46 commands with the proposed system These307

preliminary results suggest that the proposed system could sig-308

nificantly reduce the laparoscopic surgery time and the cognitive309

load required for the control of the endoscope We now plan on310

performing cadaver experiments to evaluate the deployment of311

the vision system in conditions close to the clinical reality and312

to evaluate the system with several surgeons313

42 Augmented laparoscopy thanks to an innovative314

ultrasound system315

We performed two cadaver experiments which allowed us316

to determine the optimal characteristics for the realization of317

the intraurethral ultrasound probe During these experiments318

we evaluated the (difficult) insertion of the catheter through319

the urethra and the prostate visualization It must be noted that320

our clinicians partners stressed that the rigor mortis made this321

insertion harder These experiments allowed us to find the best322

rigidity for the catheter of the probe it must be flexible enough323

Fig 8 Demonstration of the fusion of ultrasound and laparoscopic images using

passive landmarks on chicken breasts Top-left the ultrasound image top-right

the laparoscopic image The passive landmarks are indicated with the green

arrows in the laparoscopic image They are selected manually in both modalities

to perform the registration Bottom the fusion of the two imaging modalities

to be introduced in the urethra but rigid enough to avoid the 324

distortions of the catheter 325

The first ultrasound probe (2D) was designed with 64 326

piezoelements It appeared that it was not enough On the one 327

hand the probe imaged a too small part of the prostate so that 328

it was difficult to identify what was represented on the ultra- 329

sound image (Fig 7) Furthermore it was impossible to know 330

in which direction the probe was oriented On the other hand 331

we demonstrated [9] that a longer probe increases the precision 332

of the localization of the active ultrasound transducers That is 333

why the second probe designed for the project (3D) has 128 334

piezoelements This second probe has graduations located on 335

its catheter visible on Fig 2 They allow the surgeon to know 336

the probersquos insertion depth in the urethra and a line along the 337

catheter allows appreciating the torsion applied to the catheter 338

The localization of the active transducers is currently under 339

submission It is based on a global positioning system (GPS) 340

method We made some experiments on a testbench making 341

localization in water to have ideal homogeneous conditions and 342

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 7

Fig 9 Preliminary experiments of the autofluorescence detection with our testbench The autofluorescence of chlorophyll which has optical properties very similar

to Protoporphyrin IX was observed on our testbench Left direct image of a leaf right fluorescence image of the leaf at 680 nm As expected chlorophyll seems

absent in the veins

Fig 10 Specific fluorescence signal of anti-prostate-specific membrane antigen marker A Nuclear counter-staining of prostatic tissue with Hoechst (blue DNA

marker) We observe the nuclei of prostate cells regularly organized in the glandular epithelium Prostatic glands are separated by stroma B Fluorescence emission

of anti-prostate-specific membrane antigen-Fluorescein IsoThioCyanate on cellular membranes of prostatic cells C Overlay of the A and B images This figure

demonstrates the specificity on fixed prostatic tissue of a specific anti-PSMA antibody labelled with Fluorescein IsoThioCyanate We can also detect the contribution

of non-specific autofluorescence of prostatic concretions (Fig 11) (times20 magnification)

localization on a chicken breast to have more realistic condi-343

tions In both cases we were able to localize the active transducer344

with a precision less than 625 m Moreover the computation345

time is a few hundred milliseconds for a 2D localization These346

results demonstrate that the localization is fast enough to be347

implemented for a prostatectomy For the moment only a 2D 348

localization has been made with the active landmarks and the 349

3D localization is in progress Then we will be able to per- 350

form a registration with laparoscopic images using ldquoactiverdquo 351

markers 352

Fig 11 Non-specific autofluorescence observed in the prostate fixed tissues This figure shows the benefits of a near infrared marker like Cyanine 5 (excited at

633 nm) compared to Fluorescein IsoThioCyanate (excited at 488 nm) A Auto fluorescence under laser excitation at 488 nm We observe that prostate concretions

(calcified material) respond to this excitation by a strong fluorescence signal B Auto fluorescence under laser excitation at 633 nm The use of Cyanine 5 reduces

widely autofluorescence of some structures such as prostate concretions (times20 magnification)

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

8 S Voros et al IRBM xxx (2013) xxxndashxxx

Concerning the registration between laparoscopic and ultra-353

sound images we performed several testbench evaluations of354

the registration algorithm on chicken breasts The result of such355

a registration obtained by manually pairing passive markers356

visible on both the laparoscopic and ultrasound imaging modal-357

ities are presented on Fig 8 In this example we obtained a358

registration root mean square error (RMS) of 038 mm359

43 Augmented laparoscopy thanks to fluorescence360

imaging361

First experiments were performed on leaves to demonstrate362

the feasibility of the detection of chlorophyll autofluorescence363

which has optical properties very similar to PpIX (absorption at364

405 nm and emission at 680 nm) as described on Fig 9 Then365

experiments were conducted on histology slices of the human366

prostate with as we anticipated inconclusive results because367

of the expected low signal-to-noise ratio caused by the insuffi-368

cient thickness of a histology slice (3 m thickness) compared to369

the optimal use of the device (300 m thickness) These results370

motivated our biomedical research COPROST on macroscopic371

fresh prostate samples372

Concerning the immunofluorescence protocol we were able373

to demonstrate the specificity of the anti-PSMA antibody on374

fixed tissue samples labelled with FITC as illustrated by Fig 10375

We observed higher autofluorescence problems when excit-376

ing the sample at the FITC specific wavelength compared to377

an excitation at the Cyanine 5 specific wavelength (Fig 11)378

These results motivated the inclusion of both fluorophores in our379

biomedical research COPROST on macroscopic fresh prostate380

samples FITC because it was already used to demonstrate speci-381

ficity of anti-PSMA antibody and Cyanine 5 because its near382

infra-red characteristics make it a better candidate on fresh mate-383

rial (higher penetration in tissue) and will improve signalnoise384

ratio as previously demonstrated385

To our knowledge most of the optical biopsy approaches as386

previously defined have been validated either on histologic slices387

or on ratmice prostate models However these approaches have388

limitations prostatic tissue is impaired by the chemical treat-389

ment required for histologic preparation and animal models do390

not directly mimic all aspects of human prostate cancer [11] In391

short it has not been proved yet that a successful detection with392

the investigated devices on histopathology slices or on animal393

models is sufficient to guarantee a successful detection on fresh394

human tissues The experiments that will be performed in the395

frame of the COPROST protocol in 2013 will be determinant to396

confirm the feasibility of our approach on fresh tissue samples397

5 DiscussionConclusion398

The DEPORRA project allowed for the development of inno-399

vative devices and navigation prototypes in the objective of400

allowing the surgeon to see ldquobeyond the visiblerdquo during a radi-401

cal prostatectomy First evaluations of the devices and methods402

have been performed preclinically and are very encouraging for403

the development of innovative approaches to assist the surgeon

during such a complex surgery Our preliminary results have also 404

shown the limits of preclinical validation and have convinced us 405

of the necessity to launch biomedical researches that will allow 406

us to validate further the devices This process required a con- 407

sequent amount of effort and time but is mandatory for the 408

fine-tuning of our tissue characterization tools and to obtain the 409

first clinical proofs of the relevance of the developed medical 410

devices which are mandatory to perform clinical evaluations of 411

the complete navigation systems 412

We now need to push further the integration of the devices 413

into such navigation systems in order to exploit at best the infor- 414

mation provided by each modality and to determine the optimal 415

approach for displaying the relevant information to the surgeon 416

in a clinical environment This will imply a conception and 417

development effort a definition of qualitative and quantitative 418

parameters for the first assessment of the delivered medical ben- 419

efit of our developments risks analysis and biomedical research 420

preparations in order to meet the ethical instances requirements 421

Acknowledgments 422

This work has been supported by French National Research 423

Agency (ANR) through TecSan program (project DEPORRA 424

no ANR-09-TECS-006) 425

References 426

[1] Heidenreich A Bolla M Joniau S Mason MD Matveev V Mottet N et al 427

Guidelines on prostate cancer Eur Assoc Urol 2011 [www uroweb org] 428

[2] Pfitzenmaier J Pahernik S Tremmel T Haferkamp A Buse S Hohen- 429

fellner M Positive surgical margins after radical prostatectomy do 430

they have an impact on biochemical or clinical progression BJU Int 431

2008102(10)1413ndash8 432

[3] Soler L Nicolau S Schmid J Koehl C Marescaux J et al Virtual reality 433

and augmented reality in digestive surgery In Third IEEE and ACM Inter- 434

national Symposium on Mixed and Augmented Reality (ISMAR) 2004 435

p 278ndash9 436

[4] S Voros B Tamadazte P Cinquin C Fouard Systegraveme drsquoimagerie multi- 437

vision pour chirurgie laparoscopique demande de deacutepocirct de brevet FR 438

1259489 102012 439

[5] Ukimura O Ahlering TE Gill IS Transrectal ultrasound-guided 440

energy-free nerve-sparing laparoscopic radical prostatectomy J Endourol 441

200822(9)1993ndash5 442

[6] Long JA Lee BH Guillotreau J Autorino R Laydner H Yakoubi R et al 443

Real-time robotic transrectal ultrasound navigation during robotic radical 444

prostatectomy initial clinical experience Urology 201280(3)608ndash13 445

[7] Holmes III DR Davis BJ Bruce CJ Robb RA 3D visualization analysis 446

and treatment of the prostate using trans-urethral ultrasound Comput Med 447

Imaging Graph 200327339ndash49 [2003] 448

[8] Dementhon D Model-based object pose in 25 lines of code Int J Comput 449

Vis 199515(1ndash2)123ndash41 450

[9] Custillon G Voros S Cinquin P Nguyen-Dinh A Moreau-Gaudry A 451

Bidimensional localization of active ultrasound transducers for use in 452

laparoscopic prostate surgery IEEE Trans Med Imaging 2012 Q3 453

[10] Wolf R Duchateau J Cinquin P Voros S 3D tracking of laparoscopic 454

instruments using statistical and geometric modeling Med Image Com- 455

put Comput Assist Interv 20116891203ndash10 [Lecture notes in computer 456

science] 457

[11] Valkenburg KC Williams BO ldquoMouse models of prostate cancerrdquo Prostate 458

Cancer 2011201122 httpdxdoiorg1011552011895238 [Article ID 459

895238] 460

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

6 S Voros et al IRBM xxx (2013) xxxndashxxx

Fig 7 Ultrasound image of a prostate acquired with our innovative transurethral

ultrasound probe during a cadaver experiment The image was acquired with our

first prototype (64 piezoelectric elements)

surgical task once with the traditional endoscope alone and289

once with the proposed system alone (although in practice they290

can be combined to associate local and global views) The tasks291

consisted in localizing a suture needle and bringing it to a fixed292

target point The experiment was performed on porcine organs293

placed in a training box and repeated six times by the surgeon294

At each realization the needlersquos initial position was repositioned295

randomly at a distance equivalent to its initial position and the296

surgeon started randomly with the traditional endoscope or the297

proposed system In both cases a robotic endoscope holder was298

used to mobilize the endoscope and the vision system allowing299

us to record the displacements of the vision systems300

The full methodology and results are currently under sub-301

mission but to summarize the mean time required to perform302

the six experiments with the traditional endoscope was of 190 s303

compared to 245 s with the proposed system Moreover the sur-304

geon needed to give an average of 232 commands to the robotic305

scope holder to perform one stitch with the endoscope alone306

compared to 46 commands with the proposed system These307

preliminary results suggest that the proposed system could sig-308

nificantly reduce the laparoscopic surgery time and the cognitive309

load required for the control of the endoscope We now plan on310

performing cadaver experiments to evaluate the deployment of311

the vision system in conditions close to the clinical reality and312

to evaluate the system with several surgeons313

42 Augmented laparoscopy thanks to an innovative314

ultrasound system315

We performed two cadaver experiments which allowed us316

to determine the optimal characteristics for the realization of317

the intraurethral ultrasound probe During these experiments318

we evaluated the (difficult) insertion of the catheter through319

the urethra and the prostate visualization It must be noted that320

our clinicians partners stressed that the rigor mortis made this321

insertion harder These experiments allowed us to find the best322

rigidity for the catheter of the probe it must be flexible enough323

Fig 8 Demonstration of the fusion of ultrasound and laparoscopic images using

passive landmarks on chicken breasts Top-left the ultrasound image top-right

the laparoscopic image The passive landmarks are indicated with the green

arrows in the laparoscopic image They are selected manually in both modalities

to perform the registration Bottom the fusion of the two imaging modalities

to be introduced in the urethra but rigid enough to avoid the 324

distortions of the catheter 325

The first ultrasound probe (2D) was designed with 64 326

piezoelements It appeared that it was not enough On the one 327

hand the probe imaged a too small part of the prostate so that 328

it was difficult to identify what was represented on the ultra- 329

sound image (Fig 7) Furthermore it was impossible to know 330

in which direction the probe was oriented On the other hand 331

we demonstrated [9] that a longer probe increases the precision 332

of the localization of the active ultrasound transducers That is 333

why the second probe designed for the project (3D) has 128 334

piezoelements This second probe has graduations located on 335

its catheter visible on Fig 2 They allow the surgeon to know 336

the probersquos insertion depth in the urethra and a line along the 337

catheter allows appreciating the torsion applied to the catheter 338

The localization of the active transducers is currently under 339

submission It is based on a global positioning system (GPS) 340

method We made some experiments on a testbench making 341

localization in water to have ideal homogeneous conditions and 342

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 7

Fig 9 Preliminary experiments of the autofluorescence detection with our testbench The autofluorescence of chlorophyll which has optical properties very similar

to Protoporphyrin IX was observed on our testbench Left direct image of a leaf right fluorescence image of the leaf at 680 nm As expected chlorophyll seems

absent in the veins

Fig 10 Specific fluorescence signal of anti-prostate-specific membrane antigen marker A Nuclear counter-staining of prostatic tissue with Hoechst (blue DNA

marker) We observe the nuclei of prostate cells regularly organized in the glandular epithelium Prostatic glands are separated by stroma B Fluorescence emission

of anti-prostate-specific membrane antigen-Fluorescein IsoThioCyanate on cellular membranes of prostatic cells C Overlay of the A and B images This figure

demonstrates the specificity on fixed prostatic tissue of a specific anti-PSMA antibody labelled with Fluorescein IsoThioCyanate We can also detect the contribution

of non-specific autofluorescence of prostatic concretions (Fig 11) (times20 magnification)

localization on a chicken breast to have more realistic condi-343

tions In both cases we were able to localize the active transducer344

with a precision less than 625 m Moreover the computation345

time is a few hundred milliseconds for a 2D localization These346

results demonstrate that the localization is fast enough to be347

implemented for a prostatectomy For the moment only a 2D 348

localization has been made with the active landmarks and the 349

3D localization is in progress Then we will be able to per- 350

form a registration with laparoscopic images using ldquoactiverdquo 351

markers 352

Fig 11 Non-specific autofluorescence observed in the prostate fixed tissues This figure shows the benefits of a near infrared marker like Cyanine 5 (excited at

633 nm) compared to Fluorescein IsoThioCyanate (excited at 488 nm) A Auto fluorescence under laser excitation at 488 nm We observe that prostate concretions

(calcified material) respond to this excitation by a strong fluorescence signal B Auto fluorescence under laser excitation at 633 nm The use of Cyanine 5 reduces

widely autofluorescence of some structures such as prostate concretions (times20 magnification)

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

8 S Voros et al IRBM xxx (2013) xxxndashxxx

Concerning the registration between laparoscopic and ultra-353

sound images we performed several testbench evaluations of354

the registration algorithm on chicken breasts The result of such355

a registration obtained by manually pairing passive markers356

visible on both the laparoscopic and ultrasound imaging modal-357

ities are presented on Fig 8 In this example we obtained a358

registration root mean square error (RMS) of 038 mm359

43 Augmented laparoscopy thanks to fluorescence360

imaging361

First experiments were performed on leaves to demonstrate362

the feasibility of the detection of chlorophyll autofluorescence363

which has optical properties very similar to PpIX (absorption at364

405 nm and emission at 680 nm) as described on Fig 9 Then365

experiments were conducted on histology slices of the human366

prostate with as we anticipated inconclusive results because367

of the expected low signal-to-noise ratio caused by the insuffi-368

cient thickness of a histology slice (3 m thickness) compared to369

the optimal use of the device (300 m thickness) These results370

motivated our biomedical research COPROST on macroscopic371

fresh prostate samples372

Concerning the immunofluorescence protocol we were able373

to demonstrate the specificity of the anti-PSMA antibody on374

fixed tissue samples labelled with FITC as illustrated by Fig 10375

We observed higher autofluorescence problems when excit-376

ing the sample at the FITC specific wavelength compared to377

an excitation at the Cyanine 5 specific wavelength (Fig 11)378

These results motivated the inclusion of both fluorophores in our379

biomedical research COPROST on macroscopic fresh prostate380

samples FITC because it was already used to demonstrate speci-381

ficity of anti-PSMA antibody and Cyanine 5 because its near382

infra-red characteristics make it a better candidate on fresh mate-383

rial (higher penetration in tissue) and will improve signalnoise384

ratio as previously demonstrated385

To our knowledge most of the optical biopsy approaches as386

previously defined have been validated either on histologic slices387

or on ratmice prostate models However these approaches have388

limitations prostatic tissue is impaired by the chemical treat-389

ment required for histologic preparation and animal models do390

not directly mimic all aspects of human prostate cancer [11] In391

short it has not been proved yet that a successful detection with392

the investigated devices on histopathology slices or on animal393

models is sufficient to guarantee a successful detection on fresh394

human tissues The experiments that will be performed in the395

frame of the COPROST protocol in 2013 will be determinant to396

confirm the feasibility of our approach on fresh tissue samples397

5 DiscussionConclusion398

The DEPORRA project allowed for the development of inno-399

vative devices and navigation prototypes in the objective of400

allowing the surgeon to see ldquobeyond the visiblerdquo during a radi-401

cal prostatectomy First evaluations of the devices and methods402

have been performed preclinically and are very encouraging for403

the development of innovative approaches to assist the surgeon

during such a complex surgery Our preliminary results have also 404

shown the limits of preclinical validation and have convinced us 405

of the necessity to launch biomedical researches that will allow 406

us to validate further the devices This process required a con- 407

sequent amount of effort and time but is mandatory for the 408

fine-tuning of our tissue characterization tools and to obtain the 409

first clinical proofs of the relevance of the developed medical 410

devices which are mandatory to perform clinical evaluations of 411

the complete navigation systems 412

We now need to push further the integration of the devices 413

into such navigation systems in order to exploit at best the infor- 414

mation provided by each modality and to determine the optimal 415

approach for displaying the relevant information to the surgeon 416

in a clinical environment This will imply a conception and 417

development effort a definition of qualitative and quantitative 418

parameters for the first assessment of the delivered medical ben- 419

efit of our developments risks analysis and biomedical research 420

preparations in order to meet the ethical instances requirements 421

Acknowledgments 422

This work has been supported by French National Research 423

Agency (ANR) through TecSan program (project DEPORRA 424

no ANR-09-TECS-006) 425

References 426

[1] Heidenreich A Bolla M Joniau S Mason MD Matveev V Mottet N et al 427

Guidelines on prostate cancer Eur Assoc Urol 2011 [www uroweb org] 428

[2] Pfitzenmaier J Pahernik S Tremmel T Haferkamp A Buse S Hohen- 429

fellner M Positive surgical margins after radical prostatectomy do 430

they have an impact on biochemical or clinical progression BJU Int 431

2008102(10)1413ndash8 432

[3] Soler L Nicolau S Schmid J Koehl C Marescaux J et al Virtual reality 433

and augmented reality in digestive surgery In Third IEEE and ACM Inter- 434

national Symposium on Mixed and Augmented Reality (ISMAR) 2004 435

p 278ndash9 436

[4] S Voros B Tamadazte P Cinquin C Fouard Systegraveme drsquoimagerie multi- 437

vision pour chirurgie laparoscopique demande de deacutepocirct de brevet FR 438

1259489 102012 439

[5] Ukimura O Ahlering TE Gill IS Transrectal ultrasound-guided 440

energy-free nerve-sparing laparoscopic radical prostatectomy J Endourol 441

200822(9)1993ndash5 442

[6] Long JA Lee BH Guillotreau J Autorino R Laydner H Yakoubi R et al 443

Real-time robotic transrectal ultrasound navigation during robotic radical 444

prostatectomy initial clinical experience Urology 201280(3)608ndash13 445

[7] Holmes III DR Davis BJ Bruce CJ Robb RA 3D visualization analysis 446

and treatment of the prostate using trans-urethral ultrasound Comput Med 447

Imaging Graph 200327339ndash49 [2003] 448

[8] Dementhon D Model-based object pose in 25 lines of code Int J Comput 449

Vis 199515(1ndash2)123ndash41 450

[9] Custillon G Voros S Cinquin P Nguyen-Dinh A Moreau-Gaudry A 451

Bidimensional localization of active ultrasound transducers for use in 452

laparoscopic prostate surgery IEEE Trans Med Imaging 2012 Q3 453

[10] Wolf R Duchateau J Cinquin P Voros S 3D tracking of laparoscopic 454

instruments using statistical and geometric modeling Med Image Com- 455

put Comput Assist Interv 20116891203ndash10 [Lecture notes in computer 456

science] 457

[11] Valkenburg KC Williams BO ldquoMouse models of prostate cancerrdquo Prostate 458

Cancer 2011201122 httpdxdoiorg1011552011895238 [Article ID 459

895238] 460

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

S Voros et al IRBM xxx (2013) xxxndashxxx 7

Fig 9 Preliminary experiments of the autofluorescence detection with our testbench The autofluorescence of chlorophyll which has optical properties very similar

to Protoporphyrin IX was observed on our testbench Left direct image of a leaf right fluorescence image of the leaf at 680 nm As expected chlorophyll seems

absent in the veins

Fig 10 Specific fluorescence signal of anti-prostate-specific membrane antigen marker A Nuclear counter-staining of prostatic tissue with Hoechst (blue DNA

marker) We observe the nuclei of prostate cells regularly organized in the glandular epithelium Prostatic glands are separated by stroma B Fluorescence emission

of anti-prostate-specific membrane antigen-Fluorescein IsoThioCyanate on cellular membranes of prostatic cells C Overlay of the A and B images This figure

demonstrates the specificity on fixed prostatic tissue of a specific anti-PSMA antibody labelled with Fluorescein IsoThioCyanate We can also detect the contribution

of non-specific autofluorescence of prostatic concretions (Fig 11) (times20 magnification)

localization on a chicken breast to have more realistic condi-343

tions In both cases we were able to localize the active transducer344

with a precision less than 625 m Moreover the computation345

time is a few hundred milliseconds for a 2D localization These346

results demonstrate that the localization is fast enough to be347

implemented for a prostatectomy For the moment only a 2D 348

localization has been made with the active landmarks and the 349

3D localization is in progress Then we will be able to per- 350

form a registration with laparoscopic images using ldquoactiverdquo 351

markers 352

Fig 11 Non-specific autofluorescence observed in the prostate fixed tissues This figure shows the benefits of a near infrared marker like Cyanine 5 (excited at

633 nm) compared to Fluorescein IsoThioCyanate (excited at 488 nm) A Auto fluorescence under laser excitation at 488 nm We observe that prostate concretions

(calcified material) respond to this excitation by a strong fluorescence signal B Auto fluorescence under laser excitation at 633 nm The use of Cyanine 5 reduces

widely autofluorescence of some structures such as prostate concretions (times20 magnification)

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

8 S Voros et al IRBM xxx (2013) xxxndashxxx

Concerning the registration between laparoscopic and ultra-353

sound images we performed several testbench evaluations of354

the registration algorithm on chicken breasts The result of such355

a registration obtained by manually pairing passive markers356

visible on both the laparoscopic and ultrasound imaging modal-357

ities are presented on Fig 8 In this example we obtained a358

registration root mean square error (RMS) of 038 mm359

43 Augmented laparoscopy thanks to fluorescence360

imaging361

First experiments were performed on leaves to demonstrate362

the feasibility of the detection of chlorophyll autofluorescence363

which has optical properties very similar to PpIX (absorption at364

405 nm and emission at 680 nm) as described on Fig 9 Then365

experiments were conducted on histology slices of the human366

prostate with as we anticipated inconclusive results because367

of the expected low signal-to-noise ratio caused by the insuffi-368

cient thickness of a histology slice (3 m thickness) compared to369

the optimal use of the device (300 m thickness) These results370

motivated our biomedical research COPROST on macroscopic371

fresh prostate samples372

Concerning the immunofluorescence protocol we were able373

to demonstrate the specificity of the anti-PSMA antibody on374

fixed tissue samples labelled with FITC as illustrated by Fig 10375

We observed higher autofluorescence problems when excit-376

ing the sample at the FITC specific wavelength compared to377

an excitation at the Cyanine 5 specific wavelength (Fig 11)378

These results motivated the inclusion of both fluorophores in our379

biomedical research COPROST on macroscopic fresh prostate380

samples FITC because it was already used to demonstrate speci-381

ficity of anti-PSMA antibody and Cyanine 5 because its near382

infra-red characteristics make it a better candidate on fresh mate-383

rial (higher penetration in tissue) and will improve signalnoise384

ratio as previously demonstrated385

To our knowledge most of the optical biopsy approaches as386

previously defined have been validated either on histologic slices387

or on ratmice prostate models However these approaches have388

limitations prostatic tissue is impaired by the chemical treat-389

ment required for histologic preparation and animal models do390

not directly mimic all aspects of human prostate cancer [11] In391

short it has not been proved yet that a successful detection with392

the investigated devices on histopathology slices or on animal393

models is sufficient to guarantee a successful detection on fresh394

human tissues The experiments that will be performed in the395

frame of the COPROST protocol in 2013 will be determinant to396

confirm the feasibility of our approach on fresh tissue samples397

5 DiscussionConclusion398

The DEPORRA project allowed for the development of inno-399

vative devices and navigation prototypes in the objective of400

allowing the surgeon to see ldquobeyond the visiblerdquo during a radi-401

cal prostatectomy First evaluations of the devices and methods402

have been performed preclinically and are very encouraging for403

the development of innovative approaches to assist the surgeon

during such a complex surgery Our preliminary results have also 404

shown the limits of preclinical validation and have convinced us 405

of the necessity to launch biomedical researches that will allow 406

us to validate further the devices This process required a con- 407

sequent amount of effort and time but is mandatory for the 408

fine-tuning of our tissue characterization tools and to obtain the 409

first clinical proofs of the relevance of the developed medical 410

devices which are mandatory to perform clinical evaluations of 411

the complete navigation systems 412

We now need to push further the integration of the devices 413

into such navigation systems in order to exploit at best the infor- 414

mation provided by each modality and to determine the optimal 415

approach for displaying the relevant information to the surgeon 416

in a clinical environment This will imply a conception and 417

development effort a definition of qualitative and quantitative 418

parameters for the first assessment of the delivered medical ben- 419

efit of our developments risks analysis and biomedical research 420

preparations in order to meet the ethical instances requirements 421

Acknowledgments 422

This work has been supported by French National Research 423

Agency (ANR) through TecSan program (project DEPORRA 424

no ANR-09-TECS-006) 425

References 426

[1] Heidenreich A Bolla M Joniau S Mason MD Matveev V Mottet N et al 427

Guidelines on prostate cancer Eur Assoc Urol 2011 [www uroweb org] 428

[2] Pfitzenmaier J Pahernik S Tremmel T Haferkamp A Buse S Hohen- 429

fellner M Positive surgical margins after radical prostatectomy do 430

they have an impact on biochemical or clinical progression BJU Int 431

2008102(10)1413ndash8 432

[3] Soler L Nicolau S Schmid J Koehl C Marescaux J et al Virtual reality 433

and augmented reality in digestive surgery In Third IEEE and ACM Inter- 434

national Symposium on Mixed and Augmented Reality (ISMAR) 2004 435

p 278ndash9 436

[4] S Voros B Tamadazte P Cinquin C Fouard Systegraveme drsquoimagerie multi- 437

vision pour chirurgie laparoscopique demande de deacutepocirct de brevet FR 438

1259489 102012 439

[5] Ukimura O Ahlering TE Gill IS Transrectal ultrasound-guided 440

energy-free nerve-sparing laparoscopic radical prostatectomy J Endourol 441

200822(9)1993ndash5 442

[6] Long JA Lee BH Guillotreau J Autorino R Laydner H Yakoubi R et al 443

Real-time robotic transrectal ultrasound navigation during robotic radical 444

prostatectomy initial clinical experience Urology 201280(3)608ndash13 445

[7] Holmes III DR Davis BJ Bruce CJ Robb RA 3D visualization analysis 446

and treatment of the prostate using trans-urethral ultrasound Comput Med 447

Imaging Graph 200327339ndash49 [2003] 448

[8] Dementhon D Model-based object pose in 25 lines of code Int J Comput 449

Vis 199515(1ndash2)123ndash41 450

[9] Custillon G Voros S Cinquin P Nguyen-Dinh A Moreau-Gaudry A 451

Bidimensional localization of active ultrasound transducers for use in 452

laparoscopic prostate surgery IEEE Trans Med Imaging 2012 Q3 453

[10] Wolf R Duchateau J Cinquin P Voros S 3D tracking of laparoscopic 454

instruments using statistical and geometric modeling Med Image Com- 455

put Comput Assist Interv 20116891203ndash10 [Lecture notes in computer 456

science] 457

[11] Valkenburg KC Williams BO ldquoMouse models of prostate cancerrdquo Prostate 458

Cancer 2011201122 httpdxdoiorg1011552011895238 [Article ID 459

895238] 460

Please cite this article in press as Voros S et al Devices and systems targeted towards augmented robotic radical prostatectomy IRBM (2013)

httpdxdoiorg101016jirbm201301014

ARTICLE IN PRESS+ModelIRBM 252 1ndash8

8 S Voros et al IRBM xxx (2013) xxxndashxxx

Concerning the registration between laparoscopic and ultra-353

sound images we performed several testbench evaluations of354

the registration algorithm on chicken breasts The result of such355

a registration obtained by manually pairing passive markers356

visible on both the laparoscopic and ultrasound imaging modal-357

ities are presented on Fig 8 In this example we obtained a358

registration root mean square error (RMS) of 038 mm359

43 Augmented laparoscopy thanks to fluorescence360

imaging361

First experiments were performed on leaves to demonstrate362

the feasibility of the detection of chlorophyll autofluorescence363

which has optical properties very similar to PpIX (absorption at364

405 nm and emission at 680 nm) as described on Fig 9 Then365

experiments were conducted on histology slices of the human366

prostate with as we anticipated inconclusive results because367

of the expected low signal-to-noise ratio caused by the insuffi-368

cient thickness of a histology slice (3 m thickness) compared to369

the optimal use of the device (300 m thickness) These results370

motivated our biomedical research COPROST on macroscopic371

fresh prostate samples372

Concerning the immunofluorescence protocol we were able373

to demonstrate the specificity of the anti-PSMA antibody on374

fixed tissue samples labelled with FITC as illustrated by Fig 10375

We observed higher autofluorescence problems when excit-376

ing the sample at the FITC specific wavelength compared to377

an excitation at the Cyanine 5 specific wavelength (Fig 11)378

These results motivated the inclusion of both fluorophores in our379

biomedical research COPROST on macroscopic fresh prostate380

samples FITC because it was already used to demonstrate speci-381

ficity of anti-PSMA antibody and Cyanine 5 because its near382

infra-red characteristics make it a better candidate on fresh mate-383

rial (higher penetration in tissue) and will improve signalnoise384

ratio as previously demonstrated385

To our knowledge most of the optical biopsy approaches as386

previously defined have been validated either on histologic slices387

or on ratmice prostate models However these approaches have388

limitations prostatic tissue is impaired by the chemical treat-389

ment required for histologic preparation and animal models do390

not directly mimic all aspects of human prostate cancer [11] In391

short it has not been proved yet that a successful detection with392

the investigated devices on histopathology slices or on animal393

models is sufficient to guarantee a successful detection on fresh394

human tissues The experiments that will be performed in the395

frame of the COPROST protocol in 2013 will be determinant to396

confirm the feasibility of our approach on fresh tissue samples397

5 DiscussionConclusion398

The DEPORRA project allowed for the development of inno-399

vative devices and navigation prototypes in the objective of400

allowing the surgeon to see ldquobeyond the visiblerdquo during a radi-401

cal prostatectomy First evaluations of the devices and methods402

have been performed preclinically and are very encouraging for403

the development of innovative approaches to assist the surgeon

during such a complex surgery Our preliminary results have also 404

shown the limits of preclinical validation and have convinced us 405

of the necessity to launch biomedical researches that will allow 406

us to validate further the devices This process required a con- 407

sequent amount of effort and time but is mandatory for the 408

fine-tuning of our tissue characterization tools and to obtain the 409

first clinical proofs of the relevance of the developed medical 410

devices which are mandatory to perform clinical evaluations of 411

the complete navigation systems 412

We now need to push further the integration of the devices 413

into such navigation systems in order to exploit at best the infor- 414

mation provided by each modality and to determine the optimal 415

approach for displaying the relevant information to the surgeon 416

in a clinical environment This will imply a conception and 417

development effort a definition of qualitative and quantitative 418

parameters for the first assessment of the delivered medical ben- 419

efit of our developments risks analysis and biomedical research 420

preparations in order to meet the ethical instances requirements 421

Acknowledgments 422

This work has been supported by French National Research 423

Agency (ANR) through TecSan program (project DEPORRA 424

no ANR-09-TECS-006) 425

References 426

[1] Heidenreich A Bolla M Joniau S Mason MD Matveev V Mottet N et al 427

Guidelines on prostate cancer Eur Assoc Urol 2011 [www uroweb org] 428

[2] Pfitzenmaier J Pahernik S Tremmel T Haferkamp A Buse S Hohen- 429

fellner M Positive surgical margins after radical prostatectomy do 430

they have an impact on biochemical or clinical progression BJU Int 431

2008102(10)1413ndash8 432

[3] Soler L Nicolau S Schmid J Koehl C Marescaux J et al Virtual reality 433

and augmented reality in digestive surgery In Third IEEE and ACM Inter- 434

national Symposium on Mixed and Augmented Reality (ISMAR) 2004 435

p 278ndash9 436

[4] S Voros B Tamadazte P Cinquin C Fouard Systegraveme drsquoimagerie multi- 437

vision pour chirurgie laparoscopique demande de deacutepocirct de brevet FR 438

1259489 102012 439

[5] Ukimura O Ahlering TE Gill IS Transrectal ultrasound-guided 440

energy-free nerve-sparing laparoscopic radical prostatectomy J Endourol 441

200822(9)1993ndash5 442

[6] Long JA Lee BH Guillotreau J Autorino R Laydner H Yakoubi R et al 443

Real-time robotic transrectal ultrasound navigation during robotic radical 444

prostatectomy initial clinical experience Urology 201280(3)608ndash13 445

[7] Holmes III DR Davis BJ Bruce CJ Robb RA 3D visualization analysis 446

and treatment of the prostate using trans-urethral ultrasound Comput Med 447

Imaging Graph 200327339ndash49 [2003] 448

[8] Dementhon D Model-based object pose in 25 lines of code Int J Comput 449

Vis 199515(1ndash2)123ndash41 450

[9] Custillon G Voros S Cinquin P Nguyen-Dinh A Moreau-Gaudry A 451

Bidimensional localization of active ultrasound transducers for use in 452

laparoscopic prostate surgery IEEE Trans Med Imaging 2012 Q3 453

[10] Wolf R Duchateau J Cinquin P Voros S 3D tracking of laparoscopic 454

instruments using statistical and geometric modeling Med Image Com- 455

put Comput Assist Interv 20116891203ndash10 [Lecture notes in computer 456

science] 457

[11] Valkenburg KC Williams BO ldquoMouse models of prostate cancerrdquo Prostate 458

Cancer 2011201122 httpdxdoiorg1011552011895238 [Article ID 459

895238] 460