Automatic stent strut detection in intravascular OCT images using image processing and classification technique

Hong Lu a, Madhusudhana Gargesha a, Zhao Wang a, Daniel Chamie b, Guilherme F. Attizzani b, Tomoaki Kanaya b, Soumya Ray c, Marco A. Costa b, Andrew M. Rollins a, Hiram G. Bezerra b, and

David L. Wilson a,d,*

aDepartment of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, 44106, USA; bCardiovascular Imaging Core Laboratory, Harrington Heart & Vascular Institute, University Hospitals Case Medical Center, Cleveland, OH, 44106, USA; cDepartment of Electrical Engineering & Computer Science, Case Western Reserve University, Cleveland, OH, 44106, USA; dDepartment

of Radiology, Case Western Reserve University, Cleveland, OH, 44106, USA

*david.wilson@case.edu

ABSTRACT

Intravascular OCT (iOCT) is an imaging modality with ideal resolution and contrast to provide accurate in vivo assessments of tissue healing following stent implantation. Our Cardiovascular Imaging Core Laboratory has served >20 international stent clinical trials with >2000 stents analyzed. Each stent requires 6-16hrs of manual analysis time and we are developing highly automated software to reduce this extreme effort. Using classification technique, physically meaningful image features, forward feature selection to limit overtraining, and leave-one-stent-out cross validation, we detected stent struts. To determine tissue coverage areas, we estimated stent “contours” by fitting detected struts and interpolation points from linearly interpolated tissue depths to a periodic cubic spline. Tissue coverage area was obtained by subtracting lumen area from the stent area. Detection was compared against manual analysis of 40 pullbacks. We obtained recall = 90±3% and precision = 89±6%. When taking struts deemed not bright enough for manual analysis into consideration, precision improved to 94±6%. This approached inter-observer variability (recall = 93%, precision = 96%). Differences in stent and tissue coverage areas are 0.12 ± 0.41 mm2 and 0.09 ± 0.42 mm2, respectively. We are developing software which will enable visualization, review, and editing of automated results, so as to provide a comprehensive stent analysis package. This should enable better and cheaper stent clinical trials, so that manufacturers can optimize the myriad of parameters (drug, coverage, bioresorbable versus metal, etc.) for stent design.

Keywords: image processing, classification, stent detection, machine learning, intravascular OCT

1. INTRODUCTION Every year, more than 2 million people receive stent implantation by means of percutaneous coronary intervention (PCI) as a treatment of coronary artery disease (CAD). Various stent types have been developed to improve the efficacy of stent treatment. Although drug eluting stent (DES) has been proved to reduce restenosis compared to bare metal stent (BMS) 1, it is associated with late stent thrombosis (LST). Although infrequent, LST carries a mortality rate of up to 45% and a nonfatal infarction rate of 30% to 40%. Pathological studies have suggested that the absence of stent strut coverage due to delayed vascular healing is a potential surrogate metric for risk of stent thrombosis. Thus, the optimization of stent design parameters such as material, drug, mechanical design, and coating is critical for improved treatment for CAD. In addition to stent device trials, there is a need to provide analysis for treatment decisions such as second intervention and drug management.

To optimize stent designs and improve cardiovascular disease treatment, sensitive, in vivo assessments are needed for serial preclinical studies and for clinical evaluations. Intravascular Optical Coherence Tomography (iOCT) is the only imaging modality with the resolution, contrast, and speed to visualize fine luminal architecture and vessel wall response after stent implantation2. Strut tissue coverage as assessed by iOCT has become an important surrogate biomarker of stent viability3. The Cardiovascular Imaging Core Lab in the Harrington Heart & Vascular Institute, University Hospitals Case Medical Center, Cleveland, Ohio, hereafter called the Core Lab, has provided iOCT image analysis service to >20 international trials of stent devices. Manual analysis of iOCT image pullbacks is time consuming (6-16 hrs per stent), limiting the size and number of stent trial studies.

Medical Imaging 2013: Computer-Aided Diagnosis, edited by Carol L. Novak, Stephen Aylward, Proc. of SPIE Vol. 8670, 867015 · © 2013 SPIE · CCC code: 1605-7422/13/$18 · doi: 10.1117/12.2007183

Proc. of SPIE Vol. 8670 867015-1

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/12/2014 Terms of Use: http://spiedl.org/terms

We developed highly automated software to detect stent struts in iOCT images and to analyze neointima hyperplasia (NIH). A machine learning, classification approach was employed to avoid bias that appears with the use of manually developed image processing heuristics. Multiple physically meaningful image features were extracted from a large number of manually analyzed struts to train a robust classifier. Forward feature selection was performed to identify the most discriminative features and to limit overtraining. We investigated a few classification methods including logistic regression (LR), support vector machine (SVM), single decision tree, and bagged decision trees. Bagged decision trees gave significantly better results compared to the others. To measure NIH area, the stent contour was reconstructed by fitting detected struts and interpolation points from linearly interpolated tissue depths to a periodic cubic spline. Inter-analyst variability was analyzed to set a benchmark for software acceptance.

2. METHOD 2.1 Materials We analyzed 20 pullbacks acquired at 0-3 months after implantation and 20 pullbacks at 12-18 months after implantation. Images were collected by a Fourier-Domain OCT (FD-OCT) system (C7-XRTM OCT Intravascular Imaging System, St. Jude Medical, St. Paul, Minnesota). The system was equipped with a tunable laser light source sweeping from 1250 nm to1370 nm, providing 15-μm resolution along the A-line. Pullback speed was 20 mm/sec over a distance of 54.2 mm, and the interval between frames was 200μm, giving 271 total frames. Stents were imaged over 100 to 200 frames, depending upon the length of the stent. Manual ground truth data were obtained from expert analysts in the Core Lab. In a subset of data, three expert cardiologists used manual segmentation tools in Amira (www.visageimaging.com) to annotate stent struts from 3 baseline and 3 follow-up cases. These 6 cases were used to analyze inter-analyst variability, so as to provide a benchmark for software performance. Additional cases were annotated by single cardiologist analysts using the image analysis software integrated in the imaging system. 2.2 Image analysis algorithms In iOCT images, stent struts often give a bright reflection with a shadow behind it. In other cases, because of the orientation of the strut wires, only the shadow is evident. We call these bright, analyzable struts and non-bright struts, respectively. These definitions are consistent with manual analysis in the Core Lab. With non-bright struts, since there is some ambiguity as to the location of the strut, they are not used to measure strut-level tissue coverage in the Core Lab. Nevertheless, analysts in the Core Lab identify non-bright struts to help determine the stent 2D contour for area measurements. Below, we describe our method for detecting bright analyzable struts.

The proposed stent strut detection algorithm consists of 6 steps. (1) detect the expanded lumen boundary; (2) detect A-lines containing a shadow; (3) detect bright spots; logical AND of steps 1-3 giving candidate struts; (4) compute features from candidate struts; (5) classify candidate struts as either struts or else, using a bagged decision trees classifier trained on a large dataset; and (6) eliminate extra hits using a simple rule. All processing is done on polar coordinate (r,θ) iOCT images. This view is geometrically transformed to create the anatomical (x, y) view for visualization.

In the first 3 steps, candidate struts were identified using multiple image processing techniques. Since the struts always lie within certain distance to the lumen boundary, in Step 1, lumen boundary is detected automatically using dynamic programming4. Briefly, in a polar (r,θ) coordinate, we detect edges along r and then use dynamic programming to find the lumen contour having the highest cumulative edge strength from top to bottom along θ. A lumen ROI is formed by expanding the lumen boundary to certain width where it is possible to obtain a strut. Considering that struts are usually brighter than their neighborhood tissue, in Step 2, we used a morphological extended maxima detection algorithm5 to get groups of pixels brighter than their neighborhoods. Step 3 is shadow detection based upon angular intensity distribution. Along every A-line, we averaged the intensity for a predetermined number of pixels after the lumen border, creating a 1D plot of mean values. We then detected the extended minima5 of the 1D intensity profile to determine A-lines having a shadow and generate a shadow mask. Logical AND of the lumen ROI, bright spots and the shadow mask gives the candidate struts, including almost all the true struts and a large number of purposely over-called false positives(FPs). Figure 1 shows the process of candidate strut detection.

Step 4 is feature extraction from candidate struts. Features are listed in table 1. Features can be divided into 3 categories: strut region features, shadow region features and combinations of the two regions. Since struts are usually very bright and shadows are dark, intensity statistics of the two regions should be very informative (features 1-5, 9-16). Solidity and area (feature 6 and 7) are used to capture the shape characteristics of a strut. Because distance to catheter (feature 8) affects lateral resolution, struts far from the catheter have very narrow shadows with higher intensity compared to shadows of struts close to the catheter. Combination features (feature 17-19) ensure that shadow begins

Proc. of SPIE Vol. 8670 867015-2

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/12/2014 Terms of Use: http://spiedl.org/terms

Lurrby D1

EIi

right spot

detection

I\IVA

III

IIUIUdLC a

i

right after thare bolded in

*

Step 5 isreported in classificationdecision treedata and imp“double hits”when there wfrom extra reof pixels, lea

To obtaincontour is obgave a satisfyWhen there

he strut. We pen Table 1.

Strut Region F1. Maximum I2. Minimum In3. Mean Inten4. Median Inte5. Intensity Var6. Solidity 7. Area 8. Distance to

*Features in bold

s classificationthe literature

n. A bagged ds are trained oproves the stab” (Step 6). Oc

was a bright refeflection echoeaving final deten tissue coverabtained by smoying estimationisn’t any strut

erform a forwa

Features ShIntensity 9. Mntensity 10. Mnsity 11. Mensity 12. Mriance 13. I

14. P15. M

catheter 16. Sd letters are chos

n of candidate to be less se

decision trees on each replica bility of the clccasionally, a sflection alongses. In some fewected struts. ge area, we coothly connectinn of stent shapdetected in ha

ard feature sele

Figure 1. C

Table 1. Feahadow Region F

Maximum IntensiMinimum IntenMean IntensityMedian IntensityIntensity VariaPercentage of DMean of Dark AShadow depthsen by feature se

struts using bnsitive to noiclassifier creato create an en

lassifier7,8. Fosecond strut wide the shadow

w instances, ex

ompute the diffng detected stre9,10. Neverthealf of the fram

ection techniqu

Candidate strut

atures for classiFeatures ity 17nsity 18

19y

ance Dark Area Area

lection process a

bagged decisionse than stand

ates bootstrappnsemble. Bagg

ollowing classiwas found alonw, due to a reflextra bright spot

ference in arearuts with a neareless, problems

me, we omit the

ue to find the m

detection.

ification Combin

7. Difference bet8. Slope of Inten9. Percentage of

and used for clas

n trees. This idard decision tped replicas ofging reduces thification, we fung the shadow ecting object suts were pruned

s bound by therly circular curs occur when toe frame. When

most discrimin

nation Features tween Two Max

nsity Profile f Dark Pixels alo

ssification.

is a popular ctrees, giving if the training

he variance of further process

of another. Tuch as a foam

d by keeping on

e stent and lumrve. A 2D periooo few struts an there is no st

native features

xima

ong Slope

classification teimproved accudata set and

noisy predictioed results to eypically, this ocell or calcificnly the brighte

men contours. Todic cubic spliare detected in trut in a quarte

s6, which

echnique uracy of separate

ons from eliminate occurred

cation, or est group

The stent ine curve a frame. er of the

Proc. of SPIE Vol. 8670 867015-3

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/12/2014 Terms of Use: http://spiedl.org/terms

.

: ;,. .

:: .

Ïfíi

frame, we austruts in the n2.3 TrainingLeave-one-stimplantation(TP+FN), is (TP+FP), is tby analysts ismall bright compare resu

3.1 Strut det

Figure 2 sho2B). All FPs image quality

Figure 2.following

Figure 3contrast.

utomatically aneighborhood.

g and validatiotent-out cross v, respectively. a measure of tthe percentageis subjective. spot captured

ults against the

tection / classi

ows results of were removed

y is inferior du

. Detection of bg Steps 1-3, inclu

. Strut detectio. Each image in t

dd an “interpo

on experimentvalidation wasRecall (RC) a

the percentage of correctly dWhile validatby the autom

e aggregate of b

ification valida

strut detectiond after classificue to blood resi

bright, analyzabuding many FPs

on results in imthe bottom row s

olation point”

t

s performed acand precision (P

of correctly ddetected struts oting against br

mated detectionbright and non-

3. ation

n. Detection of cation (Figure 2idue, thick tissu

ble struts. (A) In. (C) Struts follo

ages with infershows detected s

by linearly int

cross 20 cases PR) are calculetected struts oof all the prediight, analyzab

n, but not defin-bright struts a

RESULTS

f candidate stru2C). Figure 3 sue coverage, an

nput image fromowing classificat

ior quality duestruts (red) from

terpolating the

at 0-3 monthslated to evaluaof all the true, icted struts. Thle struts, manyned as bright

and calculate an

uts in Steps 1-shows that strund low contras

m a baseline casetion with all FPs

e to blood residm the top row ima

e distances to

s and 20 casesate algorithm p

manually annhe call of brighy FPs actuallystruts by analyn actual precisi

-3 gave struts uts can be robust.

. (B) Detection os removed.

due, thick tissueage in the corres

lumen of the

s at 12-18 monperformance. Rotated struts. P

ht and non-brigy have shadowysts, therefore ion (PRa).

and many FPsustly detected w

of “candidate str

e coverage, andsponding column

detected

nths post RC = TP/ PR = TP/ ght struts

ws with a we also

s (Figure when the

ruts”

low n.

Proc. of SPIE Vol. 8670 867015-4

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/12/2014 Terms of Use: http://spiedl.org/terms

Table 2 sapproaches iautomatic de

3.2 Tissue co

Figure 4 shocorrect stent stent contourby automaticshows that taccuracy of t

Figure 4struts (grpoints (bl

AutomateAltman plotsdata across a

summarizes deinter-analyst a

etection perform

RCPRPRa

overage measu

ws the construcontours can b

rs are usually ically adding “ithe estimated this estimation

. Stent contourreen) is detectedlue) in images w

ed stent and ts (Figures 5&6all areas, differe

etection statistiagreement. Thmance and inte

Tabl

Automadetecti

C 90% ±3R 91% ±3a 94% ±3

urement valid

uction of stent be easily obtainaccurate onlyinterpolation pcontour goes method.

formation. In td. In the bottomwith few struts de

tissue coverag6). We used 95ences are 0.12

ics of the softwhe range of der-analyst agreele 2. Stent strut

0-3 months casatic on

Ana2 VS 1

3% 94% 3% 96% 3% 98%

dation

contour using ined (top row)y using the deteoints” based othrough the b

the top row, goom row, stent contetected. Without

e area measur50 image fram± 0.41 mm2 an

ware and inter-difference in rement of “analydetection evalu

ses alysts A

3 VS 1 95% 999% 899% 9

periodic cubic. In frames wiected struts. H

on strut depthsbeginning of s

od stent contourstours are correcthese interpolat

rements were mes containing nd 0.09 ± 0.42

-analyst agreemrecall and preysts 2 VS 1” a

uation statistics12-18 mo

Automatic detection 2 90% ±3% 989% ±6% 994% ±6% 9

c spline. With ith insufficient

However, the cos in the tissue shadows of no

s (red) are reconctly retrieved bytion points, the s

compared agaat least 1 strutmm2 for stent

ment. The perecision was 0-and “analysts 3 s onths cases

Analysts VS 1 3 VS 93% 97%97% 98%97% 98%

sufficient numt number of brontours can be (bottom row).

on-bright struts

nstructed when sy automatically stent contours wi

ainst manual at in each half and tissue area

formance of a-5% when coVS 1.”

1 % % %

mber of struts dright, analyzab

successfully cThe bottom le

s, indicating t

sufficient numbeadding interpolaill be in error.

assessments inof a frame. Coas, respectively

algorithm omparing

detected, le struts,

corrected eft panel the good

er of ation

n Bland-ollapsing y.

Proc. of SPIE Vol. 8670 867015-5

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/12/2014 Terms of Use: http://spiedl.org/terms

Figure 5. Bland-Altman plot of stent area measurement

Figure 6. Bland-Altman plot of tissue coverage area measurement.

4. CONCLUSIONS The proposed stent strut detection algorithm was validated against a large dataset to demonstrate its generality and robustness. A precision of 90±3% and a recall of 89±6% were achieved, which were comparable to inter-analyst variability and results reported in previous papers11-14. Automated measurements of stent and tissue coverage area are promising. Differences between automated and manual measurements are 0.12 ± 0.41 mm2 for stent area and 0.09 ± 0.42 mm2 for tissue coverage area. Bland-Altman plots reveal some outlier frames corresponding to significant differences between automatic and manual contours. The difference is understood from the method for creating manual

2 4 6 8 10 12 14 16-3

-2

-1

0

1

2

3

4

5

mean of automatic and manual measurements

auto

mat

ic -

man

ual

Bland-Altman plot of stent area measurements

+2std

-2std

mean

-4 -3 -2 -1 0 1 2 3-2

-1

0

1

2

3

4

mean of automatic and manual measurements

auto

mat

ic -

man

ual

Bland-Altman plot of coverage area measurements

+2std

-2std

mean

Proc. of SPIE Vol. 8670 867015-6

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/12/2014 Terms of Use: http://spiedl.org/terms

contours. In addition to bright and non-bright struts, an analyst often added “interpolation points,” in frames with few struts. Manual interpolation points could depend upon frames before and after the current frame, a process not captured in our 2D method. Another source of error is lumen detection failure in some frames, which is rare but contributes to the outliers. We are developing comprehensive software to enable fast and convenient manual review and editing in the few frames with errors. Automated results can be improved using a 3D extension of the current method.

Our results indicate that machine learning classification is well suited to the problem of stent strut analysis. It allows us to optimize our algorithm using our large database of manually analyzed stent pullbacks, which is a great advantage over manually optimized algorithms. Bagged decision trees classifier was proved working better than logistic regression and support vector machine. It is relatively robust to noise and has better accuracy than a single decision tree. The forward feature selection approach successfully allowed us to remove redundant features and limit over training.

Our algorithms should greatly reduce analysis time as compared to the fully manual method currently used, which will make stent studies much more efficient and enable much larger studies. Unattended strut detection and area measurements using software without speed optimization is about 15 minutes for 100 frames. Although automated results are promising, we would advocate analyst review of every frame to guarantee reliable results. Visual review and editing should be relatively quick. FP struts will be removed by a simple click. Current manual method usually analyzes every third frame to save time, while we can analyze every frame efficiently with our software. With much more struts detected using automatic method, FNs are relatively unimportant unless they negatively affect the stent contour. With the completion of our comprehensive software, all downstream analyses (percentage of covered, uncovered and malapposed struts, NIH thickness, etc.) can be automated efficiently. Analyst time per stent should be very much reduced from the 6-16 hours now required. Repeatable analysis with standardized software should reduce variability and improve statistical power as compared to manually performed studies suffering intra and inter-analyst variability. Therefore, stent design comparison and longitudinal studies can produce more reliable results.

ACKNOWLEDGMENTS The project described was supported by National Heart, Lung, and Blood Institute through NIH R21HL108263 and by the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant UL1RR024989. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. HL was partially supported by the Chinese Government Scholarship. ZW was partially supported by the American Heart Association predoctoral fellowship (#11PRE7320034).

REFERENCES [1] G. Guagliumi, G. Musumeci, V. Sirbu, H. G. Bezerra, N. Suzuki, L. Fiocca, A. Matiashvili, N. Lortkipanidze, A.

Trivisonno, O. Valsecchi, G. Biondi-Zoccai, and M. A. Costa, "Optical Coherence Tomography Assessment of In Vivo Vascular Response After Implantation of Overlapping Bare-Metal and Drug-Eluting Stents," 6 3, 531-539 (2010).

[2] H. G. Bezerra, M. A. Costa, G. Guagliumi, A. M. Rollins, and D. I. Simon, "Intracoronary Optical Coherence Tomography: A Comprehensive Review Clinical and Research Applications," 6 2, 1035-1046 (2009).

[3] S. Tahara, D. Chamie, M. Baibars, C. Alraies, and M. Costa, "Optical coherence tomography endpoints in stent clinical investigations: strut coverage," 6 27, 271-287 (2011).

[4] Z. Wang, H. Kyono, H. G. Bezerra, H. Wang, M. Gargesha, C. Alraies, C. Xu, J. M. Schmitt, D. L. Wilson, M. A. Costa, and A. M. Rollins, "Semiautomatic segmentation and quantification of calcified plaques in intracoronary optical coherence tomography images," 6 15, 061711 (2010).

[5] Pierre Soille, "Geodesic Transformations" in Morphological Image Analysis: Principles and Applications, 2nd Ed., (Springer-Verlag, 2004).

[6] Lior Rokach and Oded Maimon, "Feature Selection" in Data Mining with Decision trees: Theory and Application, (World Scientific, 2008).

[7] P. Buhlmann and B. Yu, "Analyzing bagging," 6 30, 927-961 (2002). [8] L. Breiman, "Bagging predictors," 6 24, 123-140 (1996). [9] N.Y.Graham., "Smoothing With Periodic Cubic Splines," 6 62, 101-110 (1983). [10] E. T. Y. Lee, "Choosing Nodes in Parametric Curve Interpolation," 6 21, 363-370 (1989).

Proc. of SPIE Vol. 8670 867015-7

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/12/2014 Terms of Use: http://spiedl.org/terms

[11] G. Unal, S. Gurmeric, and S. G. Carlier, "Stent implant follow-up in intravascular optical coherence tomography images," 6 26, 809-816 (2010).

[12] C. Kauffmann, P. Motreff, and L. Sarry, "In Vivo Supervised Analysis of Stent Reendothelialization From Optical Coherence Tomography," 6 29, 807-818 (2010).

[13] G. J. Ughi, T. Adriaenssens, K. Onsea, P. Kayaert, C. Dubois, P. Sinnaeve, M. Coosemans, W. Desmet, and J. D'hooge, "Automatic segmentation of in-vivo intra-coronary optical coherence tomography images to assess stent strut apposition and coverage," 6 (2011).

[14] G. T. Bonnema, K. O. Cardinal, S. K. Williams, and J. K. Barton, "An automatic algorithm for detecting stent endothelialization from volumetric optical coherence tomography datasets," 6 53, 3083-3098 (2008).

Proc. of SPIE Vol. 8670 867015-8

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/12/2014 Terms of Use: http://spiedl.org/terms