FUZZY SEGMENTATION OF MASSES IN DIGITAL BREASTTOMOSYNTHESIS IMAGES BASED ON DYNAMIC

PROGRAMMING

Louis Apffel†, Giovanni Palma‡§, Serge Muller‡ and Isabelle Bloch§† Ecole Centrale Paris, Grande Voie des Vignes F-92 295 Chatenay-Malabry Cedex, France

‡ GE Healthcare, 283, rue de la Miniere, 78530 Buc, France§ Telecom ParisTech - CNRS LTCI, 46, rue Barrault, 75013 Paris, France

louis.apffel@centraliens.net,{giovanni.palma, serge.muller}@ge.com, isabelle.bloch@enst.fr

Keywords: Fuzzy segmentation, Fuzzy contours, Dynamic programming,Digital breast tomosynthesis, Computer aideddetection.

Abstract: In this paper we propose a new fuzzy segmentation method to segment lesions in Digital Breast Tomosynthesis(DBT) datasets. In the proposed approach we model a contour as a path in the image. The optimal contouris defined as the path associated with a minimal cost, which isderived from the image content. Using thisformalism we present several ways to alter this cost in orderto extract several relevant contours from a singleimage. The set of contours is then used in the fuzzy contour framework to perform mass detection. Themethod has been tested on synthetic data as well as images containing lesions and provides promising results.

1 INTRODUCTION

Digital Breast Tomosynthesis (DBT) is a new 3Dimaging technique (Dobbins III and Godfrey, 2003;Wu et al., 2003) that may potentially overcome somelimitations of standard mammography such as tissuesuperimposition (Gennaro et al., 2008). Such an im-provement is gained at the expense of an increasedamount of data to be reviewed by the radiologist. Inthis context the design of a Computer Aided Detec-tion (CAD) system may help the radiologist to keepa high sensitivity in his detection and characterizationtask. Segmentation is usually a crucial step in CADsystems, and a wrong segmentation can have a disas-trous impact on the whole detection scheme. The useof fuzzy sets avoids these drawbacks (Peters, 2007).This can be done by using the fuzzy contour frame-work, which is suitable to represent several possiblecontours for one specific structure. Unfortunately,only a small amount of techniques exist in the litera-ture (Bothorel et al., 1997; Peters, 2007; Palma et al.,2008) to extract such fuzzy contours and they sufferfrom various limitations. In this paper, we introduce anew fuzzy contour extraction procedure by extendinga crisp segmentation method originally proposed forstandard mammography images, which models con-tours as paths in an image (Timp and Karssemeijer,

2004).First, in Section 2 we present the limitations of

crisp segmentation methods. Then in Section 3, werecall the formalism of fuzzy contours. In Section 4,we describe the new proposed algorithm and we dis-cuss two aspects: uncertainty and imprecision, whichare both handled by our approach. Finally we illus-trate results on both synthetic and clinical data.

2 LIMITATIONS OF CRISPSEGMENTATION

In breast imaging, crisp segmentation has many lim-itations. Indeed, segmenting some masses may beparticularly difficult when the contrast between lesionand background of the image is low. Radiologists mayhave different and various opinions on the exact loca-tion of a contour. A segmentation algorithm that pro-poses a single answer to this kind of problem is usu-ally taking an arbitrary decision on the nature of thecontour. Since the shape of the contour is suggestiveof the nature of the lesion (benign or malignant), an-ticipating its nature from a single and inaccurate con-tour could lead to a classification error.

The problem is illustrated in Figure 1 where a

7

(a) (b)Figure 1: Crisp segmentation problem: several contours canfit a given lesion and may lead to different conclusions onits class.

spiculated mass is presented with two different seg-mentation results.

3 FUZZY SEGMENTATION

The fuzzy set framework seems to be suitable to dealwith the mass segmentation problem (Peters, 2007). Itallows defining the concept of fuzzy contours, whichcan overcome several limitations of regular segmen-tation techniques by allowing not to make a decisiontoo early.

Definition 1. A fuzzy contour is a set of nested crispcontours where each contour is associated with amembership degree to the classcontour.

3.1 Imprecision and Uncertainty

In DBT images, masses may present two types of im-perfections. First, a lesion may be hard to define (i.e.there are several possible contours that can fit the le-sion boundaries) even if we are sure of its presence.This is called imprecision. Secondly, it may some-times be difficult to detect and locate a mass. In thiscase we are dealing with uncertainty. A single fuzzycontour can handle the notion of imprecision becauseit considers several potential contours. On the otherhand, uncertainty can be handled by considering sev-eral fuzzy contours.

3.2 Existing Schemes to Extract FuzzyContours

In the literature, several techniques to extract fuzzycontours have been proposed. Originally, a multi-thresholding approach was designed in order to de-tect microcalcifications in mammography (Bothorelet al., 1997). This approach has also been used inorder to segment masses in projected images used to

reconstruct DBT volumes (Peters, 2007). Unfortu-nately, because of potential superimpositions of tis-sues in projection images, this approach is not wellsuited for masses in 2D projection images.

More recently, a segmentation scheme based onthe level set framework (Osher and Fedkiw, 2002) hasbeen proposed (Peters, 2007; Palma et al., 2008). Thisapproach is taking advantage of the side effects in-troduced on the Lipschitz function, which implicitlyrepresents a contour, during the minimization stageof the contour energy. The idea is then to slice thisfunction at several levels after convergence to build afuzzy contour.

Unfortunately, tuning parameters of this segmen-tation approach is a pretty difficult task. In fact, theproblem comes from the large variability of structuresthat may be encountered in mammography. Further-more, slicing the function representing the contour atseveral levels is something that may be hard to inter-pret.

4 DYNAMIC PROGRAMMINGAND FUZZY SEGMENTATION

4.1 Original Approach

Recently, an alternative to conventional approacheswas proposed (Timp and Karssemeijer, 2004). In thisapproach, a contour is modeled as a path in an imageconverted into the polar domain. This path is associ-ated with a cost that depends on the image content.This formulation allows using well known dynamicprogramming (DP) techniques in order to efficientlysolve the problem. Several improvements have beenproposed based on this approach (Rojas and Nandi,2007).

θr

(a)

θ

r

(b)

Figure 2: Polar coordinates representation (b) of a circum-scribed lesion (a).

IMAGAPP 2010 - International Conference on Imaging Theory and Applications

8

The authors originally designed this algorithm onstandard mammography images. We eliminated theuse of the shape coefficient they propose since themasses used in our database were of numerous shapesand sizes and the research of fuzzy contours impliescontours of different radii. Concretely, the image tosegment is first converted in the polar domain. Therows and the columns of the new image represent re-spectively the distance to the lesion center and theangle around it. This representation is illustrated inFigure 2. A cost matrix is then defined on the samedomain leading to the definition of a cost function forany path linking the first to the last column:

C (c) = ∑θ∈[[0;θmax]]

M(θ,c(θ))

with M a cost matrix,θmax the index correspondingto the maximum angle (2π), andc a path, which as-sociates any angle in[0;2π[ with a radius. In our im-plementation we used the measures based on the im-age gradient along the radial direction and the optimalgray value of the contour, which were proposed in theoriginal approach (Timp and Karssemeijer, 2004).

Now to introduce some smoothness in the con-tours we may want to consider, we can restrict thesearch space to the following set of contours:

P = {c/∀θ ∈ [[0;θmax]] |c(θ)− c(θ−1)| ≤ f}

with f ∈N the parameter allowing to tune the contoursmoothness.

The segmentation problem is then solved bysearching for a contour ˆc such that:

c= argc∈P

minC (c)

This method was initially designed to segment cir-cumscribed lesions in standard mammography. Ourimplementation, which is similar as the original oneexcept for the lesion size prior, has been tested onspiculated and circumscribed lesions in DBT as it willbe presented in Section 5.

We will now present our main contribution overthe original work by proposing means to extract fuzzycontours from an image.

4.2 Penalization

To obtain several contours, the first idea was to pre-vent the algorithm from using the pixels that have al-ready been crossed by a path in the cost matrix bysetting their costs to an infinite value. This operationcan be iterated while non infinite cost paths exist inthe cost matrix.

When looking for a path in the cost matrix, theDP-based algorithm is constrained by the parameter

f along the vertical axis whose role was previouslydetailed. Thus, if we consider a pixel(θ, r) in thecost matrix, the path can reach 2f + 1 pixels fromone column to another. This parameter allows radialvariations in the polar coordinates system described in(Timp and Karssemeijer, 2004). Thus, a simple penal-ization of the pixels through which goes the first pathdoes not prevent contours from crossing each other.

Contours obtained with such a method are oftensimilar to the first contour. To force finding a contourwith a significantly different shape, it is necessary togenerate a lot of contours, which might be hard tohandle in further processing. The information con-veyed by the repetition of quasi-identical contours isrelated to the imprecision contained in the lesion wewant to segment. This correlation has some mean-ing but we may miss information related to the uncer-tainty implicitly present in some images.

In order to satisfy the inclusion criterion requiredby Definition 1, it is possible to penalize a bandnearby the path found in the cost matrix. Instead ofsetting a one-pixel band to an infinite value, we pe-nalize a band of 2f + 1 pixels. Thus, the algorithmforces the contours to be nested.

4.3 Contour Selection and Elimination

In this section, we present an approach to model un-certainty rather than imprecision. The goal is to ob-tain a set of contours different from each other. Toguarantee that the contours will be different, we de-fine a distance on the space of contours representedin polar coordinates. Letc1(θ) andc2(θ) be two con-tours, we define the distanced(r1, r2) as follows:

d(c1,c2) = maxθ

(|c1(θ)− c2(θ)|)

We can now extract representative contours us-ing the penalization technique previously presentedwith no need to use forbidden bands of pixels butskipping contours that are too close from each other.This is done iteratively by discarding all the contourswhose distance with the former representative contouris less or equal to a given threshold. This approach al-lows modeling uncertainty and produces contours thatare not nested but that exhibit differences for at leastfew points. Segmentation results obtained with thismethod are presented in Section 5.

FUZZY SEGMENTATION OF MASSES IN DIGITAL BREAST TOMOSYNTHESIS IMAGES BASED ON DYNAMICPROGRAMMING

9

4.4 Extraction of Several FuzzyContours

In order to capture both uncertainty and imprecisioncontained in a given image, we can combine the bandpenalization and the distance-based approaches. Ob-viously, we can extract for each contourc obtainedwith the second approach, a fuzzy contour by apply-ing the first method on a matrixM′ = M where thepathc is set to∞ on a large band. Doing this enablesus to have a set (uncertainty) of nested contours (im-precision).

4.5 Membership Degree

To complete the extraction of fuzzy contours, we needto assign a membership degree to each contour consti-tuting the fuzzy set. This membership degree shouldbe representative of the confidence we can have in thecontour.

The dynamic programming algorithm provides uswith a direct measurement of confidence linked to thevalues of the gradient along the contour and its graylevels: the total cost of the path (C ). This cost can beused to derive a membership value for the contour ifwe normalize it considering the whole set of contoursobtained for an image. For instance, if we obtainncontours with the associated costsCi , i ∈ [[1;n]], wecan define the membership degreeµi as follows:

µi =Cmax−Ci

Cmax−Cmin(1)

whereCmax andCmin are maxi{Ci} and mini{Ci}, re-spectively. Thus, we get a value ranging between 0and 1, with 1 being the smallest cost (hence highestmembership value) and 0 being the highest cost.

Another type of membership degree can be de-fined. Since we extract a lot of contours to constructa pool from which we extract the contour with thedistance method, we are able to create a map of con-tours density. Each time a path is derived in polarcoordinates (in the cost matrix), we set to 1 the corre-sponding pixels in the contour map. A pixel cannot becrossed twice because of the penalization. So, we ob-tain a map as represented in Figure 3(a). By averagingthe image vertically with a Gaussian filter, we obtaina blurred version as represented in Figure 3(b). Thisstep allows us to take into account the spatial den-sity of the contours, thus the resulting density map isrepresentative of the possibility of having a contourcrossing given pixels. Summing up the values of thedensity mapD along a path enables us to obtain a rep-resentative value of the confidence we can have in thecontour. The higher this value, the more confident we

can be in the contour. This total density can also benormalized to create a membership degree:

µ′i =minc′∈C

θmax

∑θ=0

D(θ,c′(θ))−θmax

∑θ=0

D(θ,ci(θ))

maxc′∈C

θmax

∑θ=0

D(θ,c′(θ))−minc′∈C

θmax

∑θ=0

D(θ,c′(θ))(2)

with C the set of extracted contours andci the consid-ered contour.

The behavior of both approaches will be discussedin Section 5.

(a)

(b)Figure 3: Computation of a contour density map. (a) Con-tour map extracted from an image. 100 contours have beenmarked on the image. (b) Density map after smoothing.

5 RESULTS

The database we used for evaluation contains 52 re-gions of interest extracted from DBT slices. Sixteenof them contain a circumscribed mass, while remain-ing ones contain a spiculated lesion. The algorithmwas first validated on these images for crisp segmen-tation. The results are presented in Section 5.1. InSection 5.2 we interpret the results obtained with ournew fuzzy segmentation approach. We focus on theresults obtained with both the penalization and thedistance and discuss the combination of the two ap-proaches. Finally we compare the two ways of con-structing membership degrees.

5.1 Crisp Segmentation on DBT Slices

In order to validate the approach, we performed anassessment of the crisp approach on two subsets of

IMAGAPP 2010 - International Conference on Imaging Theory and Applications

10

lesions (spiculated and circumscribed). This assess-ment was done using ground truth contours built by ahuman reader and a criterionP, which is the averageof 3 measures (Rojas and Nandi, 2007):

P1 =|A∩R||A∪R| similarity

P2 = 1− |R\(A∩R)||R| under-segmentation

P3 = 1− |A\(A∩R)||A| over-segmentation

whereR andA are the reference contour and the oneto be evaluated, respectively.

The evaluation was done using the leave-one-outmethod in order to learn without bias the algorithmparameters. The best results were obtained for thesegmentation of circumscribed lesion with a learn-ing step using the same type of lesions (P = 0.87).The performance obtained for spiculated lesions witha learning step using the same population was a bitlower (P= 0.71). Finally testing the approach on cir-cumscribed lesions while learning on spiculated dataresults in a small performance decrease compared tothe ideal case.

5.2 Fuzzy Segmentation

Figure 4 illustrates an example of contour extractionusing the penalization of a band on each side of thecontours to force them to be nested. Let us noticethat three of the contours are really close to eachother. This could be interesting because this modelsthe imprecision we have on the edges. Let us alsoremark that it may be interesting to get a commonportion between several contours since imprecisionmay take place only in a part of the lesion. Unfor-tunately, the penalization method prevents such a be-havior. Nonetheless, with a high enough radius quan-tization step in the polar domain, the obtained con-tours can be close enough to finally overlap when ex-pressed in the Cartesian domain.

(a) (b)Figure 4: Fuzzy segmentation (b) of a cicumscribed le-sion (a) using the penalization method.

We now present the results obtained using thedistance-based contour selection method. Figure 5 il-lustrates a clinical example with contours extracted

using the distance method. As we can see, the ad-vantage of this method is that it captures the uncer-tainty: there is a lot of variations in the proposed con-tours. Let us remark that most of these contours makesense according to the lesion to segment. Nonethelesssome are less likely to represent the lesion. Howeverthis is not an issue if we can associate those contourswith a low membership degree. In this example, thegray levels of the contours are proportional to theirmembership degree: the lower the membership de-gree (contour gray levels), the less possible the con-tour.

(a) (b)Figure 5: Fuzzy segmentation (b) of a spiculated lesion (a)using the distance-based contour selection method.

Figure 6 shows how the fuzzy contours extractionmethod relying on the two contours selection methodperforms on a synthetic image. The distance basedmethod allows retrieving four contours representingthe uncertainty in the image. Those contours appearin black boxes, corresponding to the two disks, theirunion and intersection. For each of those contours, aset of nested contours is then extracted using the bandpenalization method allowing to build a fuzzy con-tour. Here we have 4 fuzzy contours in Figure 6(b),6(c), 6(d) and 6(e). Note that for the sake of clar-ity, slight variations of the presented contours (part ofthe imprecision) that are also extracted are not shown.Let us also remark that fuzzy contours 6(b) and 6(e)are almost the same. Such a redundancy may ap-pear because we want to extract as much main con-tours as possible in the first stage. Nonetheless sucha behavior should not be a problem in real applica-tions like automatic detection/characterization of can-cer because no information is discarded.

5.3 Comments on the MembershipDegree Definition

The comparison between the two techniques of mem-bership degree computation is illustrated in Figures 7and 8. These figures present the ranking of the con-tours in a fuzzy contour according to their member-ship degrees. It is interesting to notice the position of

FUZZY SEGMENTATION OF MASSES IN DIGITAL BREAST TOMOSYNTHESIS IMAGES BASED ON DYNAMICPROGRAMMING

11

(a) (b) (c)

(d) (e)

Figure 6: Fuzzy contours extraction. (a) Synthetic imageto segment. (b) First fuzzy contour. (c) Second fuzzy con-tour. (d) Third fuzzy contour. (e) Fourth fuzzy contour. Foreach sub-figure, the black box designates the main contourretrieved by the distance based method, while the remainingcontours constitute the fuzzy contour.

the smallest contour corresponding to the kernel of thelesion. When we use Equation 1 to compute the mem-bership degrees, this contour is the worst one whilewhen Equation 2 is used, this contour is the best. Ac-tually, the membership degrees of the three best con-tours according to Equation 2 are almost the same,which is coherent with the visual quality of these con-tours. Let us also remark that this method could stillbe improved because the second and third contoursmay appear more relevant, but again since the dif-ference between the membership degrees is low, thiswould be a minor improvement.

(a) (b)

(c) (d)

(e) (f)

(g) (h)

Figure 7: Membership degree computation using Equa-tion 1. (a) Fuzzy contour to be considered. The remain-ing images are ranked in decreasing order according to themembership degree of the contour they contain. Due to thisranking, the first contour also corresponds to the result of acrisp segmentation.

6 CONCLUSIONS

We have proposed a new technique to extract fuzzycontours from images containing breast masses. Thismethod relies on previous works that model a contouras a path in the image to segment, which has beenconverted in the polar domain (Timp and Karssemei-

IMAGAPP 2010 - International Conference on Imaging Theory and Applications

12

(a) (b)

(c) (d)

(e) (f)

(g) (h)

Figure 8: Membership degree computation using Equa-tion 2. (a) Fuzzy contour to be considered. The remain-ing images are ranked in decreasing order according to themembership degree of the contour they contain.

jer, 2004; Rojas and Nandi, 2007). We extended thismethod in order to capture imprecision as well as un-certainty of lesions in mammography images or DBTslices. This is performed through two distinct contourextraction schemes, which we finally combine in or-der to get several fuzzy contours for a given structureto be segmented.

We also proposed a new technique to compute themembership degree of each contour contained in afuzzy contour. This approach relies on the idea that

location where we can extract a lot of contours arelikely to represent good contours. Using a contourdensity map, we thus derived a way to compute suchdegrees.

Because building a ground truth database forfuzzy contours is a pretty difficult task, as of today weonly evaluated our approach visually on the lesions ofour whole database. One next step may be to workwith clinical experts to review images in order to getseveral contours for these lesions. Of course to be rel-evant all the segmentations for each lesion should beperformed independently.

REFERENCES

Bothorel, S., Bouchon Meunier, B., and Muller, S. (1997).A fuzzy logic based approach for semiological analy-sis of microcalcifications in mammographic images.International Journal of Intelligent Systems, 12(11-12):819–848.

Dobbins III, J. T. and Godfrey, D. J. (2003). Digital X-raytomosynthesis: current state of the art and clinical po-tential.Physics in Medicine and Biology, 48(19):R65–R106.

Gennaro, G., Baldan, E., Bezzon, E., Grassa, M. L.,Pescarini, L., and di Maggio, C. (2008). Clini-cal performance of digital breast tomosynthesis ver-sus full-field digital mammography: Preliminary re-sults. InInternational Workshop on Digital Mammog-raphy (IWDM), pages 477–482, Berlin, Heidelberg.Springer-Verlag.

Osher, S. J. and Fedkiw, R. (2002).Level Set Methods andDynamic Implicit Surfaces. Springer.

Palma, G., Peters, G., Muller, S., and Bloch, I. (2008).Masses classification using fuzzy active contours andfuzzy decision trees. InSPIE Symposium on MedicalImaging, number 6915, pages 691509.1–691509.11,San Diego, CA, USA.

Peters, G. (2007).Computer-aided Detection for DigitalBreast Tomosynthesis. PhD thesis, Ecole NationaleSuperieure des Telecommunications.

Rojas, A. and Nandi, A. K. (2007). Improved dynamic-programming-based algorithms for segmentation ofmasses in mammograms.Medical Physics, 34:4256–4269.

Timp, S. and Karssemeijer, N. (2004). A new 2D segmenta-tion method based on dynamic programming appliedto computer aided detection in mammography.Medi-cal Physics, 31:958–971.

Wu, T., Stewart, A., Stanton, M., McCauley, T., Phillips,W., Kopans, D. B., Moore, R. H., Eberhard, J. W.,Opsahl-Ong, B., Niklason, L., and Williams, M. B.(2003). Tomographic mammography using a lim-ited number of low-dose cone-beam projection im-ages.Medical Physics, 30(3):365–380.

FUZZY SEGMENTATION OF MASSES IN DIGITAL BREAST TOMOSYNTHESIS IMAGES BASED ON DYNAMICPROGRAMMING

13