MuItiscale medial analysis of medical images

Multiscale medial analvsis of medical images

Bryan S Morse, Stephen M Pizer and Alan Liu

The Mult~scale Medial Axis is ;i means for detecting and

representin_r ob_ject shape ;1t multiple scales simultaneously.

One of its kq &tractcristics is that the scale used to measure

object properties (and hence to represent the object) is

proportion:tl to the IOGII Midth of the object. This allows it to

separate line-sc‘lle detail from larger-scale gross shape proper-

ties of the object III a manner dictated bq’ the object itself. It

works directly from image intensities :lnd does not require ;I

prior scfmcntntion 01‘ the image or explicit determination of

object bound:lrlcs. Furry (non-binary) boundary mc;lsures in-c

used to compute fu~/y medial measures. and axis points are

identified ax ridges in this fu/q medial space. This paper

presents mmc of the basic concepts of the Multiscale Medi;ll

Axis. describe\ its compu~ution. and demonstrates some

preliminary results of its application to medical images from

varncty 01‘ miaging modalities.

Keywords: scale space. medial axis, Hough transform, ridges,

shape, segmentation -___.-

Pizer’-’ and colleagues have suggested that we observe objects in images by pairing corresponding boundary points on opposite sides of the object. These pairings can then be used to compute the medial axis of the ob1ect.

?hc medial axis is 3 structure, first proposed by Blum ‘5 that captures global shape properties of an ob_ject. It i4 defined as the locus of ccntres of disks of

maximal fit within an object. By taking measurements at multiple scales. particu-

larly at scales proportional to the width of the object itself. ;I new type of medial axis is formed. This nea representation is called ;I Multiscale Medial Axis (MMA) and rcprescnts ob.ject shape at multiple scales simLlltanec,usl~. including both large-scale (gross shape prc.>perties of- the object) and small-scale (fine detail) in a

manner dictated by the image data itself. Marr and Nishihara” have suggested that this separation of detail from gross shape properties is an important clement of any visual system.

One approach to computing any axis representation is to apply boundary-sensiti\,e operators to produce ;i fuzzy (non-binary) characteristic function describing the hoLilrtitrr-illc~.r.s at each point, making some type of decision to turn this fuzzy characteristic function into ;I

binary one. and then computing the medial axis of this binary structure. Much work has been done on this skelctonization of binary object\ and has produced fairlv successful results, but with considcrablc senai- tivitj’ to detail in the cnlculated boundary. Unfortunately. this approach throus out information at carly stag of processing. thus making it unavailable at later stages and increasing the error.

Another approach. used here. is to retain the I‘LIU~

nature of the computation until the highest lcvcl possible. This is done by using the f‘wry characteristic function for boundaries to compute ;I similar fuzzy characteristic function for points on the medial axis. This approach produces an axis characteristic function that can be used by still higher order processes or by ;I

decision process that extracts the optimal binary form of the axis. We have termed this fuzzy axis characteristic function a t~wlirrl r1~.spw~.rc~ jLrr?clion or simply /~wtlirr//w.vs in the sense that it responds to medial properties. The MMA is computed as a ridge in this medial response. These ridges (axes) form ;I set of one-dimensional space- curves. as shown in F@rw 1.

This paper demonstrates the application of this type of multiscalc medial analysis to medical images. We will first present some of the motivation for using this type of imatre analysis. A more detailed discussion of- the undcrlyqng concepts can bc found elsowhcre’.‘. We Mill also describe how these Multiscale Medial Axes arc computed. A more detailed presentation of these methods can be found elscwhcre5-“. WC will then shop preliminary results demonstrating how this type of analysis can be used to form object axes in ;I variety ot medical images.

0262-8856/94/06/0327-12 (‘ 1994 Butterworth-Helnemann Ltd

Image and Vision Computing Volume 12 Number 6 July/August 1994 327

Multiscale medial analysis of medical images: Bryan S Morse et al.

scale-space curve

detail

Figure I Multiscale medial axes as shown in scale space. Height indicates object width as a function of axis position

Comparison to previous MMA research

The results presented here are produced by a boundary- based form of the MMA. Some earlier papers on the MMA’.’ and similar algorithms8 have used a single, axis-centred operator that responds well at a scale and position where the operator optimally engages the two sides of the object. This approach also computes a type of medialness response.

Such an operator must be sensitive to changes at the boundaries of the object. The response of the operator is, in a sense, the fit of the operator to a particular position in the object. An operator of the scale of best lit should produce a greater response than a slightly larger or slightly smaller operator. Similarly, an operator positioned exactly on the axis should produce a greater response than a neighbouring off-axis operator. Such a best-lit operator is termed an axis- centred operator.

Using an axis-centred operator is attractive in that it is the best lit to a region rather than to boundaries. It has several advantages and has been used for specific applications’.’ ‘. However, combinations of individual boundary-centred operators have the following advantages:

l Axis-centred operators require integration over the entire width of the object. This makes it sensitive to internal variations within the object. It is also physiologically improbable as a human visual model.

l Axis-sensitive operators, since they involve enga- ging both boundaries simultaneously, are sensitive to relative differences in the boundaries. Any differences in the strength or nature of the boundary on the two sides of the object can pull the response to one side or the other. Combinations of boundary-centred operators, because they rely more on the relative distances between the boundaries rather than the relative strengths, are not affected by such differences. While such differences may affect the strength of the medial response, they do not affect the geometry.

Boundary-centred operators can rectify the response of individual operators, producing pairings between boundary transitions of opposite polarity. For example, it can pair boundaries that are lighter than the background on one side and darker than the background on the other. This property cannot occur with simple linear operators. Boundary-centred operators, since they operate on each boundary individually, can combine boundaries of different natures to produce a medial response. That is, it can detect objects that are bounded by different types of boundaries at different parts of the object. These might include luminance boundaries, texture boundaries, line boundaries, or any other detectable type of boundary.

There is also psychophysical evidence that indicates that the human visual system (for sufficiently large separations) individually localizes each target when measuring distances between scene targets’“.“.

Another recent approach to the MMA is to extend the concept of the axis-centred operators, but to limit them by boundary measures’. If the axis-centred operator is based on Gaussian operators (as has been the case for the above-cited work), then these operators can be produced by a diffusion process. By limiting the diffusion at object boundaries, this method reduces interobject interference. This method is hybrid of both the axis-centred operators and boundary-based methods. However, it still suffers from the sensitivity of the axis-centred operators to deviations in the interior of the object. It also relies on knowing the polarity of the contrast (black to white, white to black) and cannot group objects of differing contrast polarity.

Other multiscale symmetry-based approaches from this research group” have relied on symmetries in image intensities, not on the boundaries of objects. Such approaches are severely limited for cases where the objects are not simply of different intensities (as would, for example, be the case in ultrasound images).

Boundary-based MMA does not require some preliminary segmentation of the image that produces explicit boundaries. Rather, the algorithms operate directly from a fuzzy boundary response that is computed at all points in the image directly from the image data. The boundary response at all points and at all scales is considered when computing the medial response.

Scale space

Throughout this paper, the term scale refers to the tolerance with which image locations and widths are measured.

One way of increasing the stability and concomitantly the spatial tolerances of measurements in the image is to first convolve (blur) the image with a small neighbourhood-averaging kernel. Increasingly larger-scale versions of the image are produced by blurring with successively larger kernels. It has been shown that for this scaling to behave in a reasonable way (causality,

328 Image and Vision Computing Volume 12 Number 6 July/August 1994

self-similarity of scales, etc.) any scaled operator must be a solution to the diffusion equation. The Gaussian or derivatives of the Gaussian are such solutions”.

Proper sampling of scale space requires that scales be sampled exponentially’“. In other words:

(T, = o,,h’ (I)

where r is the variable in which we will now take discrete, equally spaced samples, (TV is the initial scale of measurement and h is the scale-change factor.

All scaled operators referred to in this paper assume this type of exponential sampling of scale. The set of all /?-dimensional images at all scales produces an n + l- dimensional .SC& .S~NCC.

Relating boundary scale to object width

A requirement of any visual system, biological or artificial. is that it uses scale information appropriate to the task performed. The significance of small-scale fluctuations must be interpreted in the context of the overall scale of the task. This behaviour is incorporated into the computation of the medialness response (and hence the MMA) by using boundary operators of a scale proportional to the radius associated with the medial axis.

In other words, the scale used is approximately proportional to the width of the object (with a small adjustment to compensate for extremely small scales. decreasing in significance at larger scales. This proportionality is a key aspect of the visual mode1 upon which these algorithms are based. It represents the well-known Weber’s law relationship between separation (width) and the accuracy with which that separation is measured by the human visual system’“, 11.14.

It must be emphasized that this refers to the scale over which the boundary is localized. The underlying information (e.g. texture) may be of smaller scale (higher frequency)‘“.

This property, the use of scale proportional to the object width, is what allows the MMA to separate large- scale shape properties of the object from small-scale detail.

METHODS OF COMPUTATION

The method used here involves four steps. First. for each point in the image and at each of multiple scales. compute the boundariness of that point with respect to that scale; second. combine these results using an algorithm. similar to the Hough transformIs. to compute the medialness of every point with respect to specific object widths; third, refine this medial response using an increasingly more boundary-selective combina- tion algorithm; fourth. analyse the media1 response space and identify ridge points by examining the local geometry of the response. The remainder of this section will describe these steps in more detail.


Measuring boundaries

The first step in the analysis is to apply scaled. directional. boundary-sensitive operators to the image.

For example, a simple directional boundary function B is the change in intensity L in direction or at spatial position yH as measured at scale (T:

where (G, * L) is the convolution of the image intensity function L by a Gaussian G with standard deviation (T.

This is not, of course, the only way to detect intensity boundaries, nor are all desired boundaries in images produced by simple changes in intensity. Any suitable boundary measure will do. For example, detection of specular boundaries in ultrasound images requires a boundary measure based on intensity of the boundary itself (a ‘bar detector’) rather than intensity difference at the boundary (an ‘edge detector’).

Computing the medial response

Let M(Y I, 1.) be a measure of how medial a particular spatial position S, is with respect to a certain object half-width K. By Blum’s definition of the medial axis, the circle of radius I’ centred at 1, must be at least doublv tangent to the object boundary. (This is using a generalized definition. also proposed by Blum, whcrc the disk is not restricted to being entirely within the object.) So. for a point to be medial ~t.itl~ rr.sp~/ to (I lzu~fl~~irltl~ I’ there must be boundaries at a distance I‘ from that point. Points on the medial axis will have more than one bounclary at a distance I’. Therefore, ~1~(Y~, I.) is related to the amount of boundary response for points Y?B on a circle of radius I. ccntrcd at .\- 1. Furthermore, since this circle is tangent to the ob.ject boundary, M(I ,. 1.) depends only on the directional response for points on this circle in the direction s4 ~ So from the point _I-,~ to the centrc I, of the circle. This is illustrated in F&rw ?.

Directional

Boundary-Sensitive



As previously stated, the boundary response is measured at a scale appropriate to the size of the object. Letting the scale of the boundary operator (Q) be proportional to the radius Y gives:

r = ka (3)

where k is the proportionality constant. However, limits on the smallest practical measurement scale imply proportional limits on the smallest usable object width (radius). If one desires to work with objects smaller than this minimum radius, equation (3) may be approxi- mated by:

r=krr-L (4)

with a small constant value of c. This allows a slightly larger scale to be used at extremely small object widths where the inner scale of the image itself limits the size of meaningful scaled operators. The effect of this constant becomes negligible for larger values of r so that r/c

approaches a constant value. The images shown here were computed using a proportionality constant of k = 2 and an adjustment of c = 1.

Integrating the directional response along the points in the circle in Figure 2 shows how the medial response for the point XA and radius r is computed. The response is:

where (r + c)/k. The function W is a weighting function that controls

which boundary points contribute to which medial points. Ideally, the medialness at a point is affected only by those boundary points that are at a distance I from the medial point, in which case:

W(a’- r) = 6(d - r)

However, in discretely sampled images, there are few points that lie exactly at a distance P from any given point. To compensate for this, a wider weighting function W allows medial points to receive contribution from boundary points that are not quite exactly r

units distant. The images shown here were computed using a triangular weighting function of width 2.

Equation (5) is a continuous form of the response equation. For discrete spatial sampling, this is approxi- mated by:

This may be interpreted as the summation of the directional boundary response of the appropriate scale at all points approximately distance r from XA. Since the number of such points varies with r, the summation is normalized by l/~I%‘(j~~,~ - ?:Bll - r).

Equation (6) is the basis of an algorithms, similar to the Hough transform”, referred to as the Hough-like Medial Axis Transform (HMAT):

1. Apply boundary-sensitive operators at every position in the image at a number of scales (T;.

2. For every position XA every scale CT;,

(a)

(b)

(cl

use the boundary measurements at ;iiB and at scale cr; to compute the directional boundariness at ;yB in the direction from xB to zA, compute the contribution from the boundariness at Sg and scale CT; to the medialness at xA using the integrand of equation (6), and add this contribution to the accumulator for M(Ya, r;). where ri is proportional to CT, as given by equations (3) or (4).

The result of this algorithm is a transformation from a (X, a) boundary-response scale space to a (a, r) medial- response scale space.* Note that this response gives information not only on axis position s but also object width r. The MMA can be determined as a ridge in the scale space of this medial response’, simultaneously selecting both object middle positions and related widths (scales).

Refining the medial response

The results of the HMAT are further refined by constraining the contribution of individual boundary points. This is done through additional voting phases where each point contributes to the accumulator in a more discriminating fashion by exalnining the accumulator produced by the previous voting phase and lessening its voting where there is lesser agreement with other voters. This type of technique was first used by Gerig16 to sharpen accumulators produced by the circular Hough transform.

When applied to the previously described algorithln for computing the medial response, this approach refines the medial response space and makes the ridges more prominent. More importantly, it disassociates object boundaries and allows each boundary to contribute more discriminately to the axis for that object and to contribute less, if any. to axes for other objects. In other words, as object representations (axes) form in the medial response space, these objects begin to ‘own’ their contributing boundaries and to exert an inhibitory influence on the contribution of those boundaries to other objects. This assignment of ownership is called ~re~~~ ~~~~j~~~~~~ in that credit for the formation of the object (axis) is assigned to the boundary points which contributed to that axis.

The credit attribution algorithm is:

1. Generate the medialness accumulator using the HMAT.

2. For each potential boundary point (i.e. all points in the image) examine the locus of points to that it contributed to in the accumulator and compute the sum of these accumulator values.

3. Regenerate the accumulator, this time weighting each vote by the ratio between the previous value of

*One must be careful here when referring to M as a scale space. It is not a pure scale space since successive scales cannot be computed by diffusion from previous scales. This is due to the nonlinearity of the absolute-value operation in the boundariness measure. M should be properly referred to as a nonlinear transformation of a scale space, but the usage here should be clear.



that accumulator point and the sum of all accumulator values to which the boundary point contributed.

To write this mathematically. define the following quantities:

I.(~,~.x_,.(T) The base strength of the vote from boundary point Sg sampled at scale (T to accumulator point YVA.

.M,(\- ,, F) The value of the accumulator at position SH on iteration i.

CJY,j, 1.) The sum of the accumulator values Af,(\- !. r) for all accumulator points .\-I contributed to by boundary point yM st a distance Y on iteration

By setting each \‘ote to:

I.(SH. 1 I. (7) = /$I\-,], (T. IT , ~ T&) W( IIS., - T\.Hll ~ 1.)

as seen in equation (5). then the basic HMAT may be written as the sum of these votes:

M,j(\-,, r) = .i--

I’( -Y/j. _\‘,I, rr)dS[]

and the credit attribution algorithm may be witten as a N;eightcd sum of these votes:

The credit attribution algorithm is an iterative one because the HMAT is only one of several stages of processing in the MMA model. There is potential feedback between some of these stages, as the results of later stages are allowed to affect the (re)processing of earlier stages. The credit attribution algorithm is one form of this type of feedback the results in the accumulator affect the generation of a subsequent accumulator. Since this feedback may lead to several iterations 01‘ processing, the credit attribution algorithm must make not just ;I single regeneration of the accumulator. but ma!’ need to make several passcs through this process.

By normalizing by the sum of all contributed-to accumulator values, the above algorithm causes each boundary point to have a constant total contribution. This forces boundary points to split their contribution between cacral possible accumulator points if no clear owner can be identified. One effect of this is to make each of several ambiguous interpretations of the image weaker than a single unambiguous one. thus making the syystem more susceptible to falling into one interprcta- tion or the other based on other information in the image. Thih type of split contribution is also more consistent M.ith a neural model of the process.

Finding ridges

The next step in the analysis is to find the ridges within the medial response. This is done by examining the local geometry to find points that exhibit ridge (locally

optimal) properties. Since the rcsponsc space is three- dimensional (two spatial dimensions and the scale dimension). it is necessary to extend the landscape notion of ;I two-dimensional ridge to a t Ii i rd dimension. If medial response is thought of as the density at each point in ii three dimensional scale space. then the ridge can be thought of (somewhat fancifully) as the ‘heart of the cloud’. These ridges form ;I set of one-dimension;ll hpace CLII-LCY through the three- dimensional space.

Notice that this is quite different from finding ridge\ on a two-dimensional manifold in three-ditllcllsion~tl space”. The fourth quantity, the medial rchponsc at ;I point, must bc treated as :I separate and unique dimension. Moreover. the scale dimension must also be treated specially relative to the t\bo spatial dimensions.

Other papers from this research ~I-OLI~ have prcsentcd various definitions for finding a ridge in t\vo dimcn- sions, or in three dimensions by \clccti\;cl\ examining the projections of t~Yo-dimcnsion~il manifolds \vithin that three-dimensional space’. The ridge definition presented here is similar to one ~~scci bq Haralick” for t\vo dimensions and extended by LIS to higher dimensions and to scale spaces. The definition may be easily cvtcnded to any arbitrary tiimcnsionalit~ of space or ridge within that space.

Ridge definition A general notion in two dimensions is that a ridge point in :I function f is a local maximum in /’ along the direction of the greatest principal curvature of /. In other words, at 21 ridge point the direction of grcatcsl curvature off is the cross-ridge direction and the value of /‘at the ridge point is greater than the ncighbouring points on either side of the ridge. In higher (II) dimensions. this extends not to ;I single direction of grcatcst curvature. but ;I set 01‘ )I ~ I orthogonal directions of greatest curvature.

Specificall). for ;I function 1’ of three \ ariables .Y, .I‘ and 1, let I/., 1 3 Ii31 2 I;.;~ denote the cigcn\alues of the Hessian matrix of second derivatiws:

Then. let 7,. ~j?. & denote the corresponding eigenvectors. A point is defined to bc on a ridge in f if. at that point:

Extension to arbitrary dimensions Although not used m the images shown here, the definition may be extended to any arbitrary dimension- alit?, of space (II) and ridge (lo. For ;I function /‘of II


Multiscale medial analysis of medical images: Bryan S Morse et a/

variables, let Ii, 1 3 1221 2 . . . k /I,,,( denote the eigenvalues of the n by E matrix of second derivatives. Then, let 21) 22, . . I , Z;, denote the corresponding eigenvectors. A point is defined to be on a k-dimensional ridge infif, at that point, for all i < N - k:

2, < 0 t?j.yf=o

Extension to scale spaces Because one of the dimensions of the medial space is a scale parameter 0, the above definitions hold but the quantities must be measured differently. One of the properties of a change of scale is that one must also change one’s yardstick (unit of measure) accordingly’~. This means that the interpixel separation decreases as the scale increases. To picture this, consider viewing an image at a distance d. As the distance d changes, the perceived pixel separation varies by l/d.

Because the ridge definition given in equation (8) is based on differentiation itz ~1ediaI scale space, such differentiation must be done with respect not to an absolute difference dx, but to the relative, scaled difference dx/a. Moreover, because each scale change is itself properly defined in terms of the relative scale, differentiation with respect to scale ought not to be with respect to dg but rather to rio/o. It is not enough, however, to consider fixed-scale differences in spatial position or fixed-position differences in scale. Rather, one must use a distance metric that combines differences in both space and scale simultaneously. We here define such a distance metric ds as:

d~2(d~~~)2 -t- (dy,/cr)2 + (~~~~~)2 (9)

The most significant result of such a metric is that the geometry of scale space is non-Euclidean, a fact that should not be surprising.

First spatial derivatives in scale space are then 0:

@ . and GA. First-order differentiation with respect to

ay a! scale is similarly G----. That is, all first derivatives da

computed using numerical differentiation between adjacent pixels must be multiplied by the current scale (r.

Since scale is here sampled geometrically, differentiation of scale 0 in terms of the sampling variable 5 (I) is:

8f di 8f “a,=“~$

1 Y =-- Inb &

For scale spaces of two dimensional images, the gradient off’(x, y, 6’) is then the vector:

(10)

Second derivatives are, however, not quite so straightforward as multiple application of first-order derivatives. Using the metric defined in equations (9) and drawing on the treatment of non-Euclidean spaces in differential geometry, particularly the application of covarient derivatives, one can define a new Hessian that includes the interdependence of space and scale”. Again making the previous substitution between the geometrically sampled scale 0 and the discrete scale-sampling index i, the correct matrix of second derivatives for scale space is:

(11) Using the matrix, along with its eigenvalues and

eigenvectors, the ridge definition in scale space is identical to that for normal Euclidean space.

AIgor~t~m The ridge definition given in equation (8) is based on the dot product of the function gradient and a vector being zero. However, because the function gradient changed rapidly at or near a ridge, this dot product is rarely exactly zero at a specific pixel but rather changes rapidly from positive to negative values. The algorithm for ridge detection used detects these sign changes and labels ridge pixels accordingly.

The algorithm for finding ridge points in an n- dimensional space is as follows. For every point in the image:

1.

2.

3. 4.

5.

Compute the matrix of second derivatives by examining the neighbourhood at the point. If the function is relatively smooth, as in that produced for a scale space, this may be done with simple numerical differentiation. The derivatives are normalized according to the scale as shown in matrix

(11). Compute the eigenvalues and eigenvectors of that matrix. Sort the eigenvalues by absolute value. If any one of the n - 1 largest eigenvalues are non- negative, reject this point and move on to the next. For each of the eigenvectors Pi corresponding to the 12 - 1 largest eigenvalues: (a) Compute Q[i] = ej . Of. (b) For each of the neighbouring pixels,

(i) Compute Qn[i] = Pi Vfat that neighbour. (ii) If Q[i]QJi] < 0 label the point as a ridge.

Like most sign-change dependent algorithms, this tends to produce two-pixel thick ridges. These ridges may be further thinned by imposing the additional requirement that the value of the function (medialness) at the ridge be greater than that at the neighbouring pixel with which Q[i] is compared.

332 image and Vision Computing Volume 12 Number 6 July/August 1994

Multiscale medial analysis of medical Images: Bryan S Morse et al.

RESULTS scale space. The intensity at point (.Y. J) in subimage i represents the medialness M(.v. J‘. 0,). At small scales,

As a preliminary test of the MMA concept and medialness is only found near boundaries, so the first

algorithms. we applied the methods in the previous few scale-space slices start out similar to gradient

section to d variety of medical images of different magnitude images. At successively large scales

modalities. The results are presented on the following (increasing from left to right. top to bottom) the effect

pages (Fijyw.s 3~-7). Each of the original images is of the boundaries moves farther and farther away. At

256 x 256. except for that in Figure 7 which is scales proportional to the width of the object, the effects

128 x 128. Each image was computed for 30 scales of these boundaries superimpose to show a brighter

starting at 0~) = 1.0. scale sampling factor h = 1.1. response ~ these are the points on the MMA. Since the

radius to scale ratio k = 2 and radius to scale adjust- MMA is a space-curve through the scale space, it may

ment (’ = 1. Five iterations of the credit attribution move up or down in scale as the object widens or

algorithm were used to generate the medialness images. narrows and cannot really be seen entirely within any

The medialness for each position and scale in the individual slice of scale space.

image is presented as an array of images. each To show the spatial positions of the MMA, the ridge corresponding to a single scale within the medialness points (the brighter places in the medialness images)

a

b

Figure 3 Application of the MMA compulation to an X-ray image of a pair of hands. (a) An X-ray image of a pair of hands; (b) computed medlal- ness at each scale for the image shown in (a). Each subimage (right to left, top to bottom) shows the medialness at an increasingly larger scale: (c) ridges in (b) superimposed on the original image (a)

Image and Vision Computing Volume 12 Number 6 July/August 1994


a

b

Figure 4 Application of the MMA computation to an MRI image of a knee. (a) Original image (upper left) and medial ridges superimposed (across and down). Each successive image shows an increasingly larger set of scales. The final result is in the lower right; (b) computed medialness at each scale for the original image shown in (a). Each subimage (right to left, top to bottom) shows the medialness at an increasingly larger scale

have been projected to the image plane and superimposed on the original image in black. This type of visualization loses the width information of the axis, but such information is still retained in the representation of the MMA. The results appear somewhat noisy because these projections include MMAs at all scales, even small ones, but the properties of the MMA cause these smaller scale axes (either noise-induced or actual) to be separated in the scale dimension of the result.

To show the behaviour of the axis in the scale dimension, one of the images (Figurr 4) shows not only all of the ridges superimposed upon the original, but multiple superimpositions each with an increasingly larger set of scales. One of the subimages shows only small-scale ridges superimposed upon the original

image; the next includes additional, slightly larger scales; the next even larger scales; and so on. In this way, the progress through increasing scale can be traced as the object widens.

DISCUSSION

These preliminary results show how the MMA captures symmetries in object boundaries, both with respect to position and to object width. More importantly, they show how these symmetries are captured for both fine- scale detail and larger-scale object shape.

The first set of images (Figcrvr 3) are for an X-ray



b

Figure 5 Application of the MMA

the skull and brain: (b) tom~uted meddness at each scale for the image shown in (a). Each subimage (right to lcf, top to bottom) shows the medialness at an increasingly larger scale: (c) ridges in (hi superimposed on the ortgtrnal image (a)

image of a pair of hands. The object symmetries are seen quite plainly here in the long symmetries of the fingers. Notice how the MMA of the small vertically oriented, rectangle on the left side of the image is the same as the well-known medial axis for rectangles.

The second set of images (Figttre 4) are for an MRI image of a knee and show how the MMA behaves through scale. Each of the successive axis-overlays show the MMA at increasingly large scales (object widths). At the smaller scales, only corners and small objects are detected by the MMA. At increasingly larger scales, the axis moves up through scale space to capture symmetries of larger parts of the object. This is seen most clearly by following the progression of the axis along the ~-idenir~~ part of the upper bone.

The third set of images (F’&.K~~ 5) are for an MRI

image of the brain and show the relationship between small-scale detail and larger-scale object properties. The image shows both individu~~l axes for the gyri of the brain and collective, larger scale. axes for the gyri as a group and for the brain as a whole. The fourth set of images (F@WCJ h), again of the brain. shows ;I similar behaviour.

The fifth set of images (F@w 7) are for a scintigram of the chest and show the robust behaviour of the MM.4 algorithtns even under extreme noise conditions. Individual ribs are represented by the small-scale components of the axes. while the spaces between the rib cage and sternum are represented by larger-scale axes without regard to the smaller individual ribs.

These results denlonstrate one of the apparent problems of the MMA: the richness of the rcpresenta-



a

b

Figure 6 Application of the MMA computation to a different MRI image of the skull and brain. (a) A different MRI image of the skull and brain; (b) computed medialness at each scale for the image shown in (a). Each subimage (right to left, top to bottom) shows the medialness at an increasingly larger scale; (c) ridges in (b) superimposed on the original image (a)

tion found by capturing all boundary symmetries. These images were processed without any assumptions about the image content and thus all possible symmetries were allowed. As a result, object boundaries that do not belong together were sometimes paired. However, the questions of what belongs together cannot always be answered without additional information or some type of cognitive process. For interactive systems or early stages of processing, we feel that it is better for a computer vision system to err on the side of including these ambiguities rather than eliminating potentially correct solutions.

This problem may be constrained within certain limited domains. In X-ray images of bone structures, for example, the boundary operator could be modified to only include black (outside) to white (inside) transi-

tions rather than both directions. This is easily done by changing the absolute value operation in equation (2) to one that simply zeroed negative values.

There should also be ways to improve the selectivity of boundary-pairing beyond the present credit attribution algorithm. By extending credit attribution to include interscale competition and other constraints, even better groupings should be possible. Still, some potential groupings will remain ambiguous without additional information or constraints on the problem domain.

The MMA groups multiply objects into composite ones when they blur together at larger scales. As seen in Figure 4, when a large-scale portion of the axis is projected on to the image it may appear to cross a gap between two clearly defined objects. This is because the


Multiscale medlal analysis of medical images: Bryan S Morse et a/.

a

b

Figure 7 Application of the MMA c~lrnput~lio~l to a scintigram of the chest. (a) A scintigram image of a clhest; (h) computed mediainess at each scale for Ihe image shown in (a). Each subimagc (right to left. top to bottom) shows the medialness at an increasingly larger scale: (c) ridges m (b) supcrinyxaed on the original image (2 1

gap itself blurs away when viewed at the scale of the objects. Although this is exactly the desired result at the scale of the object group. objects should remain separate when examined at their individual scales. Non-uniform methods of blurring such as variable conductance diffusion may be the key to solving this type of problem. Such non-uniform blurring would cause the boundary measurement operators to be restricted when the b~~undary is encroached upon by other nearby boundaries.

One practical drawback of the present MMA imple- mentation is its computational complexity due to the combinatoric relationship between potential boundary points (all point in the image) and medial points (again. all points in the image). We are presently exploring ways to reduce the computation given known limits on object

size. However. this is still new research and we are more interested in exploring the yualilative aspects of the MMA to verify its usefulness ah ;I means of object representation.

CONCLUSION

We feel that the Multiscale Medial Axis is a useful tool for analysing medical images. Rccause it captures shape properties of the object at multiple scales simultu- neously, and does so in a manner relative to the object’s width, it is able to represent both fine-scale detail and larger-scale proper-tics of object shape. Because this detail is often caused by noise or by minor



variations introduced through different imaging modalities, this ability to capture larger, more stable properties of objects makes these axes useful for such tasks as object identification and image registrationy,20,“.

As shown in these preliminary results, larger-scale representations may include collective groupings of smaller objects. This type of behaviour is exactly the hierarchical object representation intended for the MMA. Although there is still much work to do with the MMA, both as a visual model and as a collection of computer vision algorithms, the results here are extremely promising.

ACKNOWLEDGEMENTS

The authors would like to acknowledge the contribu- tions of Christina Burbeck, James Coggins and Dan Fritsch in the development of the MMA model and algorithms; Jonathan Marshall for his help with the credit attribution algorithm; and David Eberly, Christine Scharlach and Robert Gardner for their help with the formulation of the ridge definition used here. Thanks also to Guido Gerig for his helpful discussions regarding his own work with the Hough transform. We would also like to thank the reviewers for Irzfhmation Processing in Medical Imaging ‘93 and those for Image and Vision Computing for their helpful suggestions.

This research was supported in party by a fellowship from the UNC Graduate School and by NIH grant #PO1 CA47982.

REFERENCES

I Pizer S M, Coggins, J M, Burbeck, C A. Morse, S and Fritsch, D Inluge ohjec,t de.vcription ~~~ifhout e.up/icif edgeyfi’nding. Technical Report TR91-049, Department of Computer Science, University of North Carolina at Chapel Hill (1991)

2 Pizer, S M. But-beck, C A. Coggins, J M, Fritsch, D S and Morse, B S Ol~iect shupc h&w houndur~~ .~hape: .suIe-spucc mrdicd rr.~rs. Technical Report TR92-023, Department of Computer Science. University of North Carolina at Chapel Hill (1992)

3

4

5

6

7

8

9

IO

II

I2

I3

I4

15

16

17 18

I9

20

21

Blum, H and Nagal. RN Shape description using weighted symmetric axis features’, Pa/c. Rrco~n,, Vol IO (1978) pp l67- 180 Marr, D and Nishihara. H K ‘Representation and recognition of the spatial organization of three-dimensional shapes’. ~roc. Roy. Sot. Land. B, Vol 200 ( 197X) pp 269-294 Morse, B S Modifi’cution of’ houndury influence on nwdiulnrxs hi u&i/ uttrihuriun. Technical Report TR92-024, Department of Computer Science, University of North Carolina at Chapel Hill (1992) Morse, B S, Pizer, S M and But-beck. C A A Ha&-like m&d

czui.c wsponsr function. Technical Report TR9l-044. Department of Computer Science. University of North Carolina at Chapel Hill (1991) Fritsch, D S. Coggins, J M and Pizer. S M ‘A multiscale medial description of greyscale image structure’. m Adwnw.v in Intel/i-

grnr Robotic S~xfems, hrrll+ynt Rohot.v rmd Computrr Vi,sion X. A/gori/hms and Techniques. SPIE. USA (199 I) Crowley. J L and Parker. A C ‘A representation of shape based on peaks and ridges in the difference of low-pass transform’, IEEE Truns. PAMI. Vol 6 No 2 (March 1984) pp 156-170 Fritsch. D S ‘Image registration by MMA’ in R A Robb (ed), Visuukrrtion in Biomediwl Computing ( VBC ‘YZ). Vol 1808. SPIE, USA (I 992) Burbeck. C A ‘Position and spatial frequency in large-scale localization judgments’, Vision Rrs., Vol 27 No 3 ( 1987) pp 417~ 429 Levi. D M and Westheimer. G ‘Spatial-interval discrimination in the human fovea: What delimits the interval‘?’ J Opr. SW. Am. A, Vol4 No 7 (July 1987) pp 13041313. Gauch. J M and Pizer. S ‘A multiresolution intensity axis of symmetry and its application to image segmentation’, fEEE Truns. PAMI, Vol I3 (1991) ter Haar Romeny. B M and Florack, L ‘A multiscale geometric model of human vision’, in B Hendee and P N T Wells (eds). Perwption of’ Visual Infurmution, Springer-Verlag, Berlin (1991) Southall, J P C (ed) Helmholtr ‘.s Trcariw on Ph~~sio/ogicu/ Optics.

volume III, Dover, New York (1962) Hough. P V C Method crntl mecms of wcogniring compku puttrms, U.S. Patent 3.069.654 (1962) Gerig G ‘Linking image-space and accumulator-space: a new approach for object recognition’, Proc. First Int. Con/. on

Comput. Vision (ICCV ‘87) (1987) pp 112~122 Koenderink, J J Solid Shape, MIT Press, Cambridge, MA (I 990) Haralick, R M ‘Ridges and valleys in digital images‘, Comput.

Vision, Graph. & lmuge Prowss., Vol 22 (I 983) pp 28-38 Eberly, D, Gardner, R. Morse. B and Pizer, S ‘The differential geometry of scale space’, to appear in GrornefryDriom Diffkion.

Kluwer, Holland (1994) Fri tsch. D The mulri.wuk wwdiul uxis and its upplicurion.~ in irwge wgistration, Technical Report TR93-058, University of North Carolina at Chapel Hill (1993) Pizer, S M, Fritsch, D S. Morse, B S. Eberly. D H and Liu, A ‘Multiscale medial axis approaches for object definition and registration in medical images’. Pwc~. Int. Conf: on Volumr Imtipr Pror~cwrng (VIP ‘Y3) (1993) pp l-4


MuItiscale medial analysis of medical images

Documents

Transcript of MuItiscale medial analysis of medical images