p051 Automated analysis of the American College of Radiology mammographic accreditation phantom...

Automated analysis of the American College of Radiology mammographicaccreditation phantom images

Kenneth W. Brooksa)The Department of Radiation Oncology, The Emory Clinic, 1365 Clifton Road, Atlanta, Georgia 30322

Jon H. TruebloodMedical College of Georgia, Department of Radiology, Medical Physics Section, 1459 Laney WalkerBoulevard, Room AE-2018, Augusta, Georgia 30912-3950

Kimberlee J. KearfottDepartment of Nuclear Engineering, University of Michigan, 108 Cooley Lab, North Campus, Ann Arbor,Michigan 48109-2104

Daryl T. LawtonDigital Imaging and Graphics Group, Microsoft Corporation, Redmond, Washington 98052

~Received 27 January 1995; accepted for publication 11 February 1997!

A significant metric in federal mammography quality standards is the phantom image qualityassessment. The present work seeks to demonstrate that automated image analyses for AmericanCollege of Radiology~ACR! mammographic accreditation phantom~MAP! images may be per-formed by a computer with objectivity, once a human acceptance level has been established.Twelve MAP images were generated with different x-ray techniques and digitized. Nineteen medi-cal physicists in diagnostic roles~five of which were specially trained in mammography! viewed theoriginal film images under similar conditions and provided individual scores for each test object~fibrils, microcalcifications, and nodules!. Fourier domain template matching, used for low-levelprocessing, combined with derivative filters, for intermediate-level processing, provided translationand rotation-independent localization of the test objects in the MAP images. The visibility classi-fication decision was modeled by a Bayesian classifer using threshold contrast. The 50% visibilitycontrast thresholds established by the trained observers’ responses were: fibrils 1.010, microcalci-fications 1.156, and nodules 1.016. Using these values as an estimate of human observer perfor-mance and given the automated localization of test objects, six images were graded with thecomputer algorithm. In all but one instance, the algorithm scored the images the same as thediagnostic physicists. In the case where it did not, the margin of disagreement was 10% due to thefact that the human scoring did not allow for half-visible fibrils~agreement occurred for the othertest objects!. The implication from this is that an operator-independent, machine-based scoring ofMAP images is feasible and could be used as a tool to help eliminate the effect of observervariability within the current system, given proper, consistent digitization is performed. ©1997American Association of Physicists in Medicine.@S0094-2405~97!00205-8#

Key words: mammography, phantom images, image processing, medical imaging

y

oeamtgi

naon

res

e aivedgeresour.is

adiveas-

ocu-elityom

I. INTRODUCTION

An integral portion of the American College of Radiolog~ACR! mammography accreditation program~MAP! is thephantom image quality test, which alone accounts for 25%the MAP’s failures. An additional 10% fail due to both thphantom image quality criterion and a clinical mammogrcriterion.1 The MAP phantoms contain clinically relevantest objects designed to represent typical breast patholofibrils ~or fibers!, microcalcification groups~or specks!, andnodules~or masses!. The objects decrease in size and cotrast sufficiently to demonstrate a visibility threshold usingtypical clinical image technique and viewing apparatus. Fcertification, a facility’s phantom image is analyzed indepedently by three American Board of Radiology~ABR! certi-fied medical physicists who are experienced in mammogphy quality control and trained in reading phantom imag

709 Med. Phys. 24 (5), May 1997 0094-2405/97/24(5)/7

f

es:

-

r-

a-.

The criteria used by the readers has evolved to includsystem of partial scores commensurate with the perceobject visibility as well as a system of deductions for imaartifacts. The three independent object visibility net scoare averaged and must demonstrate visibility for at least ffibers, three microcalcification groups, and three masses2 Arequirement for using phantoms to monitor image qualitynonvariable, consistent viewers.3 However, there are twosources of variability in the MAP process which could leto inconsistent accreditation failure. First, the perceptvariability among medical physicists which has beensessed in previous research.4–6 Second, variabilities in thephantom manufacturing processes have also been dmented in the literature5,7 and may be compounded onto thviewer inconsistency. Ultimately the phantom image quaevaluation test for accreditation, as well as the phant

70909/15/$10.00 © 1997 Am. Assoc. Phys. Med.

aog

ioctheua,astceimetyudpsoheth

regeiprsimothfway-isermses

tioen. Iutneks

-s

c

ueca-

esenthends as

en-hyus,

hed ahasage

r,

ter,sing

tom

710 Brooks et al. : ACR mammographic accreditation phantom images 710

manufacture itself, involve human variabilities, the ramifictions of which may not be acceptable in a national mammraphy program.

The authors have previously proposed a computer-visphantom image analysis approach to mitigate these effe6

Generally such approaches perform poorly, compared tomans, when interpreting complex scenes requiring highorder human skills associated with memory and the viscortex.8 However, the phantom image test objects are stdard geometric shapes on a uniform background whichcomparison to actual patient mammograms, are simplesuggest that perhaps an automated, computer-vision syfor evaluating the phantom images can match human actance levels for test objects in mammography phantomages. Similarly, other researchers5 have demonstrated thsuccess of statistical methods for comparing each faciliaccreditation image against a standard image. In their strelative contrast of the two largest microcalcification grouand three largest nodules was measured on MAP imagevarying quality and a linear, least-squares relationship thigh-quality reference image was assumed. Though tquantitative image measurements did not include all oftest objects in the MAP phantom image~three of the fivenodules and two of the five microcalcification groups weutilized!, the human observer responses for these tarwere well predicted by the quantitative, linear relationshwith much less overall variance than the human observe

We chose another approach to quantitative phantomages analysis. The present work stemmed from earlierserver experiments which demonstrated variability inMAP phantom evaluation process.4 The complete problem oscoring image artifacts was not included in this study. Hoever, all test object groups and all sizes were included ineffort to encounter the human visibility threshold. We hpothesize that if this human visibility contrast thresholdquantified, then a visibility decision based on it may be pformed by an autonomous computer algorithm. Since mamography is predominantly performed via x-ray film, tranparency film digitization devices, appropriate for thphantom evaluation process, have been analyzed andcessful technology identified.9 A description of the phantomand imaging systems used are provided in the next secAlso, the human observer MAP phantom image experimtal design and threshold contrast results are summarizedage processing algorithms which were developed to amatically locate all MAP phantom image test objects apredict their visibility relative to threshold contrast are dscribed and compared to human observers for these tas

II. METHODS

A. Phantom

A commercially available breast phantom@RadiationMeasurement Incorporated~RMI! Model-156 Breast Phantom, Middleton, WI 53562# which meets MAP standardwas used. This standard mammographic phantom~SMP! isconstructed of a 10 cm310 cm34.5 cm thick acrylic blockwith a removable, tissue-equivalent wax insert in one fa

Medical Physics, Vol. 24, No. 5, May 1997

--

ns.u-r-aln-inndemp--

’sy,sofaire

ts

.-b-e

-n

---

uc-

n.-m-o-d-.

e

~serial #312 156 type 4 phantom!. The insert is embeddedwith various sizes of nylon fibers to simulate soft-tissedges, aluminum oxide particles to simulate microcalcifitions, and water-density masses to simulate tumors.10 Theseobjects represent common breast pathologies and are prin sizes that range from being easily visible to invisible in tphantom film image. The wax insert is 0.4 cm thick acontains the fibers, microcalcification groups, and massedepicted in Fig. 1.

B. Mammographic systems

Two dedicated mammography units were used. The Geral Electric ~Senograph 600T Series HF MammograpMachine, General Electric, 92137 Issy les MoulineaFrance! and the LoRad~M-II Mammography Machine, Lo-Rad Medical Systems, Danbury, CT 06811! machines areboth equipped with Molybdenum anodes and filtration. TGeneral Electric machine has a 0.3 mm focal spot size an65 cm source-to-image distance was chosen. The LoRada 0.3 mm focal spot size and a fixed 50 cm source-to-imdistance. All images were taken using standard 18324 cmmammography film~Min R E, Kodak Company, RochesteNY 14445! with a mammographic screen~Min R, KodakCompany, Rochester, NY 14650! and developed with thesame undedicated darkroom film processor~Kodak RPX-omat automatic processor, Kodak Company, RochesNY 14650!. A set of eleven representative phantom filmwere selected from 50 films which were generated us

FIG. 1. Schematic diagram of the mammographic accreditation phan~MAP! showing the locations and relative sizes of the features.

elth

reethey

lite

Cimurfn

e

oann-ot

e-icbl

h-

tion

de-eticalingges

ack

. Toanthef atheat,ce,ela-heileheringign-e-s ofD50r,ireith

ydel-the-ingx-ap-:

ornts,lo-of

son-ject

rv


these two machines and various techniques. The main stion criteria used was the background optical density incenter of the film, which ranged from 0.61 to 2.50. Sincewide range of film densities is acceptable in the MAP, psumably because radiologists preferences span a rangbackground densities, films were selected which spannedrange of qualities expected from facilities participating in tACR MAP @private communication, ACR MammographAccreditation Physicist reader~1992!#. The techniques arelisted in Table I. The machines are subject to routine quacontrol including the ACR MAP certification for both of thdedicated units.

A specimen radiography machine~SRM! ~Faxitron series,43807N X-Ray system, Hewlett-Packard, Pruneridge,95014-9826! was used to produce a reference phantomage. This machine is designed to operate for long exposwithout tube damage. It has a source-to-image distance ocm. The same type of mammography film was utilized awas also processed with an undedicated processor~KodakRP X-omat automatic processor, Kodak Company, Rochter, NY 14650!. The remaining film~Table I, No. 5! wasobtained using a nonclinical technique with a 10 min expsure on a typical specimen radiography unit with the phtom’s wax insert placed directly on the film, without an itensifying screen or cassette. The purpose for using a nmammographic machine and technique for this film wasmaximize object visibility and produce a film which reprsents the upper bound of image quality. The eleven clintechnique films all have less subject contrast and morethan film No. 5.

TABLE I. System configurations and exposure techniques for the obseexperiments.

FilmNo. X-ray unit kVp mAs

Opticaldensity atfilm center

1 LoRad 35 6 1.102 LoRad 30 13 1.213 LoRad 26 40 1.444 LoRad 28 12 1.625 SRMa 20 600 2.516 General

Electric30 115 0.99b

7 GeneralElectric

30 82 0.66b

8 GeneralElectric

22 400 1.11

9 GeneralElectric

26 125 1.25

10 GeneralElectric

35 12 0.61

11 GeneralElectric

26 82 0.96

12 GeneralElectric

26 74 0.69

aSRM5specimen radiography machine.bIncludes 1.3 cm scattering media placed on top of phantom.


ec-ea-ofhe

y

A-es56d

s-

--

n-o

alur

C. Film digitization

A previous study investigated the various digitization tecnologies which could be used for this imaging problem.9 Thesmallest objects in the images are the last microcalcificagroup ~;160/mm diameter!. From Nyquist sampling fre-quency considerations, this indicates a device capable oflivering an 80mm spot size or smaller. Similarly, the rangof optical densities represented spans nearly three opdensity units, which indicates a device capable of deliverinto the thousands of unique gray values. The digital imawere generated with a cooled charge-coupled device~CCD!~Photometrics, Model 2300, Phoenix, AZ! providing 203332045 each at 12 bits. The camera was installed in a BlContainment Box~BCB! with a previously specified lightsource and detector arrangement.11 Those details are brieflysummarized here. The field of view~FOV! of the camerawas adjusted to encompass the phantom image boundaryminimize stray light, the light source was collimated byopaque tray with a 10 cm square opening directly overcenter of the light source. The light source consisted obank of fluorescent cool-white light bulbs arranged alongedges of a two-foot square to yield an approximately flsymmetrical light source in the center. The light souropaque tray, and CCD camera were all adjusted in their rtive positions to yield maximum brightness values in tcenter of the brightest object in the phantom image whminimizing the exposure time and maintaining the FOV. TCCD chip and the square opening were registered dudigitization. This arrangement assured the consistent alment of images during digitization without regard to the dgree of rotation of the phantom image relative to the edgethe film. It also allowed the full spatial resolution of the CCto sample the 10 cm square which yielded approximatelymm pixel in each direction. A Macintosh IIfx computewhich interfaced with the CCD camera, was used to acquand store the images on an optical platter for processing wa standard UNIX workstation and C compiler.

D. Model-based vision computation

The algorithm for analyzing SMP digitized film images mabe characterized as a constrained, two-dimensional, mobased recognition technique. The problem domain foralgorithm involves processing two-dimensional digital images of specific test objects in the SMP images, localizthe objects, and estimating their visibility according to eperimentally measured observer data. The objects andproximate locations are defineda priori. The shapes includerectangular-shaped fibers slanted at645°, circular-shapedmicrocalcifications, and larger, disklike simulated tummasses. In order to accomplish the localization requiremethe approach taken must not drastically alter the spatialcation of the shapes in the images. Constrained rotationthe fibers~45° rectangles! as well as translation of the fiberand other shapes must be allowable. Given these initial cstraints, a template matching scheme was utilized for oblocalization.

er

sp4-

geerrieyaa

o

o

riocoTbcy

imhetm

ncyheingro-

is-

n

n-nys

ng

tiveectthededrayheal-cor-redro-estinem-

t beriermall. In


1. Frequency domain template matching

Figure 2 depicts the template matching process in thetial domain. The digitized images are 2033 pixels by 20pixels ~10 cm square FOV! by 12 bits/pixel due to the resolution requirements for sampling the 160mm microcalcifica-tion group. This dictates a prohibitively large object imaand template image for a spatial domain approach. Thfore, the template matching approach utilized a fast Foutransform ~FFT!, decimation in time, or Cooley–Tukealgorithm.12 The only assumptions are that the array sizessquare and are evenly-divisible by 2. These conditions hbeen met by the proposed digitization scheme. Equation~1!provides the mathematical definition of the continuous, twdimensional Fourier transform, where

F~u,v !5E2`

` E2`

`

f ~x,y!exp@2 j2p~ux1vy!#dx dy.

~1!

The variablesu andv are the associated frequency compnents for thex and y variables. The functionsf (x,y) andF(u,v) are the Fourier transform pairs. The parametej5 A21 is the standard imaginary number. The convoluttheorem provides the means for using frequency domainrelation as an alternative to a spatial domain approach.theorem states that the spatial domain convolution, givenf (x,y)* g(x,y), is equivalent to the corresponding frequendomain relation,F(u,v) • G(u,v), as shown in Eq.~2!:

f ~x,y!* g~x,y!⇔F~u,v !•G~u,v !,

F~u,v !•G~u,v !⇔ f ~x,y!* g~x,y!, ~2!

f ~r ,u1u0!⇔F~v,w1w0!.

This is practically the process of centering the templateage over the first pixel of the object image, multiplying ttemplate’s values by each value underneath it, replacingoriginal image pixels by the product, and moving the te

FIG. 2. Spatial domain schematic depicting the correlation off (x,y) andtemplateg(x,y) at point (s,t) ~adapted from Ref. 13!.


a-5

e-r

reve

-

-

nr-hey

-

he-

plate across the image in the spatial domain. In the frequedomain, this can be accomplished by simply multiplying ttwo Fourier-transformed functions and inverse transformthe product. The continuous version of the convolution pcess is given by Eq.~3!:

f ~x,y!* g~x,y!5E2`

` E2`

`

f ~a,b!g~x2a,y2b!da db.

~3!

Since an image is formed of quantized gray values, Eq.~3!must be cast into discrete form. The two-dimensional, dcrete Fourier transform is given by Eq.~5!, which followsfrom application of Eq.~4! to Eq. ~1!:

f e~x,y!5H f ~x,y!, 0<x<A21 and 0<y<B21

0, A<x<M21 and B<y<N21,~4!

ge~x,y!5H g~x,y!, 0<x<C21 and 0<y<D21.

0, C<x<M21 and D<y<N21,

F~u,v !51

N (y50

N21

(x50

N21

f ~x,y!expF2 j2p~ux1vy!

N G . ~5!

Equation~4! allows f (x,y) and g(x,y) to become discretearrays with finite bounds of sizeA by B andC byD, respec-tively. Equation~6! follows from application of Eq.~4! toEq. ~3!. The valuesM andN are the assumed periodicity ithe x andy directions, respectively:

f e~x,y!* ge~x,y!51

N (m50

M21

(n50

N21

@ f e~m,n!

•ge~x2m,y2n!#,~6!

M>A1C21, N>B1D21.

The issue of wrap-around error is of concern with the covolution process. This issue is treated in detail in mareferences.12,13 To completely avoid this error, the imageshould be adjusted until the conditions forM andN in Eq.~6! are met. It is sufficient to summarize this effect by statithat both the image@ f (x,y)# and template@g(x,y)# need tobe zero-padded out to the maximum positive or negaduration of the objects of interest. For instance, if an objof interest occupied the middle-half of an image, andtemplate was sized similarly, the template must be expanby adding zero values to perimeter locations until its arsize is larger by one-half the dimension of the object. Timage must also be increased in this fashion or, if it isready large enough to meet this requirement, the sameresponding locations must be either zero padded or ignoas they will be corrupted from wrap-around error. The pcess of zero-padding effectively selects a window of interthrough which to view the image. This is unavoidablepractice since most images are of objects which are thselves finite in extent~i.e., nonzero mean!. Unless the imageis band-limited and periodic, all spatial frequencies cannocompletely recovered after forward and reverse Foutransformations have taken place. The effect adds a sdegree of blur to the inverse-transformed resultant image

alaorca

rneatthser

nter

uetolit-i

ophiniti-enss

herierheac-ew

he

d atof

-

r the

er-nfre-heingthe5,for

. 6,theasbe-pre-larentt inancor-asrer-

d

en-e-nalreval-mforesingberfecttheoidd a 1

al-


the current context, this loss of very high-frequency informtion is negligible since the Fourier results are used for retive correlation coefficient estimation only and this errdoes not affect the original image values or the contrastculation.

Since the FFT is a symmetric linear operator, arbitratranslation and rotation of the test images are maintaithroughout Fourier space processing. This is demonstrby Fig. 3 where the power spectrum rotates along withobject, assuring that the FFT will not alter the original tranlation and rotation of the image information. The powspectrum display@Eq. ~7!# allows visualization of the Fouriefrequency domain information. The termsR(u,v) andI (u,v) are

P~u,v !5R2~u,v !1I 2~u,v !, ~7!

The real and imaginary components of the transformed fution, F(u,v). This is also evident in the polar coordinarepresentation of the convolution theorem statement pvided by the last line of Eq.~2!.

2. Object localization algorithm

The implementation of the Fourier convolution techniqas well as other model-based techniques are combined insingle algorithm referred to as the Mammography QuaControl Program~MQCP!. Figures 4 and 5 depict the algorithmic data and control flow. The main assumption madeMQCP is that the digitized input images are cropped ortimally digitized at the apparent edge of the wax insert. Tassumption is readily met in practice by the CCD device athe BCB arrangement. If the film is translated during digzation, the result will become evident in the displacemreport but this will not add error to the localization proce

FIG. 3. ~a! Simple object in spatial domain,~b! power spectrum of simpleobject, ~c! simple object rotated 45°, and~d! power spectrum of rotatedsimple object.


--

l-

ydede-r

c-

o-

ay

n-sd

t

unless the film is shifted more than about 1 cm or more. Trotation- and translation-independent nature of the Foutransform localization methods provides this immunity. T1 cm limit is due to the constraints of the phantom manufturer whereby any displacement error of more than a fmillimeters is sufficient for rejection@Private Communica-tion, RMI-156 Breast Phantom Product Manager~1993!#.Thus, these constraints were utilized in MQCP to limit tsearch areas.

The Fourier-domain template matching approach usethe start of the MQCP provides only a partial measureobject localization. Following Fig. 6, the control flow:~1!reads the image and its dimensions and scale;~2! extracts thefirst subimage and performs its FFT;~3! either reads a precomputed mask-FFT~shaded lines in Fig. 6! for the particu-lar shape or generates a zero-centered, binary mask foshape and performs the FFT;~4! the object FFT and themask FFT are then multiplied, element-by-element, to pform the convolution;~5! the inverse-FFT is performed othe product image;~6! the resultant image is a matrix ocorrelation coefficient values indicating the degree of corlation the template exhibited for the value’s location in toriginal image. This process is shown with correspondimages in Figs. 6, 7, and 8 for the three shapes. Forsecond portion of the MQCP control flow, shown in Fig.the correlation surface resulting from part 1 is searchedmaxima in the search ranges determineda priori and theselocations are stored. Typical maxima are indicated in Figs7, and 8. Since one objective for fibers is to measureamount of rotation, the ideal angle for a particular fiber wused to create the mask and minor angular differencestween the mask and the actual grayscale image do not apciably affect measurement of the underlying fiber anguplacement error. The same argument holds for displacemerror of the other shapes. This effect should be apparenFig. 7, where only a portion of the speck group is used asinput image and the approach localizes the specks in arectly registered, correlation surface image. This is truelong as displacement is no more than the manufactuspecified 1 cm in any direction for all three shapes~includingindividual specks!. The MQCP was not designed or testefor objects displaced beyond this limit.

The fiber correlation surface, shown in Fig. 6, is esstially a ridge of high values which are somewhat noisy, dpending upon the noise and artifact levels in the origiimage. Multi-element, unidirectional derivative filters wetested to determine which would demonstrate the peakues of fiber correlation or maximum ridge. The results frothe filter testing were used to select the optimal filter sizeMQCP’s localization of fiber peak values. The peak valuof fiber correlation can deviate from a straight line dependon the image noise and the amount of dislocation of the firelative to the subimage lateral boundaries. The latter efis due to the frequency-domain errors associated withFFT convolution approach as discussed previously. To avthese effects, the peak values are only considered arouncm square vicinity of the ideal,a priori fiber location. Theimage noise may still cause a discontinuity in the ridge v

aryene area of


FIG. 4. Images corresponding to Mammography Quality Control Program~MQCP! control flow for fibers. The original fiber image and an appropriate binmask are Fourier transformed~the power spectra of each are shown below each original image! and multiplied in the frequency domain. The product is thinverse Fourier transformed to yield the convolution image. The convolution image is depicted by scaling the correlation coefficients to 8 bits. Thmaximum correlation is evident by the darkest area in the vicinity of the original fiber.

isl ale

op

t-

e

luecytodi-

ues which fall inside this spatial constraint. This effecthandled by iteratively selecting portions of the ridge untisection, as long as the particular fiber is wide, gives angfrom a least-squares fit to within 10° of thea priori 45°. Ifthe constraint is not met after ten iterations, MQCP will stsearching and default to thea priori location as the centroidcoordinates. Equation~8! mathematically describes the leassquares fit used by MQCP. The angle was taken as theverse tangent of the slope from the least-squares fit whxi5peak locationi , x coordinate;yi5peak locationi , y co-ordinate;n5total number of peak values:14


s

in-re

angle5tan21F X(i51

n

xiyi2S (i51

n

xi(i51

n

yi D C(i51

n

xi2

S (i51

n

xi D 2n

G . ~8!

The peak values should include the numerically largest vaof correlation in the image. However, a minor discrepancould arise from application of the derivative filter usedfind the peak values. If this occurs, the final centroid coornates for a fiber are taken to be thex coordinate from the

an.to 8 bits.


FIG. 5. Images corresponding to Mammography Quality Control Program~MQCP! control flow for speck groups. The original speck group image andappropriate binary mask are Fourier transformed~the power spectra of each are shown below each original image! and multiplied in the frequency domainThe product is then inverse Fourier transformed to yield the convolution image. The convolution is depicted by scaling the correlation coefficientsThe area of maximum correlation is evident by the darkest area in the vicinity of the original fiber.

isbam

revi-ctedg.mernersthe

maximum correlation location and they coordinate is takenas the peak value corresponding to thex coordinate. Sincethe specks and masses are rotationally symmetric, thereangular error component. The ideal locations for all 16 ojects are schematically depicted in Fig. 9. These locationsmeasured from the edges of the image, thus it is assu


no-reed

that the input images are cropped at the wax edges as pously discussed. The centroid coordinates were selewhich were within 1 cm of the ideal locations shown in Fi9 and the maximum of the correlation surface in the sasearch area. The speck groups are arranged in the coand center of a regular pentagon. The ideal location of


FIG. 6. Part 1— Mammography Quality Control Program~MQCP! algorithmic data and control flow.

thloebrodo

thCeur-

e

mn

cltea

les

edi-

-f ahathehetedt

tesob-d theork.at agory,d toThe6,alsosh-s tod ton-

center of the speck group is assumed to coincide withlocation shown in Fig. 9 where specks occur. The idealcations of the remaining specks in a given group are msured relative to the coordinates of a pentagon centered athe coordinates shown in Fig. 9. The displacement erwere measured by the standard distance formula appliethe MQCP-located centroid coordinates and the ideal codinates for a particular shape.

3. Object visibility

Once the location of a particular shape is estimated,contrast of the shape is determined. The contrast for MQis calculated by Eq.~9!, whereS5signal, average gray valuof located object;B5background, average value of area srounding located object; dark5dark current of camera system ~if applicable!:11

C5S2dark

S1B2dark. ~9!

The background area dimensions were selected whichcompassed enough background area to provide a stableerage gray-value for the region. The MQCP utilizes a coparable area of surround, or greater, on each dimensiothe object as suggested by Chesters.15 The MQCP indicatesthe areas used as signal and background by marking theest outside pixel black at the border of each region of inest. This computer graphics feature allows the MQCP loc


e-a-outrstor-

eP

-

n-av--of

os-r-l-

ization results to be quickly analyzed for accuracy. Sampof this process are provided in the results section.

Final classification of the localized objects is performby a two-hypothesis~binary! Bayesian classifier. The classfication variable is threshold contrast,Ct, and the decisionfor visibility is assumed to be at 50% visibility. Implementation of this classifier is accomplished by comparison oparticular shape’s contrast with the threshold value for tshape and establishing visibility if it is greater than tthreshold and nonvisibility if it is less than the threshold. Tobserver data were taken from previously presenmeasurements.4 Section III contains the determination of Cfor the three shapes used in MQCP.

Testing of MQCP against humans for ACR passing rawas performed for six images which had a moderateserver response rate. That is, about half as many passeimages as failed them. Human observers in the present wdid not include scoring of artifacts or partial object visibilityThe moderate response rate for these films assured thhuman threshold was encountered in each shape catebecause a portion of the human observer population hafail the images for the response rate to be moderate.images used were digitized from film numbers 1, 2, 3, 5,and 10 of those listed in Table I. The same images wereused to train the binary classifier by establishing the threold contrasts for each shape. While using the same imageboth establish a threshold and test the threshold could leabias in the classifier, the objective of this work is to demo

naryenmaximum


FIG. 7. Images corresponding to Mammography Quality Control Program~MQCP! control flow for masses. The original mass image and an appropriate bimask are Fourier transformed~the power spectra of each are shown below each original image! and multiplied in the frequency domain. The product is thinverse Fourier transformed to yield the convolution image. The convolution is depicted by scaling the correlation coefficients to 8 bits. The area ofcorrelation is evident by the darkest area in the vicinity of the original fiber.

nttriat

raujefile

ane-hthh

ntemleonan

as aity

ndngler’sen-. 9.thater-lsoes ater-lityorinlylmups

strate that automated processing is feasible. Establishmea universally applicable threshold or other decision mederived from actual ACR cases is necessary for futurevancement of such an approach, but it is not addressed inwork.

For displacement error testing, three SMP wax insewere used from a selection of rejected SMPs from the phtom vendor. These inserts were specifically rejected becaof their displacement errors. Each image represents a retion based upon one shape. The known errors consist of:1, the fifth fiber is significantly rotated past 45°; film 2, ththird speck group has a severely displaced speck; and filmthe last mass is severely displaced. The errors were qufied by physically estimating displacement or rotational dviation from estimated ideal conditions. The SMP film images were digitized and the pixels manually counted. Tideal conditions were estimates used to demonstrateMQCP could quantify errors. Comparison of MQCP and tmeasured errors are provided in the results in Table II.

After MQCP has estimated location, errors, contrast, avisibility for all shapes, final output reports are generasummarizing this information. Sample output results froMQCP for a test image are provided in the results in TabIII and IV. They include displacement errors and rotatierrors for fibers, displacement errors for speck groups


ofcd-his

tsn-sec-m

3,ti--

eate

dd

s

d

masses, and contrast information for the above as wellfinal ACR passing decision based on the object visibilscores.

III. RESULTS

A. Automated localization performance

Table II contains the comparison between MQCP ameasured distances for displacement errors as well as aerror for the three films which failed the phantom vendoquality control requirements. The distances given are ctroid displacement distances, measured relative to FigBoth the angle errors and the distance errors indicateMQCP is capable of tracking displacements and angularror relative to any arbitrary reference frame. This is aevident from the results shown in Fig. 10 for film No. 5. Thobserver passing response for this film was unanimou100%.4 These results are indicative of the localization pformance expected from MQCP when a very good quaimage is provided and high-resolution digitization used. Ffilm No. 2, MQCP’s localization performance is shownFig. 11. These results are indicative of a film which is onmarginally meeting the ACR passing rate criterion. This fiaveraged less than 50% passing rate from all of three gro


FIG. 8. Part 2—Mammography Quality Control Program~MQCP! algorithmic data and control flow.

si-cts

ncyin

rob-

rohidufeed

andcom-the

FIG. 9. Ideal centroid coordinates for Mammography Quality Control Pgram ~MQCP! displacement error calculations. The lateral edges of tideal reference frame corresponds to the edges of a square mask usedthe digitization of each phantom image. This provided a convenient reence frame and allowed the spatial resolving capabilities of the CCD camto operate only on the image contents without wasted space around theedges.


of 30 observers each.4 The image is noisy and all of theobjects are not necessarily visible to the human eye.

B. Automated vision performance

The results from training MQCP’s binary decision clasfier are shown in Fig. 10. The low spatial frequency obje~fibers and masses! exhibit very similar contrast visibilityand are distinct from the response for high spatial frequeobjects~specks! for the observers tested. These results areagreement with similar results presented previously.15 Thethreshold contrast values corresponding to the decision p

-sringr-raark

TABLE II. Mammography Quality Control Program~MQCP! localizationperformance compared to phantom vendor quality control. The angledistance measurements were made manually on the film image. Theputed~MQCP! estimates for angle and distance were made relative toideal reference frame shown in Fig. 9.

Filmno. Error Angle~°!

MQCPangle~°!

Distance~cm!

MQCPdistance

~cm!

1 fiberangle

72 69.7 n/a n/a

2 speck n/a n/a 1.64 1.783 mass n/a n/a 1.93 2.04

dtes therelativesed for

2288977965


Medical Physics, Vo

TABLE III. Mammography Quality Control Program~MQCP! summary output report. Contrast, visibility, andisplacement error for film 5. The first half of this table gives the computer measured contrast and indicapass/fail decision. The last half provides an example summary displacement report. The displacement isto the ideal reference frame outlined in Fig. 9. This shows how a computerized approach could be udocumenting manufactured locations for quality assessment.

Contrast and visibility reportFibers Contrast~min. 1.010! Visible

1 1.093 Yes2 1.079 Yes3 1.069 Yes4 1.060 Yes5 1.034 Yes6 1.031 Yes

Number of fibers visible: 6 ~ACR requirement: 4!

SpeckGroups

Contrast~min. 1.156!1 2 3 4 5 6 Visible

1 1.276 1.329 1.342 1.327 1.318 1.309 Yes2 1.570 1.578 1.544 1.534 1.323 1.602 Yes3 1.502 1.546 1.578 1.571 1.610 1.611 Yes4 1.338 1.332 1.441 1.360 1.453 1.375 Yes5 1.130 1.132 1.101 1.107 1.067 1.089 NoNumber of speck groups visible: 4 ~ACR Requirement: 3!

Masses Contrast~min. 1.016! Visible1 1.229 Yes2 1.121 Yes3 1.098 Yes4 1.057 Yes5 1.023 Yes

Number of masses visible: 5 ~ACR requirement: 3!

Displacement report~mm from ideal!

Fiber No. Right~1!Left~2!

Up~1!Down~2!

Centroiddistance

Rotation angle~degrees!

1 21.498 20.005 1.498 45.3242 6.840 0.277 6.846 246.6853 11.094 20.756 11.120 45.9874 20.277 21.366 20.323 243.9535 21.545 210.643 10.755 44.6786 4.305 25.432 6.93 245.000

Centroid distanceSpeck

group No.1 2 3 4 5 6

1 13.507 13.493 14.962 15.441 13.515 11.62 19.902 19.325 21.255 21.453 19.265 17.83 12.599 10.662 12.337 15.005 14.330 12.24 13.405 12.598 14.697 15.892 14.956 12.95 18.485 16.799 18.746 15.410 15.756 15.7

Mass No.Right~1!Left~2!

Up~1!Down~2!

Centroiddistance

1 18.587 212.174 22.2192 20.136 219.103 19.1043 0.418 219.291 20.3314 12.784 219.291 23.1435 18.164 217.930 25.523

6;P

nVar

nforack-me

on

ability of 50% visibility were: fibers, 1.010; specks, 1.15and masses, 1.016. These values were utilized in MQCtest for visibility of located objects.

The results from testing MQCP with six test images acomparing them to human judgments are shown in TableThe contrast, visibility, and displacement error results

l. 24, No. 5, May 1997

to

d.e

summarized in Tables III and IV for the two images showin Figs. 11 and 12, respectively. Localization graphicsthese images indicate the areas used for signal and bground in the contrast calculation. The MQCP gave the saobject scores as humans for film 5~Fig. 10.! This was evi-dence of the ability of the system to correctly operate

dtes therelativesed for

3146657506


Medical Physics, Vo

TABLE IV. Mammography Quality Control Program~MQCP! summary output report. Contrast, visibility, andisplacement error for film 5. The first half of this table gives the computer measured contrast and indicapass/fail decision. The last half provides an example summary displacement report. The displacement isto the ideal reference frame outlined in Fig. 9. This shows how a computerized approach could be udocumenting manufactured locations for quality assessment.

Contrast and visibility reportFibers Contrast~Min. 1.010! Visible

1 1.019 Yes2 1.014 Yes3 1.009 No4 1.002 No5 1.003 No6 1.005 No

Number fibers visible: 2 ~ACR requirement: 4!

SpeckGroups

Contrast~Min. 1.156!1 2 3 4 5 6 Visible

1 1.123 1.201 1.111 1.076 1.167 1.105 No2 1.195 1.193 1.122 1.162 1.083 1.165 Yes3 1.166 1.234 1.236 1.229 1.245 1.178 Yes4 1.101 1.123 1.131 1.169 1.138 1.145 No5 1.115 1.085 1.142 1.099 1.115 1.074 NoNumber speck groups visible: 2 ~ACR requirement: 3!

Masses Contrast~Min. 1.016! Visible1 1.061 Yes2 1.023 Yes3 1.021 Yes4 1.003 No5 1.007 No

Number of masses visible: 3 ~ACR requirement: 3!

Displacement report~mm from ideal!

Fiber No.Right~1!Left~2!

Up~1!Down~2!

Centroiddistance

Rotation angle~degrees!

1 2.493 6.991 7.422 45.5672 6.418 0.230 6.422 245.7493 8.700 0.324 8.706 45.1344 11.451 6.991 13.416 246.9875 23.469 0.812 3.563 46.0056 8.296 0.390 8.305 246.003

Centroid distanceSpeck

group no. 1 2 3 4 5 61 10.957 11.0691 12.404 12.802 10.924 9.12 17.200 16.713 18.468 18.612 16.860 15.63 10.876 9.066 10.413 13.049 12.710 10.84 12.480 10.183 12.161 13.514 12.717 10.65 14.778 13.563 20.137 15.850 15.165 13.3

Mass No. Right~1!Left~2!

Up~1!Down~2!

Centroiddistance

1 17.742 29.732 20.2362 22.155 216.991 17.1273 4.681 217.460 18.0774 9.920 211.310 15.0445 21.028 212.577 24.503

bereeteg

was-thees

extremely good images. Film 2 gave conflicting resultstween MQCP and the trained diagnostic physicists. Theson film 2 failed was due to the fact that only half of thfourth fiber is actually visible. The observers were instructo make a judgment between visible or nonvisible, thou

l. 24, No. 5, May 1997

-a-

dh

40% of the trained observers commented that the fiberhalf-visible while viewing this film. Thus the contrast is lowered by an appreciable amount, causing it to drop belowthreshold. In this instance, the binary classifier still comvery close to reaching the same decision as humans~off by

simade

lutesedithlmint,in am-liza-ltersghw-ingco-ivendofned

rv


10%!. The specks and masses for film 2 registered passcores with MQCP as they did with humans. A more coplex classifier, such as a multi-hypothesis decision rule, mprovide closer results in these cases. The MQCP passefailed the remaining films the same as the human observ

FIG. 10. Mammography Quality Control Program~MQCP!-measured con-trast versus percent visibility for fibers, specks, and masses. The obsewere specialists trained in diagnostic physics.


ng-yorrs.

IV. DISCUSSION

Since current SMP designs do not provide an absoreference frame such as fiducial markers, the methods uhad to locate the test objects autonomously. To help wthis, a digitization mask was created which cropped the fiimage at the apparent edge of the wax insert. At this pothe images were positioned so that the shapes wereknown order. Having properly digitized the images, the iage processing was performed in two stages: object location and object visibility. First, object localization is cruciasince no attempt to model visual responses with compucan work without first finding gray values which have a hiprobability of being related to the correct objects. Thus, lolevel processing utilizing Fourier domain template matchwas employed to provide a registered map of correlationefficients. Intermediate-level processing utilized derivatfilters operating on the correlation coefficient map to filocal maxima. This terminology is consistent with thatMaar.16 These results look promising for salient localizatioof the test objects in SMP images. The algorithms perform

ers

FIG. 11. Mammography Quality Control Program~MQCP! localization, film 5.


FIG. 12. Mammography Quality Control Program~MQCP! localization, film 2.

lityssa

at

trastble

bitllyt setifier.s inatege-ly.ep-e

las-

theys-t in

at least as well as humans and without human variabiThis was to be expected since the computer has accemuch more information than the human eye can processis in agreement with other published results.5 The final stageof processing was the high-level classification which wmodeled by a Bayesian classifier using threshold contras

TABLE V. Mammography Quality Control Program~MQCP! passing rateresults compared to human observers.

Film No. MQCP judgmentTrained-physicist ACR

passing rate

1 Fail ~,50% probability! 40%passed2 Fail ~,50% probability! 60%passed3 Pass~.50% probability! 100%passed5 Pass~.50% probability! 100%passed6 Fail ~,50% probability! 20%passed10 Fail ~,50% probability! 40%passed


.tond

sas

measured from the target observer group. Threshold conhas previously been identified as a useful predictor variafor estimating human visibility.5,17 The performance byMQCP, coupled with a cooled CCD 2033 by 2045 by 12camera digitizer, is in good agreement overall with speciatrained human observers. However, the same image teswas used to establish the threshold and test the classWhile using the same images to do this could lead to biathe classifier, the objective of this work was to demonstrthat automated processing is feasible, not to establish aneric classifier threshold which would apply universalNow that this has been shown, any model of human perction that is effective may be used for the classification. Ware currently considering other models of human visual csification for both the test objects and artifacts.

In any system useful for a national program, there isadditional requirement of artifact analysis. The current stem of deductions and half-visible object scores was no

emthteecuafotininsinecor

ulgitufeitbldnoilla

ol

--

i-

2-5

a-rver

ivehys.

-ys.

CR

o--ys-

in

ri-

ry,

in

or


place at the time our original observer measurements wtaken and we have no set of human responses for the alunlimited random shapes presented by artifacts. We feelthe role for MQCP is to preprocess the images to find theobjects. After the objects are located, then subtract thfrom the image and process the remaining image area foing on artifacts. We are currently working to establish a dtabase of known artifacts from which to devise a methodat least developing a computer-assisted tool for documenartifacts and deducting points. In addition, we are extendthis work by investigating phantom design enhancementprovide an all-encompassing quality assessment includspectral information, entrance exposure information, dirmeasurement of noise power, and MTF to provide mquantitative, objective measures of image quality.

In summary, machine interpretation is plausible and cobe used in conjunction with the current MAP phantom imaevaluation process to improve consistency and objectivThe techniques used for object recognition and scoring cobe extended to image artifacts augmented by decision inence rules. Digitization devices which meet the requistechnical performance criterion are commercially availaand must be used consistently. Such an approach coulused to assist with elimination of manufacturing errors aprovide an autonomous screening tool for future mammgraphic facility accreditation. The ultimate significance wbe apparent when the implementation of such an approprovides more consistent, high-quality mammograms.

a!Electronic-mail: [email protected]. C. Martin and J. B. Smathers, ‘‘Requirements of the American Clege of Radiology Mammography Accreditation Program,’’ inCurrentRegulatory Issues in Medical Physics, proceedings of an American College of Medical Physics Symposium~Medical Physics Publishing, Madison, WI, 1992!.


reostatstms--rggtogte

dey.ldr-eebed-

ch

-

2P. A. Wilcox, ‘‘American College of Radiology Mammography Accredtation Program,’’ in24th Annual Conference on Radiation Control, Pro-ceedings of a CRCPD Conference, Orlando, FL, CRCPD-Pub. 9~1992!, pp. 145-152.3NCRP Report #85,Mammography~National Council on Radiation Pro-tection and Measurements, 1989!.4K. W. Brooks, J. H. Trueblood, and K. J. Kearfott, ‘‘Subjective evalutions of mammographic accreditation phantom images by three obsegroups,’’ Inv. Radiol.29, 42–47~1994!.5D. P. Chakraborty and M. P. Eckert, ‘‘Quantitative versus subjectevaluation of mammography accreditation phantom images,’’ Med. P22, 133–143~1995!.6K. W. Brooks, J. H. Trueblood, K. J. Kearfott, and D. T. Lawton, ‘‘Automated analysis of mammographic quality control images,’’ Med. Ph20, 881 ~1993!.7R. K. Cacak, ‘‘Inconsistencies in mammographic phantoms used for Aaccreditation,’’ Radiology181, 288 ~1991!.8A. Rose,Vision: Human and Electronic~Plenum, New York, 1973!.9J. H. Trueblood, S. E. Burch, K. J. Kearfott, and K. W. Brooks, ‘‘Radigraphic film digitization,’’ inDigital Imaging Proceedings of 1993 Summer School, edited by W. R. Hendee and J. H. Trueblood, Medical Phics Monograph 22~Medical Physics Publishing Co, Madison, WI, 1993!.

10T. G. Langer, E. S. de Parades, S. Agarwal, and D. Smith, ‘‘QAMammography, Part 1: QA Physics,’’ Applied Radiology~1992!.

11J. P. Peng, K. J. Kearfott, J. H. Trueblood, and T. B. Kingsbury, ‘‘Vaable illumination light box for film digitization,’’ Med. Phys.18, 832~1991!.

12W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. FlanneNumerical Recipes in C, The Art of Scientific Computing, 2nd ed.~Cam-bridge U.P., Cambridge, 1992!.

13R. C. Gonzalez and R. E. Woods,Digital Image Processing~Addison–Wesley, New York 1992!.

14R. L. Scheaffer and J. T. McClave,Probability and Statistics for Engi-neers, 2nd ed.~Duxbury, Boston, 1986!.

15M. S. Chesters, ‘‘Human visual perception and ROC methodologymedical imaging,’’ Phys. Med. Biol.37~7! ~1992!.

16D. Maar,Vision ~W. Freeman, New York, 1982!.17A. Burgess, ‘‘Visual signal detection. III. On Bayesian use of priknowledge and cross correlation,’’ J. Opt. Soc. Am.2, 1498–1507~1985!.

p051 Automated analysis of the American College of Radiology mammographic accreditation phantom...

Documents

Transcript of p051 Automated analysis of the American College of Radiology mammographic accreditation phantom...