IEEE TRANSACTIONS ON COMPUTERS, VOL. C-31, NO. 10, OCTOBER 1982

Content-Addressable Read/Write Memories forImage Analysis

WESLEY E. SNYDER, SENIOR MEMBER, IEEE, AND CARLA D. SAVAGE

Abstract-Two common problems in image analysis are de-scribed-the region-labeling problem and the clustering problem. Bothare shown to be instances of a search-and-rename problem which canbe solved in parallel by a system architecture inherently suitable forVLSI implementation.

That architecture, a novel type of content-addressable memory, isdescribed, and its application to search-and-rename problems is dis-cussed.

Index Terms-Clustering, content-addressable memory, datastructures, graph algorithms, image analysis, minimum spanning tree,region labeling.

I. INTRODUCTION

THE recent advances in microelectronics have made itfeasible to consider new system architectures for solving

data manipulation problems. In particular, it is no longernecessary or desirable to separate the functions of memory andcomputing as is done in a von Neumann machine. The newarchitectures which are evolving share two main principles:they make use of regular structures which distribute the ele-ments of memory and computation over the area of the system;and they tend to be applications-driven, and therefore, in somesense, "special purpose."

By use of the word "application" here, we mean a particularmathematical problem or class of problems, rather than thespecific task environment in which the mathematical operationis to be used. In this paper, we will show that there is a largeclass of image analysis problems which can be solved with aparticular set of mathematical tools. These are problems suchas region identification and clustering, which we will show tobe instances of a search-and-rename problem.

Section II describes general techniques for solving search-and-rename problems. In Section III, these techniques areapplied to the problems of region identification and clustering.We will then show, in Section IV, that a particular ipemory/processing architecture, based on a new type of content-ad-dressable architecture, provides a highly parallel, efficientsolution to these problems, which is likewise easily imple-

Manuscript received January 4, 1982. This work was supported in part bythe National Aeronautics and Space Administration under Grant NAG 1-20,in part by the U.S. Army Research Office under Contract DAAG29-82-K-0070, and in part by the DFVLR.W. E. Snyder is with the Department of Electrical Engineering, North

Carolina State University, Raleigh, NC 27650.C. D. Savage is with the Department of Computer Science, North Carolina

State University, Raleigh, NC 27650.

mented in VLSI. Related work in the literature is discussedin Section V.

II. A DATA STRUCTURE FOR DISJOINT SETS

A data structure can be viewed as a structured collection ofobjects, together with operations to be performed on the col-lection. A STACK, for example, is a data structure whosestructured collection of objects is an ordered sequence of ele-ments, one end of the sequence designated as "top," with theoperations of PUSH (add an element to the top), POP (deletethe top element), TOP (read the top element), and EMPTY (testfor empty). The operations can be classified as "updates,"which alter the structured collection of objects (PUSH, POP)and "retrieves," which only return information about thecollection (TOP, EMPTY).The data structure which is of significance for our purposes

is the following. The objects are disjoint sets of integers in therange 1, - - *, n. There are two operations: FIND (X), a retrieveoperation which returns the name of the set containing theinteger X, and UNION (A, B, C), an update operation whichreplaces sets A and B of the collection of disjoint sets by theset C = A u B. Posed in terms of a search-and-rename prob-lem, FIND (X) corresponds to a search for the name of the setcontainingX and UNION (A, B, C) corresponds to renamingboth A and B as C, that is, every element previously in the setnamed A or the set named B is now, instead, in a set named C.We show in the next section that such an approach can be usedto solve the region-labeling problem.The disjoint set data structure has been studied extensively

as a component of sequential and parallel algorithm design formany combinatorial problems [1], [6], [11]. Very efficientimplementations for it exist: the disjoint sets can be representedin such a way that any intermixed sequence ofm UNION andFIND operations can be performed in time practically linearin m [14].The disjoint set data structure arises in a natural way in

several problems. These include computing EQUIVALENCEof Fortran variables, testing equivalence of finite state auto-mata, and determining connectivity properties of graphs [ 1 ].We focus now on graph connectivity, which has applicationsto both region labeling and clustering in images, as describedin Section III.A graph G = (V, E) consists of a finite set V = I,* nIof

vertices and a set E of pairs of vertices, called edges. A pathin G is a sequence voeovIel ... vkekVk+ 1 where, for 0 < i <k + I, v1 e Vand, forO < i < k, ei isan edge inE joiningvertices vi and vi+ 1. The path joins vertices vo and Vk+ 1. G is

0018-9340/82/1000-0963$00.75 © 1982 IEEE

963

IEEE TRANSACTIONS ON COMPUTERS, VOL. C-31, NO. 10, OCTOBER 1982

said to be connected if every pair of vertices of G is joined bya path.

This definition of a graph does not require an edge betweenany two vertices. It does not even require that a path via othervertices exist. A graph may thus consist of several sets of ver-tices which are themselves connected, but are not connectedto other sets. Such sets are called "connected components."An example of an undirected graph with three connectedcomponents is shown in Fig. 1.The disjoint set data structure can be used to determine if

G is connected in the following way. Maintain disjoint sets ofvertices of G. Initially, each vertex v is in a set by itself, named"v." The edges of G are scanned sequentially. When an edgee = (u, v) is scanned, determine to which sets u and v currentlybelong. If they are in different sets, combine these into a singleset. That is, perform the sequence of operations

A- FIND (u)

B - FIND (v)IF A /- B THEN UNION (A, B, C)

where C can be chosen to be either A or B. It can be showninductively that if at some point in this procedure E' is the setof edges processed so far, then vertices u and v are in the sameset if and only if they are joined by a path in the graph G' = (V,E'). Thus, upon termination of this procedure, G is connectedif and only if all vertices are in the same set. If G is not con-nected, the algorithm finds the connected components of G.

If G has m edges and n vertices, a sequence of at most 2mFINDS and n - 1 UNIONS suffices to solve this problem. Aswe will show in the next section, this technique is particularlyappropriate for solving the region-labeling problem in a rasterscan fashion. An efficient sequential software implementationof the disjoint set data structure will allow this entire sequenceof operations to be performed in time which is essentially linearin m [14]. A linear-time hardware implementation of thedisjoint set data structure on a one-dimensional systolic arrayof n cells is presented in [11 ]. In Section IV, we show that aCAM-based architecture is well suited to a parallel imple-mentation of the disjoint set data structure.A spanning tree of a connected graph G = (V, E) is a

subgraph T = (V, E') ofG (that is, E' c E) which is connectedand contains no cycles. The procedure described above to de-termine connectivity of G can be modified to find the edge setE' of a spanning tree of G. We simply include in E' exactlythose edges which, when scanned, join vertices in distinctsets.

If each edge e of G is assigned a cost, we can associate withany spanning tree T of G a cost equal to the sum of the costsof the edges in T. A minimum spanning tree for G is aspanning tree of minimum cost. Computing a minimumspanning tree is the basis for the cluster algorithm describedin Section IlI. It can be shown [1] that if the edges of G arescanned in order of nondecreasing cost, the spanning treeprocedure described above will actually produce a minimumspanning tree.To summarize this section, each of the following problems

can be solved by maintaining disjoint sets of integers in therange 1, *, n and performing only the operations of UNIONand FIND on the sets: determining connected components (or,

Fig. 1. A unidirected graph with three connected components.

in fact, the equivalence classes of an equivalence relation),finding a spanning tree of a graph, and computing a minimumspanning tree.

III. APPLICATIONS TO IMAGE ANALYSIS

One central problem arising in scene analysis is the parti-tioning of an image into a set of meaningful regions. Theseregions are composed of all pixels that have similar attributes(such as gray level) and that are connected to each other. Oneof the most powerful approaches to this task is "region grow-ing." This technique is initiated by choosing a pixel whichmeets the criteria for inclusion in a region. It then proceeds byexamining all adjacent neighbors of the pixel and comparingthat pixel with the neighbor in question. Typical measures ofsimilarity include the magnitude of the neighboring pixel'sgray level or the relative contrast between the pixel and itsneighbor under consideration for inclusion in the region. Thisprocess is repeated recursively for all newly accepted pixelsuntil no new pixels can be added to the region. Then a newregion is "grown" around a pixel which has not been yet as-signed to a region. When all pixels have been assigned to re-gions, the algorithm terminates. Since the region-growingtechnique always results in closed regions, this technique isoften preferable to other techniques which are based on edgedetection or line fitting.

The algorithm given in the previous paragraph is recursivein structure; however, data typically enter an imaging systemin a line-by-line raster scan fashion. Snyder and Cowart [13]have shown that a structure can be developed to perform la-beling in a raster scan manner as the data enter. An alternativeapproach to high-speed labeling has been proposed in a reportto the Army Night Vision Laboratory [9].The algorithm for region partitioning presented in [13] is

a region-growing technique based on the concept of equiva-lence relationships between the pixels of an image. Two pixelsa and b are defined to be equivalent [designated R(a, b)] ifthey belong to the same region of an image. This relationshipcan be shown to be reflexive [R(a, a)], symmetric [R(a, b) 4

R(b, a)], and transitive [R(a, b) and R(b, c) > R(a, c)].The transitive property enables all pixels in a region to be

determined by considering only local adjacency properties. Inthis algorithm, each pixel will be compared first to the pixelto its left and then to the pixel above itself in a left-to-right,

964

SNYDER AND SAVAGE: MEMORIES FOR IMAGE ANALYSIS

top-to-bottom raster scan fashion. The assignment of a regionlabel to a pixel results from the comparison operation. Fig. 2demonstrates the situation that can arise as a result of thiscomparison. Pixels in a simple binary image are being labeledin a raster scan order. The region labeling proceeds in astraightforward manner until the equivalence R(1, 2) is dis-covered at the pixel designated by the question mark.Now, the transitive labeling property comes into play. Since

region 1 has been discovered to be equivalent to region .2, wewould like to relabel all pixels whose label is a 2, and changethat label to a 1. (We use the convention that when distinctregions i and j are discovered to be equivalent, the labels of thepixels in the region with the larger label are the ones which areupdated.)

This problem can be solved using the disjoint set datastructure of Section II. Imagine the n pixels of the image to belabeled 1, . , n. Initially, each pixel p is in a region by itself,labeled p. In a raster scan fashion, we determine whether ad-jacent pixels should be included in the same region. Wheneverit is determined that pixels p and q should be included in thesame region, we perform the following sequence of opera-tions.

Pixels enter the system in a top-to-bottom, left-to-rightmanner. When a new pixel enters, it is first compared to thepixel above and to its left. (This means that the image memoryreally only needs to be \In + 1 words long, assuming n pixelsin a square array.) If the pixel in question belongs to the sameregion as either of these, it is labeled by placing the appropriatenumber in the corresponding location in the label memory.

Should this labeling be ambiguous, as in the case of Fig. 2,then a UNION operation is done by updating the equivalencememory.

Since video data enter the system in a serial manner, thespeed of operation is never required to exceed the input speed.Labels are chosen for pixels by doing a FIND lookup and usingthe lowest numbered equivalent label. Use of the lowestnumber guarantees a minimum number of labels in oneequivalent set. Furthermore, by following this convention, andresolving each equivalence as it occurs, the "chaining" ofequivalences never occurs. Instead, each equivalent label isalways directly mapped to the lowest numbered label.The operations mentioned above are performed by the

high-speed processor, subject to the following flowchart:

A FIND (p)B.* FINP (q)IF A #£ B THEN C - minimum ofA and B;

UNION (A, B, C).

Upon termination, each disjoint set corresponds to a region.Since each of the n pixels is compared to at most two adja-

cent pixels, the region-labeling problem has been solved byperforming a sequence of at most 4n FINDS and n - 1UNIONS. As described in Section II, efficient sequentialimplementation of the disjoint set data structure will allow thisentire sequence of operations to be performed in time almostlinear in n. In Section IV, we propose a hardware parallelimplementation.

It should be mentioned here that the region-growing tech-

1

11

1

1

1

1

1

1

2

2

2

2

2

2

2

1 I E|1 1

Fig. 2.

nique could just as easily be based on a neighborhood of eightpixels surrounding a given pixel rather than only four adjacentpixels. Then, as pixels are examined in a raster scan fashion,each pixel will be compared to the pixel to its left and then toits three neighbors above. In this case, the region-labelingproblem can be solved by performing a sequence of at most 8nFINDS and n - 1 UNIONS.

B. Clustering

In many pattern recognition and/or image processing ap-plications, data can be determined to be in clusters in somemeasurement or parameter space. For example, the Houghtransform [7] maps points in Cartesian space which are col-linear or nearly collinear to a cluster in the parameter space.Likewise, sets of pixels representing the same type of vegetationform clusters in the multidimensional space defined by amultispectral scanning satellite such as LANDSAT.A technique often used for finding and identifying such

clusters is based on computing a minimum spanning tree(MST). Consider the graph G whose vertex set is the set ofpixels under consideration and whose edge set consists of edgesjoining every pair of distinct vertices. The cost of the edgejoining pixels p and q is just the "distance" between p and qaccording to the metric of interest. The technique consists ofsimply determining the MST for this graph, and then dividingthat tree into clusters by breaking the longest edge. Fig. 3(from Duda and Hart [4]) shows the results of applying thistechnique to some sample data.As described in Section II, the disjoint set data structure can

be used to find a minimum spanning tree, and therefore willsolve the clustering problem.

In the next section, we discuss an architectural approachwhich provides a solution for region labeling and clusteringand, in fact, for any problems which cFan be solved by thedisjoint set manipulations UNION and FIND.

IV. A UNIFIED ARCHITECTURAL APPROACH

We have shown that certain image analysis problems arevery similar in that they can be solved with the same simple setof data structure operations. In this section, we will discuss anarchitectural mechanism for solving these problems, an ar-chitecture which exhibits the high throughput required forreal-time operations. This architecture is based on the con-tent-addressable memory (CAM).

A. Content-Addressable MemoryDifferent forms of content-addressable memory may be used

in computer technology. These forms have arisen in part dueto the varied expectations people have [8], including

1) filing of documents by association,2) production of intelligent function and systems,

965

IEEE TRANSACTIONS ON COMPUTERS, VOL. C-31, NO. 10, OCTOBER 1982

(a)

(b)

(c)Fig. 3.

3) replacement of conventional random access memoriesby structures from which operands could be addressed by datacontents, and

4) programmed sequential control operations.A content-addressable memory can be defined as a memory

in which memory cells are activated according to the datastored in those cells rather than by the address of the cell. Datamanipulation by content-addressable or "associative" lookupis not a new idea since the concept has been around since atleast 1956 [12]; however, associative processors were not putto practical use until the development of PEPE [3] andSTARAN [10]. We do not survey the literature in this areahere, since an excellent survey is provided in [ 16]. The recentadvances in VLSI make the implementation of associativesystems more feasible now, since solutions now exist for thetwo major problems: implementation cost and interrogationdrive (loading of data buses).

In this section, we discuss a particular type of content-addressable memory system which implements the disjoint setdata structure in a natural way. That system is of the "fullyparallel word organized" (to borrow the notation of Yu andFung [16]) class of machine, and thus can be compared toPEPE. Our philosophy for the development of VLSI systemsis very strongly applications-driven. While PEPE is certainlymotivated by a particular application, missile tracking [2],each processing element is in some sense a general-purposeprocessor with associative lookup capabilities. Our designphilosophy is. motivated by specific mathematical operationswhich need to be performed to solve general problems. Thus,the associative processing we discuss is motivated by and tai-

lored specifically to those mathematical operations. That is,our hardware design implements the disjoint set data structure,and is therefore an appropriate design for solving any problemwhich can be solved efficiently by maintaining disjoint sets andperforming UNIONS and FINDS. Region labeling and clus-tering are two important examples of such problems.

B. An Example System: Region Labeling

In this section, we show how a content-addressable chipcould be used in a practical, high-speed image analysis system[ 1 3] . The operation to be performed is region labeling, that is,marking all pixels in the same region with the same labelnumber.The system makes use of three memories, as shown in Fig.

4.The image memory contains the intensity values corre-

sponding to scene brightness. Such a memory is typically512 X 512 X 8.The label memory holds, for each pixel, a number describing

to which region that pixel belongs. The word length in thismemory depends upon how many different region labels areallowed. A 512 X 512 X 10 bit memory, for example, allowsup to 1024 different regions. An 18 bit memory would allowevery pixel to have a unique label, and while this is worst case,it is so unlikely as to be unreasonable.The equivalence memory, a dual-mode CAM, is the heart

of the labeling system. It contains the equivalence relationsbetween region labels. That is, location i of this memory con-tains the smallestj such that region i and region j are the same.In Fig. 4, regions 1, 3, 4, and 9 are all equivalent, as are 2, 5,and 8.A FIND operation thus consists of two memory reads: "FIND

to what region the pixel at (124, 215) belongs" consists ofreading the label memory at 124, 215 and presenting thenumber stored there to the equivalence memory as an address.The value in that address is presented to the CPU as the correctlabel.The UNION operation makes use of content addressability

to update the equivalence memory. If two regions, i and j, aredetermined to be equivalent (as regions labeled 1 and 2 in Fig.2), then the high-speed processor updates the equivalencememory as follows:

1) a = FIND(i)2) j = FIND(j)3) y = min (a, /)4) replace all occurrences of max (a, 3) in the equivalence

memory by y.Step 4) is implemented with a content-addressable write.The equivalence memory requires as many words as there

are regions. If that number is m, then Iog2 m bits are requiredper word. We have observed that m never exceeds about 2000(for 512 X 512 images), even for very "busy" scenes.

Using queued memory fetches and pipelining, one can seethat this sequence reduces to 4 ,usteps/pixel. Assuming 512X 512 resolution and video rates (100 ns/pixel), we can allow25 ns/microstep. Already, integrated circuits are being builtwhich meet these speed requirements. For example, some newchips use 80 MHz on-chip clock rates, and 100 MHz off-chiprates are being discussed.

966

SNYDER AND SAVAGE: MEMORIES FOR IMAGE ANALYSIS

20

00 N/C01 LT10 GT11 EQ

,ATn

CPU

addressbus

01234

m 56789

10

EquivalenceMemory(CAM)

Fig. 4.

This section has discussed, in detail, an architecture whichmakes use of content-addressable search to perform UNION-FIND operations. The nodes of the graph in this case are

simply the pixels in an image and the edges the adjacency/similarity property between pixels. Each region in the imagethus corresponds to a disjoint set. FIND is implemented by a

simple lookup, and UNION requires a content-addressableread/write cycle. The next section will discuss how thatread/write operation could be built into a chip.

databus

Fig. 5.

C. A Content-Addressable Chip toImplement Disjoint SetsA special CAM can be used to store disjoint sets and per-

form UNIONS and FINDS. The two-phase CAM chip wepresent here (Fig. 5) functions either as a conventional memoryor a content-addressable memory, depending on the state ofan input bit MODE. If MODE = 0, then conventional RAMoperations are enabled, utilizing the address -bus and R/Wlines in the usual way. If MODE = 1, then two-phase contentaddressable writing is enabled.When MODE = 1, then during the first phase, the contents

of each memory location are compared to the externally es-tablished contents of the data bus. The comparison may be ongreater than, less than, or equal to. In this case, we use equality.Associated with each cell is a flag bit which is set if the com-parison is successful. During the second phase, another, pre-sumably different, number is externally supplied to the databus, and every memory cell whose flag bit is set reads andstores the value on the data bus.The disjoint set data structure can be implemented as fol-

lows. Assume that the CAM consists of n word locations whereeach location (address) corresponds to a distinct integer in therange 1-* - , n. The location corresponding to integer i willstore the label of the set containing i (in the collection ofdisjoint sets), which initially is i itself. Associated with eachlocation is a flag bit. The operation FIND (X) is performed bysetting MODE to 0 and using the CAM as a conventional RAMto read the integer stored in location X. To perform UNION(A, B, C) where C is the minimum ofA and B, we do the fol-lowing. Let M be the maximum of A and B.

1) Place M on the data bus, set MODE to 1, and strobephase 1.

2) Place C on the data bus, with MODE still set to 1, andstrobe phase 2.

Thus, a UNION or a FIND can be performed in constanttime, independent of the number of elements stored. Thismeans, for example, that the region-labeling problem for animage of n pixels can be solved in 0(n) hardware operations,

967

IEEE TRANSACTIONS ON COMPUTERS, VOL. c-31, NO. 10, OCTOBER 1982

which is the speed required for raster-scan input applications.In addition, the CAM architecture described here can be usedboth for storing and processing data to minimize data flowbetween the CAM and the host machine.

V. CONCLUSION

In this paper, we approached certain problems in imageanalysis by identifying the data structures involved in solvingthe problems, and then proposing an architecture to efficientlyimplement the data structure operations. As a result, we haveshown that these problems can all be solved by the same typeof computing/memory architecture. The use of a content-addressable structure provides parallel search and updatefunctions, allowing speed reductions down to constant time peroperation. The full range of potential for these architectureshas yet to be explored.

Related work by Hall [5] proposed that, with VLSI,CAM-based structures with enhanced instruction sets forgeneral-purpose processing will be feasible.

REFERENCES

11I A. V. Aho, J. E. Hopcroft, and J. D. Ullman. TheWDesign and AnalysisofComputer Algorithms. Reading, MA: Addison-Wesley, 1974.

[2] J. A. Cornell, "Parallel processing of ballistic missile defense radar datawith PEPE," presented at IEEE COMPCON. 1972.

[3] B. A. Crane, M. J. Gilmartin, J. H. Huttenhoff, P. T. Bux. and R. R.Shively. "PEPE computer architecture," presented at IEEE COMP-CON. 1972.

[4] R. 0. Duda and P. E. Hart, Pattern Classification and Scene Analysis.New York: Wiley, 1973.

[5] J. S. Hall, "A general purpose CAM-based system," in VLSI Systemsand Computations, H. T. Kung, B. Sproull, and G. Steele, Eds.Computer Science Press, 1981, pp. 379-388.

[6] J. E. Hopcroft and J. D. Ullman, "Set merging algorithms," SIAM J.Comput., vol. 2, no. 4, pp. 294-303, 1973.

[7] P. V. Hough, "Method and means for recognizing complex patterns,"U.S. Patent 3 069 654, Dec. 18, 1962.

[8] T. Kohonen, Content-Addressable Memories. New York: Springer-Verlag, 1980.

[91 D. L. Milgram, A. Rosenfeld, T. Willett, and G. Tisdale, "Algorithmsand hardware technology for image recognition," Final Rep. to U.S.Army Night Vision Lab., Mar. 1978.

[10] J. A. Rudolph, "A production implementation of an associative arrayprocessor: STARAN," in Proc. AFIPS 1972 Fall Joint Comput.Conf

[11] C. D. Savage, "A systolic data structure chip for connectivity problems,"in VLSI Systems and Computations, H. T. Kung, B. Sproull, and G.Stelle, Eds. Computer Science Press, 1981, pp. 296-300.

[12] A. Slade and H. 0. McMahon, "A cryotron catalog memory system,"in Proc. Eastern Joint Comput. Conf New York: AIEE, 1957.

[13] W. E. Snyder and A. Cowart, "An interative approach to region grow-ing," IEEE Trans. Pattern Anal. Machine Intell., to be published.

[ 14] R. E. Tarjan, "Efficiency of a good but not linear set union algorithm,"J. Ass. Comput. Mach., vol. 22, no. 2, pp. 215-225, 1975.

[15] K. J. Thurber and L. D. Wald, "Associative and parallel processors,"Comput. Surveys, Dec. 1975.

[16] S. S. Yau and M. S. Fung, "Associative processor architectures-Asurvey," Comput. Surveys, Mar. 1977.

Wesley E. Snyder (S'71-M'75-SM'80) receivedthe B.S. degree from North Carolina State Uni-versity, Raleigh, in 1968 and the M.S. and Ph.D.degrees from the University of Illinois, Urbana, in1971 and 1975, respectively.He was an employee of the IBM Corporation,

and has been a Consultant to a number of indus-tries and to the United Nations in the area of auto-

#'e;j_ _ mation. He has also had brief affiliations with theNASA Langley Research Center and the WestGerman Air and Space Agency, Munich. He is

currently an Associate Professor of Electrical Engineering, North CarolinaState University, where he has been a part of the Image Analysis Group,working on modeling and the processing of images displaying motion.

Carla D. Savage received the Ph.D. degree inmathematics from the University of Illinois, Urba-na, in 1977.

She spent the 1977-1978 academic year at theUniversity of Texas, Austin, as an IBM Postdoc-toral Fellow. She is currently an Assistant Profes-

- l sor in the Department of Computer Science,North Carolina State University, Raleigh.

968