Deformation Energy for Size Functions

Pietro Donatini1, Patrizio Frosini1, and Claudia Landi2

1 Dipartimento di Matematica, Universit�a di Bologna, P.zza P.ta S. Donato 5,I-40126 Bologna, Italy, Tel.: ++39 51 354478, Fax: ++39 51 354490

donatini@dm.unibo.it, http://www.dm.unibo.it/~donatinifrosini@dm.unibo.it, http://www.dm.unibo.it/~frosini

2 Dipartimento di Matematica, Universit�a di Pisa, Via Buonarroti 2,I-56127 Pisa, Italy, Tel.: ++39 50 844229, Fax: ++39 50 844224

landi@mail.dm.unipi.it

Abstract. Size functions are functions from the real plane to the naturalnumbers useful for describing shapes of objects. They allow to translatethe problem of comparing shapes to the problem of comparing functions,that is a much simpler task. In order to perform the comparison betweensize functions we present a method to measure the energy necessary todeform size functions into each other. Minimizing such an energy allowsfor a measure of the similarity between shapes. Some experimental resultsconcerning the comparison of free hand-drawn sketches are shown.

1 Introduction

Size functions are a mathematical tool originally conceived for representingshapes but applicable for studying all data that can be seen as topological spaces.Size functions have already been successfully used to deal with various recogni-tion problems (see, e.g., [1], [2], [3], [4], [13], [14], [15], [16], [17], [18]). Basically,a size function is the output of a transform whose input is a topological spacetogether with a real function de�ned on it. The topological space is the signal(e.g. B/W image, colour image, sound wave) under study, and the real functionde�ned on it is the criterion used to study the signal. Each size function is anatural valued function of two real variables.

The purpose of this paper is to provide a mathematical method for measuringthe energy necessary to deform size functions into each other. This gives a mea-sure of the similarity of the corresponding topological spaces. In order to do so,size functions are preliminarily transformed into simpler objects, precisely intoparticular formal series. This representation contains almost the same amountof information about the space under study as the original size function doesbut it is much easier to handle. Then we show how to assign an energy to anydeformation of a formal series associated with a size function. This way we de�nea deformation energy for size functions. Finally, we measure the extent to whichtwo spaces resemble each other by minimizing the energy necessary to deformthe corresponding size functions into each other.

This framework for the comparison of size functions is treated mainly froma theoretical point of view, nevertheless some experiments on B/W images arepresented.

We shall con�ne ourselves to the study of a method for measuring the simi-larity between shapes, leaving out the problem of classi�cation.

2 Size functions

LetM be a subset of some Euclidean space. We shall call any continuous function' :M! IR a measuring function. M is the signal we want to study and ' is afunction chosen according to the properties of the signal we are interested in.

The size function of the pair (M; ') is a function `(M;') : IR2 ! IN[f+1g.

For every pair (x; y) 2 IR2 consider the set of points inM at which ' takes valuesmaller or equal to x. Two such points are considered equivalent if they eithercoincide, or can be connected in M by a path at whose points ' takes valuesmaller or equal to y. Then `(M;')(x; y) is de�ned to be equal to the number ofequivalence classes so obtained.

For an example of size function see Figure 1. In this example the set M is acurve in the plane. By choosing the function ordinate of the point as measuringfunction, one obtains the size function shown on the right. The number displayedin each region of the domain of the size function denotes the value taken by thesize function in that region.

Fig. 1. A plane curve and the associated size function with respect to the measuringfunction ordinate of the point: '(x; y) = y. The number displayed in each region of thedomain of the size function denotes the value taken by the size function in that region.Continuous lines represent lines made of discontinuity points for the size function.

We remark that computing a size function is equivalent to counting the num-ber of connected components of a graph: this gives the complexity of our task(see [6], [7]).

From the de�nition of size function one has immediately that if for y < x

there exists an in�nite number of points P 2 M such that y < '(P ) � x then`(M;')(x; y) = +1. On the other hand, ifM is a �nite union of compact locallyarcwise connected components then for x < y we have `(M;')(x; y) < +1 forevery measuring function. Since this assumption is not very restrictive, in whatfollows we shall stick to it. Therefore in general we shall be interested only inthe behaviour of size functions in the set S0 = f(x; y) 2 IR2 : x < yg.

Another easy consequence of the de�nition of size functions is their mono-tonicity: size functions are non-decreasing in the �rst variable and non-increasingin the second one.

For more details and results about size functions we refer the reader to [5],[7], [8], [9], [12], [15] and [16]. Algorithms for the computation of size functionscan be found in [6] and [17]. The resistance of size functions under noise andocclusions is treated in [11].

3 Size functions and formal series

Independently of the pair (M; '), discontinuities of size functions satisfy somegeneral properties. The most remarkable, but not immediate, one is that if `(M;')

is discontinuous at (x; y) with x < y, then either x is a discontinuity point for`(M;')(�; y) or y is a discontinuity point for `(M;')(x; �) or both the statementshold.

Moreover, for x < y, discontinuities in the variable x propagate downwardto the diagonal � = f(x; y) 2 IR2jx = yg and discontinuities in the variable ypropagate toward the right up to the diagonal.

For every size function it is then possible to select special points in S0, calledcornerpoints, from which discontinuity points propagate horizontally and ver-tically toward the diagonal. More precisely, cornerpoints are de�ned as thosepoints p = (x; y), with x < y, satisfying the following property: if we denote by��;�(p) the number

`(M;')(x+ �; y � �)� `(M;')(x+ �; y + �)�

`(M;')(x� �; y � �) + `(M;')(x � �; y + �);

it must hold �(p)def= minf��;�(p) : � > 0; � > 0; x + � < y � �g > 0. We shall

call �(p) the multiplicity of p. It can be shown that �(p) � 0 for every p 2 S0.

Under our assumptions on M a size function has at most a countable set ofcornerpoints in S0. Actually, for any �xed � > 0, the region S� = f(x; y) 2 IR2 :x < y � �g contains only �nitely many cornerpoints.

In Figure 2 we show that the only cornerpoints for the size function depictedon the left are the points a; b; c. All these cornerpoints have multiplicities equal

Fig. 2. A size function on the left and the corresponding cornerpoints (a; b; c) andcornerlines (s) on the right. The resulting formal series is �0(`) = s+ a+ b+ c.

to 1 but it is easy to construct examples with points having multiplicity greaterthan 1.

Analogously one can select special vertical lines whose points above the di-agonal are discontinuity points for the size function. These lines are called cor-nerlines and are de�ned as those lines l, with equation x = k, k 2 IR, for whichit holds

�(l)def= min

�>0;k+�<y`(M;')(k + �; y)� `(M;')(k � �; y) > 0:

We shall call �(l) the multiplicity of l. From the monotonicity of size functionsit follows that �(l) � 0 for every vertical line l. Our assumptions on M implythat a size function can have at most a �nite number of cornerlines.

The only cornerline for the size function of Figure 2 is the line s with mul-tiplicity equal to 1. In general, it can be shown that every arcwise connectedcomponent ofM generates a cornerline with multiplicity 1. Since di�erent com-ponents can give rise to the same cornerline, one can obtain cornerlines withmultiplicity greater than 1.

The importance of cornerpoints and cornerlines of a size function lies in thefact that for almost every point p = (�x; �y) lying above the diagonal, the valueof the size function at p is equal to the sum of the multiplicities of cornerpoints(x; y) with x < �x and y > �y and of cornerlines x = k with k < �x. That is to saythat cornerpoints and cornerlines determine the size function almost everywhere.

Let us now denote by R the set of all lines with equation x = k, with k 2 IR.For every � � 0, we shall call formal series in S� [ R any object of the form� =P

m(X)X with m(X) 2 ZZ, X varying in S� [ R and m(X) 6= 0 only for

a countable set of X 's. The set of X 's such that m(X) 6= 0 is called the supportof the formal series.

It is then natural to de�ne a map �� from the set of all size functions intothe set of formal series in S� [ R. For � > 0 it su�ces to set �� equal to themap which takes a size function `(M;') into the formal series

P�(X)X , where

X varies in the set of all cornerpoints belonging to S� and of all cornerlines for`(M;'), while �(X) is its multiplicity (cf. Figure 2). For � = 0 one can take �0to be the map \extending" all the maps �� for every � > 0. For every � � 0, ��induces a well de�ned map ~�� from the set L� of all size functions quotiented bythe relation of coincidence almost everywhere in S� into the set of formal seriesin S� [ R.

Theorem 1. For every real number � � 0 the map ~�� : L� �! S� [ R isinjective.

As a consequence it remains proven that size functions can be represented asparticular formal series of points and lines of the plane.

Furthermore, for every � � 0, it is possible to characterize a particular subsetof the set of formal series in S� [R on which ~�� turns out to be also surjective.

For the proofs of the results stated in this section we refer to [10].

4 Deformation energy

It follows from Theorem 1 that we can reduce the problem of assigning an energyto the deformation of a size function to the problem of giving an energy to thedeformation of the associated formal series. This is clearly an easier task sinceformal series allow a concise and algebraic representation of size functions.

The �rst step in order to assign an energy to the deformation of a size functionis therefore that of de�ning what a deformation of the associated formal seriesis. A very natural way to do this is illustrated by the example in Figure 3. Insuch an example we consider two formal series �1 = s + a + b + c and �2 =s0 + a0 + b0 obtained as images of two size functions by the map �0 of Section3. The discontinuity points of the two size functions are represented by dashedand continuous lines respectively. We deform �1 into �2 by transforming the lines into the line s0, the points a and b into a0 and b0 respectively, and by sendingc onto the diagonal (this last action has the same meaning as \destroying" thepoint c).

This example can be generalized as follows.

Let us �x a positive real number �. Then every formal series obtained asimage of a size function by the map �� has �nite support in S�[R. For the sakeof simplicity in what follows the considered formal series will always have �nitesupport in S� [ R. Nevertheless our approach can be extended to the generalcase when the support is not �nite. Moreover, we shall consider each point phaving multiplicity m(p) as m(p) distinct points.

b'

c

y

s s' x

aa'

b

Fig. 3. A possible way to deform a size function, represented by dashed lines, intoanother one, represented by continuous lines.

De�nition 1. We shall call elementary deformation of a formal series � any ofthe following transformations:(a) Identity transformation leaving � unchanged.(b) Transformation taking � to ��X+X 0 with X 6= X 0 and either both X andX 0 belonging to S� or both belonging to R.(c) Transformation taking � to � +X with X 2 S� [ R.(d) Transformation taking � to � �X with X 2 S� [ R.

Remark 1. We recall that coe�cients of formal series are allowed to be negative.Applying an elementary deformation of type (b) means transforming either apoint p into a di�erent point p0 or a vertical line l into a di�erent line l0. Anelementary deformation of type (c) has the e�ect of creating either a new pointor a new line. Analogously, an elementary deformation of type (d) has the e�ectof eliminating either a point p or a line l.

De�nition 2. We shall call deformation of a formal series any �nite orderedsequence of elementary deformations of the considered formal series. We shalldenote by T (�) the result of the deformation T applied to the formal series �.Given two deformations T and S, the juxtaposition ST means applying �rst Tand then S.

Remark 2. The e�ect of each deformation of formal series is independent of theorder in which elementary deformations are carried out.

Remark 3. The inverse of an elementary deformation of type (b) is again anelementary transformation of the same type. The inverse of an elementary de-formation of type (c) is an elementary deformation of type (d) and viceversa.

In general, the inverse of a deformation T will be denoted by the symbolT�1.

Remark 4. Given two formal series (with �nite support) there exists an in�nitenumber of deformations transforming such formal series into each other. In fact,if T and S are deformations of a formal series �, then T (�) = TSS�1(�) butT 6= TSS�1.

The following step is that of de�ning what the energy of a deformation is.

De�nition 3. For every deformation T of a formal series, the associated en-ergy E(T ) is a real number greater than or equal to 0 satisfying the followingproperties:i) if T is the identity transformation, then E(T ) = 0;ii) if T = T1T2 � � �Tn then E(T ) =

Pn

i=1 E(Ti);iii) E(T ) = E(T�1).We allow the possibility that E(T ) is equal to +1.

As an example of deformation energy, one can consider the following one: ifT is an elementary deformation of type (b) sending a point p to a point p0 (resp.a line l to a line l0) then E(T ) is equal to the Euclidean distance between p andp0 (resp. l and l0); if T is an elementary deformation of type (c) or (d) creatingor destroying a point p then E(T ) is equal to the Euclidean distance between p

and the diagonal �; if T is an elementary deformation of type (c) or (d) creatingor destroying a line l then E(T ) is equal to +1.

The deformation energy thus de�ned will be used for the experiments de-scribed in the following section.

The last step consists of deducing a measure of the similarity between twoformal series from the measure of the energy necessary to deform them into eachother.

De�nition 4. For every pair of formal series �1 and �2 the similarity between�1 and �2 is the number �(�1; �2) = inffE(T )g where T runs in the set of allthe deformations taking �1 to �2.

Remark 5. We point out that if E is the deformation energy de�ned in the aboveexample, then inffE(T )g is actually a minimum. Anyway, in general inffE(T )gneeds not to be attained.

Let us �nally observe that � is actually well suited for a comparison betweenformal series, and hence between size functions, because it satis�es the followingproperties, making of � a pseudo-distance.

Proposition 1. The following properties hold:1) for every pair of formal series �1 and �2, �(�1; �2) � 0;

2) for every formal series �, �(�; �) = 0;3) for every pair of formal series �1 and �2, �(�1; �2) = �(�2; �1);4) for every triad of formal series �1, �2 and �3, �(�1; �3) � �(�1; �2) +�(�2; �3).

Proof. Property 1) is trivial. Since�(�; �) � 0 for every formal series �, property2) is easily proven by taking T equal to the identity transformation and byapplying i) of De�nition 3.

Property 3) is a direct consequence of property iii) of De�nition 3.Finally, property 4) is proven by observing that for every sequence of positive

real numbers f�igi2IN with limi!+1 �i = 0, there exists a sequence of deforma-tions fT 0igi2IN taking �1 to �2 and verifying the inequality �(�1; �2) � E(T 0i )��i.Analogously, there exists a sequence of deformations fT 00i gi2IN taking �2 to �3and verifying the inequality �(�2; �3) � E(T 00i ) � �i. Thus for every i 2 IN itholds that �(�1; �3) � E(T 00i T

0i ) = E(T 0i ) + E(T

00i ) � �(�1; �2) +�(�2; �3) + 2�i.

The desired inequality is obtained for i! +1.

5 Experimental results

We have tested the method here described in the task of recognizing B/W im-ages. More precisely, we have considered 15 drawings belonging to three classesof objects: scissors, open end wrenches, screwdrivers. For each drawing we havecomputed the size function of its outer edge with respect to the measuring func-tion distance from the center of mass. Then size functions have been transformedinto formal series. These two operations have been carried out by means of theprogram SketchUp (for more details about this classi�er of free-hand drawnsketches we refer the reader to [1]). Finally the similarity between images hasbeen calculated by using the deformation energy suggested in Section 4.

The results are displayed in Table 1, showing that the deformation energycan be used to easily distinguish the considered shapes. Indeed, the averagesimilarity between size functions corresponding to drawings of the same class,displayed in the gray rectangles, is smaller than that between drawings belongingto di�erent classes.

6 Conclusions

In this paper we have shown how an energy minimization paradigm can be suc-cessfully used to compare size functions. This approach transforms the problemof shape comparison into a minimization problem with respect to deformationsof �nite sets of points and lines of the plane, that is a much simpler task. Aconcrete application of this technique has been given.

7 Acknowledgements

Research accomplished under support of CNR{GNSAGA, of ASI, and of theUniversity of Bologna, funds for selected research topics.

A1 A2 A3 A4 A5 B1 B2 B3 B4 B5 C1 C2 C3 C4 C5

A1 0 55,65 54,94 59,17 46,43 71,07 93,88 76,96 74,35 61,27 96,47 87,14 100,67 86,98 87,98

A2 55,65 0 29,42 47,17 31,56 116,07 132,99 108,2 128,05 102,34 115,8 120,8 111,6 112,6 121,89

A3 54,94 29,42 0 59,1 35,93 114,45 139,23 116,46 125,34 105,53 134,48 128,64 139,24 126,39 126,03

A4 59,17 47,17 59,1 0 31,32 114,23 124,01 107,05 122,58 98,14 106,96 97,17 102,34 103,83 101,21

A5 46,43 31,56 35,93 31,32 0 100,78 121,21 102,35 108,33 90,04 114,68 98,91 107,33 109,14 113,19

B1 71,07 116,07 114,45 114,23 100,78 0 26,31 20,72 16,19 17,73 52,95 45,27 55,2 45,47 44,47

B2 93,88 132,99 139,23 124,01 121,21 26,31 0 38,96 22,01 39,11 42,74 36,69 43,29 34,68 34,28

B3 76,96 108,2 116,46 107,05 102,35 20,72 38,96 0 32,62 19,4 48,79 56,61 49,3 54,37 49,6

B4 74,35 128,05 125,34 122,58 108,33 16,19 22,01 32,62 0 28,7 50,15 39,21 50,32 41,9 41,69

B5 61,27 102,34 105,53 98,14 90,04 17,73 39,11 19,4 28,7 0 53,42 48,42 56,17 47,39 45,24

C1 96,47 115,8 134,48 106,96 114,68 52,95 42,74 48,79 50,15 53,42 0 18,59 13,37 11,48 8,51

C2 87,14 120,8 128,64 97,17 98,91 45,27 36,69 56,61 39,21 48,42 18,59 0 17,89 13,7 11,48

C3 100,67 111,6 139,24 102,34 107,33 55,2 43,29 49,3 50,32 56,17 13,37 17,89 0 19,56 14,63

C4 86,98 112,6 126,39 103,83 109,14 45,47 34,68 54,37 41,9 47,39 11,48 13,7 19,56 0 5

C5 87,98 121,89 126,03 101,21 113,19 44,47 34,28 49,6 41,69 45,24 8,51 11,48 14,63 5 0

SCISSORS OPEN END WRENCHES SCREWDRIVERS

Table 1

References

1. Collina, C., Ferri, M., Frosini, P., Porcellini, E.: SketchUp: Towards QualitativeShape Data Management. In: Chin, R., Pong, T. (eds.): Proc. ACCV'98, LectureNotes in Computer Science 1351, Vol. I, Springer-Verlag, Berlin Heidelberg NewYork (1998) 338{345

2. Donatini, P., Frosini, P., Lovato, A.: Size Functions for Signature Recognition.In: Proc. SPIE Workshop \Vision Geometry VII" 3454, (1998) 178{183

3. Ferri, M., Frosini, P., Lovato, A., Zambelli, C.: Point Selection: A New Compari-son Scheme for Size Functions (With an Application to Monogram Recognition).In: Chin, R., Pong, T. (eds.): Proc. ACCV'98, Lecture Notes in Computer Science1351, Vol. I, Springer-Verlag, Berlin Heidelberg New York (1998) 329{337

4. Ferri, M., Lombardini S., Pallotti, C.: Leukocyte Classi�cation by Size Func-tions. In: Proceedings of the second IEEEWorkshop on Applications of ComputerVision, IEEE Computer Society Press, Los Alamitos, CA (1994) 223{229

5. Frosini, P.: Measuring Shapes by Size Functions. In: Proc. of SPIE, IntelligentRobots and Computer Vision X: Algorithms and Techniques, Boston, MA 1607(1991), 122-133

6. Frosini, P.: Discrete Computation of Size Functions. J. Combin. Inform. SystemSci. 17(3-4), (1992) 232{250

7. Frosini, P.: Connections Between Size Functions and Critical Points. Math. Meth.Appl. Sci., 19 (1996) 555{596

8. Frosini, P., Landi, C.: Size Functions and Morphological Transformations. ActaAppl. Math., 49 (1997) 85{104

9. Frosini, P., Ferri, M.: Range Size Functions. In: Proceedings of the SPIE's Work-shop \Vision Geometry III", 2356 (1994), 243{251

10. Frosini, P., Landi, C.: Size Functions and Formal Series. Submitted for publica-tion.

11. Frosini, P., Landi, C.: Size Theory as a Topological Tool for Computer Vision.To appear in Pattern Recognition and Image Analysis.

12. Frosini, P., Pittore, M.: New Methods for Reducing Size Graphs. Intern. J. Com-puter Math., 70 (1999) 505{517,

13. Uras, C., Verri, A.: Studying Shape through Size Functions. In: Y. O, A. Toet,D. Foster, H. Heijmans, P. Meer (editors) Shape in Picture, NATO ASI Series F126, Springer-Verlag, Berlin Heidelberg (1994), 81{90

14. Uras, C., Verri, A.: On the Recognition of the Alphabet of the Sign Languagethrough Size Functions. In: Proc. XII IAPR International Conference on PatternRecognition, Jerusalem (Israel) II, IEEE Computer Society Press, Los Alamitos,CA (1994), 334{338

15. Verri, A., Uras, C., Frosini, P., Ferri, M.: On the Use of Size Functions for ShapeAnalysis. Biol. Cybern., 70 (1993) 99{107

16. Verri, A., Uras, C.: Metric-Topological Approach to Shape Representation andRecognition. Image Vision Comput., 14 (1996) 189{207

17. Verri, A., Uras, C.: Computing Size Functions from Edge Maps. Internat. J.Comput. Vision, 23 (1997) 1{15

18. Verri, A., Uras, C.: Invariant Size Functions. In: J. L. Mundy, A. Zisserman, D.Forsyth (editors) Applications of Invariance in Computer Vision, Lecture Notesin Computer Science 825, Springer-Verlag, Berlin Heidelberg (1994), 215{234