Signal Processing 75 (1999) 1}10

Repulsive attractive network for baseline extraction ondocument images

E. OG ztop, A.Y. MuK layim, V. Atalay, F. Yarman-Vural*

Department of Computer Engineering, Middle East Technical University, TR-06531 Ankara, Turkey

Received 26 July 1996; received in revised form 9 November 1998

Abstract

This paper describes a new framework, called repulsive attractive (RA) network for baseline extraction on documentimages. The RA network is an energy minimizing dynamical system, which interacts with the document text imagethrough the attractive and repulsive forces de"ned over the network components and the document image. Experimentalresults indicate that the network can successfully extract the baselines under heavy noise and overlaps between theascending and descending portions of the characters of adjacent lines. The proposed framework is applicable to a widerange of image processing applications, such as curve "tting, segmentation and thinning. ( 1999 Elsevier Science B.V.All rights reserved.

Zusammenfassung

Diese Arbeit beschreibt ein neues Rahmenprogramm, genannt &Repulsive Attractive (RA) network for baseline extraction'bei Dokument Abbildungen. Das RA Netzwerk ist ein energieminimierendes dynamisches System. Dieses wird mit derAbbildung des Dokumenttextes durch die anziehenden und absto{ende KraK fte, de"niert uK ber Netzwerk-Komponenten,und dem Dokumentbild in Beziehung gesetzt. Experimentelle Resultate zeigen, da{ das Netzwerk die Grundliniezwischen aufsteigenden und absteigenden Buchstabenbereichen unter schwierigen Rauschbedingungen und bei UG berlap-pung erfolgreich extrahieren kann. Das vorgestellte Programm ist anwendbar auf weite Bereiche der Bildverarbeitung,wie Kurvenanpassung, Segmentierung und AusduK nnung. ( 1999 Elsevier Science B.V. All rights reserved.

Re2 sume2

Cet article deH crit un nouveau cadre de travail, appeleH re& seau re&pulsif attractif pour l'extraction de la ligne de base sur desimages de documents. Let reH seau reH pulsif attractif est un systeme dynamique minimisant une eH nergie, qui interagit avecl'image du texte du document par des forces attractives et reH pulsives deH "nies sur les composantes du reH seau et sur l'imagedu document. Des reH sultats expeH rimentaux indiquent que le reH seau peut extraire avec succes les lignes de base sous unbruit important et en preH sence de recouvrements entre les parties ascendantes et descendantes des caracteres de deuxlignes adjacentes. Le scheHma proposeH est applicable a une large gamme d'applications de traitement d'images, tellesl'ajustement de courbes, la segmentation et l'a$nement. ( 1999 Elsevier Science B.V. All rights reserved.

Keywords: Baseline; Repulsive attractive network; Hough transform; Energy minimization; Active model

*Corresponding author. E-mail: vural@ceng. metu.edu.tr

0165-1684/99/$ } see front matter ( 1999 Elsevier Science B.V. All rights reserved.PII: S 0 1 6 5 - 1 6 8 4 ( 9 8 ) 0 0 2 2 0 - 5

1. Introduction

It is well known that a crucial step in docu-ment image analysis is the identi"cation of thebaselines. On a document image, baselines notonly give important information about the layoutstructure, but also provide e!ective clues to sub-sequent steps of document analysis, such as op-tical character recognition. Baseline extractionproblem becomes complicated in handwrittendocuments where the ascending and descendingportions of the characters between adjacent linesoverlap. Additional complexity is introduced whenthe characters overlay on a curved or slanted base-line and the documents are contaminated withnoise.

In this study, a new method is presented forextracting the baselines. The method is an ap-plication of a framework called repulsive at-tractive (RA) network [11]. RA Network canbe described as an energy minimizing dynamicalsystem [9] and it is inspired from the universallaw of gravitation where the masses attract eachother according to their weight and distanceamong them. In this particular application inwhich the aim is to extract the baselines ofa document image, the pixels of the image andthe baselines are considered as if they were masses;each pixel attracts the baselines proportional toits gray value and inversely proportional to thesquare of the distance between them. The baselines,themselves, repel each other with a magnitude in-versely proportional to the square of the distancebetween them. The new method is applicable toa wide range of documents and overcomes manyproblems faced during the baseline extractionprocess, such as thresholding, noise sensitivity, in-tolerance to font-style variations and skew angles.Some other possible applications of the RA net-work in image processing can be thinning andsegmentation.

In the next section, the available techniques forbaseline extraction problem are reviewed. In Sec-tion 3, the baseline extraction using RA Network isdescribed. The network is tested on various textimages in Section 4. Finally, Section 5 concludesthe paper, indicating the direction of the futurework.

2. Baseline extraction

In general, a baseline is de"ned to be a curveor line on which the characters overlay on a docu-ment image. Standard baseline extraction methodsinclude Hough transform [5], least-squaresmethods [1], horizontal projection pro"les [7],run-length smearing and use of typographical in-formation [8].

Among all, the Hough transform and its variantsare the most widely used methods [10]. In thesemethods, the document image is "rst transformedto the (o,h) plane. o indicates the length and h indi-cates the slope of the normal line to a pixel in polarcoordinates. Then, a two-dimensional search is ac-complished for detecting the peaks in the transformplane. For a document image having curved base-line and skew angle Hough transform works suc-cessfully. However, it has certain drawbacks thatmay limit its use. One problem is the quantizationof the input image by extracting the geometricfeatures, such as center of mass of each character.This process introduces an uncertain amount oferror into the result yielding unsatisfactory solu-tions for the characters with ascending and de-scending portions. Secondly, the complexity of thesearch for peaks in the (o,h) plane is a seriousdrawback. In addition, it can be di$cult to puta criterion for identifying the peaks [3,4].

There is a vast amount of variation of the least-squares methods for "tting lines or curves toa given set of data points [6]. The major limitationof these methods is the sensitivity to the noisecontamination.

The most popular baseline extraction method forprinted text is the use of horizontal projection pro-"le which simply obtains the histogram of the im-age on the y-axis and identi"es the baseline as thepeak points of the histogram [12]. Obviously, it ishighly sensitive to skew angles which requiresa strong preprocessing stage for normalization.

A common baseline extraction method for bi-nary images is to use run length smearing. It isbased on the run length codes which consists ofa start address of each string of 1's, followed by thelength of that string. This method is, also, highlysusceptible to the noise contamination and skewangle [2].

2 E. O$ ztop et al. / Signal Processing 75 (1999) 1}10

Finally, use of typographical information forbaseline extraction heavily depends on the skewangle, font style and size. It is particularly de-veloped for machine printed texts [8].

3. The repulsive attractive (RA) network for baselineextraction

In this section, the repulsive attractive (RA) net-work for baseline extraction is described.

A repulsive attractive network is identi"ed by thetriple (Y,g,U) where Y is a vector space, g is a real-valued function de"ned on Y and U is the set ofunits. A unit ;

i3U is composed of sub-units, u

ij.

Each sub-unit is associated with a position ora weight vector in Y.

The dynamics of the repulsive attractive networkis determined by the following forces:f The internal force, f */5, that exists among the

sub-units belonging to the same unit. This forcegives units the tendency to have certain shape ororientation.

f The repulsive force, f 3%1, that exists among thesub-units of di!erent units.

f The attractive force, f !55, that is exerted by thepoints ofY with a magnitude proportional to thevalue of g at these points.

The repulsive attractive network can be associatedwith the document text image. The baselines, beingcurves, are approximated as connected line seg-ments. The triple (Y,g,U) of the RA networkfor baseline extraction is speci"ed as follows (seeFig. 1):f Y denotes the document image embedded into

R2. More precisely, Y"M(x,y)3R2 D & twopoints (x

0,y

0), (x

1,y

1) on the image such that

x0)x)x

1and y

0)y)y

1N

f U"M;0,;

1,2,;

nN denotes the set of curves

that approximates the baseline where n is thenumber of baselines, and the sub-units u

ijare the

connection points on the curves.f g :YPZ, where g(p) is the intensity value of the

image at pixel p if p3Z2, 0 otherwise.The internal force of the RA network enforces thesub-units of the same baseline to lie on a horizontalline. The forces acting on the document image isde"ned at three levels:

Fig. 1. The forces acting on the sub-unit u12

(only a sample fromeach type of force is shown).

At level 1, the internal, repulsive and attractiveforces among the sub-units and pixels are de"nedas follows: For each sub-unit u

ij3;

i, the received

internal force between the sub-units of the sameunit is

f */5(uil,u

ij)

"u(uil,u

ij)/d2(u

il,u

ij), ∀u

il3;

i, jOl, (1)

the repulsive force between the sub-units of distinctbaselines is

f 3%1(ulk,u

ij)"u(u

lk,u

ij)/d2(u

lk,u

ij), ∀u

lk3;

l, iOl,

(2)

and the attractive force between the image pixelsand sub-unit u

ijis

f !55(p,uij)"u(p,u

ij)g(p)/(d2(p,u

ij)#g), ∀p3Y. (3)

In the above equations d is the Euclidean dis-tance function, u(P

1,P

2) is the unit vector directed

from P2

towards P1

and g is a real constant.At level 2, the internal, repulsive and attractive

forces are aggregated to generate net forces

FNET~*/5ij

" +v|Iij

f */5(v,uij), (4)

FNET~3%1ij

" +v|Rij

f 3%1(v,uij), (5)

FNET~!55ij

"+p|Y

f !55(p,uij), (6)

E. O$ ztop et al. / Signal Processing 75 (1999) 1}10 3

where

Iij"Mu

ikD kOjN and R

ij"Mu

lkD iOlN.

At level 3, the net forces are further aggregated togenerate the total net force

FNETij

"aFNET~*/5ij

#bFNET~3%1ij

#cFNET~!55ij

, (7)

where a, b, c are real coe$cients.The RA network framework de"nes the internal,

repulsive and attractive forces among the sub-unitsand the pixels in order to extract the baselines ona document. A unit evolves dynamically via themovements of its sub-units. The displacement ofthe sub-units is determined by the forces acting onit and the total net force is used to update theposition vector associated with each sub-unit. Inthis particular application, only the vertical compo-nents are taken into account. The vertical compon-ent is simply multiplied by a constant scale value to"nd out the amount of change to be performed onthe vertical position of each sub-unit. The energyfunction is de"ned as the integral of the forces. Ata local minimum of the energy function, the sub-units do not change their positions. Each baseline isassumed to be a local minimum of the energyfunction. If a unit is su$ciently close to a baseline,then it will be attracted by the local minimum andthis situation is called as the local convergence. Theglobal convergence yields the extraction of all thebaselines on a document. In order to guarantee theglobal convergence, a virtual unit is allocated ontop of the document. Also, bottom of the documentis considered to behave as a virtual textline. As soonas a unit is stuck to the bottom of the image, theglobal convergence is achieved. This indicates thatall of the existing baselines have been extracted.

4. Algorithm for baseline extraction

In order to simulate the system described in thepreceeding section, the following algorithm is used:

1. Choose network parameters: a, b, c.Initialize o, e and M.Initialize U"Mu

0N, u

0located at the top edge of

the image.2. Let d"0

3. For k"1,2,2,M do1. 3.1. For each sub-unit u

ijof the newly added unit

3.1.1. Compute FNETij

3.1.2. Let Fy

be the vertical component ofFNETij

3.1.3. Let *y"oFy

3.1.4. Update the vertical position of sub-unit u

ijby adding *y

3.1.5. Let d"d#*y2/M4. If d'e goto step 2, else continue5. If the coordinates of the last added unit collapse

with the bottom of the document image then stop.Else add a new unit to U, located just below

the last added unit.6. Goto step 2.

At the initial step, parameter values for internal,repulsive and attractive forces are chosen and theparameters that control the convergence are in-itialized.

The behavior of the forces in the network aredetermined by the values of the parameters a, b, c.The internal force, enforces the sub-units of thesame unit to lie on a horizontal line. Therefore,when the coe$cient of the internal force a, is largerelative to b and c, the RA network looses its abilityto detect curved baselines. On the other hand, if a istaken to be relatively small, then the local proper-ties of the document image dominates the e!ect ofneighboring sub-units, causing zigzags on the base-line. If the coe$cient of repulsive force b, is relative-ly large, then the in#uence of text pixels on thebaseline decreases. Repulsive force guarantees thatthe distance between the sub-units of di!erent unitson the same column does not come close to 0. Onthe other hand, as b gets smaller, multiple unitstend to capture the same baseline. In fact, thisparameter is proportional to the font size. Theamount of attraction that the text pixels exert onthe sub-units is controlled by the parameter for theattractive force, c. If c is taken to be smaller com-pared to a and b, then the input to the network isreceived as an empty page image. If c is large, thenthe attractive force overrides the repulsive force.This may cause accumulation of more than oneunit on the same line.

o is a constant for controlling the step size of theposition updates. When o gets small, the simulation

4 E. O$ ztop et al. / Signal Processing 75 (1999) 1}10

yields "ner results in terms of the shape of thebaseline. However, it causes slow converge. e isa constant to control the local convergence. Itshould be close to 0. The very "rst unit is introduc-ed just below the top of the document image. At thesecond step, coordinates of the sub-units of thenewly introduced unit are updated M times. This isrepeated until the local convergence is reached. Thelocal convergence criterion checks whether there isany change in the positions of the sub-units. This isevaluated by computing the average square dis-placements after a set of M updates. The algorithmterminates when the coordinates of the lastly intro-duced unit collapse with the bottom of the docu-ment as a result of a local convergence. Otherwise,a new unit is added just below the last one.

5. Results

The performance of the RA network as a baselineextractor is tested on various document images bya simulation program using the C language, underX Windows environment.

The gray-level images are selected among thedocuments from the ancient Ottoman archives(written in Arabic) and from the Latin handwrittenand printed documents. The heavy noise on thedocuments are due to the aging, ink smearage andlow quality of the paper. Furthermore, the hand-written documents have high curvature on thebaselines with considerable overlaps on the ascend-ing and descending portions of the characters.

One of the major superiority of the RA networkis that it does not require any preprocessing stagefor noise reduction and binarization. The sampledocuments are directly fed to the RA network, forbaseline extraction. In most of the documents, thenoise pixels have relatively light gray values com-pared to text pixels and the number of the textpixels exceed the number of the noise pixels. In thiscase, the attractive force of the text pixels domin-ates the attractive force of the noise pixels. Asa result the RA network becomes insensitive tonoise.

In the RA network for baseline extraction, thesub-units are evenly placed on a unit. The choice ofthe number of sub-units is not a critical parameter.

A complex baseline, which does not have a con-stant curvature, needs relatively more sub-unitsthan a simple baseline with constant curvature. Forexample, a linear baseline can be extracted simplyby two sub-units. The number of sub-units mayinitially be chosen large. Then, by checking curva-tures locally on the constructed unit, a kind ofpruning algorithm on the sub-units can be applieduntil any change is detected on the curvatures. Thenetwork is composed of units and sub-units. Thereexists interactions among all of the sub-units, andalso among all of the sub-units and the pixels of theimage. Therefore, whenever a new unit is introduc-ed, its sub-units a!ect the sub-units of already exist-ing units and vice versa. However, since the existingunits are stuck to the local minima, they do notexhibit important changes. Thus, the update is per-formed only for the sub-units of a newly introducedunit. The "rst unit appears just below the top of thedocument. It will be repelled by the top and attrac-ted by the "rst textline. As soon as the local conver-gence is obtained for the "rst unit, second unit isinitialized just below the "rst one. At this moment,repulsive force from the "rst unit pushes the secondone, so that the second unit is captured by theattractive force of the second textline. More speci"-cally, the displacement is determined by the netforce which is composed of internal, attractive andrepulsive forces. Internal forces and attractive for-ces are almost the same for the units discussedabove. However, since these two units repel eachother, the "rst unit goes up and the second unitgoes down. The "rst unit cannot go further, becauseit is also repelled by the top of the document.

In the implementation of the algorithm, M istaken as 10 and e is taken as 0.25 which is decidedexperimentally. During the simulations, the valueof g was "xed to 10, which is relatively large com-pared to other parameters in Eq. (3). If g is chosenrelatively small, then vertical displacements maybecome large (particularly when the distance be-tween a sub-unit and a pixel is close to 0) whichyields jumps rather than smooth displacements.Thus, the unit does not get stuck to a local min-imum. The position at which a new unit is added&just below' the previous one depends on the base-line spacing and the resolution of the documentimage. However, in the implementation, it is "xed

E. O$ ztop et al. / Signal Processing 75 (1999) 1}10 5

Fig. 2. The evolution of the RA network for a handwritten Ottoman text.

to a value of 20 pixels which works "ne for a mod-erate range of documents. The parameters a, b, cand the number of sub-units will be discussed be-low in detail.

The evolution of the RA network for a hand-written Ottoman text is presented in Fig. 2. Taking

the parameters a"600, b"550, c"50 alongwith 15 sub-units on each unit enables the networkto capture the high curvature in the baselines. Fig.2(a) shows the state of the network, when the localconvergence is achieved for the "rst unit. The sec-ond unit is introduced just below the "rst one as

6 E. O$ ztop et al. / Signal Processing 75 (1999) 1}10

Fig. 3. Average square change in the displacement (change in vertical position) of the sub-units of a unit.

indicated in Fig. 2(b). Fig. 2(c) and (d) present someintermediate results for the second unit. The base-line extracted by the second unit is shown in Fig.2(e). Finally, Fig. 2(f ) presents the state when theglobal convergence is achieved.

In order to show the experimental convergenceof the system, evolution of the position updates foran Ottoman text (cf. Fig. 2) is given as a plot in Fig.3. The y-axis is the average square change in thedisplacement of the sub-units of a unit while thex-axis represents the set of iterations (M]numberof subunits"10]15"150). After the "rst set ofiterations, the average square change drops drasti-cally, which indicates in fact, the local convergence.In the algorithm, this fact is tested by the compari-son to e which is "xed to 0.25. Local convergence isachieved in about 10 ms on a PowerPC processor-based computer.

One may expect that the values of the para-meters, particularly those of a, b and c, depend onthe complexity of the document which may bea!ected by the alphabet in which the document iswritten. The complexity may include: noise level,curvature, change of curvature, regularity of writ-ing, overlaps on the ascending and descending por-tions of the characters, etc. However, during the

experimentation it is noted that the RA network isnot sensitive to the changes in the parameters.A wide range of values of the network parameters iscapable of baseline extraction. For example, thetriple (a"5000, b"550, c"50) with 15 sub-unitssuccessfully extracs baselines of all of the presenteddocuments.

Fig. 4 shows a handwritten Ottoman text. Notethat the baselines are quite close to each other.There is a substantial amount of overlap betweenthe adjacent baselines because of ascending anddescending portions of the characters which is quitecommon in handwritten documents. This situationis highly di$cult to handle with standard baselineextraction methods.

Fig. 5 shows a handwritten Latin text with heavynoise and fair amount of overlap between the char-acters of adjacent lines. The simulated RA network,however is quite insensitive to such disturbancesand captures the curvatures of the baselines. Fig.6 shows a printed Latin text. The RA networkeasily "nds the baselines with the above indicatedparameter values.

Finally, the result of one of the most popularbaseline extraction method, namely the horizontalprojection pro"les is demonstrated for comparison

E. O$ ztop et al. / Signal Processing 75 (1999) 1}10 7

Fig. 4. Extracted baselines by RA network for a handwritten Ottoman text.

Fig. 5. Baseline extraction by RA network for a handwritten Latin text.

Fig. 6. RA network extracts baselines easily for a printed Latin text.

8 E. O$ ztop et al. / Signal Processing 75 (1999) 1}10

Fig. 7. Baseline extraction by using horizontal projectionpro"les.

purpose. In Fig. 7, the Ottoman handwritten docu-ment is presented with the extracted baselines bya simple projection method where the image isdivided into vertical strips and a local maximum issought inside the strips. This method requires bi-narization of the document image. Additionally,the method depends on many parameters such asthe search for a local maximum value of the projec-tions. It is clear that the curvature of the extractedbaselines do not exhibit a stable behavior on thedocument image.

RA network or any of the methods mentioned inSection 2 can be applied in a straightforward man-ner to the baseline extraction problem of machineprinted text documents. However, the major prob-lem is the baseline extraction in complex environ-ments, for example, when handwritten text hasirregularities or high skew or even when thereis overlapping of ascending and descending por-tions of characters of adjacent lines. In this case,algorithms do not provide meaningful results.Additionally, a quantitative assessment of the per-formance of the employed method is very di$cult,simply because the identi"cation of the real base-line becomes subjective, depending on the problemdomain.

6. Conclusion

In this study, the RA network for baseline extrac-tion on document images is described. It is based

on an energy minimization principle. This networkmay be incorporated into a complete documentprocessing system as a part of a page layout extrac-tor or it may be used to provide information toa character recognizer.

There are many advantages of RA network base-line extraction method: RA network completelyeliminates the binarization problem. Moreover,employing gray-level images improves the perfor-mance in contrast to the other methods. One of themost serious problems in many image processingtasks is noise sensitivity. The existing methods forbaseline extraction also su!ers from this problem.However, since the attraction of the text pixelsdominate the attraction of the noise pixels on thebaseline, the proposed method is insensitive tonoise. Not only a given set of parameters works "nefor a wide range of documents, but also smallvariations in these parameters does not a!ect per-formance of the method. In the proposed method,the baselines are not restricted to straight lineswhich is the situation for most handwritten docu-ments. Additional di$culty in identi"cation ofbaselines is introduced in hand written documentsby the overlapping portions of ascending and de-scending characters between the baselines. The RAnetwork baseline extraction method handles theproblem of overlapping elegantly.

Finally, the principles used in the RA networkbaseline extraction method can be used to developnew techniques for various image processing tasks,such as segmentation and thinning.

References

[1] H.K. Aghajan, T. Kailath, SLIDE: subspace-based linedetection, T-PAMI 16 (11) (1994) 1057}1073.

[2] A. Amin, O!-line arabic character recognition: the state ofthe art, Pattern Recognition 31 (5) (1998) 517}530.

[3] V. Atalay, M. OG zc7 ilingir, N. Yalabmk, Computer recogni-tion of Ottoman text, in: Proc. ISCIS V, Capadoccia,Turkey, 1990.

[4] R.O. Duda, P.E. Hart, Use of Hough transform to detectlines and curves in pictures, Comm. ACM 15 (1) (1972)11}15.

[5] L.A. Fletcher, R. Kasturi, A robust algorithm for textstring separation from mixed text/graphic images, T-PAMI 10 (1988) 910}918.

E. O$ ztop et al. / Signal Processing 75 (1999) 1}10 9

[6] R.M. Haralick, L.G. Shapiro, Computer and Robot Vision,Addison-Wesley, Reading, MA, 1992, Vol. 1, pp. 602}627.

[7] A.K. Jain, Fundamentals of Digital Image Processing,Prentice-Hall, Englewood Cli!s, NJ, 1989.

[8] J. Kanai, Text-line extraction using character prototypes,in: Workshop on Syntactic and Structural Pattern Recog-nition, Newjersey, 1990, pp. 182}191.

[9] M. Kass, A. Witkin, D. Terzopoulos, Snakes: active con-tour models, Internat. J. Comput. Vision 1 (1988) 321}331.

[10] V.F. Leavers, Which Hough transform?, CVGIP: ImageUnderstanding 58 (2) (1993) 250}264.

[11] E. OG ztop, A.Y. MuK laymm, S.S. GuK l, F. Yarman-Vural, V.Atalay, Repulsive attractive network for baseline extrac-tion, Technical Report TR.95-1, Dept. of Comp. Eng.,METU, February 1995.

[12] A.A. Atmcm, F.T. Yarman-Vural, A heuristic algorithm forcharacter recognition of Arabic script, Signal Processing62 (1997) 87}99.

10 E. O$ ztop et al. / Signal Processing 75 (1999) 1}10