Uncertainty propagation and the matching of junctions as feature groupings

Uncertainty Propagation and the Matching ofJunctions as Feature Groupings

Xinquan Shen and Phil Palmer

AbstractÐThe interpretation of the 3D world from image sequences requires the identification and correspondences of key features in

the scene. In this paper, we describe a robust algorithm for matching groupings of features related to the objects in the scene. We

consider the propagation of uncertainty from the feature detection stage through the grouping stage to provide a measure of

uncertainty at the matching stage. We focus upon indoor scenes and match junctions, which are groupings of line segments that meet

at a single point. A model of the uncertainty in junction detection is described, and the junction uncertainty under the epipolar constraint

is determined. Junction correspondence is achieved through matching of each line segment associated with the junction. A match

likelihood is then derived based upon the detection uncertainties and then combined with information on junction topology to create a

similarity measure. A robust matching algorithm is proposed and used to match junctions between pairs of images. The presented

experimental results on real images show that the matching algorithm produces sufficiently reliable results for applications such as

structure from motion.

Index TermsÐFeature tracking, junction, uncertainty propagation, junction matching, junction topology, topological matching, epipolar

geometry.

æ

1 INTRODUCTION

RELIABLE detection and tracking of low-level featuresacross sequences of images is crucial to many high-level

computer vision tasks such as structure from motion [29],stereo vision [7], object tracking [25], and 3D shapereconstruction [6]. For realistic applications, near real timeprocessing is required. Since operations such as featurematching often have a complexity proportional to thesquare of the number of features involved, it is importantto only consider significant features for scene interpretation.Different types of features are used for different contexts,but common choices include corners [19], [18], edges [15],[4], and line segments [21], [20].

Junctions, as feature groupings, have both characteristics

of point features and line features [22], [2], [8]. A junction in

the image consists of a grouping of line segments (called

junction segments) and a point (called junction location)

describing a corner structure. The matching of junctions

then provides simultaneous correspondences on both

corners and line segments. Since junctions are geometrically

complex image primitives, effective matching criteria can be

derived for establishing the correspondence between junc-

tions in two images. Also, for a given junction in an image,

the number of its matching candidates in another image is

often small. Thus, matching junctions will be both efficient

and reliable.

A considerable body of work on junction detection andapplications has been reported. Techniques for detectingjunctions can be categorized into two approaches: 1) detect-ing line segments and grouping line segments into ajunction [2], [9]; 2) defining a template model of junctionsand treating junction detection as template matching [8],[22]. The first approach is often accurate in determining thedirection of junction segment, while the second approachtends to be more accurate in determining the junctionlocation. Both approaches are sensitive to noise in deter-mining the end-points of junction segments.

Approaches for establishing the correspondence offeatures between images have been extensively investi-gated. Reported work attacked the correspondence problemon various features including points and edges [14], [24],[10], [16] and lines [5], [30], [17], [1], [26]. The informationon camera calibration and camera motion/displacementbetween two images is used to derive geometrical con-straints on features in two or three images. The epipolarconstraint [7] determined from two images provides arelation between points in two images, while the trifocaltensor [12], determined from three images, provides aconstraint on points and lines across the image set.Geometrical constraints derived from two images onlyprovide a necessary condition for matching features. Areliable way to eliminate matching ambiguities is to match agroup of features [13], [1], [27], although information aboutphysical objects (e.g., opaque objects) and camera motion(e.g., small camera motion) may also be used to detect somefalse matchings.

In this paper, we discuss the uncertainty of junctions andits role in feature matching. The principal source ofuncertainty arises from the uncertainties of the end-pointsof the associated line segments. The uncertainty of epipolarconstraint on junctions is then discussed by determining an

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 22, NO. 12, DECEMBER 2000 1381

. X. Shen is with Altera European Technology Center, Holmers Farm Way,High Wycombe, HP12 4XF, UK. E-mail: [email protected].

. P. Palmer is with the School of Electronic Engineering, Information Theoryand Mathematics, University of Surrey, Guildford GU2 7XH, UK.E-mail: [email protected].

Manuscript received 8 July 1999; revised 31 May 2000; accepted 1 Sept. 2000.Recommended for acceptance by S. Sarkar.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference IEEECS Log Number 110206.

0162-8828/00/$10.00 ß 2000 IEEE

uncertainty model associated with two correspondingjunction segments. Based on this uncertainty model, a basicsimilarity measure characterizing a matching likelihood fortwo junctions is formulated. For each pair of junctions, astructural matching between their associated junctiontopologies is also determined by finding a maximalmatched structure from two topology networks. The basicsimilarity measure is then combined with the topologicalmatching to create a new similarity measure associated witha pair of junctions. We demonstrate that this similaritymeasure is robust in detecting the true matching from a setof candidate matchings. A matching algorithm for findingcorresponding junctions in two images is then developedbased on this similarity measure and the computationalcomplexity of the algorithm is analyzed.

The context of this work is in connection with anautonomous mobile robot exploring an unknown environ-ment. The success of the robot's navigation and reconstruc-tion of its environment is determined to a large extent bythe accuracy of the matching process. A fast algorithm isrequired as the image processing chain needs to beperformed in near real time. The matching algorithm alsohas to be robust to noise, as the scene interpretation isderived from it. Robust matching can be achieved bymatching dense feature sets between images (e.g., [28]), butthis is too computationally expensive to be used in ourapplication. Other methods (e.g., [5], [26]) exist whichmatch fewer, higher level features, but, to achieve robust-ness, the matching criterion becomes more expensive tocompute. In this paper, we try to satisfy both requirementsby matching groupings of features. The number of featuresused in the matching must be reduced to a minimumwithout losing geometric information useful for 3D recon-struction. We have therefore focused attention on thematching of junctions which usually define boundaries ofobjects in the scene. We believe this is one of the majornovelties of this work.

We want to fully exploit the information available fromthe mobile robot and, therefore, we assume the epipolarconstraint can be computed from the odometry of the robot.Many more accurate methods for computing the epipolargeometry from the images exist [31], but this would add tothe computational load and ignores the odometry informa-tion that is available. The odometry from the robot is notvery accurate and, therefore, the matching strategy needs tobe robust to errors in derived epipolar constraint. For thisreason, we use junctions as the matching primitive andexploit junction topology to achieve a robust matchingalgorithm. We do not use the trifocal tensor as itexaggerates the noise from the odometry. Experimentshave been applied to real image sequences taken by amobile robot. The results are quantitatively satisfactory andsufficient for applications such as structure from motion.

The rest of the paper is organized as follows: Section 2describes the junction detection process. The probabilitymodel characterizing the uncertainty of the end-points ofthe line segments is described. Section 3 discusses theepipolar constraint on junctions. An uncertainty modelassociated with two corresponding junction segments isdeveloped. Section 4 describes the method for matching

junctions by using the uncertainty information associatedwith the junctions and the information of junction topology.Experimental results and the evaluation of the matchingalgorithm is presented in Section 5. Finally, Section 6presents our conclusions.

2 FEATURE EXTRACTION

In this section, we analyze the principal error sources indetermining locations of junction features. The featureextraction process consists of edge detection using anoptimal filter [23], followed by a Hough transform linefinder [21]. The Hough transform detects linear features,but not the end-points of line segments. An optimizationscheme is employed to obtain an estimate of line parametersup to a subpixel accuracy [20]. Uncertainty in the lineparameters is very small as lines in the images usually havea significant number of edge pixels associated along them.

No matter what method is used for the extraction of linesegments, the principal uncertainty in the line features is inthe locations of the end-points of each segment. In theabsence of any further information, the final edge pixelsassociated with the line segment are projected onto theHough line to determine their end-points, which means thatthese locations are estimated from a single measurement,the final edge pixel, whereas the line parameters areestimated from the set of associated edge pixels.

Our starting point, therefore, is a set of line segmentswith uncertain end-points. From these, we hypothesize thatpairs of line segments may be extended to join at a commonpointÐjunction location. This idea is also extended toseveral line segments all meeting at a common point. Toevaluate the likelihood of the hypothesis being true, weneed to model the uncertainty of the end-points associatedwith each line segment.

2.1 Edge Probability along a Line Segment

We define a line segment in the image as a pair of imagepoints �pp0; pp1� , where ppi � �xi yi�T , i � 0; 1, are the locationsof the two end-points of the segment, and define thedirection of this line segment, nn, as follows:1

nn � �nx ny�T � pp1 ÿ pp0

kpp1 ÿ pp0k; �1�

where k � k is the vector norm. We can determine a unitvector qq orthogonal to nn so that the position of any pointalong the line segment can be expressed as

pp � �qq � tnn; �2�where t is a parameter along the line and � is theperpendicular distance from the origin to the line. Theend-points are then associated with two positions t1 and t2.

Line segments may be located in images by a variety ofmethods, but each associates a set of pixels with thesegment. The length of the line segment is determined bythe probability of finding an ªedgeº pixel associated withthis line segment in a section of length dt. Ignoringsystematic lighting variations, this probability should be

1382 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 22, NO. 12, DECEMBER 2000

1. In the following discussions, we always use a bold-faced symbol todenote a vector or matrix.

independent of where the length dt is chosen along the line.It then follows that the probability must just be proportionalto the length dt, i.e.:

Pdt � �dt; �3�where � is a constant. The probability of finding no edge

pixels along a segment of length L of a line in an image is

then proportional to �1ÿ �dt�L. We can characterize this by

the continuous probability density function:

f�L� � �eÿ�L: �4�This is the probability density that the projection of theworld line extends beyond the detected end-points of thesegment in the image by a distance L. In practice, a linesegment may also be detected with extraneous edge pixelsbeyond the projection of the physical line due to numericalerrors involved in the line detection and the noise in theimage. We model the end-points in this case by a similarprobability with a different parameter in the exponent.Typically, we expect this parameter to be much larger sothat the probability of significantly shortening a linesegment is very small.

Consequently, the uncertainty in the location of eachend-point of a line segment in an image is described by thefollowing probability density function:2

f�t� � eÿ�t; t � 0e�t; t � 0;

��5�

where � and � are parameters determining the strength ofthe uncertainty in extending and shortening the end-pointalong the line segment by a distance t.

2.2 Junction Detection

A junction in the image is defined by a junction location pp0

associated with a set of junction segments with eachsegment running from pp0 to a point ppi (called junctionend-point), i � 1; 2; . . . . Fig. 1 shows a triple junction withjunction location pp0, junction end-points pp1, pp2, and pp3, andsegment directions nn1, nn2, and nn3. The detection of ajunction in an image therefore requires the grouping of linesegments belonging to the same junction and then thedetermination of the junction location.

Given two line segments ��1; nn1� and ��2; nn2� defined by(2), we hypothesize that they meet at the point ofintersection:

pp0 � �1qq1 � �2�nn1 � qq2� ��1 ÿ �2�qq1 � qq2��nn2 � qq1�

�nn1 � nn2��

nn1; �6�

where ª�º denotes dot product of two vectors. We compare

this location with end-points of the two line segments from

the line detector and use (5) to evaluate the probability to

extend/shorten the line segment to this location. The

probability for a junction is then given by the product of

probabilities that the end-point of each line segment can be

changed to reach the point of intersection. A junction is then

determined if this probability is greater than a preselected

threshold.Junctions containing more than two line segments are

determined by examining each intersection point. In

keeping with our earlier assumption, we only consider

N-junctions where the N line segments can be extended/

shortened to intersect with each other with a probability

that exceeds the threshold. This guarantees that the changes

in the line parameters (say � and �) are small. The junction

location is then determined by the probability weighted

average position of the intersection of these line segments.

The probability that this is a junction is given by the sum of

the individual joint probabilities as this is the probability

that it is a junction, rather than the probability that it is a

junction of N line segments.

3 EPIPOLAR CONSTRAINT ON JUNCTIONS

In this section, we discuss the epipolar constraint on

junction end-points. We investigate the propagation of

uncertainties in junctions to the constraint. The derived

uncertainty model on the epipolar constraint is used in

matching stage to create a similarity measure on two

junctions. In the following discussions, we will use an

undashed symbol (e.g., pp) to denote an entity in the first

image, and a dashed symbol (e.g., pp0) to denote an entity in

the second image.

3.1 Epipolar Geometry

Projections of a scene point in two images can be related by the

epipolar constraint which can be expressed as follows [7]:

pp0

1

� �TFF

pp1

� �� 0; �7�

where pp and pp0 are the projections of the scene point in the

first and second images, respectively, and FF is a 3� 3

matrix, called the fundamental matrix [7], which is

determined from the camera parameters and camera

motion/displacement between the two images. Geometri-

cally, the epipolar constraint states that point pp0 lies on the

line defined by

FFpp1

� �;

which is called the epipolar line associated with the point pp.

Symmetrically, point pp lies on the epipolar line defined by

FFT pp0

1

� �:

SHEN AND PALMER: UNCERTAINTY PROPAGATION AND THE MATCHING OF JUNCTIONS AS FEATURE GROUPINGS 1383

2. The factor �� is ignored for simplicity as it would not effect our result

in later discussions.

Fig. 1. Representation of junctions.

3.2 Uncertainty Propagation to Epipolar Lines

Let J � fpp0; pp1; . . . ; ppNg be a junction detected in image 1,

where pp0 is the junction location and ppi is the end-point of

the ith junction segment with orientation nni, 1 � i � N .

From our discussion in Section 2, the estimated end-point ppiis randomly distributed around the true end-point ~ppi along

the direction nni and, thus, can be expressed as follows:

ppi � ~ppi � tnni; �8�where t is a random variable describing a signed distance

from ~ppi to ppi along the direction nni (t is positive if the

increase of the distance is in the same direction as nni) (see

Fig. 2a). The distribution of ppi is characterized by the

probability density function f�t� defined in (5). Let J 0 �fpp00; pp01; . . . ; pp0Ng be a junction detected in image 2 which we

hypothesize should match the junction J in image 1 in the

way that the segment pp0ppi matches the segment pp00pp0i,

1 � i � N , and lppi and l~ppi denote epipolar lines in image 2

associated with the detected end-point ppi and true

(unknown but fixed) end-point ~ppi in image 1, respectively.

Assuming that pp0e is the intersection point between the line

pp00pp0i and the epipolar line lppi ,

3 then pp0e can be expressed as

follows:

pp0e � pp0t � snn0i; �9�where pp0t is the intersection point between the line pp00pp

0i and

the epipolar line l~ppi , and s is the signed distance from pp0t to

pp0e (see Fig. 2b). Clearly, s is a function of the random

variable t. The distribution of pp0e is then characterized by the

probability density function of the random variable s, g�s�.In general sense, g�s� reflects the propagation of the

uncertainty in a junction end-point to the epipolar line

associated with the end-point.We want to determine the probability density function

g�s� for a pair of corresponding junction segments. Since s is

a function of the random variable t, we have [3]:

g�s� � f�t� dtds

�� : �10�

The problem is now reduced to determining the relation

between s and t.From the fact that pp0t is on the epipolar line associated

with the point ~ppi, we have

pp0t1

� �TFF

~ppi1

� �� pp0e ÿ snn0i

1

� �TFF

ppi ÿ tnni1

� �� 0: �11�

Substituting

aab

� �� FF ppi

1

� �and

uuv

� �� FF nni

0

� �;

where aa and uu are 2� 1 vectors into the above equation and

noticing that

pp0e1

� �TFF

ppi1

� �� 0;

we have

t s uu � nn0iÿ �ÿ uu � pp0e � v

ÿ �� ÿ s aa � nn0iÿ � � 0: �12�Vector

aab

� �defines the epiplolar line lppi .

4 We assume, for the moment,

that this epipolar line is not parallel to the segment pp00pp0i, i.e.,

aa � nn0i 6� 0. We can then rewrite (12) as

t�sK ÿD� ÿ s � 0; �13�where


Fig. 2. Epipolar constraint on junction segments. pp0ppi and pp00pp0i are two junction segments with directions nni and nn0i in image 1 and 2, respectively. t is

a random variable describing a signed distance from the true end-point ~ppi to its estimated position ppi. lppi and l~ppi are epipolar lines associated with ppiand ~ppi, respectively. pp0e and pp0t in image 2 are intersection points between the line pp00pp

0i and the epipolar lines lppi and l~ppi , respectively. s is a random

variable describing a signed distance from pp0t to pp0e and �s is a signed distance from pp0i to pp0e.

3. When the context is clear, we do not distinguish an infinite line from aline segment, and use the notation pp0ppi to denote either an infinite linedefined by two points pp0 and ppi or a line segment determined by its twoend-points pp0 and ppi.

4. We assume ppi is not the epipole in image 1 (i.e., lppi is defined).Otherwise, we can simply use a new point ppi ��tnni and then compensatepp0e by ÿ�snn0i.

D � uu � pp0e � v

aa � nn0i; K � uu � nn

0i

aa � nn0i: �14�

Geometrically, D � 0 means that the epipolar line

associated with the point pp0e, lpp0e is parallel to the line pp0ppi(i.e., pp0e is a vanishing point and ppi is located at infinity),5

while K � 0 means that the epipolar lines associated with

the points along the line pp0ppi are all parallel.6 Since ppi is a

point in the image, D must be nonzero.Equation (13) is sufficient for a unique solution of t. This

can be seen from the fact that sK ÿD 6� 0. In fact, according

to (13), sK ÿD � 0 implies that s � 0, which then implies

that D � 0, contradicting the fact that D 6� 0. Thus, we have

t � s

sK ÿD �15�

dt

ds� ÿ D

�sK ÿD�2 : �16�

In order to determine the function g�s�, we need to

determine the domain associated with s and t since function

f�t� has two branches corresponding to t � 0 and t < 0,

respectively. We first prove the following fact:

Fact. When line segments pp0ppi and pp00pp0i are two junction

segments corresponding to the same 3D structure in the scene,

the following condition must be true:7

D < 0 and sK ÿD > 0: �17�

Proof Equation (16) indicates that t is monotonic with

respect to s (monotonically increases if D < 0 and

monotonically decreases if D > 0). When pp0ppi and pp00pp0i

are two corresponding junction segments, they are theprojections of the same 3D line in the scene. Point pairs(ppi, pp

0e) and (~ppi, pp

0t) then describe two 3D points on that

3D line, respectively. Thus, t should be continuous andmonotonically increase with s, i.e., D < 0. According to(15), t � 0 when s � 0. Therefore, t and s have the samesign, implying that sK ÿD > 0. tu

Based on the condition (17), the domain associated with sand t can be correspondingly determined:

s 2 �ÿ1;�1� and t 2 �ÿ1;�1�; if K � 0s 2 �DK ;�1� and t 2 �ÿ1; 1

K�; if K > 0s 2 �ÿ1; DK� and t 2 � 1

K ;�1�; if K < 0:

The fact that s approaches infinity when t approaches 1K

means that the point corresponding to t � 1K , ppv � ~ppi � 1

K nni,is the position where the line pp0ppi vanishes in image 1 (seeFig. 3a). Similarly, the point corresponding to s � D

K ,pp0v � pp0t � D

K nn0i, is the position where the line pp00pp

0i vanishes

in image 2 (see Fig. 3b).Substituting (5), (15), and (16) into (10) and incorporating

the above analysis on the relation between s and t, we have

g�s� �ÿ D�sKÿD�2 e

ÿ�ssK�D; s � 0

ÿ D�sKÿD�2 e

�ssK�D; s � 0:

8<: �18�

It has been found that, when sK < 0 (i.e., when s is

increased/decreased toward the vanishing point pp0v), g�s�does not monotonically decrease within the range �0; DK� if

� < ÿ2K and the range �DK ; 0� if � < 2K, respectively.8 This

arises from the situation that the estimated end-point ppi is

allowed to be distributed within a large range (due to a

relatively small � or �) beyond the true end-point ~ppi, while

point pp0e in image 2 can only be distributed within the small

range beyond pp0t (due to a relatively large K). This then

suggests that the uncertainty model (in fact, the parameters


Fig. 3. Vanishing positions of 2D lines in images. (a) The projection of the 3D line ÿ vanishes at the point ppv in image 1. The value of t corresponding

to ppv is 1K . (b) The projection of the 3D line ÿ vanishes at the point pp0v in image 2. The value of s corresponding to pp0v is D

K .

5. This can be seen from the fact that uu � pp0e � v � FFT pp0e1

� �� Tnni0

� �and

FFT pp0e1

� �defines the epipolar line lpp0e in image 1. The exception case that pp0e is

the epipole in image 2 is not considered.

6. This can be seen from the fact that uu � nn0i � 1� nn0iT FF

pp1

� �ÿ pp0

1

� �� ,

where pp is a point on the line pp0ppi, and � is the distance from the point pp0 to

pp.7. This is a necessary conditions for the correspondence of two junction

segments. In the junction matching stage (see Section 5), the matchingstrength associated with two junctions is taken as zero if either of theconditions is violated. 8. In fact, g�s� has a maximum at s �

DK �1� �

2K�; � < ÿ2KDK �1ÿ �

2K�; � < 2K:

�

of the model) associated with the observed end-point ppi is

inappropriate and provides no information on the location

of the corresponding true end-point in the other image. We

use fixed model parameters for all detected features in

order to formulate a similarity measure based on g�s� to

quantify a matching likelihood associated with two junc-

tions. When the above case occurs, we simply ignore the

constraint from the first image, and take g�s� � ÿ 1D eÿ�s for

s � 0 and g�s� � ÿ 1D e

�s for s � 0, respectively. The factor

ÿ 1D is introduced so that g�s� has the same value at s � 0 as

other cases. Consequently, g�s� has the following form:

g�s� �

ÿ D�sKÿD�2 e

ÿ �ssKÿD; s � 0 and � � ÿ2K

ÿ 1D eÿ�s; s � 0 and � < ÿ2K

ÿ D�sKÿD�2 e

�ssKÿD; s � 0 and � � 2K

ÿ 1D e

�s; s � 0 and � < 2K:

8>>>><>>>>: �19�

The above form of g�s� is derived under the assumption

that the epipolar line lppi is not parallel to the segment pp00pp0i.

However, when segments pp0ppi and pp00pp0i correspond to the

same 3D line in the scene, the only case for the epipolar line

lppi being parallel to the segment pp00pp0i is that the line lppi is

colinear with the segment pp00pp0i when both pp0ppi and pp00pp

0i are

on the epipolar plane [7]. In this case, we can always treat pp0eas the point pp0t (i.e., s � 0). Thus, when lppi is parallel to pp00pp

0i,

we simply take g�s� � ÿ 1D if lppi is colinear with pp00pp

0i and

g�s� � 0 otherwise.

3.3 Uncertainty Propagation to Epipolar Constraint

To quantify how likely two junction segments pp0ppi and pp00pp0i

actually describe the same 3D structure, we determine a

probability H�ppi; pp0i� that ppi and pp0i are the estimated

positions of true end-points ~ppi and ~pp0i on two corresponding

line segments. Obviously, pp0t � ~pp0i if pp0ppi and pp00pp0i are

corresponding line segments. Thus, H�ppi; pp0i� is equivalent

to a probability that both pp0i and pp0e are the estimated

position of ~pp0i. In the general sense, H�ppi; pp0i� reflects an

uncertainty propagation from junction end-points to the

epipolar constraint on these end-points.When pp0ppi and pp00pp

0i are corresponding line segments, the

distributions of positions pp0i and pp0e are characterized by

probability density functions f�sÿ �s� and g�s�, respectively,

where �s � ÿ aa�pp0i�baa�nn0i is the signed distance from pp0i to pp0e along

the direction nn0i (see Fig. 2b). Thus, given a value of s (i.e.,

given a ~ppi and ~pp0i), the probability that both pp0i and pp0e are the

estimated position of ~pp0i can be taken as:9

f�sÿ �s�g�s��s�2: �20�This is a function of the variable s. Since true end-points ~ppiand ~pp0i are unknown, the probability H�ppi; pp0i� is then

defined as the maximum value of this function, i.e.,

H�ppi; pp0i� � maxs

f�sÿ �s�g�s��s�2n o

� maxs

h�s�f g� �

��s�2;

�21�where h�s� � f�sÿ �s�g�s�.

It should be noted that the probabilityH�ppi; pp0i� is derived

based on the epipolar constraint from image 1 to image 2.

We can also derive a similar probability H0�ppi; pp0i�, also

characterizing the correspondence between ppi and pp0i, using

the epipolar constraint from image 2 to image 1.The behavior of the function h�s� depends on domains of

s and �s which are uniquely determined by the positional

relation among points pp0i, pp0e, and pp0t on the line pp00pp

0i. Fig. 4

shows all possible positional configurations of points pp0i, pp0e,

and pp0t. It is easy to see that both functions f�sÿ �s� and g�s�monotonically decrease with s in domains s � 0 and �s � 0

(Fig. 4a) and 0 � �s � s (Fig. 4b), respectively, and mono-

tonically increase with s in domains s � 0 and �s � 0 (Fig. 4d)

and s � �s � 0 (Fig. 4e), respectively. The maximum of h�s�for these cases occurs at either s � 0 or �s � 0. Thus, it is

sufficient to determine H�ppi; pp0i� by considering the function

h�s� in domains 0 � s � �s (Fig. 4c) and �s � s � 0 (Fig. 4f)

only. According to (5) and (19), we have

h�s� �

ÿ D�sKÿD�2 e

��sÿ�s�ÿ �ssKÿD; 0 � s � �s and � � ÿ2K

ÿ 1D e��ÿ��sÿ��s; 0 � s � �s and � < ÿ2K

ÿ D�sKÿD�2 e

ÿ��sÿ�s�� ssKÿD; �s � s � 0 and � � 2K

ÿ 1D e��ÿ��s��s; �s � s � 0 and � < 2K:

8>>>><>>>>:�22�

We now determine the maximum value of the function

h�s�. We approach the problem by considering whether the

value of K is zero.

A. K � 0.

1. 0 � s � �s: h�s� is monotonic with respect to s. Themaximum of h�s� occurs at either s � 0 or s � �s. Thecorresponding maximum value hmax can be deter-mined as follows:

hmax � ÿ 1D eÿ��s; if � � ÿ�=D

ÿ 1D e

��s=D; if � � ÿ�=D:�

�23�

2. �s � s � 0: By analogy to case 1, we have

hmax � ÿ 1D e

��s; if � � ÿ�=Dÿ 1

D eÿ��s=D; if � � ÿ�=D:

��24�

B. K 6� 0.

1. 0 � s � �s and � < ÿ2K: h�s� is monotonic withrespect to s. The maximum of h�s� occurs at eithers � 0 or s � �s, leading to:

hmax � ÿ 1D eÿ��s; � � �

ÿ 1D eÿ��s; � � �:

��25�


9. This is a probability element [3] corresponding to the probability thatthe value of the random variable s is located within small interval�sÿ�s=2; s��s=2�. In numerical implementation, we can compute thisprobability by giving a suitable value to the differential element �s, e.g.,�s � 1.

2. 0 � s � �s and � � ÿ2K: h�s� has a local maximum/minimum. By solving s from the equation @h�s�

@s � 0and noticing the fact that sK ÿD > 0, we have

sm �DK �

1��1ÿ��D=K2p

� ; K > 0

DK �

1ÿ��1ÿ��D=K2p

� ; K < 0;

8<: �26�

where sm denotes the location of the local max-imum/minimum of h�s�.

It can be verified that h�sm� is the maximumwhen K < 0. However, since functions g�s� andf�sÿ �s� are discontinuous at s � 0 and s � �s,respectively, sm may be located outside the interval�0; �s�, indicating that h�s� is actually monotonicwithin the interval �0; �s�. Thus, hmax is determinedas follows:

hmax � h�sm�; if K < 0 and 0 � sm � �smaxfh�0�; h��s�g; otherwise:

��27�

3. �s � s � 0 and � < 2K: By analogy to case 1, we have

hmax � ÿ 1D e

��s; � � �ÿ 1

D e��s; � � �:

��28�

4. �s � s � 0 and � � 2K: By analogy to case 2, we have

hmax � h�sm�; if K > 0 and 0 � sm � �smaxfh�0�; h��s�g; otherwise;

��29�

where sm is determined as follows:

sm � D

Kÿ 1ÿ ��

1ÿ ��D=K2p

�; K > 0: �30�

Obviously, the probability H is also a function of �s. Tojustify our use of H to characterize the uncertainty of anepipolar constraint on junction segments, we need to showthat, for any �s1 and �s2 such that j�s1j � j�s2j, H��s1� � H��s2�.This is easily seen from the fact that h monotonicallydecreases with respect to the absolute value of �s, i.e., h��s1� �h��s2� when j�s1j � j�s2j. It then follows that H describes a

strength of matching between line segments pp0ppi and pp00ppi0,which is what we require.

4 MATCHING JUNCTIONS

In this section, we present a method for establishing thecorrespondence of junctions in two images. Two junctionsare considered as a candidate matching only if they havethe same number of junction segments. The uncertaintymodel for the epipolar constraint on junction segments isused to derive a similarity measure associated with eachcandidate matching. The junction matching is then deter-mined based on the similarity value on each candidatematching.

4.1 Similarity Based on Two Junctions

Let J � fpp0; pp1; . . . ; ppNg and J 0 � fpp00; pp01; . . . ; pp0Ng be two

junctions in the first and second images, respectively. End-

points in a junction are numbered so that the junction

segments are organized in a clockwise order. We construct a

function ��J; J 0� which measures the similarity between J

and J 0 under the epipolar constraint. For two junctions J 01and J 02 in the second image, ��J; J 01� > ��J; J 02� then

indicates that J and J 01 are more likely to be a pair of

matched junctions (corresponding to the same 3D structure

in the scene) than J and J 02.

The epipolar constraint on two images provides a

necessary condition for a pair of matched junctionsÐif J

and J 0 are images of the same 3D structure in the scene,

epipolar lines lppi and lpp0i should pass through points pp0i and

ppi, respectively. In practice, this condition may not be

satisfied due to the uncertainties in junction detection.

However, it does suggest that we can assume a high

probability that pp0i and ppi satisfy the epipolar constraint if J

and J 0 are a pair of matched junctions. We use this

knowledge to construct the similarity function ��J; J 0�.Let d and d0 be distances from pp0 and pp00 to epipolar lines

lpp00

and lpp0, respectively. Consider

�1 � 0:5�eÿwd � eÿwd0 �; �31�where the parameter w is chosen so that eÿwR � 0:01, where

R is a threshold for the largest possible distance from the

location of one junction to the epipolar line associated with


Fig. 4. Positional configurations of points pp0i, pp0e, and pp0t on the line pp00pp

0i.

the location of another corresponding junction. Clearly, �1

characterizes a likelihood that pp0 and pp00 satisfy the epipolar

constraint.10 J and J 0 are considered to be unmatched if

either d or d0 is greater than R.To evaluate the strength of matching between two

junction segments, we use the probability model H derivedin the previous section. Let H�ppi; pp0j� and H0�ppi; pp0j� be theprobabilities characterizing the correspondence between ppiand pp0j (see Section 3.3). Consider

M�k� � 1

N

XNi�1

H�ppi; pp0j�H0�ppi; pp0j�h i

; 0 � k < N;

j � ��i� kÿ 1�modN� � 1:

�32�

M�k� then measures a similarity between J and J 0 whensegments pp0ppi are chosen to be matched with segments pp00pp

0j.

Value k indicates which segment in J 0 matches the firstsegment pp0pp1 in J . The similarity between J and J 0 based onthe correspondence of junction segments is then defined as

�2 � maxkfM�k�g; 0 � k < N: �33�

By combining �1 and �2, the similarity function ��J; J 0� isdefined as:

��J; J 0� � 0:5��1 � �2�: �34�For consistency, we simply assign a zero value to ��J; J 0� ifJ and J 0 are considered as unmatched either due to d > Ror d0 > R.

Similarity function ��J; J 0� then provides a measure onthe likelihood that the junction J in image 1 matches withthe junction J 0 in image 2. Basically, ��J; J 0� tends to belarge if J and J 0 describe the same 3D structure in the scene.However, function � is derived from the epipolar con-straint, which only provides a necessary condition for twocorresponding junctions. Two unmatched junctions mayalso satisfy the epipolar constraint and lead to a large valueof �.

4.2 Similarity Based on Junction Topologies

Let J 01 and J 02 be two junctions in image 2 which both satisfythe epipolar constraint with a junction J in image 1.According to the discussion in the previous section, thesimilarity function � provides no information for discrimi-nating J 01 and J 02 from each other based on the junction J .However, this ambiguity between J 01 and J 02 in matchingwith J may be eliminated by investigating neighboringfeatures of junctions. In this section, we aim to determine anew similarity measure ��J; J 0� which characterizes alikelihood of matching between J and J 0 by comparingtopologies of a cluster of junctions associated with J and J 0,respectively.

The topology of junctions in an image can be describedby a graph [11] in which each vertex represents a junctionand each edge linking two vertices corresponds to the linesegment connecting the junctions (see Fig. 5). A connectedsubgraph then describes the topology of a junction network

in which each junction shares a line segment with anotherjunction. Let GJ and GJ 0 be two maximal connectedsubgraphs containing the junction J in image 1 and junctionJ 0 in image 2, respectively.11 A structural matching betweenjunction networks associated with J and J 0 can thus bedetermined by establishing an edge mapping between GJ

and GJ 0 . Clearly, each edge in a graph characterizes astructure containing two junctions sharing a line segmentand a pair of edges from two graphs can be considered for amapping only if their corresponding structures (linesegments) can be matched. Ideally, we need to find amapping which describes a maximal structural matchingbetween two junction networks. In general, this is still anNP-problem since there may be more than two junctionsconnected to the same line (e.g., junctions 2, 3, and 4 inFig. 5). In this work, we determine a mapping bysimultaneously traversing GJ and GJ 0 starting from verticesJ and J 0, respectively. At each pair of correspondingvertices (the first such pair is �J; J 0�), an optimal mappingbetween currently unmapped edges emitted from thecurrent vertices is determined based on the values of thesimilarity function � at the junctions associated with thecorresponding edges. The traversal then proceeds to thenext pair of corresponding vertices according to thedetermined local edge mapping. The procedure is repeateduntil no more edges need to be processed. The computa-tional complexity of this algorithm is in polynomial time.Since similarity function � usually has a good discrimina-tion between a true matching and a false matching, thedetermined mapping is often a good approximate to theglobally optimal one which describes the maximal structur-al matching between two junction networks. In thefollowing discussion, we first present the method fordetermining a mapping between edges emitted from twovertices in GJ and GJ 0 , respectively, and define a similaritymeasure associated with each pair of mapped edges. Themapping between GJ and GJ 0 for computing the similaritymeasure ��J; J 0� is then determined by recursively applyingthe local edge mapping.

We use JiJj to denote an edge linking vertices Ji and Jj,

and <JiJj; J0iJ0j> to denote a mapping between the edge

JiJj and the edge J 0iJ0j. A mapping <JiJj; J

0iJ0j> is called


Fig. 5. A graph description of the topology of junctions in an image.

(a) Junctions in an image. (b) The corresponding graph describing the

topology of these junctions.

10. This likelihood measure decreases exponentially as d and d0 increase.To be more tolerant of the inaccuracy of the epipolar constraint, we mayalternatively use a linear measure �1 � 0:5 2ÿ �d� d0�=R� �.

11. Since each vertex in the graph represents a junction, vertices arenotated by junction notations.

plausible if ��Ji; J 0i� > 0, ��Jj; J 0j� > 0, and ��Ji; J 0i� and

��Jj; J 0j� are obtained by matching line segment JiJj to J 0iJ0j.

Let EJ be a set of edges emitted from the vertex J in GJ and

EJ 0 be a set of edges emitted from the vertex J 0 in GJ 0 . A

mapping between two sets of edges can be represented by a

set containing each individual pair of mapped edges. Let

C�EJ;EJ 0 � be a mapping between EJ and EJ 0 , and

determined in a recursive way as follows:

C�EJ;EJ 0 � �f<JJi; J 0J 0j>g[C�EJÿfJJig; EJ 0ÿfJ 0J 0jg�

if '�JJi;J 0J 0j;EJ;EJ 0� � True

; otherwise;

8><>:�35�

where [ denotes the union of two sets, ; denotes the emptyset, and ' is a predicate function. For JJi 2 EJ andJ 0J 0j 2 EJ 0 , the function '�JJi; J 0J 0j; EJ ; EJ 0� is true if thefollowing conditions are satisfied:

1. Edge mapping <JJi; J0J 0j> is plausible.

2.

a. I f <JJk; J0J 0j> , k 6� i, is also plausible ,

��Jk; J 0j� � ��Ji; J 0j�.b. I f <JJi; J

0J 0k> , k 6� j i s a l so plaus ib le ,��Ji; J 0k� � ��Ji; J 0j�.

The set C�EJ;EJ 0 �, then defines an optimal mapping

between EJ and EJ 0 . The predicate function ' enforces a

one-to-one mapping describing a structural matching

between junction networks. The first condition of the

function ensures that the mapping follows the structure of

junctions, while the second condition ensures that the pair

producing the largest similarity value � is chosen if there

exists multiple plausible mappings associated with an edge.

For a structural matching described by a mapping

<JJi; J0J 0j> , a similarity measure quantifying this matching

is defined as 0:5��J; J 0� � ��Ji; J 0j��.From now on, we always assume that EJ and EJ 0 only

contain those edges which currently have not been mappedto other edges. The mapping between graphs GJ and GJ 0 isdetermined by simultaneously traversing GJ and GJ 0

starting from vertices J and J 0, respectively, and applyingthe above edge mapping scheme to each pair of verticessimultaneously reached during the traversals. First, amapping C�EJ;EJ 0 � is determined. For each pair<JJi; J

0J 0j> 2 C�EJ;EJ 0 �, the traversals are stepped tovertices Ji and J 0j, respectively. A mapping C�EJi ; EJ 0j� isfurther determined for those unmapped edges emitted fromJi and J 0j, respectively. The operation is recursively appliedto each pair of mapped edges until no more edges need to beprocessed. The similarity measure ��J; J 0� is defined as��J; J 0� if C�EJ;EJ 0 � � ;. Otherwise, ��J; J 0� is computed byaccumulating the matching similarity associated with eachpair of mapped edges. The operation for simultaneouslytraversing GJ and GJ 0 can be implemented using a stack. Anillustrative algorithm for determining ��J; J 0� is as follows:

1. If C�EJ;EJ 0 � � ;, ��J; J 0� � ��J; J 0�. Stop.

2. Initialization

a. Mark all edges in GJ and GJ 0 as unmapped,b. For each pair <JJi; J

0J 0i> 2 C�EJ;EJ 0 �Push JJi and J 0J 0i onto stack and mark them as

mapped.c. ��J; J 0� � 0.

3. While the stack is not empty

3.1. Pop two edges from the stack ) JiJj and J 0iJ0j

3.2. For each pair <JjJk; J0jJ0k> 2 C�EJj ; EJ 0j�

a. ��J; J 0� � ��J; J 0� � 0:5��Jj; J 0j� � ��Jk; J 0k��.b. Push JjJk and J 0jJ

0k onto the stack and mark

them as mapped.

In practical situations, most of the junctions have at mostthree line segments. Situations where an edge has multipleplausible mappings are rare cases and usually only involvea small number of candidate mappings. Thus, the averagecomputational time for determining the set C can beconsidered as a constant time if the time for computingthe function � is taken as a unit. The time for traversing agraph is proportional to the number of edges in the graph.Let NGJ

and NGJ 0 be the number of edges in graphs GJ andGJ 0 , respectively. The time for determining a graphmapping by simultaneously traversing GJ and GJ 0 is lessthan O�2 �minfNGJ

;NGJ 0 g�. Thus, the computational com-plex i ty of t he a lgor i th m i s of t he order ofO�minfNGJ

;NGJ 0 g�. It should be noted that ��J; J 0� ��J; J 0� if GJ or GJ 0 is trivial, i.e., only contains one vertex.Also, ��J; J 0� still reflects a necessary condition for detect-ing corresponding junctions as it is derived from ��J; J 0�.However, since ��J; J 0� also uses the information of junctiontopology, a matching decision based on ��J; J 0� will bemore reliable than the one based on ��J; J 0�.4.3 Physical Constraints

Two uncorresponding junctions may still have a largesimilarity value �. To further eliminate ambiguities, weexplore some constraints derived from a priori informationabout the camera motion and properties of objects in theenvironment.

We assume that the object is opaque. Thus, the order ofjunction segments would not change from one image to thenext image. For two corresponding junctions, J and J 0, thisknowledge can be expressed as follows:

nni � nni�1� � � nn0i � nn0i�1

ÿ �> 0; 1 � i < N; �36�

where ª�º denotes the cross product of two vectors.In our application, images are captured by a mobile robot

with a high capture rate on images. The camera motion(and, therefore, the disparity) between two images is small.This implies that the difference in orientation between twocorresponding junction segments is also small. Therefore,two corresponding junctions should satisfy the followingconstraint for two corresponding segments:

nni � nn0i � 0; 1 � i � N: �37�Any two junctions which do not satisfy the above two

constraints are considered as an unmatched pair whose


associated similarity function � has a zero value. It should

be pointed out that the incorporation of the above two

constraints only eliminates part of the matching ambiguity.

Since junctions are complex image features, most of the

obvious false matchings are, however, eliminated by using

the similarity measure � combined with the above two

constraints.

4.4 The Algorithm

Given a junction J in image 1 and a set of candidate

junctions J 01; J02; . . . ; J 0M in image 2, J 0k is chosen to be

matched with J , if the following conditions are satisfied:

��J; J 0k� > �1��J; J 0k� � max

if��J; J 0i�g

minif ��J; J 0k� ÿ ��J; J 0i�� g > �2;

8><>: �38�

where �1 is a threshold for rejecting a matching candidate

which has a similarity measure not greater than this

threshold and �2 is a threshold for discriminating a true

matching from other candidate matchings.Let J � fJ1; J2; . . . ; JPg be P junctions in image 1, and

J 0 � fJ 01; J20 ; . . . ; J 0Qg be Q junctions in image 2. The

algorithm for finding all matched pairs of junctions is as

follows:

1. For each junction Ji 2 J1.1. For each junction J 0j 2 J 0

1.1.1. If ��Ji; J 0j� > �1, put fJi; J 0j; ��Ji; J 0j�g into list S.2. For each �Ji; J 0j� 2 S

2.1. For each �Ji; J 0k� 2 S, k 6� j2.1.1. If ��Ji; J 0j� < ��Ji; J 0k� or j��Ji; J 0j� ÿ ��Ji; J 0k�j � �2

Mark �Ji; J 0j� as an unmatched pair and skip the

following steps.2.2. For each �Jk; J 0j� 2 S, k 6� i

2.2.1. If ��Ji; J 0j� < ��Jk; J 0j� or

j��Ji; J 0j� ÿ ��Jk; J 0j�j � �2Mark �Ji; J 0j� as an unmatched pair and skip the

following steps.

2.3. Mark �Ji; J 0j� as a matched pair.

The algorithm requires evaluating the similarity function

� for PQ times in Step 1 for finding all candidate matchings

for each junction in J . Let Nc be the average number of

candidate matchings for each junction in J . Assuming P �Q (otherwise, we just need to exchange the role of J and J 0in the algorithm), the list S then contains at most NcP items

which are ordered based on the ordinal number of junctions

in J . Given an access to an item describing a candidate

�Ji; J 0j�, the average time for finding all candidates �Ji; J 0k�,k 6� j is Nc and the time for finding all candidates �Jk; J 0j�,k 6� i is NcP . The computational time in Step 2 for scanning

S to find candidates satisfying conditions in (38) is then no

more than NcP �Nc �NcP �. In practice, Nc is normally less

than 5 and Nc � P . So, N2c P

2 will be in the same order as

PQ. Therefore, the computational time of the algorithm is of

the order of O�PQ�.

5 EXPERIMENTAL RESULTS

In this section, we present some experimental results onreal images. The experiments are devised to investigatethe performance of the junction matching algorithmdeveloped based on the uncertainty model introducedfor junctions. The matching operation is applied to imagesequences captured using a mobile robot with knownodometry. The camera of the robot is calibrated, but notaccurately, and the radial distortion is also ignored.Junctions in each image of the sequence are determinedbased on a set of line segments provided by existingfeature extraction algorithms [23], [20], [21].

Our first experiment is applied to a sequence of 10 imagestaken as the robot moves across a lab. Each image has a size of720� 576. The motion of the camera is approximatelycalibrated so that a fairly accurate epipolar constraint(fundamental matrix) can be computed. The matchingalgorithm is applied to all V- and Y-junctions with parametervalues R � 6, �1 � 0:0, and �2 � 0:1. Fig. 6 shows matchedjunctions in image pairs �0; 1�, �2; 3�, �4; 5�, �6; 7�, and �8; 9�.Each pair of matched junctions is labeled with a number attheir locations and the correspondence of junction segmentsis indicated by highlighting the corresponding segments withthe same gray-level (white or black). The matching operationis fast. On average, it spends less than 0:2 second CPU time onan Ultra-Sparc with a 300MHz CPU for each pair of images.The performance of the matching operation is summarized inTable 1. It can be seen that most of the detected matchings arecorrect matchings. False matchings are junction pair 15 inimage pair (0, 1), 0 and 10 in image pair (2, 3), 13 and 16 inimage pair (4, 5), and 21 in image pair (6, 7).12 Allmismatchings arise from the situation where the junction tobe matched is either not detected or poorly detected due to thenoise in the image. The matching algorithm relies on the basicsimilarity measure � derived based on the uncertainty ofjunctions under the epipolar constraint which only provides anecessary condition for two corresponding junctions. Theo-retically, the algorithm will not be effective to the situationwhen two candidate junctions have similar junction topologyand both satisfy the epipolar constraint with the junction to bematched. In practice, this situation rarely happens forgeometrically complex features such as junctions.

The similarity function ��J; J 0� is derived by accumulatingthe basic similarity measure � on a structural matchingbetween two junction networks. To investigate the advantagefor incorporating a matching on junction topologies, wecompare the performance of junction matchings produced byusing � and � as the similarity measure in the matchingcriterion (38), respectively. We also compare the matchingresults obtained by using different values of tuning para-meters to investigate the effect of these parameters to theperformance of the algorithm. The matching algorithm hasthree tuning parameters R, �1, and �2 (see (31) and (38)).Parameter �1 is a threshold for rejecting a matching hypoth-esis which has a similarity measure not greater than this


12. Junction 15 in images 0 and 1 is located below the junction 11. Thelabel for junction 0 in image 3 is covered by the label for junction 2. The labelfor junction 16 in image 5 is covered by the label for junction 21. Labels forjunction 21 in images 6 and 7 are covered by the labels for junction 22,respectively.


Fig. 6. Experimental result of junction matching applied to a sequence of 10 images. The matched V- and Y-junctions in image pairs �0; 1�, �2; 3�,�4; 5�, �6; 7�, and �8; 9� are shown by labeling their locations and highlighting their corresponding line segments.

threshold while �2 is a threshold for discriminating a truematching from other candidate matchings. A value less than0:5 is typical for the parameter �1. In most of our experiments,the value of �1 is simply set to zero, meaning that allmatching candidates are considered in the first instance.Parameter R controls the distance (search radius) from theepipolar line for searching candidate junctions. Its valuereflects the allowed error in epipolar constraint. Obviously,a robust matching algorithm should be insensitive to thechange of these parameters. Table 2 presents results ofjunction matching on the image pair (4, 5) shown in Fig. 6by using � and � as the similarity measure in the matchingcriterion (38) with a fixed �1 � 0 and different values forparameters �2 and R, respectively. For each set of parametervalues, numbers of total detected matchings, correctmatchings, and mismatchings are considered. It can beseen that, as the threshold �2 increases, the number ofmatching detected by � is quickly reduced, while theperformance of � keeps nearly the same. This means that �is more discriminative than � among candidate matchingsand, thus, is more powerful in eliminating possibleambiguities. The sensitivity to the change of search radiusR is not significant to both � and �. In general, matchingalgorithm using the similarity measure � is more robustthan using the basic similarity measure � only in terms ofthe sensitivity to the change of tuning parameters.

The next experiment demonstrates how the use of arobust matching algorithm can compensate poor knowl-edge of the camera motion. Fig. 7 shows the junctionmatching applied to a sequence of 20 images taken as therobot moves along a corridor. Each image has a size of720� 484. The camera motion between successive images isnot calibrated. The odometry information provided by the

robot is directly used to calculate the epipolar constraint.Parameter values of the matching algorithm are chosen asR � 10, �1 � 0:25, and �2 � 0:1. The reason for using largeparameter values is the inaccuracy of the fundamentalmatrix directly determined from the odometry information.In this experiment, the matching algorithm is also appliedto T-junctions together with V- and Y-junctions. T-junctionsoften describe the occlusion between two objects (e.g., theocclusion between the back wall and side walls of thecorridor). Matching T-junctions allows us to track thoseocclusions. The result is presented for image pairs �0; 1�,�4; 5�, �8; 9�, �12; 13�, and �16; 17� by labeling each matchedpair of junctions in corresponding images. It can be seenthat the matching algorithm still performs very well eventhough the epipolar constraint derived from odometryinformation is not very accurate.

The matching algorithm also performs well even whenthe odometry is complex. Fig. 8 shows the experimentapplied to a sequence of 15 images taken as the robotªlooksº around the corner of a lab. The motion of thecamera involves both rotation and translation. Each imagehas a size of 320� 240. Again, odometry of the robot isdirectly used to calculate the epipolar constraint. Thematching algorithm is applied to V-, Y-, and T-junctionswith parameter values chosen as R � 10, �1 � 0:0, and�2 � 0:1. The result is presented for image pairs �0; 1�, �3; 4�,�6; 7�, �9; 10�, and �12; 13�.

6 CONCLUSION

We have presented a method for the detection andmatching of junctions extracted from two images. We havemodeled the principal error sources from the feature


TABLE 2Performance of Junction Matchings on Image Pair (4, 5) Shown in Fig. 6 by Using � and � as the Similarity

Measure in the Matching Criterion (38) with a Fixed �1 � 0 and Different Values for Parameters �2 and R, Respectively

TABLE 1Performance of the Junction Matching on Image Pairs Shown in Fig. 6

extraction algorithms and used this to define a matching

uncertainty which has then been incorporated into a

similarity function used to determine junction correspon-

dences. The matching criterion is based upon the


Fig. 7. Experimental result of junction matching applied to a sequence of 20 images. The matched V-, Y-, and T-junctions in image pairs �0; 1�, �4; 5�,�8; 9�, �12; 13�, and �16; 17� are shown by labeling their locations and highlighting their corresponding line segments.

well-known epipolar constraint, which only provides anecessary rather than sufficient condition for matching. We

have then extended the similarity function to include localtopology in the matching process. This is a straightforward


Fig. 8. Experimental result of junction matching applied to a sequence of 15 images. The matched V-, Y-, and T-junctions in image pairs �0; 1�, �3; 4�,�6; 7�, �9; 10�, and �12; 13� are shown by labeling their locations and highlighting their corresponding line segments.

extension of the matching algorithm, but provides a much

greater robustness to the procedure.We have demonstrated the robustness of the matching

algorithm using real data taken from a mobile robot. Theunfiltered odometry from the robot is used to compute theepipolar constraint. Junctions have been successfullymatched between each successive frames in the sequences,which have included a wide range of camera motions. Theuncertainty in the camera motion does not affect the results asthe robustness of the method allows a significant tolerance inthe parameter settings, which more than compensates.

ACKNOWLEDGMENTS

This work is supported by the grant from EPSRC (grantGR/L34099). The authors would also like to express theirthanks to members in the Centre for Vision, Speech, andSignal Processing, University of Surrey, for their supportand valuable discussions. This work was done whenXinquan Shen was with the School of Electronic Engineer-ing, Information Theory & Mathematics, University ofSurrey, United Kingdom.

REFERENCES

[1] N. Ayache and B. Faverjon, ªEfficient Registration of StereoImages by Matching Graph Descriptions of Edge Segments,º Int'lJ. Computer Vision, vol. 1, no. 2, pp. 107-131, 1987.

[2] D.J. Beymer, ªFinding Junctions Using the Image Gradient,º Proc.Computer Vision and Pattern Recognition, pp. 720-721, June 1991.

[3] Z.W. Birnbaum, Introduction to Probability and MathematicalStatistics. Harper & Brothers, 1962.

[4] J. Canny, ªA Computational Approach to Edge Detection,º IEEETrans. Pattern Analysis and Machine Intelligence, vol. 8, no. 6,pp. 679-698, Nov. 1986.

[5] Y. Chang and J.K. Aggarwal, ªLine Correspondences fromCooperating Spatial and Temporal Grouping Processes for aSequence of Images,º Computer Vision and Image Understanding,vol. 67, no. 2, pp. 186-201, 1997.

[6] L. Dreschler and H.-H. Nagel, ªVolumetric Model and 3-DTrajectory of a Moving Car Derived from Monocular TV FrameSequences of a Street Scene,º Computer Graphics and ImageProcessing, vol. 20, pp. 199-228, 1982.

[7] O. Faugeras, Three Dimensional Computer Vision. MIT Press, 1993.[8] W. Forstner, ªA Framework for Low Level Feature Extraction,º

Proc. Computer Vision±ECCV '94, J.-O. Eklundh, ed., vol, I, pp. 383-394, 1994.

[9] W. Freeman and E. Adelson, ªJunction Detection and Classifica-tion,º Proc. Assoc. for Research in Vision and Opthalmology, 1991.

[10] W. Grimson, ªComputational Experiments with a Feature BasedStereo Algorithm,º IEEE Trans. Pattern Analysis and MachineIntelligence, vol. 7, no. 1, pp. 17-34, Jan. 1985.

[11] F. Harary, Graph Theory. Addison-Wesley, 1969.[12] R. Hartley, ªLines and Points in Three ViewsÐA Unified

Approach,º Proc. ARPA Image Understanding Workshop, 1994.[13] R. Horaud and T. Skordas, ªStereo Correspondence through

Feature Grouping and Maximal Cliques,º IEEE Trans. PatternAnalysis and Machine Intelligence, vol. 11, no. 11, pp. 1,168-1,180,Nov. 1989.

[14] X. Hu and N. Ahuja, ªMatching Point Features with OrderedGeometric, Rigidity, and Disparity Constraint,º IEEE Trans.Pattern Analysis and Machine Intelligence, vol. 16, no. 10, pp. 1,041-1,049, Oct. 1994.

[15] D. Marr and E. Hildreth, ªTheory of Edge Detection,º Proc. RoyalSoc., London, vol. B207, pp. 187-217, 1980.

[16] J. Mayhew and J. Frisby, ªPsychophysical and ComputationalStudies towards a Theory of Human Stereopsis,º ArtificialIntelligence, vol. 17, pp. 349-385, 1981.

[17] G. Medioni and R. Nevatia, ªSegment-Based Stereo Matching,ºComputer Vision, Graphics, and Image Processing, vol. 31, pp. 2-18,1985.

[18] H.H. Nagel, ªDisplacement Vectors Derived from Second OrderIntensity Variations in Image Sequences,º Computer Vision,Graphics, and Image Processing, vol. 21, pp. 85-117, 1983.

[19] J.A. Noble, ªFinding Corners,º Image and Vision Computing, vol. 6,no. 2, pp. 121-128, 1988.

[20] P.L. Palmer, J. Kittler, and M. Petrou, ªAn Optimising Line FinderUsing a Hough Transform Algorithm,º Computer Vision, Graphics,and Image Processing: Image Understanding, vol. 67, no. 1, pp. 1-23,1997.

[21] P.L. Palmer, M. Petrou, and J. Kittler, ªA Hough TransformAlgorithm with a 2D Hypothesis Testing Kernel,º Computer Vision,Graphics, and Image Processing: Image Understanding, vol. 58, no. 2,pp. 221-234, 1993.

[22] L. Parida, D. Geiger, and R. Hummel, ªJunctions: Detection,Classification, and Reconstruction,º IEEE Trans. Pattern Analysisand Machine Intelligence, vol. 20, no. 7, pp. 687-698, July 1998.

[23] M. Petrou and J. Kittler, ªOptimal Edge Detectors for RampEdges,º IEEE Trans. Pattern Analysis and Machine Intelligence,vol. 13, pp. 483-491, 1991.

[24] F. Pla and J. Marchant, ªMatching Feature Points in ImageSequences through a Region-Based Method,º Computer Vision andImage Understanding, vol. 66, no. 3, pp. 271-285, 1997.

[25] I.D. Reid and D.W. Murray, ªActive Tracking of Foveated ClustersUsing Affine Structure,º Int'l J. Computer Vision, vol. 18, pp. 41-60,1996.

[26] C. Schmid and A. Zisserman, ªAutomatic Line Matching acrossViews,º Proc. Computer Vision and Pattern Recognition, pp. 666-671,June 1997.

[27] V. Venkateswar and R. Chellappa, ªHierarchical Stereo andMotion Correspondence Using Feature Groupings,º Int'l J.Computer Vision, vol. 15, pp. 245-269, 1995.

[28] G. Wei, W. Brauer, and G. Hirzinger, ªIntensity- and Gradient-Based Stereo Matching Using Hierarchical Gaussian Basis Func-tions,º IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20,no. 11, pp. 1,143-1,159, Nov. 1998.

[29] J. Weng, T.S. Huang, and N. Ahuja, Motion and Structure fromImage Sequences. Berlin: Springer-Verlag, 1993.

[30] Z. Zhang, ªEstimating Motion and Structure from Correspon-dences of Line Segments Between Two Perspective Images,º Proc.Fifth Int'l Conf. Computer Vision, pp. 257-262, 1995.

[31] Z. Zhang, ªDetermining the Epipolar Geometry and Its Un-certainty: A Review,º Int'l J. Computer Vision, vol. 27, no. 2, pp. 161-195, 1998.

Xinquan Shen received the BSc degree incomputer science and the MSc degree incomputer vision from Changsha Railway Uni-versity, China in 1982 and 1988, respectively.From 1991 to 1995, he was involved in researchin computer vision at the School of ComputerStudies, The University of Leeds, United King-dom, and received the PhD degree from theUniversity of Leeds in 1995. From 1995 to 1998,he was a research fellow at the University of

Birmingham, United Kingdom. From 1998 to 2000, he was a researchfellow at the University of Surrey, United Kingdom. He is a guestprofessor at Changsha Railway University, China, and is currently withAltera European Technology Centre, United Kingdom. His researchinterests are computer vision and image processing.

Phil Palmer received his BSc degree fromLeicester University in physics with astrophysicsin 1976 and a PhD degree from the University ofCambridge in astronomy in 1981. He is currentlythe deputy director of the Surrey Space Centre,head of the Astrodynamics Team at the SpaceCentre, and head of the mobile robotics group atthe Centre for Vision, Speech, and SignalProcessing at Surrey University. His currentinterests in the field of computer vision center

on 3D environment modeling using an autonomous robot, whichincludes aspects of active vision, feedback control, and 3Dmodeling. Dr. Palmer has published widely in the areas ofastronomy, computer vision, pattern recognition, remote sensing,and astronautics. He is a member of the British Machine VisionAssociation and a member of the IEE.


Uncertainty propagation and the matching of junctions as feature groupings

Documents

Transcript of Uncertainty propagation and the matching of junctions as feature groupings