Direct Least Square Fitting of Paracatadioptric Line Images

JoaoP. Barreto HelderAraujo

Instituteof SystemsandRobotics Instituteof SystemsandRoboticsDept.of ElectricalandComputerEngineering Dept.of ElectricalandComputerEngineering

Universityof Coimbra Universityof Coimbra3030Coimbra- PORTUGAL 3030Coimbra- PORTUGAL

jpbar@isr.uc.pt helder@isr.uc.pt

Abstract

Paracatadioptricsensors combinea parabolic shapedmir-ror and a camera inducing an orthographic projection.Such a configuration providesa wide field of view whilekeepinga singleeffectiveviewpoint. In general the para-catadioptricimageof a line is a coniccurve. Theestimationof line imagesis an importantsubjectfor applicationssuchas reconstructionand visual control of motion. Howeverthe estimationof the coniccurveswhere lines are mappedis hard to accomplish. In general only a small arc of theconic is visible in the image andconventionalconicfittingtechniquesare unableto correctlyestimatethecurve. Thispapershowsthat line imagescan be accurately estimatedbyconstrainingthesearch space. A coniccurveis thepara-catadioptricimage of a line if, andonly if, theimage of thecircular pointslie on the curveand two certain pointsareconjugatewith respectto theconic. Consideringthespaceof all coniccurves,theline imageslie in a linear subspacewhich dependson the systemcalibration. The paracata-dioptric projectionof a line canestimatedbyfitting a conicin thesubspaceto thedatapoints. Theproposedapproachis computationallyefficientsincethefitting problemcanbesolvedby aneigensystem

1 Introduction

Theapproachof combiningmirrorswith conventionalcam-erasto enhancesensorfield of view is referredto ascata-dioptric imageformation. The useof catadioptricsystemsto achievepanoramicvision is simpleandfastenablingthecaptureof dynamicscenes.The entire classof catadiop-tric configurationsverifying the fixed viewpoint constraintisderivedin [1]. Panoramiccentralcatadioptricsystemscanbebuilt by combininganhyperbolicmirror with a perspec-tive cameraand, a parabolicmirror with an orthographiccamera(paracatadioptricsensor).The constructionof theformerrequiresa carefulalignmentbetweenthemirror and

theimagingdevice. Thecameraprojectioncentermustpo-sitionedin the outerfocusof the hyperbolicreflective sur-face.Theparacatadioptriccamerais easierto constructbe-ing broadlyusedin applicationsrequiringomnidirectionalvision.

The paracatadioptricimageformation can be modeledby astereographicprojectionfrom asphere,centeredin theeffective viewpoint, into a plane[2]. Theplaneandthe fi-nal catadioptricimagearerelatedby acollineationdepend-ing on the mirror parametersandcameraintrinsic matrix.For theparticularcaseof a parabolicsystemthis projectivetransformationis always affine. The systemis calibratedwhenever thecollineationis known. Thecalibrationcanbeperformedusingtheimageof threeor morelinesin generalpositionin theworld asexplainedin [3]. Theparacatadiop-tric sensormapsa line in theworld into aconiccurvein theimageplane[2]. In generalonly a small arc of the coniclocusis visible in the imageandconventionalconicfittingtechniquesareunableto correctlyestimatethecurve [6].

Thepresentpaperproposesaconicfitting algorithmthatcopeswith theocclusionproblemandaccuratelyestimatesparacatadioptricline images. The approachis specificforline projectionin a parabolicsystemandassumesthat thesensoris calibrated. If is is true that any line mapsinto aconic,it is not truethatany conicis theimageof a line. Weprove thata coniccurve is the paracatadioptricimageof aline if, andonly if, the imageof the circular points lie onthecurve andtwo certainpointsareconjugatewith respectto theconic. Theparacatadioptriccameramapslinesin thesceneinto coniccurveswhichbelongto alinearsubspaceinthespaceof all conics.By constrainingthesearchspacetheline imagecanbeaccuratelydetermined.Thecorrespond-ing locusis estimatedby fitting a conic in the subspacetothe datapoints. The approachis computationallyefficientsincethefitting problemcanbesolvedby aneigensystem.Theproposedalgorithmcanbeusefulfor many applicationssuchas3D reconstructionandvisualcontrolof motionus-ing paracatadioptricimages.

1

Zc

OcYc

cX

n

Ο^

H c

X

Z

Y

ΩO

O

Π

x

x

Ω

x

worldpoint

Ξ

Paracatadioptric Image Plane

Ω^

P

Figure1: Model for paracatadioptricimageformation

2 The Paracatadioptric Camera

This sectionreviews the imageformationmodel for para-catadioptricsystems[2] and introducesthe necessaryandsufficient conditionsfor a conic curve being the paracat-dioptric imageof a line.

2.1 The Mapping Model

Assumea paracatadioptricsystemcombininga parabolicmirror with latus rectum

, andan orthographiccamera.

The principal axis of the cameramustbe alignedwith thesymmetryaxisof theparaboloid.Theparacatadioptricpro-jectioncanbemodeledby a stereographicprojectionfromanunitary sphere,centeredin the effective viewpoint, intoaplane asshown in Fig. 1.

Theworld point shown in Fig. 1 is imagedat point intheparacatadioptricimageplane.Themappingcanbede-scribedasfollows. To eachvisiblescenepoint correspondsan orientedprojective ray , joining the 3Dpointwith theprojectioncenter . Theprojectiveray inter-sectsthe unit spherein a singlepoint . Considera point"! , with coordinates$#%&#%')(* , which lies in the unitarysphere.To each correspondsan orientedprojective ray+ joining "! with theintersectionpoint . Thenon-linearmapping, (equation1) correspondsto projectingthescenein theunity spheresurfaceandthenre-projectingthepointson the sphereinto a plane from a novel projectioncen-ter ! . Pointsin catadioptricimageplane areobtainedaftera collineation - ! of 2D projectivepoints

+ . Equation2 shows that -.! dependson the intrinsic parameters/.!of theorthographiccamera,andon the latusrectumof the

parabolicmirror.

, 0124365 87139:7139;7< (1)

= /.!>?A@ # ## @ ## # (

BCD EGF HIKJ

+ (2)

ConsidertheplaneL M$N1#; goingthroughtheeffec-tiveviewpoint asdepictedin Fig. 1 ( NOM$PRQ%PRSPRTU ).The paracatadioptricimageof any line lying on L is theconic curve V . The line in the sceneis projectedinto agreatcircle in the spheresurface. This greatcircle is thecurve of intersectionof plane L , containingboth the lineandtheprojectioncenter , andtheunit sphere.Thepro-jective rays

+ , joining ! to pointsin thegreatcircle, formacentralconesurface.Thecentralcone,with vertex in ! ,projectsinto theconic

+Vin plane (equation3). Sincethe

imageplaneand arerelatedby collineation- ! , theresultof equation4 comesin a straightforward manner. Noticethatconic

+Vis degeneratedwhenever PRT)6# . If theline is

co-planarwith theprincipalaxisof theorthographiccamerathenthecorrespondingparacatadioptricimageis alsoaline.

+V >? 'WP 7T # PRQXPRT# 'WP 7T PRSUPRTPRQYPRTZPRSUPRT P 7TBC

(3)

V >?\[ ]_^]a` b^ b_cBC - !ed V - !XdRf (4)

2.2 The Image of a Line

Considerthe schemeof Fig. 1 for the mappingof a lineby a paracatadioptricsensor. PlaneL , containingboth theline andtheeffective viewpoint , intersectsthesphereina greatcircle. The mappingfrom the sphereto plane is a stereographicprojection.Thestereographicprojectionmapsany circle in thesphereinto a circle in theplane[7].Thusthe greatcircle is projectedinto

+Vwhich is circle as

canbeverifiedby inspectingequation3. Pointsin plane aremappedinto pointsin the imageby a collineation -.! .Noticethatfor theparabolicsituation-.! is alwaysanaffinetransformation(equation2). Sinceanaffine transformationdoesnotchangethetypeof conic,thentheparacatadioptricimageof a line V is alwaysacircle/ellipse(equation4).

Considerthe following points lying on plane :+ g h(eij#; , +k lh(e<'Wij#; , +m lh(e#n<'Wih and+- h(e#%ih . Points

+ gand

+karethecircularpointsof theplane.

It is well known thatany circle mustgo throughthecircu-lar points. Since

+Vis always a circle, then it is true that+ g +V + g o# and

+k +V +k o# . Moreover from equation3 arises

2

that+m +V +- p# . Points

+mand

+- areconjugatewith re-spectto the conic curve

+V. Assumethat collineation -.!

mapspoints+ g,+k,+m

and+- into points g , k , m and - .

Proposition: A conic curve V is the paracatadioptricim-ageof a line in thesceneif, andonly if, it containspoints gand k ( g V g q# , k V k q# ), andpoints m , - are conju-gatewith respectto V ( m V - 6# ).

Proof: Thecircle+V

mustgo throughthecircularpoints+ g,+k. Sincecollineation -.! preservesincidence,thenboth

points g , k lie on the parabolicline image V . Moreovertheprojective transformationalsopreservesthe cross-ratioandpole/polarrelations.Sincepoints

+m,+- areconjugate

with respectto+V

, then m , - arealsoconjugatewith re-spectto V . Thus,if V is aparacatadioptricline image,thenit must verify g V g r# , k V k s# and m V - s# .Thederivedconditionsarenecessary, neverthelessit is notclear that they are sufficient. By sufficient we meanthatif a coniccurve in theparacatadioptricimageplaneverifiestheset constraints,thenit is thelocuswhereacertainline inthesceneis projected.Noticethat,neglectingthescalefac-tor, theconiccurve

+Vprovidedby equation3 is a function

of@

independentparameters.These@

degreesof freedom(DOF) areassociatedwith the poseof plane L containingtheoriginal line andtheeffectiveviewpoint (Fig. 1). Sincein generala coniccurve has u DOF, thenwe mustbe ableto find t , andnomorethan t , independentconstraints.Thisprovesthesufficiency of thestatement.

3 Fitting a Conic Curve to ImagePoints

Assumetheparacatadioptricimageof a line in space.Theline is projectedin a coniccurve V (equation4). Theconiccurve V canbeparameterizedby a point v in w)x [7]

v92 [ ] ` ^ b c (5)

Considerthe set of imagepoints yz*|:e|U with~ (e @< . If thepointslie onconic v thentheequalityof equation6 mustbe verified. However equation6 holdsonly in ideal circumstances.In generalthe datapoints yarecorruptedwith noiseand vo# .>? | 7 f

@ | f | f | 7f @ | f @ | f (| 77 @ | 7 | 7 | 77 @ | 7 @ | 7 (...

......

......

...| 7 @ | | | 7 @ | @ | (

BCD EF H

v6# (6)

A conicfitting algorithmdeterminesthe curve thatbestfits thedataaccordinglyto a certaindistancemetric. There

areseveralapproachesto estimatea coniccurve usingim-agepoints[6]. They differ betweeneachotherby thecrite-ria, or metric,that is minimizedin thefitting process.Thissectionreviewssomewell known conicfitting methodsandcomparestheir performance.

3.1 Conic Fitting Methods

Wefocusexclusivelyondirectmethodsfor whichthefittingproblemcanbesolvednaturallyby aneigensystem

3.1.1 Linear Least Squares (LMS)

ConsidertheA

designmatrix providedin equation6. TheLMS methodestimatestheconiccurve v thatmin-imizesthealgebraicdistance v v underthecon-straint v vz2( . Theobjectivefunctionis providedin equa-tion 7 wherethe constraintis introducedusinga Lagrangemultiplier

8W v v v3 v v'(* (7)

Theconiccurve v thatminimizes W

is determinedbysolving the eigensystem v Kv . The minimizer istheeigenvectorcorrespondingto thesmallesteigenvalueofmatrix .

3.1.2 Approximate Mean Squares (AMS)

The approximatemean square metric has been intro-duced by Taubin in [5]. The proposedconic fittingmethodminimizes the algebraicdistanceunder the con-straint vG 3 vz( where and arethe partial derivativesof . The correspondingobjectivefunctionis

¡ W v v v¢3 v 3 vA'O(*(8)

The minima of equation8 can be determinedanalyt-ically by solving the generalizedeigensystem v 3 v . The conic curve estimationisprovidedby the eigenvectorcorrespondingto the smallesteigenvalue.

3.1.3 Direct Least Square Fitting of Ellipses (FF)

Sincetheconic+V

is a circle andthe transformation- ! isaffine (equations2 and3), thenthe paracatadioptricimageof a line V mustbea circle/ellipse.Theestimationmethodproposedin [4] is ellipsespecific. Considerthe following"\

matrix

3

0 2 4 6 8 10 12 14 160

1

2

3

4

5

6

7

Noise Standard Deviation (pixel)

Mea

n E

rror

(pi

xel)

MEAN ERROR FOR THE PRINCIPAL POINTS

LMSAMSFF

0 50 100 150 200 250 300

100

101

102

Occluded Arc (degree)

Mea

n E

rror

(pi

xel)

MEAN ERROR FOR THE PRINCIPAL POINTS

LMSAMSFF

Figure2: Comparingthe performanceof the conic fittingmethods.Top: Robustnessto noisewithoutocclusion.Bot-tom: Robustnessto occlusionin thepresenceof noisewithstandarddeviation £ @ (pixel)

¤ >?¥ ¥ ¦§¥§¥¨¥¥ª©W«¬¥¨¥§¥¨¥¦ ¥ ¥¨¥§¥¨¥¥ ¥ ¥¨¥§¥¨¥¥ ¥ ¥¨¥§¥¨¥¥ ¥ ¥¨¥§¥¨¥

B C

If a conic v verifies v ¤ v­®( then it mustbe a cir-cle/ellipse. The conic fitting methodproposedin [4] esti-matesthecurveby minimizing thealgebraicdistanceto thedatapointsunderthe constraint v ¤ v¯( . The resultantobjectivefunctionis providedbelow wheretheconstraintisintroducedusinga Lagrangemultiplier°U° v v v=3 v ¤ v±'(* (9)

It canbe provedthat the conic v which minimizes °U°

andverifiestheconstraintis theeigenvectorcorrespondingto the singlepositive eigenvalueof the generalizedeigen-system vz ¤ v [4].

3.2 Performance Evaluation

An arc of a test conic with pre-definedamplitudeis uni-formly sampledby (#Y# points. Two dimensionalgaussiannoisewith zeromeanandstandarddeviation £ is addedtoeachsample.Thesesamplesarethedatapointsusedby thedifferentestimators.The principal pointsof the estimatedcurvearecomparedwith thegroundtruth andthemeaner-ror is computedover (*#Y# runsof eachexperiment.

Considerthe resultsdepictedin Fig. 2. The graphicatthetop shows therobustnessto noiseof thedifferentmeth-ods. In this situationthereis no occlusionand the entireconicis sampled.All thealgorithmsexhibit asimilargrace-ful degradationin the presenceof increasingnoise. TheFF methodis the one that seemsto be more robust. Thegraphicat thebottomshowsthebehavior in thepresenceofocclusion.Thenoisestandarddeviation is

@pixelsandthe

amplitudeof thesampledarcdecreasesfrom t #;² (theen-tire conic)up to

#e² (occlusionof t #Y#;² ). In all methodstheestimationresultsuffersanabruptdegradationwhentheoc-clusionis higherthan

@Y@ #e² . NeverthelesstheAMS methodseemsto betheonethatbetterdealswith occlusion.

4 Direct Least Square Fitting ofParacatadioptric Line Images(CATPARB)

In generala conic has u degreesof freedomand can beparametrizedby a point in w x . Neglecting the scalefac-tor the spaceof all conicshasfive dimensions. The ap-proachesdescribedin theprevioussectionsearchtheentirespacefor the conic thatbestfits the data. However not allconicscanbetheparacatadioptricimageof aline. Considerpoints g 2i Q i S i T h , k M ~ Q ~ S ~ T , m ³ Q h³ S ³ T hand - ´¶µnQn&µ8S;&µ8T*h introducedin section2.2. As dis-cusseda coniccurve v is theparacatadioptricprojectionofa line if, andonly if, ·v is null with · the t O matrixprovidedin equation10.

>? i 7Q @ i Q i S i 7S~ 7Q @ ~ Q ~ S ~ 7S³YQ;µnQ_³YQX³YS3¸µnQeµnS¨³YSXµnS@ i¹QXi¹T @ iS*iT i 7T@ ~ Q ~ T @ ~ S ~ T ~ 7T³YQº³YT13µnQYµ8T»³YS*³YT13¸µ8SXµnT»³YTUµ8T

BCD EF H¼

v9o# (10)

The paracatadioptricline image v must lie in the nullspaceof matrix · . Thenull spaceof · is a linearsubspacein thespaceof all coniccurves. Our approachfits thedataby the conic curve, lying in this subspace,that minimizes

4

thealgebraicdistanceto theimagepoints.Considerthesin-gularvaluedecompositionof matrix · .

· 6½A¾À¿ Matrices½ , ¾ and ¿ haverespectively dimensiont t ,t and

. ¿ is full rankandorthonormal( ¿ dÀf o¿\ ).

The threelast columnsof ¿ are an orthonormalbasisofthe null spaceof · . Considerthe changeon the baseofrepresentationvÁ6¿ v . If v belongsto the null spaceof matrix · , thenthe correspondingvÁ hasthe followingstructure

v Á M$#n#%&#% ^eà bUà cºÃD EF HÄ (11)

The algebraicdistancebetweenconic v and the datapoints is

^ v v . Rewriting the algebraicdistancein termsof thenew coordinatesarises vÁ ¿ ¿\ vÁ .Takinginto accountthestructureof vÁ (equation11)comesthat

^ oÅeÆÇÆÈÅ with Æ thebottomright t t submatrixof¿ ¿\ . We aim to determinethesolution Å which mini-mizesthealgebraicdistance undertheconstraintÅ ÅÉ2( .Theobjective functionis

ÅÊ oÅ Æ ÆÈÅË3 $Å ÅÈ'(* (12)

Theminimaof theobjectivefunction

is theeigenvectorof matrix ÆÇÆ correspondingto the smallesteigenvalue.Thefinal conic v is computedby replacingÅ in equation11andmaking v9Ì¿¢ vÍÁ .5 Experiments

Accordingly to the resultspresentedin section3.2, the FFalgorithmhasthebestperformancein thepresenceof noiseandthe AMS methodis the mostrobust to occlusion.Thepresentsectioncomparesthesetwo methodswith the pro-posedCATPARB algorithm.Noticehoweverthat,while theFF andAMS conic fitting methodsare generic,the CAT-PARB algorithm is specific to estimatethe conic curveswherelines in the scenearemappedby a paracatadioptriccamera.We startby describingthesimulationschemeusedto generateartificial data. The performanceof the AMS,FF andCATPARB methodsin estimatingparacatadioptricline imagesis compared. Experimentsusing real imagesarealsopresented

5.1 Simulation Scheme

Assumea paracatadioptricsystemwith a field of view(FOV) of (*ÎY#;² . Theparacatadioptricimageof a line is gen-eratedasfollows. As depictedin Fig. 1 to aline in thescene

ξ = 1α = 1.1 ; s

k = 30 ; f = 240

(cx,c

y) = (330,238)

N = 20

Θ = 25°

σ = 5 (pixel)

AMSFFCATPARB

Figure3: Estimatingparacatadioptricline imageusingdif-ferentmethods

correspondsa plane L with normal N . The normalis uni-tary andrandomlychosenfrom an uniform distribution inthesphere.Eachnormaldefinesa planethat intersectstheunit spherein agreatcircle. Noticethathalf of thegreatcir-cle is within thecamerafield of view (theFOV is (Îe#e² ). AnangleÏ , lessor equalto theFOV, is chosento betheampli-tudeof thearcthatis actuallyvisible in theparacatadioptricimage.Thearcis randomlyanduniformly positionedalongthe part of the greatcircle which is within the FOV. Thevisible arc is uniformly sampledby a fixed number Ð ofsamplepoints. The eachsamplepoint correspondsa pro-jective ray . Thesampleraysareprojectedusingequation1 andtransformedusing2 with thechosenintrinsic param-eters.Two dimensionalgaussiannoisewith zeromeanandstandarddeviation £ is addedto eachimagepoint . Fig.3 is an exampleof a simulatedimageof a randomlygen-eratedparacatadioptricline image. The visible arc hasanamplitudeof

@ u ² andis sampledby@ # points. The gaus-

siannoiseaddedto thesampleshasa standarddeviation ofu pixels. As a final remarknoticethat theamplitudeof thevisible arc is measuredin the greatcircle whereplane Lintersectsthe sphere,andnot in the coniccurve wheretheline is projected. In generalthe visible angleof the para-catadioptricline imageis muchlessthan Ï .5.2 Comparing AMS, FF and CATPARB

Methods

Considerthe first graphicon Fig. 4 which comparestheperformanceof the threemethods.The datapointsarear-tificially generatedusingthe simulationschemedescribedabove. An arcwith anamplitudeÎe#e² is uniformly sampledby # points. Eachmethodfits a conic curve to the data

points. The estimatedconic is comparedwith the groundtruthandtheRMSerrorin theprincipalpointsis computed

5

0 2 4 6 8 10 12 14 1610

−1

100

101

102

Noise Standard Deviation (pixel)

RM

S E

rror

(pi

xel)

RMS ERROR FOR THE PRINCIPAL POINTS

AMSFFCATPARB

0 2 4 6 8 10 12 14 160

10

20

30

40

50

60

Noise Standard Deviation (pixel)

Prin

cipa

l Poi

nts

RM

S E

rror

(pi

xel)

N = 45 ; Θ = 90°

N = 25 ; Θ = 45°

N = 10 ; Θ = 45°

N = 20 ; Θ = 25°

Figure4: Estimationof paracatdioptricline images. Top:The performanceof AMS, FF and CATPARB methods( Ð #n Ï ÎY#e² ). Bottom: Theperformanceof theCAT-PARB algorithm.

over (*#Y# runsof eachexperiment. The estimationresultsdramaticallyimprovewhenusingtheCATPARB algorithm.Thegraphicat thebottomof Fig. 4 shows thebehavior ofthe proposedapproachin the presenceof increasingnoisefor different valuesof Ð and Ï . As expectedthe perfor-manceis worsewhenthenumberof samplesand/ortheam-plitudeof thevisiblearcdecrease.

5.3 Experiments with Real Images

Fig. 5 shows a calibratedparacatadioptricimagewith res-olution (ºÑº# @Y@ Ñ @ . Theconiccurve wherea line is pro-jectedhasonly

@independentdegreesof freedom(proposi-

tion in section2.2). Thustwo imagepointsareenoughtocorrectlydeterminea paracatadioptricline image.We haveselectedby handtwo pointslying ontheconiclocuswhereacertainline in thesceneis projected.Theestimationresultsusingthe proposedCATPARB algorithmcanbe observedin Fig. 5. Notice that in generala coniccurve canonly beestimatedusing u or moredatapoints.

Figure5: Estimatinglines in a real image.Thecrossesarethepointsselectedby hand

6 Summary and Conclusions

The article presentsa specificmethodto estimatelines incalibratedparacatadioptricimages.Thealgorithmusesthepropertiesof parabolic image formation to constrainthesearchspace. The performaceof the proposedapproachis comparedwith othergenericconicfitting methodsusingbothsimulateddataandreal images.

References

[1] S.BakerandS.Nayar, “A theoryof catadioptricimageforma-tion,” in Proc.of IEEEInternationalConferenceonComputerVision, Bombay, 1998,pp.35 – 42.

[2] JoaoP. BarretoandHelderAraujo, “Geometricpropertiesofcentralcatadioptricline images,” in Proc.of EuropeanCon-ferenceon ComputerVision, Copenhagen,Denmark,May2002.

[3] C. GeyerandK. Daniilidis, “Paracatadioptriccameracalibra-tion,” IEEE Trans.on PatternAnalysisandMachine Intelli-gence, vol. 24,no.4, April 2002.

[4] A. Fitzgibbon,M. Pilu andR. Fisher, “Direct LeastSquaresFitting of Ellipses,” IEEETrans.onPatternAnalysisandMa-chineIntelligence, vol. 21,no.5, May 1999.

[5] G. Taubin, “Estimationof PlanarCurves,SurfacesandNon-planarSpaceCurvesDefinedby Implicit Equations,” IEEETrans.onPatternAnalysisandMachineIntelligence, vol. 13,no.11,November1991.

[6] Z. Zhang, “Parameterestimationtechniques:A tutorial withapplicationto conic fitting,” INRIA Raportde Recherche n2676, October1995.

[7] J. G. SempleandG. T. Kneebone,Algebraic ProjectiveGe-ometry, ClaredonPress,1998.

6