Critical Motions for Auto-CalibrationWhen Some Intrinsic Parameters Can Vary �Fredrik Kahl (fredrik@maths.lth.se), Bill Triggs(bill.triggs@inrialpes.fr) and Kalle Åström(kalle@maths.lth.se)Centre for Mathematical Sciences,Mathematics (LTH),P.O. Box 118,SE-221 00 Lund, SWEDENINRIA Rhône-Alpes,655 avenue de l'Europe,Montbonnot, 38330,FRANCEAbstract.Auto-calibration is the recovery of the full camera geometry and Euclidean scenestructure from several images of an unknown 3D scene, using rigidity constraintsand partial knowledge of the camera intrinsic parameters. It fails for certain specialclasses of camera motion. This paper derives necessary and su�cient conditions forunique auto-calibration, for several practically important cases where some of theintrinsic parameters are known (e.g. skew, aspect ratio) and others can vary (e.g.focal length). We introduce a novel subgroup condition on the camera calibrationmatrix, which helps to systematize this sort of auto-calibration problem. We showthat for subgroup constraints, criticality is independent of the exact values of theintrinsic parameters and depends only on the camera motion. We study such criticalmotions for arbitrary numbers of images under the following constraints: vanishingskew, known aspect ratio and full internal calibration modulo unknown focal lengths.We give explicit, geometric descriptions for most of the singular cases. For example,in the case of unknown focal lengths, the only critical motions are: (i) arbitraryrotations about the optical axis and translations, (ii) arbitrary rotations about at mosttwo centres, (iii) forward-looking motions along an ellipse and/or a correspondinghyperbola in an orthogonal plane. Some practically important special cases are alsoanalyzed in more detail.� Supported by the ESPRIT Reactive LTR project 21914, CUMULIc 2000 Kluwer Academic Publishers. Printed in the Netherlands.

finalcms.tex; 17/05/2000; 11:16; p.1

1. IntroductionOne of the core problems in computer vision is the recovery of 3D scenestructure and camera motion from a set of images. However, for certaincon�gurations there are inherent ambiguities. This kind of problemwas already studied in optics in the early 19th century, for example,by Vieth in 1818 and Muller in 1826. Pioneering work on the subjectwas also done by Helmholtz. See [13] for references. One well-studiedambiguity is when the visible features lie on a special surface, calleda critical surface, and the cameras have a certain position relativeto the surface. Critical surfaces or �gefährlicher Ort� were studied byKrames [21] based on a monograph from 1880 on quadrics [32]. See alsothe book by Maybank [24] for a more recent treatment. Another well-known ambiguity is that when using projective image measurements, itis only possible to recover the scene up to an unknown projective trans-formation [8, 10, 35]. Additional scene, motion or calibration constraintsare required for a (scaled) Euclidean reconstruction. Auto-calibrationuses qualitative constraints on the camera calibration, e.g. vanishingskew or unit aspect ratio, to reduce the projective ambiguity to a sim-ilarity. Unfortunately, there are situations when the auto-calibrationconstraints may lead to several possible Euclidean reconstructions. Inthis paper, such degeneracies are studied under various auto-calibrationconstraints.In general it is possible to recover Euclidean scene information fromm � 3 images by assuming constant but unknown intrinsic parametersof a moving projective camera [26, 7]. Several practical algorithms havebeen developed [39, 2, 30]. Some of the intrinsic parameters may evenvary, e.g. the focal length [31], or the focal length and the principalpoint [14]. In [29, 15] it was shown that vanishing skew su�ces for aEuclidean reconstruction. Finally in, [16] it was shown that given atleast 8 images it is su�cient if just one of the intrinsic parameters isknown to be constant (but otherwise unknown).However, for certain camera motions, these auto-calibration con-straints are not su�cient [42, 1, 40]. A complete categorization of thesecritical motions in the case of constant intrinsic parameters was givenby Sturm [36, 37]. The uniformity of the constant-intrinsic constraintsmakes this case relatively simple to analyze. But it is also somewhatunrealistic: It is often reasonable to assume that the skew actually van-c 2000 Kluwer Academic Publishers. Printed in the Netherlands.finalcms.tex; 17/05/2000; 11:16; p.2

ishes whereas focal length often varies between images. While the case ofconstant parameters is practically solved, much less is known for otherauto-calibration constraints. In [43], additional scene and calibrationconstraints are used to resolve ambiguous reconstructions, caused by a�xed axis rotation. The case of two cameras with unknown focal lengthsis studied in [12, 28, 4, 20]. For the general unknown focal length case,Sturm [38] has independently derived results similar to those presentedhere and in [20, 19].In this paper, we generalise the work of Sturm [37] by relaxing theconstraint constancy on the intrinsic parameters. We show that for alarge class of auto-calibration constraints, the degeneracies are inde-pendent of the speci�c values of the intrinsic parameters. Therefore, itmakes sense to speak of critical motions rather than critical con�gura-tions. We then derive the critical motions for various auto-calibrationconstraints. The problem is formulated in terms of projective geometryand the absolute conic. We start with fully calibrated cameras, and thencontinue with cameras with unknown and possibly varying focal length,principal point, and �nally aspect ratio. Once the general description ofthe degenerate motions has been completed, some particular motionsfrequently occurring in practice are examined in more detail.This paper is organized as follows. In Section 2 some backgroundon projective geometry for vision is presented. Section 3 gives a formalproblem statement and reformulates the problem in terms of the abso-lute conic. In Section 4, our general approach to solving the problem ispresented, and Section 5 derives the actual critical motions under vari-ous auto-calibration constraints. Some particular motions are analyzedin Section 6. In order to give some practical insight of critical and near-critical motions, some experiments are presented in Section 7. Finally,Section 8 concludes. 2. BackgroundIn this section, we give a brief summary of the modern projective formu-lation of visual geometry. Also, some basic concepts in projective andalgebraic geometry are introduced. For further reading, see [6, 24, 33].A perspective (pinhole) camera is modeled in homogeneous co-ordinates by the projection equationx ' PX (1)where X = (X;Y;Z;W )T is a 3D world point, x = (x; y; z)T is its2D image, P is the 3 � 4 camera projection matrix and ' denotesequality up to scale. Homogeneous coordinates are used for both image

finalcms.tex; 17/05/2000; 11:16; p.3

and object coordinates. In a Euclidean frame, P can be factored, usinga QR-decomposition, cf. [9], asP ' K [R j �Rt] where K = 24f f s u00 f v00 0 1 35 : (2)Here the extrinsic parameters (R; t) denote a 3� 3 rotation matrixand a 3�1 translation vector, which encode the pose of the camera. Thecolumns of R = [r1r2r3] de�ne an orthogonal base. The standard base isde�ned by e1 = (1; 0; 0)T ; e2 = (0; 1; 0)T ; e3 = (0; 0; 1)T . The intrinsicparameters in the calibration matrix K encode the camera's internalgeometry: f denotes the focal length, the aspect ratio, s the skewand (u0; v0) the principal point.A camera for which K is unknown is said to be uncalibrated.It is well-known that for uncalibrated cameras, it is only possible torecover the 3D scene and the camera poses up to unknown projectivetransformation [8, 10, 35]. This follows directly from the projectionequation (1): Given one set of camera matrices and 3D points thatsatis�es (1), another reconstruction can be obtained from

PX = (PT ) (T�1X) = ~P ~X;where T is a non-singular 4 � 4 matrix corresponding to a projectivetransformation of P3.A quadric in Pn is de�ned by the quadratic formXTQX = 0;where Q denotes a (n+ 1) � (n+ 1) symmetric matrix and X denoteshomogeneous point coordinates. The dual is a quadric envelope, givenby �TQ�� = 0; (3)where � denotes homogeneous coordinates for hyper-planes of dimen-sion n�1 that are tangent to the quadric. For non-singular matrices, itcan be shown that Q ' (Q�)�1 (see [33] for a proof). A quadric with anon-singular matrix is said to be proper. Quadrics with no real pointsare called virtual. In the plane, n = 2, quadrics are called conics. Wewill use C for the 3 � 3 matrix that de�nes the conic points xT Cx = 0and C� for its dual that de�nes the envelope of tangent lines lTC�l = 0(where C� ' C�1). The image of a quadric in 3D-space is a conic, i.e.the silhouette of a 3D quadric is projected to a conic curve. This canbe expressed in envelope forms as

C� ' PQ�PT : (4)finalcms.tex; 17/05/2000; 11:16; p.4

Projective geometry encodes only cross ratios and incidences. Prop-erties like parallelism and angles are not invariant under di�erent pro-jective coordinate systems. An a�ne space, where properties like par-allelism and ratios of lengths are preserved, can be embedded in aprojective space by singling out a plane at in�nity �1. The points on�1 are called points at in�nity and be interpreted as direction vectors.In P3, Euclidean properties, like angles and lengths, are encoded by sin-gling out a proper, virtual conic on �1. This absolute conic 1 givesscalar products between direction vectors. Its dual, the dual absolutequadric �1, gives scalar products between plane normals. �1 is a 4�4symmetric rank 3 positive semide�nite matrix, where the coordinatesystem is normally chosen such that �1 = diag(1; 1; 1; 0). �1 is �1'sunique null vector: �1�1 = 0. The similarities or scaled Euclideantransformations in projective space are exactly those transformationsthat leave 1 invariant. The transformations that leave �1 invariantare the a�ne transformations. The di�erent forms of the absoluteconic will be abbreviated to (D)AC for (Dual) Absolute Conic.Given image conics in several images, there may or may not be a3D quadric having them as image projections. The constraints whichguarantee this in two images are called the Kruppa constraints [22].In the two-image case, these constraints have been successfully appliedin order to derive the critical sets, e.g. [28]. For the more general caseof multiple images, the projection equation given by (4) can be used foreach image separately.3. Problem FormulationThe problem of auto-calibration is to �nd the intrinsic camera pa-rameters (Ki)mi=1, where m denotes the number of camera positions.In general, auto-calibration algorithms proceed from a projective re-construction of the camera motion. In order to auto-calibrate, someconstraints have to be enforced on the intrinsic parameters, e.g. vanish-ing skew and/or unit aspect ratio. Thus, we require that the calibrationmatrices should belong to some proper subset G of the group K of 3�3upper triangular matrices. Once the projective reconstruction and theintrinsic parameters are known, Euclidean structure and motion areeasily computed.For a general set of scene points seen in two or more images, thereis a unique projective reconstruction. However, certain special con�g-urations, known as critical surfaces, give rise to additional ambiguoussolutions. For two cameras, the critical con�gurations occur only if bothcamera centres and all scene points lie on a ruled quadric surface [24].finalcms.tex; 17/05/2000; 11:16; p.5

Furthermore, when an alternative reconstruction exists, then there willalways exist a third distinct reconstruction. For more than two cameras,the situation is less clear. In [25], it is proven that when six scene pointsand any number of camera centres lie on a ruled quadric, then thereare three distinct reconstructions. If there are other critical surfaces isan open problem.We will avoid critical surfaces by assuming unambiguous recoveryof projective scene structure and camera motion. In other words, thecamera matrices and the 3D scene are considered to be known up to anunknown projective transformation. We formulate the auto-calibrationproblem as follows: If all that is known about the camera motionsand calibrations is that each calibration matrix Ki lies in some givenconstraint set G � K, when is a unique auto-calibration possible? Moreformally:Problem 3.1. Let G � K. Then, given the true camera projections(Pi)mi=1, where Pi = Ki[Ri j �Riti ] and Ki 2 G, is there any projec-tive transformation T (not a similarity) such that ~Pi ' PiT has de-composition ~Pi = ~Ki[ ~Ri j �~Ri~ti ] with calibration matrices ~Ki lying inG?Without constraints on the intrinsic parameters T can be chosenarbitrarily, so auto-calibration is impossible. Also, T is only de�nedmodulo a similarity, T ! T �R t0 1� ;as such transformations leave K in the decomposition P = K[R j �Rt ]invariant. Based on the above problem formulation, we can de�ne pre-cisely what is meant by a motion being critical.De�nition 3.1. Let G � K and let (Pi)mi=1 and (~Pi)mi=1 denote two pro-jectively related motions, with calibration matrices (Ki)mi=1 and (~Ki)mi=1,respectively. If the two motions are not related by a Euclidean trans-formation and Ki; ~Ki 2 G, they are said to be critical with respect to G.A motion is critical if there exists an alternative projective motionsatisfying the auto-calibration constraints. Without any additional as-sumptions, it is not possible to tell which motion is the true one. Onenatural additional constraint is that the reconstructed 3D structureshould lie in front of all cameras. In many (but by no means all) casesthis reduces the ambiguity, but it depends on which 3D points areobserved.

finalcms.tex; 17/05/2000; 11:16; p.6

According to (4) the image of the absolute conic 1 is,!�i ' Pi�1PTi ' Ki[Ri j �Riti ] �I 00 0� [Ri j �Riti ]TKTi = KiKTi : (5)Thus, knowing the calibration of a camera is equivalent to knowing itsimage of 1. Also, if there is a projectively related motion (~Pi)mi=1, thenthe false image of the true absolute conic is the true image of a �false�absolute conic:~!�i ' ~Pi�1~PTi = ~PiT�1T�1T TT�T ~Pi = Pi�fPTi ;where �f = T�1T T is some dual, virtual quadric of rank 3. Thisobservation allows us to eliminate the �false� motion (~Pi)mi=1 from theproblem and work only with the true Euclidean motion, but with a falseabsolute dual quadric �f .Problem 3.2. Let G � K. Then, given the true motion (Pi)mi=1, wherePi = Ki[Ri j �Riti ] and Ki 2 G, is there any other proper, virtual conic�f , di�erent from �1, such that Pi�fPTi ' ~Ki~KTi , where ~Ki 2 G?Given only a 3D projective reconstruction derived from uncalibratedimages, the true 1 is not distinguished in any way from any otherproper, virtual planar conic in projective space. In fact, given any suchpotential conic �f , it is easy to �nd a `rectifying' projective transforma-tion that converts it to the Euclidean DAC form �1 = diag(1; 1; 1; 0)and hence de�nes a false Euclidean structure. To recover the true struc-ture, we need constraints that single out the true 1 and �1 from allpossible false ones. Thus, ambiguity arises whenever the images of somenon-absolute conic satisfy the auto-calibration constraints. We call suchconics potential absolute conics or false absolute conics. They arein one-to-one correspondence with possible false Euclidean structuresfor the scene.A natural question is whether the problem is dependent on the actualvalues of the intrinsic parameters. We will show that this is not thecase whenever the set G is a proper subgroup of K. Fortunately, accord-ing to the following easy lemma, most of the relevant auto-calibrationconstraints are subgroup conditions.Lemma 3.1. The following constrained camera matrices form propersubgroups of the 3� 3 upper triangular matrices K:(i) Zero skew, i.e. s = 0.(ii) Unit aspect ratio, i.e. = 1.

finalcms.tex; 17/05/2000; 11:16; p.7

(iii) Vanishing principal point, i.e. (u0; v0) = (0; 0).(iv) Unit focal length, i.e. f = 1.(v) Combinations of the above conditions.Independence of the values of the intrinsic camera parameters isshown as follows:Lemma 3.2. Let Gi 2 G for i = 1; :::;m, where G is a proper subgroupof K. Then, the motion (Pi)mi=1 is critical w.r.t. G if and only if themotion (GiPi)mi=1 is critical w.r.t. G.Proof. If (Pi)mi=1 is critical with the alternative motion ( ~Pi)mi=1 andcalibrations Ki; ~Ki 2 G, then clearly (GiPi)mi=1 and (Gi ~Pi)mi=1 are alsocritical, because GiKi; Gi ~Ki 2 G by the closure of G under multipli-cation. The converse also holds with G�1i , by the closure of G underinversion.Camera matrices with prescribed parameters do not in general forma subgroup of K, but it su�ces for them to be of the more general formK0K where K0 is a known matrix and K belongs to a proper subgroup ofK. For example, the set of all camera matrices with known focal lengthf has the form 24f 0 00 f 00 0 135241 � �0 � �0 0 135 :The invariance with respect to calibration parameters simpli�es things,especially if one chooses Gi = K�1i for i = 1; :::;m. With this in mind,we restrict our attention to proper subgroups of K and formulate theproblem as follows.Problem 3.3. Let G � K be a proper subgroup. Then, given the truemotion (Pi)mi=1 for calibrated cameras, where Pi = [Ri j �Riti ], is thereany other false absolute conic �f , di�erent from �1, such that[Ri j �Riti ]�f [Ri j �Riti ]T ' ~Ki~KTi ;where ~Ki 2 G?

finalcms.tex; 17/05/2000; 11:16; p.8

4. ApproachWe want to explicitly characterize the critical motions (relative cameraplacements) for which particular auto-calibration constraints are insuf-�cient to uniquely determine Euclidean 3D structure. We assume thatprojective structure is available. Alternative Euclidean structures corre-spond one-to-one with possible locations for a potential absolute conicin P3. Initially, any proper virtual projective plane conic is potentiallyabsolute, so we look for such conics � whose images also satisfy thegiven auto-calibration constraints. Ambiguity arises if and only if morethan one such conic exists. We work with the true camera motion ina Euclidean frame where the true absolute conic 1 has its standardcoordinates.Several general invariance properties help to simplify the problem:Calibration invariance: As shown in the previous section, if the auto-calibration constraints are subgroup conditions, the speci�c parametervalues are irrelevant. Hence, for the purpose of deriving critical motions,we are free to assume that the cameras are in fact secretly calibrated,Ki = I, even though we do not assume that we know this. (All thatwe actually know is Ki 2 G, which does not allow some image conics!�i 6= I to be excluded outright).Rotation invariance: For known-calibrated cameras, Ki can be set toidentity, and thus the image !�i = I of any false AC must be identicalto the image of the true one. Since

Pi�fPTi ' I ) RPi�fPTi RT ' RRT = I;hold for any rotation R, the image !�i is invariant to camera rota-tions. Hence, criticality depends only on the camera centres, not ontheir orientations. More generally, any camera rotation that leaves theauto-calibration constraints intact is irrelevant. For example, arbitraryrotations about the optical axis and 180� �ips about any axis in theoptical plane are irrelevant if (a; s) is either (1; 0) or unconstrained,and (u0; v0) is either (0; 0) or unconstrained.Translation invariance: For true or false absolute conics on the planeat in�nity, translations are irrelevant so criticality depends only oncamera orientation.In essence, Euclidean structure recovery in projective space is a mat-ter of parameterizing all of the possible proper virtual plane conics, thenusing the auto-calibration constraints on their images to algebraicallyeliminate parameters until only the unique true absolute conic remains.More abstractly, if C parameterizes the possible conics and X the cam-era geometries, the constraints cut out some algebraic variety in (C;X)finalcms.tex; 17/05/2000; 11:16; p.9

space. A constraint set is useful for Euclidean structure from motionrecovery only if this variety generically intersects the subspacesX = X0in one (or at most a few) points (C;X0), as each such intersection rep-resents an alternative Euclidean structure for the reconstruction fromthat camera geometry. A set of camera poses X is critical for theconstraints if it has exceptionally (e.g. in�nitely) many intersections.Potential absolute conics can be represented in several ways. Thefollowing parameterizations have all proven relatively tractable:(i) Choose a Euclidean frame in which �f is diagonal, and expressall camera poses relative this frame [36, 37]. This is symmetrical withrespect to all the images and usually gives the simplest equations. How-ever, in order to �nd explicit inter-image critical motions, one mustrevert to camera-based coordinates which is sometimes delicate. Thecases of a �nite false absolute conic and a false conic on the plane atin�nity must also be treated separately, e.g. �f = diag(d1; d2; d3; d4)with either d3 or d4 zero.(ii) Work in the �rst camera frame, encoding �f by its �rst image !�1and supporting plane (nT ; 1). Subsequent images !�i ' Hi !�1 HTi aregiven by the inter-image homographies Hi = Ri + ti nT where (Ri; ti)is the ith camera pose. The output is in the �rst camera frame andremains well-de�ned even if the conic tends to in�nity, but the algebrarequired is signi�cantly heavier.(iii) Parameterize �f implicitly by two images !�1; !�2 subject to theKruppa constraints. In the two-image case this approach is both rel-atively simple and rigorous � two proper virtual dual image conicssatisfy the Kruppa constraints if and only if they de�ne a (pair of)corresponding 3D potential absolute conics � but it does not extendso easily to multiple images.The derivations below are mainly based on method (i) .5. Critical MotionsIn this section, the varieties of critical motions are derived. In mostsituations, the problem is solved in two separate cases. One is whenthere are potential absolute conics on the plane at in�nity, �1, and theother one is conics outside �1. If the potential conics are all on �1, it isstill possible to recover �1 and thereby obtain an a�ne reconstruction.Otherwise, the recovery of a�ne structure is ambiguous, and we say thatthe motion is critical with respect to a�ne reconstruction.The following constraints on the camera calibration are considered:(i) known intrinsic parameters,finalcms.tex; 17/05/2000; 11:16; p.10

(ii) unknown focal lengths, but the other intrinsic parameters known,(iii) known skew and aspect ratio.These constraints form a natural hierarchy and they are perhaps themost interesting ones from a practical point of view. In Section 3, itwas shown in that it is su�cient to study the normalized versions ofthe auto-calibration constraints, since critical motions are independentof the speci�c values of the intrinsic parameters. That is, when some ofthe intrinsic parameters are known, e.g. the principal point is (10; 20),we may equivalently analyze the case of principal point set to (0; 0).The corresponding camera matrices give rise to subgroup conditionsaccording to Lemma 3.1.5.1. Known intrinsic parametersWe start with fully calibrated perspective cameras. The results may notcome as a surprise, but it is important to know that there are no otherpossible degenerate con�gurations.Proposition 5.1. Given projective structure and calibrated perspectivecameras at m � 3 distinct �nite camera centres, Euclidean structurecan always be recovered uniquely. With m = 2 distinct camera centres,there is always exactly a twofold ambiguity.Proof. Assuming that the cameras have K = I does not change thecritical motions. The camera orientations are irrelevant because anyfalse absolute conic must have the same (rotation invariant) images asthe true one. Calibrated cameras never admit false absolute conics on�1, as the (known) visual cone of each image conic can intersect �1in only one conic, which is the true absolute conic. Therefore, considera �nite absolute conic �f , with supporting plane outside �1. As allpotential absolute conics are proper, virtual and positive semi-de�nite[34, 37], a Euclidean coordinate system can be chosen such that �f hassupporting plane z = 0, and matrix coordinates�f = 2664d1 0 0 00 d2 0 00 0 0 00 0 0 d43775 :Since the cameras are calibrated, K = I, and their orientations areirrelevant, R = I, the conic projection (4) in each camera becomes[I j �t] �f [I j �t]T ' I , 24d1 + d4t21 d4t1t2 d4t1t3d4t1t2 d2 + d4t22 d4t2t3d4t1t3 d4t2t3 d4t23 35 ' I:finalcms.tex; 17/05/2000; 11:16; p.11

y

x

z

T

z

x

y

x

y

x

zzy

2

1optical centre

t 1 2t t

tFigure 1. A twisted pair of reconstructions.As the conic should be proper, both d4 6= 0 and t3 6= 0, which givest1 = t2 = 0. Thus the only solutions are t� = (0; 0;�z) and �f 'diag(1; 1; 0; 1=z2) for some z > 0. Hence, ambiguity implies that thereare at most two camera centres, and the false conic is a circle of imag-inary radius i z, centred in the plane bisecting the two camera cen-tres.In the two-image case, the improper self-inverse projective transfor-mation T = 26641 0 0 00 1 0 00 0 0 z0 0 1=z 03775interchanges the true �1 and the false �f , according toT �f T T ' �1and takes the two projection matrices P� = R�[I j �t�] toP�T�1 = P� and P+T�1 = �R+ h�1 �1 1 j �t+i :While the �rst camera remains �xed, the other has rotated 180� aboutthe axis joining the two centres. This twofold ambiguity corresponds ex-actly to the well-known twisted pair duality [23, 18, 27]. The geometryof the duality is illustrated in Figure 1.The `twist' T represents a very strong projective deformation thatcuts the scene in half, moving the plane between the cameras to in�nity,see Figure 2. By considering twisted vs. non-twisted optical ray intersec-tions, one can also show that it reverses the relative signs of the depths,so for one of the solutions the structure will appear to be behind one

finalcms.tex; 17/05/2000; 11:16; p.12

t t 21

absolute

visual

optical centre

supportingplanes

potential

conics

cone

Figure 2. Intersecting the visual cones of two image conics satisfying the Kruppaconstraints generates a pair of 3D conics, corresponding to the two solutions of thetwisted pair duality.camera, cf. [17]. To conclude, Proposition 5.1 states that any two-viewgeometry has a `twisted pair' projective involution symmetry and anycamera con�guration with three or more camera centres has a uniqueprojective-to-Euclidean upgrade.5.2. Unknown focal lengthsIn the case of two images and internally calibrated cameras modulounknown focal lengths, it is in general possible to recover Euclideanstructure. Since we know that the solutions always occur in twisted pairs(which can be disambiguated using the positive depth constraint), it ismore relevant to characterize the motions for which there are solutionsother than the twisted pair duality. Therefore, the two-camera case willbe dealt with separately, after having derived the critical motions forarbitrary many images.5.2.1. Many imagesIf all intrinsic parameters are known except for the focal lengths, thecamera matrix can be assumed to be K = diag(f; f; 1) which in turnimplies that the image of a potential absolute conic satis�es!� = KKT ' 24� 0 00 � 00 0 135 ; for some � > 0: (6)We start with potential absolute conics on �1.Potential absolute conics on �1Let Cf denote a 3 � 3 matrix corresponding to a false absolute conicfinalcms.tex; 17/05/2000; 11:16; p.13

(in locus form) on the plane at in�nity. Since Cf is not the true one,Cf 6' I. The image of Cf is according to (4)! ' RCf RT : (7)Notice that criticality is independent of translation of the camera.Two cameras are said to have the same viewing direction if theiroptical axes are parallel or anti-parallel.Proposition 5.2. Given �1 and known skew, aspect ratio and prin-cipal point, then a motion is critical if and only if there is only oneviewing direction.Proof. Choose coordinates in which camera 1 has orientation R1 = I.Suppose a motion is critical. According to (6) and (7), this implies thatCf ' diag(1; 1; 1 + �) = I + �e3eT3 for some � > �1. For camera 2, letR2 = [ r1 r2 r3] and apply (7),

I + �r3rT3 ' 241 0 00 1 00 0 �35 ; for some � > 0:This implies that r3 = �e3, and in turn, R2 = � � � 0� � 00 0 �1� which is equiv-alent to a �xed viewing direction of the camera. Conversely, suppose theviewing direction is �xed, which means that Ri = � � � 0� � 00 0 �1� for i > 1.Then, it is not possible to disambiguate between any of the potentialabsolute conics in the pencil Cf (�) ' I + �e3eT3 , since RiCf RTi = Cf .Potential absolute conics outside �1Assume we have a critical motion (Ri; ti)mi=1 with the false dual absoluteconic �f . If the supporting plane for �f is �1, the critical motion isdescribed by Proposition 5.2, so assume that �f is outside �1. As inthe proof of Proposition 5.1, one can assume without loss generalitythat a Euclidean coordinate system has been chosen such that �f hassupporting plane z = 0, and matrix coordinates�f = 2664d1 0 0 00 d2 0 00 0 0 00 0 0 d43775 :The image of �f is according to (4),Pi�fPTi ' !�i , RiCf RTi ' !�i ; where Cf = h d1 d2 0 i+ d4titTi : (8)

finalcms.tex; 17/05/2000; 11:16; p.14

optical centrescritical ellipse

critical hyperbola

Figure 3. Two orthogonal planes, where one plane contains an ellipse and the othercontains a hyperbola.A necessary condition for degeneracy is that Ri should diagonalize Cf tothe form (6), i.e. the matrix Cf must have two equal eigenvalues. As itis always possible to �nd an orthogonal matrix that diagonalizes a real,symmetric matrix [5], all we need to do is to �nd out precisely when Cfhas two equal eigenvalues. Lemma A.1 in the Appendix characterizesmatrices of this form.Applying the lemma to Cf in (8), with �1 = d1, �2 = d2, �3 = 0 and� = d4, results in the following cases:(i) If d1 6= d2, thena. t1 = 0 and t22d1d4 + t23(d1 � d2)d4 = (d1 � d2)d1, orb. t2 = 0 and t21d2d4 + t23(d2 � d1)d4 = (d2 � d1)d2.These equations describe a motion on two planar conics for whichthe supporting planes are orthogonal. On the �rst plane, the conicis an ellipse, while on the other the conic is a hyperbola (dependingon whether d1 > d2 or vice versa), see Figure 3.(ii) If d1 = d2, then t1 = t2 = 0 and t3 arbitrary.Notice that the second alternative in case (ii) of Lemma A.1 doesnot occur, since it implies tTe3 = 0, making Cf rank-de�cient. Also,case (iii) is impossible, since �3 = 0 = �1 = �2.It remains to �nd the rotations that diagonalize Cf . Since rotationsaround the optical axis are irrelevant, only the direction of the opticalaxis is signi�cant. Suppose the optical axis is parameterized by thecamera centre t and a direction d, i.e. ft + �dj� 2 Rg. Any point onfinalcms.tex; 17/05/2000; 11:16; p.15

the axis projects to the principal point,[R j �Rt] �t+ �d1 � ' 2400135, d ' RT 2400135 :The direction d should equal the third row of R, which correspondsto the eigenvector of the single eigenvalue of Cf . Regarding the proofof Lemma A.1, it is not hard to see that the eigenvectors are v '(0; t2 d1; t3 (d1 � d2) ) and v ' (t1 d2; 0; t3 (d1 � d2) ) in the two sub-cases in (i) above. Geometrically, this means that the optical axis mustbe tangent to the conic at each position, as illustrated in Figure 4(b).Similarly in (ii), it is easy to derive that v = (0; 0;�1), which meansthat the optical axis should be tangent to the translation direction, cf.Figure 4(c). An exceptional case is when Cf has a triple eigenvalue,because then any rotation is possible. However, according to Proposi-tion 5.1, it occurs only for twisted pairs. To summarize, we have proventhe following.Proposition 5.3. Given known intrinsic parameters except for focallengths, a motion is critical w.r.t. a�ne reconstruction if and only ifthe motion consists of (i) rotations with at most two distinct centres(twisted pair ambiguity), or (ii) motion on two conics1 (one ellipse andone hyperbola) whose supporting planes are orthogonal and where theoptical axis is tangent to the conic at each position, or (iii) translationalong the optical axis, with arbitrary rotations around the optical axis.The motions are illustrated in Figure 4. In case (i) and (ii), theambiguity of the reconstruction is twofold, as there is only one falseabsolute conic, whereas in case (iii) there is a one-parameter family ofpotential planes at in�nity (all planes z=constant). Case (iii) can beseen as a special case of the critical motion in Proposition 5.2, whichalso has a single viewing direction, but arbitrary translations.5.2.2. Two imagesFor two cameras, projective geometry is encapsulated in the 7 degreesof freedom in the fundamental matrix, and Euclidean geometry in the5 degrees of freedom in the essential matrix. Hence, from two projec-tive images we might hope to estimate Euclidean structure plus twoadditional calibration parameters. Hartley [10, 11] gave a method forthe case where the only unknown calibration parameters are the focallengths of the two cameras. This was later elaborated by Newsam et. al.1 The actual critical motion is the conics minus the two points where the ellipseintersects the plane z = 0, since the image !� is non-proper at these points.finalcms.tex; 17/05/2000; 11:16; p.16

(a)

potential

conic

t 1 t 2

absolute

optical centre

(b)

conic

potentialabsolute

optical centrescritical ellipse

critical hyperbola

(c)

conic

potential

optical centres

absolute

Figure 4. Critical motions for unknown focal lengths: (a) A motion with two �xedcentres. (b) A planar motion on an ellipse and a hyperbola. (c) Translation alongthe optical axis. See Proposition 5.3.finalcms.tex; 17/05/2000; 11:16; p.17

(a)

yx

x

z z y

t t 21

y

(b)

z

x

z

x

yt

t

1

2

Figure 5. Critical con�gurations for two cameras: (a) Intersecting optical axes. (b)Orthogonal optical axis planes. See Proposition 5.4.[28], Zeller and Faugeras [41] and Bougnoux [4]. All of these methodsare Kruppa-based. We will derive the critical motions for this case basedon the results of the previous sections.Proposition 5.4. Given zero skew, unit aspect ratio, principal pointat the origin, but unknown focal lengths for two cameras, then a motion(in addition to twisted pair) is critical if and only if (i) the optical axesof the two cameras intersect or (ii) the plane containing the optical axisof camera 1 and camera centre 2, is orthogonal to the plane containingoptical axis of camera 2 and camera centre 1.Proof. Cf. [28]. Suppose a motion is critical. Regarding Proposition 5.2,we see that if there is only one viewing direction, the optical axes areparallel and intersect at in�nity, leading to (i) above. Examining thethree possibilities in Proposition 5.3, we see that the �rst one is thetwisted pair solution. The second one, either both cameras lie on thesame conic (and hence their axes are coplanar and intersect) or one lieson the hyperbola, the other on the ellipse (in which case their opticalaxes lie in orthogonal planes) leading to (ii). Conversely, given any twocameras with intersecting or orthogonal-plane optical axes, it is possibleto �t (a one-parameter family of) conics through the camera centres,tangential to the optical axes.The two critical camera con�gurations are shown in Figure 5.5.3. Known skew and aspect ratioConsider the image !� = KKT of �1. Inserting the parameterizationof K in (2) into its dual !, it turns out that !12 = � sf2 . Since f and never vanish, requiring that the skew vanishes is equivalent to !12 = 0.The constraint can also be expressed in envelope form using !� ' !�1,!12 = 0 or dually !�12!�33 � !�13!�23 = 0: (9)finalcms.tex; 17/05/2000; 11:16; p.18

If in addition to zero skew, unit aspect ratio is required in K, it isequivalent to !11 = !22. This follows from the fact that !11 = 1f2 and!22 = 1f2 . The constraint can also be transfered to !�,!11 � !22 = 0 or dually !�33(!�22 � !�11) + !�213 � !�223 = 0: (10)Analyzing the above constraints on ! in locus form, results in thefollowing proposition when the plane at in�nity �1 is known.Proposition 5.5. Given �1, a motion is critical with respect to zeroskew and unit aspect ratio if and only if there are at most two viewingdirections.Proof. For each image we have the two auto-calibration constraints (9),(10) with ! given by (7). Choose 3D coordinates in which the �rstcamera has orientation R1 = I. The image 1 constraints become simplyC11�C22 = C12 = 0, so we can parameterize Cf with C11, C12 and C13.Given a subsequent image 2, represent its orientation R2 by a quaternionq = (q0; q1; q2; q3), evaluate its two auto-calibration constraints, andeliminate C11 between them to give:(q20+q23)(q21+q22) ((q0q1+q2q3)C13 + (q0q2�q1q3)C23)=0One of the 3 factors must vanish. If the �rst vanishes the motion is anoptical axis rotation, q21+q22 = 0. If the second vanishes it is a 180� �ipabout an axis orthogonal to the optical one, q20+q23 = 0. In both casesthe viewing direction remains unchanged and no additional constraintis enforced on Cf . Finally, if the third factor vanishes, solving for Cf interms of q gives a linear family of solutions of the form

Cf ' � I + � (o1 oT2 + o2 oT1 ) (11)where o1 = (0; 0; 1)T and o2 = (the third row of R2(q)) are the twoviewing directions and (�; �) are arbitrary parameters. Conversely, givenany potential AC Cf 6' I, there is always exactly one pair of realviewing directions o1;o2 that make Cf critical under (11). The linearfamily �0 I + �0 Cf contains three rank 2 members, one for each eigen-value � of Cf (with �0=�0 = ��). Explicit calculation shows that eachrank 2 member can be decomposed uniquely (up to sign) into a pairof viewing direction vectors o1;o2 supporting (11), but only the paircorresponding to the middle eigenvalue is real. (Coincident eigenvaluescorrespond to coincident viewing directions and can be ignored). Hence,no potential AC Cf can be critical for three or more real directionssimultaneously.finalcms.tex; 17/05/2000; 11:16; p.19

Table I. Summary of critical motions in auto-calibration.Auto-calibration Critical motions Reconstructionconstraint ambiguityKnown calibration twisted pair duality projectiveUnknown focal length (i) optical axis rotation a�nebut otherwise known (ii) motion on two planar conics projectivecalibration (iii) optical axis translation projectiveUnknown focal length (i) intersecting optical axes projective(two images only) (ii) orthogonal optical axis planes projectiveZero skew and (i) two viewing directions a�neunit aspect ratio (ii) complicated algebraic variety projectiveFor potential absolute conics outside �1 things are more compli-cated. For each image, there are two auto-calibration constraints. Soin order to single out the true absolute conic (which has 8 degrees offreedom), at least 4 images are necessary. For a given �f the polynomialconstraints in (9) and (10) determine a variety in the space of rigidmotions. We currently know of no easy geometrical interpretation ofthis manifold.It is easy to see that given a critical camera motion, the ambiguityis not resolved by rotation around the camera's optical axis.5.4. SummaryA summary of the critical motions for auto-calibration under the auto-calibration constraints studied is given in Table I. The reconstructionambiguity is classi�ed as projective if the plane at in�nity cannot beuniquely recovered, and a�ne if it is possible. As mentioned earlier, thetwisted pair duality is not a true critical motion, since the positive-depthconstraint can always resolve the ambiguity.6. Particular MotionsSome critical motions occur frequently in practice. In this section, aselection of them is analyzed in more detail.

finalcms.tex; 17/05/2000; 11:16; p.20

6.1. Pure rotationIn the case of a stationary camera performing arbitrary rotations, no 3Dreconstruction is possible. There always exist many potential absoluteconics outside �1.However, it is still possible to recover the internal camera calibration,provided there are no potential absolute conics on �1, cf. [37]. Propo-sition 5.2 and Proposition 5.5, regarding critical motions and potentialACs on �1 tells us when such auto-calibration is possible for a purelyrotating camera.6.2. Pure translationIf a sequence of movements only consists of arbitrary translations andno rotations, all proper, virtual conics on �1 are potential absoluteconics. Still, one could hope to recover the plane at in�nity correctly,and thus get an a�ne reconstruction.Proposition 6.1. Let (ti)mi=1 be a general sequence of translations,where m is su�ciently large. Then, the motion is(i) always critical w.r.t. a�ne reconstruction under the constraints zeroskew and unit aspect ratio.(ii) not critical w.r.t. a�ne reconstruction under the constraints zeroskew, unit aspect ratio and vanishing principal point.Proof. (i) We need to show that there exists a potential DAC �f outside�1, which is valid for all (ti)mi=1. Choose a coordinate system such thatPi = �

I j � ti�. Then for instance �f = diag(1; 1; 0; 1) is a potential DAC(multiply Pi�fPTi to get !� and check that it ful�lls (9) and (10)). (ii)follows directly from Proposition 5.3.Note that translating only along the optical axis in case (ii) above resultsin a critical motion.6.3. Parallel axis rotationsSequences of rotations around parallel axes with arbitrary translationsare interesting in several aspects. They occur frequently in practice andare one of the major degeneracies for auto-calibration with constantintrinsic parameters [36, 43]. See Figure 6.It follows directly from Proposition 5.5 that given zero skew, unitaspect ratio and general rotation angles, the �xed-axis motion is notcritical unless it is around the optical axis. If we further add the vanish-ing principal point constraint, the optical axis remains critical accordingfinalcms.tex; 17/05/2000; 11:16; p.21

Figure 6. Rotations around the vertical axes with arbitrary translations.to Proposition 5.2. If we know only that the skew vanishes, we have thefollowing proposition.Proposition 6.2. Let (Ri; ti)mi=1 be a general motion whose rotationsare all about parallel axes, where R1 = I and m is su�ciently large.Given �1, the motion is critical w.r.t. zero skew if and only if therotation is around one of the following axes:(i) (0; �; �) or (�; 0; �),(ii) (1; 1; 0) or (1;�1; 0),where each � denotes an arbitrary real number.Proof. Let Cf denote a false AC on �1. The zero skew constraint in(9) using the parameterization in (7) gives C12 = 0 for camera 1. Anarbitrary rotation around a �xed axis (q1; q2; q3) can be parameterizedby �(q1; q2; q3), � 2 R. Inserting this into the zero skew constraint in (9)yields a polynomial in R[�]. Since � can be arbitrary all coe�cients ofthe polynomial must vanish. The solutions to the system of vanishingcoe�cients are the ones given above.Some of these critical axes may be resolved by requiring that thecamera calibration should be constant. In [37], it is shown that parallelaxis rotations under constant intrinsic parameters are always criticalfinalcms.tex; 17/05/2000; 11:16; p.22

and give rise to the following pencil of potential absolute conics:Cf (�) = I + �[q1; q2; q3][q1; q2; q3]T : (12)Combining constant intrinsic parameters, and some a priori knownvalues of the intrinsic parameters, some of the critical axes are stillcritical.Corollary 6.1. Let (Ri; ti)mi=1 be a general motion with parallel axisrotations, where R1 = I and m is su�ciently large. Given �1, andconstant intrinsic parameters, the following axes are the only ones stillcritical:(i) (0; �; �) and (�; 0; �) w.r.t. zero skew,(ii) (0; 0; 1) w.r.t. zero skew and unit aspect ratio,(iii) (0; 0; 1) w.r.t. an internally calibrated camera except for focal length.Proof. (i) Using the potential ACs in (12) in the proof of Proposi-tion 6.2, one �nds that the only critical axes remaining under thezero skew constraint are (0; �; �) and (�; 0; �). (ii) and (iii) are provedanalogously. 7. ExperimentsIn practice, a motion is never exactly degenerate due to measurementnoise and modeling discrepancies. However, if the motion is close toa critical manifold it is likely that the reconstructed parameters willbe inaccurately estimated. To illustrate the typical e�ects of criticalmotions, we have included some simple synthetic experiments for caseof two cameras with unknown focal lengths but other intrinsic param-eters known. We focus on the question of how far from critical the twocameras must be to give reasonable estimates of focal length and 3DEuclidean structure [20]. The experimental setup is as follows: two unitfocal length perspective cameras view 25 points distributed uniformlywithin the unit sphere. The camera centres are placed at (�2;�2; 0)and (2;�2; 0) and their optical axes intersect at the origin, similar tothe setup in Figure 5(a). Independent Gaussian noise of 1 pixel standarddeviation is added to each image point in the 512 � 512 images.In the experiment, the elevation angles are varied, upwards for theleft camera and downwards for the right one, so that their optical axesare skewed and no longer meet. For each pose, the projective structure

finalcms.tex; 17/05/2000; 11:16; p.23

0 5 10 15 20 25 30 35 40 45

101

102

Unknown focal bundle: −

Foc

al L

engt

h E

rror

(%

)

Elevation Angle (degrees)

0 5 10 15 20 25 30 35 40 450

0.05

0.1

0.15

Unknown focal bundle: −Calibrated bundle: −x−

3D P

oint

Err

or

Elevation Angle (degrees)Figure 7. Relative errors vs. camera elevation for two cameras.and the fundamental matrix are estimated by a projective bundle ad-justment that minimises the image distance between the measured andreprojected points [3]. Then, the focal lengths are computed analyti-cally with Bougnoux' method [4]. For comparison, a calibrated bundleadjustment with known focal lengths is also applied to the same data.The resulting 3D error is calculated by Euclidean alignment of the trueand reconstructed point sets.Figure 7 shows the resulting root mean square errors over 100 trialsas a function of elevation angle. At zero elevation, the two opticalaxes intersect at the origin. This is a critical con�guration accordingto Proposition 5.4. A second critical con�guration occurs when theepipolar planes of the optical axes become orthogonal at around 35�elevation. Both of these criticalities are clearly visible in both graphs.For geometries more than about 5-10� from criticality, the focal lengthscan be recovered quite accurately and the resulting Euclidean 3D struc-ture is very similar to the optimal 3D structure obtained with knowncalibration.

finalcms.tex; 17/05/2000; 11:16; p.24

8. ConclusionIn this paper, the critical motions in auto-calibration under several auto-calibration constraints have been derived. The various constraints onthe intrinsic parameters have been expressed as subgroup conditions onthe 3�3 upper triangular camera matrices. With this type of condition,we showed that the critical motions are independent of the speci�cvalues of the intrinsic parameters.It is important to be aware of the critical motions when trying toauto-calibrate a camera. Additional scene or motion constraints mayhelp to resolve the ambiguity, but clearly the best way to avoid de-generacies is to use motions that are �far� from critical. Some syntheticexperiments have been performed that give some practical insight to thenumerical conditioning of near-critical and critical stereo con�gurations.AcknowledgmentsThis work was supported by the European Union under Esprit projectLTR-21914 Cumuli. We would like to thank Sven Spanne for construct-ing the proof of Lemma A.1. AppendixLemma A.1. Let A be a real, symmetric 3� 3 matrix of the formA = �1e1eT1 + �2e2eT2 + �3e3eT3 + �ttT ;where e1 = (1; 0; 0), e2 = (0; 1; 0), e3 = (0; 0; 1), t a non-zero real3 vector and � a non-zero real scalar. Let �1,�2 and �3 be given realscalars. Then, necessary and su�cient conditions on (t,�) for A to havetwo equal eigenvalues can be divided into three cases:(i) If �1 6= �2 6= �3, then tTei = 0 for at least one i (where i = 1; 2or 3). Furthermore, � can take the values:� = (�i � �j)(�i � �k)t2j(�i � �k) + t2k(�i � �j) ;for any i for which tTei = 0 (where j 6= k 6= i).(ii) If �i = �j 6= �k, thena. tTei = tTej = 0 and � arbitrary, or

finalcms.tex; 17/05/2000; 11:16; p.25

b. tTek = 0 and � = �k��it2i+t2j .(iii) If �1 = �2 = �3, then t and � arbitrary.Proof. It follows from the Spectral Theorem [5] that if A is real andsymmetric with two equal eigenvalues �, then there is a third eigen-vector v and a scalar � such that A = �I + �vvT . (The eigenvaluecorresponding to v is �+ �). This gives�1e1eT1 + �2e2eT2 + �3e3eT3 + �ttT � �vvT � �I = 0:Multiplying this matrix equation with e1, e2 and e3, results in threevector equations, (�1 � �)e1 + �t1t� �v1v = 0(�2 � �)e2 + �t2t� �v2v = 0(�3 � �)e3 + �t3t� �v3v = 0: (13)To prove (i), assume �1 6= �2 6= �3. The orthogonal bases e1, e2 ande3 are linearly independent and cannot all be linear combinations of tand v, so one of �i � � must vanish, and thereby exactly one. Suppose� = �1. Then, �t1t� �v1v = 0:If one of the coe�cients is non-zero, then t and v would be linearlydependent. However, this is impossible because e2 and e3 are linearlyindependent and �2 � � 6= 0, �3 � � 6= 0. Analogously, � 6= 0 becauseotherwise e2 and e3 would be linearly dependent according to (13).Therefore tTe1 = 0.If t is orthogonal to e1 and � = �1, some easy calculations yield that� must be chosen as� = (�1 � �2)(�1 � �3)t22(�1 � �3) + t23(�1 � �2) ;which is also su�cient.When two or three of �i (i = 1; 2; 3) are equal, similar argumentscan be used to deduce (ii) and (iii).References1. Armstrong, M., A. Zisserman, and R. Hartley: 1996, `Self-Calibration fromImage Triplets'. In: European Conf. Computer Vision. Cambridge, UK, pp.3�16.finalcms.tex; 17/05/2000; 11:16; p.26

2. Åström, K. and A. Heyden: 1996, `Euclidean Reconstruction from ConstantIntrinsic Parameters'. In: Int. Conf. Pattern Recognition. Vienna, Austria, pp.339�343.3. Atkinson, K. B.: 1996, Close Range Photogrammetry and Machine Vision.Whittles Publishing.4. Bougnoux, S.: 1998, `From Projective to Euclidean Space Under any PracticalSituation, a Criticism of Self-Calibration'. In: Int. Conf. Computer Vision.Mumbai, India, pp. 790�796.5. Cullen, C. G.: 1966, Matrices and Linear Transformations. Addison-WesleyPublishing Company.6. Faugeras, O.: 1993, Three-Dimensional Computer Vision. MIT Press, Cam-bridge, Mass.7. Faugeras, O., Q.-T. Luong, and S. Maybank: 1992, `Camera self-calibration:Theory and experiments'. In: European Conf. Computer Vision. SantaMargherita Liguere, Italy, pp. 321�334.8. Faugeras, O. D.: 1992, `What can be seen in three dimensions with an uncal-ibrated stereo rig?'. In: European Conf. Computer Vision. Santa MargheritaLiguere, Italy, pp. 563�578.9. Golub, G. and C. Van Loan: 1989, Matrix Computation. The Johns HopkinsUniversity Press, Baltimore.10. Hartley, R.: 1992, `Estimation of Relative Camera Positions for UncalibratedCameras'. In: European Conf. Computer Vision. Santa Margherita Liguere,Italy, pp. 579�587.11. Hartley, R.: 1993, `Extraction of Focal Lengths from the Fundamental Matrix'.Unpublished manuscript.12. Hartley, R., R. Gupta, and T. Chang: 1992, `Stereo from uncalibrated cameras'.In: Conf. Computer Vision and Pattern Recognition. Champaign, USA, pp.761�764.13. Helmholtz, H.: 1867, Handbuch der Physiologischen Optik. Leibzig: Voss.14. Heyden, A. and K. Åström: 1997, `Euclidean Reconstruction from Image Se-quences with Varying and Unknown Focal Length and Principal Point'. In:Conf. Computer Vision and Pattern Recognition. San Juan, Puerto Rico, pp.438�443.15. Heyden, A. and K. Åström: 1998, `Minimal Conditions on Intrinsic Parametersfor Euclidean Reconstruction'. In: Asian Conf. Computer Vision, Vol. II. HongKong, pp. 169�176.16. Heyden, A. and K. Åström: 1999, `Flexible Calibration: Minimal Cases forAuto-calibration'. In: Int. Conf. Computer Vision. Kerkyra, Greece, pp. 350�355.17. Heyden, A. and G. Sparr: 1999, `Reconstruction from Calibrated Cameras -A New Proof of the Kruppa Demazure Theorem'. Journal of MathematicalImaging and Vision 10(2), 123�142.18. Horn, B. K. P.: 1987, `Motion Fields Are Hardly Ever Ambiguous'. Int. JournalComputer Vision 1(3), 239�258.19. Kahl, F.: 1999, `Critical Motions and Ambiguous Euclidean Reconstructionsin Auto-Calibration'. In: Int. Conf. Computer Vision. Kerkyra, Greece, pp.469�475.20. Kahl, F. and B. Triggs: 1999, `Critical Motions in Euclidean Structure fromMotion'. In: Conf. Computer Vision and Pattern Recognition. Fort Collins,USA, pp. 366�372.finalcms.tex; 17/05/2000; 11:16; p.27

21. Krames, J.: 1941, `Zur Ermittlung eines Objectes aus zwei Perspectiven (EinBeitrag zur Theorie der gefährlichen Örter).'. Monatch. Math. Phys. 49, 327�354.22. Kruppa, E.: 1913, `Zur Ermittlung eines Objektes aus zwei Perspektiven mitinnerer Orientierung'. Sitz-Ber. Akad. Wiss., Wien, math. naturw. Kl. AbtIIa(122), 1939�1948.23. Longuet-Higgins, H. C.: 1988, `Multiple Interpretations of A Pair of Images ofA Surface'. Phil. Trans. Royal Soc. London A 418, 1�15.24. Maybank, S.: 1993, Theory of Reconstruction from Image Motion. Springer-Verlag, Berlin, Heidelberg, New York.25. Maybank, S. and A. Shashua: 1998, `Ambiguity in Reconstruction from Imagesof Six Points'. In: Int. Conf. Computer Vision. Mumbai, India, pp. 703�708.26. Maybank, S. J. and O. D. Faugeras: 1992, `A theory of self calibration of amoving camera'. Int. Journal Computer Vision 8(2), 123�151.27. Negahdaripour, S.: 1990, `Multiple Interpretations of the Shape and Motionof Objects from Two Perspective Images'. IEEE Trans. Pattern Analysis andMachine Intelligence 12(11), 1025�1039.28. Newsam, G., D. Huynh, M. Brooks, and H.-P. Pan: 1996, `Recovering UnknownFocal Lengths in Self-Calibration: An Essentially Linear Algorithm and Degen-erate Con�gurations'. In: Int. Arch. Photogrammetry & Remote Sensing, Vol.XXXI-B3. Vienna, Austria, pp. 575�80.29. Pollefeys, M., R. Koch, and L. Van Gool: 1998, `Self-Calibration and MetricReconstruction in spite of Varying and Unknown Internal Camera Parameters'.In: Int. Conf. Computer Vision. Mumbai, India, pp. 90�95.30. Pollefeys, M., L. Van Gool, and M. Oosterlinck: 1996a, `The Modulus Con-straint: A New Constraint for Self-Calibration'. In: Int. Conf. PatternRecognition. Vienna, Austria, pp. 349�353.31. Pollefeys, M., L. Van Gool, and M. Proesmans: 1996b, `Euclidean 3D Recon-struction from Image Sequences with Variable Focal Lengths'. In: EuropeanConf. Computer Vision. Cambridge, UK, pp. 31�42.32. Schröter, H.: 1880, Theorie der Ober�ächen zweiter Ordnung und derRaumkurve dritter Ordnung als Erzeugnisse projektvischer Gebilde. Leibzig:Teubner.33. Semple, J. G. and G. T. Kneebone: 1952, Algebraic Projective Geometry.Clarendon Press, Oxford.34. Spain, B.: 1960, Analytical Quadrics. Pergamon Press.35. Sparr, G.: 1991, `An algebraic-analytic method for a�ne shapes of point con-�gurations'. In: Scandinavian Conf. on Image Analysis. Aalborg, Denmark,pp. 274�281.36. Sturm, P.: 1997a, `Critical motion sequences for monocular self-calibrationand uncalibrated Euclidean reconstruction'. In: Conf. Computer Vision andPattern Recognition. San Juan, Puerto Rico, pp. 1100�1105.37. Sturm, P.: 1997b, `Vision 3D Non Calibrée: Contributions à la ReconstructionProjective et Étude des Mouvements Critiques pour l'Auto-Calibrage'. Ph.D.thesis, Institut National Polytechnique de Grenoble.38. Sturm, P.: 1999, `Critical Motion Sequences for the Self-Calibration of Camerasand Stereo Systems with Variable Focal Length'. In: British Machine VisionConf. Nottingham, UK, pp. 63�72.39. Triggs, B.: 1997, `Autocalibration and the Absolute Quadric'. In: Conf.Computer Vision and Pattern Recognition. San Juan, Puerto Rico, pp. 609�614.finalcms.tex; 17/05/2000; 11:16; p.28

40. Van Gool, L., T. Moons, M. Proesmans, and M. Van Diest: 1994, `A�ne re-construction from perspective image pairs obtained by a translating camera'.In: Int. Conf. Pattern Recognition. Jerusalem, Israel, pp. 290�294.41. Zeller, C. and O. Faugeras: 1996, `Camera Self-Calibration from Video Se-quences: the Kruppa Equations Revisited'. Technical Report 2793, INRIA,Sophia-Antipolis, France.42. Zisserman, A., P. Beardsley, and I. Reid: 1995, `Metric Calibration of a StereoRig'. In: IEEE Workshop on Representation of Visual Scenes. Cambridge Ma,USA, pp. 93�110.43. Zisserman, A., D. Liebowitz, and M. Armstrong: 1998, `Resolving Ambiguitiesin Auto-Calibration'. Phil. Trans. Royal Soc. London A 356(1740), 1193�1211.

finalcms.tex; 17/05/2000; 11:16; p.29