Machine Vision Tools for CAGDV. Koivunen J-M. VezienABSTRACT. In this paper, the problem of constructing geometric models from dataprovided by 3-D imaging sensors is addressed. Such techniques allow for rapid model-ing of sculptured free-form shapes and generation of geometric models for existing parts.In order for a complete data set to be obtained, multiple images, each from a di�erentviewpoint, have to be merged. A technique stemming from the Iterative Closest Point(ICP) method for estimating the relative transformations among the viewpoints is de-veloped. Computational solutions are provided for estimating shape from noisy sensorymeasurements using representations that conform with commonly used representationsfrom Computer Aided Geometric Design (CAGD). In particular, NURBS and triangularsurface representations are applied in shape estimation. The surface approximations arere�ned by the algorithms to meet a user-de�ned tolerance value.Keywords: 3-D machine vision, shape estimation, geometricmodeling, registration, splines.1 IntroductionCommon industrial machine vision applications are, for example, visual inspection, roboticassembly and bin picking tasks. There are, however, a few new emerging application areassuch as Free-Form Fabrication (FFF) and Rapid Prototyping (RP). In particular, we areinterested in developing machine vision based tools for Computer Aided Geometric Design(CAGD).In manufacturing, geometry plays an important role in shape design, in production ofinput for engineering analysis tools and in simulation of di�erent manufacturing stages.The geometric design and manufacturing phases are tightly interconnected, since a par-ticular design is eventually made into a product.In this paper, computational solutions for constructing a shape model of an artifactfrom 3-D sensory measurements are provided. The goal is to provide a rapid geometricmodeling service. Such service is useful as a design aid. Especially, in the case of free-form shapes the shape design by hand is relatively time consuming and usually requiresextensive knowledge about the mathematical background of the modeling primitives, suchas splines. These modeling techniques provide an initial shape model. The initial model,however, typically evolves and requires modi�cations, no matter whether it is producedby an experienced designer or automatically from sensor measurements.The two main problems in constructing shape models from 3-D data are: acquisition ofa complete 3-D data set; and shape estimation from noisy data. The �rst issue requires astrategy for positioning the sensor with respect to the object as well as the estimation ofrelative transformation among the images acquired from di�erent viewpoints. The secondone concentrates on techniques for processing and analyzing noisy 3-D sensor data inorder to create a geometric model of a part. Both problems are addressed in this paper.1

Currently, there is no single representation or method in CAGD that would be best forevery design task. Therefore, we employ multiple representations in shape estimation tocapture various shapes e�ciently. In particular, triangulations, triangular spline, Non-Uniform Rational B-Spline (NURBS) and trimmed surface representations are used herefor constructing models for free-form shapes. These representations allow for approximat-ing surfaces of di�erent geometry and topology. Moreover, the shape representations arerelated to commonly used design primitives which provides a means for data sharing withvarious manufacturing subsystems.The organization of this paper is as follows. In section 2, data acquisition problemis addressed and a technique for view registration is developed. The technique is anextension of the ICP algorithm [3]. In section 3, computational solutions developed forshape model estimation are described. In section 4, examples of model construction usingreal and simulated range data are given. Finally, section 5 summarizes the paper.2 Acquisition of 3-D dataOptical range-�nding techniques provide only partial 3-D data because the emitted sig-nals do not reach all the object surfaces when measurements are acquired from a singleviewpoint at a time. Therefore, a strategy for positioning the sensor with respect tothe object (or vice versa) is required to obtain measurements from all the surfaces of anartifact.In order to merge range data sets from multiple viewpoints their registration, i.e., therelative rotation and translation among the views, must be estimated and the integra-tion of the views performed. For surveys on di�erent registration techniques the readeris referred to [4, 3]. A good estimate of the transformation among the viewpoints cansometimes be obtained directly using a high accuracy translation-rotation table. Other-wise, the transformation has to be determined and re�ned from observations. In suchcases, a closed-loop system for obtaining a complete 3-D data set is desirable in orderto estimate registration accurately as well as to adapt the data acquisition to arbitraryobject geometry and topology. The problem is illustrated in Figure 1.Figure 1: Data fusing from multiple viewpoints. The registration estimates the relativerotation and translation parameters among the viewpoints and expresses the completedata set in a common coordinate frame.Physical measurements are subject to noise. In the case of range data, the actual noisedistribution may di�er from Gaussian. Furthermore, there may occur greatly deviatingobservations, i.e. outliers, due to the material or orientation of the surface. The rawrange data have to be �ltered in order to separate the desired and undesired parts of thesignal in contaminated sensor data. Non-linear �ltering techniques are used because thespectra of the signal and the noise overlap to such an extent that it is not possible toseparate them using �lters that have frequency selective behavior without distorting thesignal severely. Moreover, bias can be avoided even if the signal is not linear or wherelocal minima or maxima occur [15].2.1 Estimating registration among di�erent viewpointsThe registration technique presented here is an extension of the Iterative Closest Point(ICP) algorithm proposed by Besl and McKay [3]. The original technique was devel-2

oped for registering data from a single view with a priori known model. No explicitfeature-to-feature correspondences are required for matching. Furthermore, the techniqueis computationally e�cient and any model primitive can be used providing that thereexists a method for computing distances between a data point and a primitive. Only theestimation of the relative transformation is addressed here.In a model construction task the registration problem is more complicated since there isno a priori known part model available. The internal representation used as a model has tobe build incrementally by adding data from each view after registration. An approximatetransformation between the views is available from the sensor setup. It provides a good�rst estimate for the the minimization, and allows to exclude points from matching ifthey are not visible in the representation registered so far. Otherwise, those points wouldcause large errors in registration.Figure 2: Principle of Iterative Closest Point matching procedure. The data set P isregistered against model M, by iteratively searching for the closest point projection setX .The technique matches a collection of points from one set of raw data with a set of modelprimitives. Given a model M= fmkjk = 1; :::;Mg containing M primitives, consider aset of N points P= fpiji = 1; :::; Ng. A set X= fxi:kg is then computed, such that eachxi:k is de�ned as the closest point of the closest primitive mk to pi. Note that the matchP !M is not 1:1, because several points can select the same feature as the closest one(see �gure 2). Once a match P ! X is established, the registration state vector q, whichconsists of a rotation quaternion qR and a translation vector qT , can be estimated. Amean square error criterion [3]:e(q) = 1N NXi=1 kxi:k �RqRpi � qTk2 (1)is minimized. R denotes the 3�3 rotation matrix generated from the rotation quaternion.Let q0 be the state vector solution that is applied on the set P. The closest primitivesearch is iterated on the displaced set, leading to a new P ! X matching. The obtainedconsecutive displacements q are cumulated into a �nal solution (qR; qT ), until the errormeasure e(q) or the relative error change between two consecutive iterations �ee is belowa user-de�ned threshold value. In practise, the latter criterion is dimensionless and easierto set, with typical values ranging between 0.01 and 0.001.The primitives employed in the distance computation have to be chosen. Surface trian-gulations are a good choice because they are not limited by the spatial arrangement of theparametric space; they can model scattered data as well as surfaces of arbitrary topology;and are numerically stable. In particular, optimal Delaunay triangulation which possessesempty minsphere property is appealing for this purpose. In the case of triangulated data,the centers of gravity (pi's) of the triangles in one image form the set P, and these pointsare registered against the triangles in the model registered so far.In order to �nd valid points for matching, the angle between the surface normals ischecked after the displacement as follows:nxi:k �RqR(npi) > cos(�)where nxi is the normal estimated at point xi, and � is a threshold angle. Points where theangle between the normals exceeds the threshold value are rejected at each iteration. At3

iteration k, the displacement q is recomputed until no point violates the normal validitycheck above. In general, invalid points are rejected rapidly, so that only two or three ICPpasses are necessary at each iteration. Although we have found out that a �xed value5� < � < 10� was a reasonable guess for our application, the rejection angle couldbe dynamically set, and lowered as the matching process converges towards the exactregistration q.Although convergence to the nearest minimum is insured in the standard ICP proce-dure, it is no longer guaranteed if one rejects points according to the above criterion. Inparticular, one has to make sure the quality of the initial estimate is reasonably good.Otherwise, too may points of data set P may get rejected (in that case the mean squareerror e tends to increase or even diverge rapidly). This phenomenon remains easy tomonitor because it almost always occur in the very �rst stages of the registration run.3 Shape estimation3.1 Issues on shape representationIdeally, the shape estimation should extract the geometry and the topology of an objectfrom the sensor data. Currently, there is no single representation that would be appropri-ate in all situations. Therefore, multiple representations are necessary in order to recovervarious shapes e�ciently. In addition, the representations have to be closely related tomodeling primitives commonly used in CAD systems and in a standard product dataexchange format (e.g. IGES) or a technique for converting them to such representationshave to exist. Otherwise, the model cannot be imported into a design system or sharedin a concurrent engineering environment.In this paper, we are interested in producing geometric models for free-form surfaces.Tensor product splines are used for surfaces where a rectangular parametric grid is ap-propriate, i.e., when a bijective mapping between the parametric domain and physicalcoordinates exists. Triangular representations are not limited by the spatial arrangementof the parametric space. Triangulations and triangular splines are employed here becausethey allow for representing surfaces of arbitrary topology and estimating shape from scat-tered data. The obtained shape estimates are re�ned by the �tting algorithms to meet auser-de�ned tolerance value. For surveys on di�erent shape representations in CAGD thereader is referred to [28, 21, 24, 19]The choice of shape representations does not depend only on the class of shapes oneneeds to model but also on the other design automation processes (e.g., CAD, CAE,CAM). As an example, if the production is done using conventional machining processes,the shape should be de�ned using design primitives which convey structural informationabout the part and consequently make engineering analysis and process planning easier.Moreover, primitives conveying structural information facilitate signi�cant data reductionin shape estimation. As an example, superellipsoid surfaces are used in [18] to detectprimitive solids and capture global shape properties such as symmetry. Quadric surface�tting [12, 25] used in computer vision and graphics domains can be used similarly.In the case Free-Form Fabrication (FFF) processes based on layered technology suchas stereolithography, the process planning is basically independent of part complexityand very little structural information is required in geometric model. As an example, arotationally symmetric shape can be de�ned as a collection of cross-section curves thatcorrespond to di�erent layers. For conventional manufacturing processes, the geometry4

is represented by a surface of revolution which allows for mapping the shape directlyto a machining operation on a CNC lathe. Figure 3 shows a cylindrical pin de�ned bycross-sectional layers and and by a surface of revolution design primitive.Figure 3: The shape of a cylindrical pin is modeled using (a) a collection of cross-sectionallayers and (b) using a surface of revolution design primitive.3.2 TriangulationA surface triangulation is constructed in order to describe polygonal and irregular shapesas well as surfaces of arbitrary topology. Moreover, triangles are used as shape primi-tives in image registration. Vertices in the triangulation are used as domain vertices insmooth surface reconstruction based on triangular spline patches. Triangular primitivesare included in IGES [14] as Finite Element entities which allows for data sharing.A k/n-triangulation is a collection of k-simplices in n-dimensional space. A Delaunaytriangulation satis�es the property that the interior of the minsphere of a k-simplex con-tains no vertex of any k-simplex. The minsphere is the smallest (n�1)-dimensional spherewhich passes through k + 1 vertices of a k-simplex. The technique used here follows theGeneralized Delaunay Triangulation algorithm presented in [13] with k = 2 and n = 3.It starts with one k-simplex and new triangulation points are inserted one at the time.Di�erent insertion operations are executed based on the location of insertion point rela-tive to the triangulation. The process continues until all points have been inserted or theremaining points cannot be inserted without violating the empty minsphere property.The triangulation is adaptively re�ned using the L2 norm between points and triangles.Points are added and removed to meet a user-de�ned tolerance value tol. The triangula-tion is �rst performed using a sparse sampling and in subsequent steps more points areincorporated until the required tolerance is achieved. A point is candidate for additionif the distance d to the nearest triangle T is > tol. Between each sampling interval, thetriangulation vertex list is scanned for possible removals. For each point p in the trian-gulation, let T be the set of 2-simplices sharing p. p is tentatively removed and T isretriangulated into T 0. p is de�nitively removed only if the distance of p to T 0 is lessthan tol. The tolerancing step tends to produce plenty of small triangles if there are dis-continuities on the surface. In the presence of edges a suboptimal Constrained Delaunaytriangulation where edges are not allowed to cross discontinuities on the surface may bemore appropriate [6].3.3 Smooth surface �tting over triangulationA 2/3 Delaunay triangulation provides a planar approximation of a surface. Often onewants to model a shape with smooth surfaces. Commonly this is done by using tensorproduct splines such as NURBS and the continuity (C1; C2) is de�ned as di�erentiabilitywith respect to the parameterization. However, the rectangular arrangement of the para-metric domain and strict C continuity de�nition do not allow for representing surfacesof arbitrary topology. Triangular splines provide a convenient modeling tool for suchsurfaces and allow for shape estimation from scattered data. Moreover, the continuityproperties at the joints can be relaxed to geometric tangent plane (G1) continuity (see[11]) to overcome the problem. 5

Figure 4: (a) The arrangement of control points for a cubic Bezier triangle, where b030,b300 and b003 are the domain vertices from 2/3 triangulation, and (b) subdivision of atriangular patch using modi�ed Clough-Tocher split.A smooth surface representation is constructed on top of the optimal 2/3 triangulation(or constrained Delaunay triangulation). The shape is represented by a collection of Beziertriangles where the vertices of the 2/3 triangulation are the domain vertices for the Bezierpatches. A Bezier triangle of degree n is de�ned using barycentric coordinates (u; v; w)as follows [11]: Xjij=n biBni (u; v; w) (2)where jij = i+ j + k so that i + j + k = n and i; j; k � 0. Bni (u; v; w) are the Bernsteinpolynomials Bni (u; v; w) = n!i! j! k!uivjwk;where u; v; w � 0 and u+ v + w = 1. The control net formed using coe�cients bi in (2)consists of 12(n+1)(n+ 2) control points forming a triangular structure. The coe�cientsare estimated through a local interpolation process instead of approximation. Therefore,the resulting surface passes through the vertices of the underlying triangulation. Thecontrol net in the case of cubic patch is depicted in Figure 4(a).The 2/3 triangulation can be done using a less strict tolerance value since a cubic patchhas more degrees of freedom to de�ne each triangle. For each patch, the bi coe�cientsin (2) have to be estimated by determining the 3 boundary curves. The intermediatecoe�cients along each boundary are obtained through cubic Hermite interpolation usingthe domain vertices and the tangent plane at each vertex. The direction vector at eachvertex is the orthogonal projection of the vector (AB) connecting the domain vertices Aand B to the estimated tangent plane at each vertex. The lengths of the direction vectorsare normalized to jjABjj2, i.e., using the L2 norm between the endpoints. The value ofthe remaining coe�cient is set to: b111 = 14(b201+ b102+ b021+ b012+ b210+ b120)� 16(b300+b030 + b003).The estimation of the tangent plane at each vertex is a key factor for obtaining a highquality surface interpolant from real sensor data. This is often neglected in computergraphics and CAD literature. In particular, surface curvatures (employed for example in[7]) are di�cult to estimate reliably from noisy data (see [1]). If the data are smooth in theneighborhood of a vertex and noise are Gaussian distributed, an e�cient estimate for thetangent plane can be obtained using a least squares method. In reality, however, the errordensity is often not Gaussian and the vertex may be in the vicinity of an edge. Therefore,the tangent plane at each vertex is determined by estimating covariance matrix � in arobust manner for a small number of neighboring observations using the IMCD (IterativeMinimum Covariance Determinant) technique [16]. In a neighborhood of N observationsP (i) = (xi; yi; zi)T and robust center estimate bP c, a robust covariance estimate b�:b� = 1h� 1 hXi=1(P (i) � bP c)(P (i) � bP c)T (3)is computed for h = [N=2] + 1 out of N observations using the following error criterion:minimize jdet(b�)j(h:N):6

The normal of the tangent plane is obtained from the eigenvector ~� corresponding to thesmallest eigenvalue �min(b�) of the covariance matrix estimate. This technique producesreliable results in the presence of both impulsive and nonimpulsive noise as well as nearedges.Cubic triangular patches do not o�er enough degrees of freedom to obtain overallsmooth (G1 or C1) surface. Therefore, each patch is subdivided (through modi�ed Clough-Tocher split [10]) by splitting it into 3 minitriangles at the centroid (u = 13 ; v = 13 ; w = 13)as depicted in Figure 4(b). The coe�cients are then modi�ed in a two stage process.First, in each minitriangle, the coe�cients on the next row parallel to the boundary areadjusted to meet the continuity requirement cross the boundary. The internal coe�cientsare then corrected in order to preserve the continuity within the subdivided triangle. Incase of G1 continuity, the degree of a cubic minitriangle have to be elevated into quarticin order to have a cubic boundary curve. A degree n patch is thus written as a degreen+ 1 patch with coe�cients ci so that [11]:Xjij=n biBni (u; v; w) = Xjij=n+1 ciBn+1i (u; v; w): (4)This operation increases the number of parameters signi�cantly and some degrees offreedom remain unset after meeting the continuity requirement which may introduceunwanted oscillations. The concept of approximately G1 continuous surface ("-G1) allowsthe use of lower order patches [7].The G1 continuity is constructed asymmetrically here using Farin's condition [9] andemploying the following heuristic: the coe�cients of the triangle with smaller area aremodi�ed to establish the continuity with the triangle with larger area. This operation isjusti�ed by the fact that as a result of our adaptive triangulation, larger triangles can beconsidered more reliable than smaller ones. More sophisticated ways have been presentedfor setting the continuity condition over the boundary [21]. In practice, however, it isvery di�cult to estimate the tangent �elds cross the boundary reliably from real noisy [1]or sparse data and consequently it is di�cult to set the coe�cients in a balanced way.3.4 Tensor product spline approximationB-splines are widely used for geometric modeling because of their local control and con-tinuity properties. Furthermore, they are included in a standard product data exchangeformat [14] which facilitates data sharing.A NURBS (Non-UniformRational B-Spline) surface is de�ned as a bivariate polynomialfunction of parameters u and v as follows:S(u; v) = nXi=1 mXj=1Bhi;jNi;k(u)Mj;l(v); (5)where Ni;k and Mj;l are the basis functions of order k and l, and the Bhi;j 's are thehomogeneous coordinates (xi;j; yi;j; zi;j; hi;j) of the control points. n and m de�ne thenumber of control points in each parameter direction. The basis functions are de�nedrecursively (see [11]). An open end condition is used.In shape estimation we have no a priori knowledge about the complexity of the underly-ing surface and consequently we do not know the size of the control point mesh (n and m in(5)) required for approximating it. We estimate n and m from data, whereas in CAD andComputer Graphics domains it is common to have the user provide this information. It is7

important to have an appropriate number of control vertices. If there are too few controlpoints, the �t is not likely to converge; conversely, if there are too many control pointsunwanted oscillations may be introduced. An initial estimate for the control point meshsize is obtained by a local surface characterization process which subdivides the surfacesinto geometrically homogeneous (planar, elliptic, hyperbolic or parabolic) patches usingthe coe�cients (L;M;N) of the second fundamental form (II = Ldu2+2Mdudv+Ndv2)from di�erential geometry [20]. The surface type is determined based on the sign of thediscriminant LN �M2. The maximum number of homogeneous surface patches in eachparameter direction yields an estimate for the number of control points needed in eachparameter direction [17]. The subsequent processing steps re�ne the estimate to improvethe accuracy.Tensor product B-spline surfaces require a rectangular parametric grid. Such a gridis provided directly in some sensor arrangements, for example, systems that use a high-precision rotating platform and express all the measurements in a single cylindrical coor-dinate system. In general, however, the samples do not form a parametric grid directlyand a resampling procedure has to be applied. The approach used here is related tothe lofting techniques used in design. A NURB curve is �t to samples on each scan-line, and the required number of samples for the parametric grid are interpolated. If thedata are scattered, a surface triangulation has to be performed and isoparametric linesinterpolated.The obtained shape model is re�ned where the error distance exceeds a user-de�nedtolerance value. The whole spline �tting procedure including the re�nement proceeds asfollows:1. An initial estimate for n and m in (5) is obtained through surface characterizationbased on di�erential geometry.2. Positions of the control points are estimated minimizing the least squares error cri-terion using (6).3. The (u; v)-parameterization is re�ned using (7).4. The positions of the control points are estimated using the new parameterization.5. If the error exceeds a user given tolerance value, a knot and consequently a controlpoint is added to that location.6. The position of the new control point is modi�ed to meet the tolerance value using(8).The locations of the control points of the approximating NURBS surface are computedby minimizing error in least squares sense. Now the Bhi;j's from equation (5) have to beestimated, and S(u; v)'s are the data points. All the weights hi;j are originally set to1:0 because the S(u; v)'s are physical measurements. Using matrix representation, thesolution is [27]: [B] = [[C]T [C]]�1[C]T [S]; (6)where elements of C are Ci;j = Ni;kMj;l, S is the matrix of data points, and B is theobtained control point mesh.The parameterization is re�ned to make the error at each point orthogonal to theapproximating surface. This provides an optimal estimate when L2 norm is used. The8

re�nement in parameter u for point Pi;j is computed as follows:unewi = uoldi + � [(ui+1 � ui�1)=2]; (7)where � = (Pi;j � S(ui; vj)) Su(ui; vj)jSu(ui; vj)j ;and Su is partial derivative with respect to u. The re�nement in parameter v is performedsimilarly. The total length of each parametric line ui and vj is normalized to 1.0. Our ini-tial estimate for the mesh size is not necessary adequate and large errors may occur wherethe surface undergoes rapid changes. Reparameterization alone may not be su�cient forobtaining the required accuracy.In areas where large errors occur, a �ner control point mesh is required. The exibility ofthe surface is increased �rst by inserting new knots, and consequently, control points usingthe Oslo-algorithm [5]. Let the unre�ned NURBS surface be de�ned using n�m controlpoints Bi;j and basis functions Qi;j(u; v) as follows: S(u; v) = Pni=1Pmj=1Bi;jQi;j(u; v). Asa result of the knot insertion, exactly the same shape is de�ned using p�q control verticesFi;j (p � n and q � m) as follows: S(u; v) = Ppi=1Pqj=1 Fi;jRi;j(u; v), where Ri;j(u; v) arethe basis functions. The positions of the new control points generated by knot insertionare then modi�ed [23] in order to obtain the required accuracy. Given a point (u; v) onthe surface, a distance D of the required modi�cation, and a direction vector V for themodi�cation, the new location F newi;j of a control point Fi;j is determined as follows:F newi;j = Fi;j + �V; where � = DjV jRi;j(u; v): (8)Curve or surface discontinuities can be introduced by inserting a knot with multiplicityequal to the order of the B-spline.In the case of approximating surfaces with branches or holes, trimmed surfaces areemployed. A trimmed B-spline is a regular B-spline surface where certain parts of thesurface are marked invalid. An example of a trimmed NURBS surface is depicted inFigure 5. The intersection curves of parametric surfaces are computed using subdivisionFigure 5: A trimmed NURBS surface.based techniques [5]. The trimming is presented in procedural form and the intersectioncan be approximated less accurately for visualization purposes and with high accuracy formachining operations (see, [22]). Trimmed surfaces are included in IGES standard andadvanced solid modelers [14, 2].4 ExamplesIn this section, examples of view registration and shape estimation are presented. Regis-tration estimates the relative rotation and translation among the views. An example ofregistration of free-form shapes is given using 4 views. The �rst triangulation is used as areference set, and the registration is performed using the barycenter of each 2-simplex fromthe next view as points pi. The surface normal at each point is obtained directly from theplane equation of the 2-simplex. Range data of the 4 views as well as the correspondingtriangulation results are illustrated in Figure 6. The image resolution is �x = �y = 1:0and �z = 0:026. All images are triangulated using tolerance value tol = 1:3. In theexperiments, the approximate rotations used as initial estimates are set 5 degrees o� from9

the actual value. Typically 100 iterations are required to obtain su�cient accuracy. Theerrors in pairwise registrations among the 4 views are given in Table 1.Figure 6: Range data of the free-form object from 4 di�erent viewpoints. The relativerotation around vertical axis between viewpoints 1 and 2 is 20 degrees, viewpoints 2and 3, 40 degrees, and viewpoints 3 and 4, 40 degrees. The Delaunay triangulationscorresponding to each view are illustrated below.Table 1: Errors in rotation (in degrees), translation and the average residual errors for 3pairwise registrations. The residual error is the the distance between each point used forregistration and the closest triangular primitive.In order to show the �nal registration result, isoparametric contours obtained fromthe free-form registration experiment above are used as an input to the NURBS �ttingprocedure. The approximating spline surface covering more than 180� view is depicted inFigure 7.Figure 7: The approximating NURBS surface for the data from the registration exampleon free-form shape. Two di�erent side views and a view from above are shown. Hiddenlines are not removed.Multiple representations are used in surface approximation in order to model variousshapes e�ciently. Widely used real range data [26] are employed in the shape estimationexamples. The image resolution is 256-by-256 pixels and 16 bit quantization is used forthe depth values. The Face Mask data are approximated in order to show the abilityto describe very complicated shapes whereas the Hand data describes a surface wherethe rectangular parametric space of a tensor product B-spline surface is not appropri-ate. Therefore, the shape is approximated using the trimmed surface representation. Thebounding curve is found by Canny step edge detector [8]. The original data sets, the ap-proximating NURBS surfaces and the domain vertices of the Bezier triangles are depictedin Figure 8.Unlike parametric surfaces, triangulations are not limited by the rectangular or spher-ical arrangement of the parametric domain. The triangular spline �tting is using a 2/3triangulation as a starting point. A constrained Delaunay triangulation would providemore reliable starting point since edges of triangles do not extend over discontinuities.The estimation of tangent planes at each vertex of the triangulation is crucial for highquality shape estimation. Mean absolute error (MAE) was computed for each triangularspline patch at barycentric coordinates (u; v; w = 13). The error �gures are given using theleast squares and robust estimates of the tangent plane at each vertex in Table 2. A moredetailed comparison in the presence of di�erent types of noise can be found in [15, 16]. Ifthe assumption on Gaussian distributed noise does not hold or if there are discontinuitiespresent in the neighborhood the robust technique produces reliable results. On smoothregions, however, the output variance is higher. In subdivided triangles, many degreesof freedom remain unset after satisfying the continuity condition which contributes toincreased variance. This is apparent especially with G1 continuous patches where thedegree of the triangles is elevated into quartic.The NURBS approximations are re�ned to meet a user-de�ned tolerance value. First,the parameterization is re�ned to make the error orthogonal to the signal. If the required10

Figure 8: The test images and the obtained triangular and NURBS approximations. Onlythe domain vertices are shown for the surface consisting of triangular Bezier patches: a)the Face Mask and b) the Hand are real range data.Table 2: Mean absolute error (MAE) values for triangular spline approximation for theFace Mask and Hand data. Least squares (LSQ) and robust estimates of tangent planesare used. The error is given before and after subdivision.accuracy is not obtained everywhere by re�ning the parameterization, new knots (andcontrol points) are inserted to such locations to increase the exibility of the surface. Thepositions of the new control points are then modi�ed. Figure 9(a) depicts the improvementin RMS error for a pro�le from the Face Mask data as the parameter re�nement is iterated.The pro�le, the approximating spline and the corresponding error distances before andafter the re�nement are shown in Figures 9(b)-9(d). The re�nement is done using tolerancevalue 0.7 mm. Three new knots are inserted, and subsequently the locations of the newcontrol points modi�ed.Figure 9: (a) The improvement in RMS error for the sample pro�le as the parameterizationis re�ned iteratively. (b) The original pro�le from the Face Mask data and the obtainedB-spline (dashed line), and (c) the corresponding error distances. (d) shows the B-splinere�nement result using 0.7mm tolerance value, and (e) the error distances, respectively.The obtained shape model is produced both in Alpha 1 [2] procedural modeling lan-guage and in IGES format. The actual part geometry is generated by executing theAlpha 1 procedure. Such procedure is also easy to modify as the design evolves.5 SummaryIn this paper, the problem of constructing geometric models from data acquired by 3-Dimaging sensors is addressed. In particular, computational solutions for free-form shapeestimation from noisy sensor data are provided. In addition, a technique for estimatingthe relative transformation among the views without a priori model is developed. Thetechnique extends the ICP algorithm by using additional constraints to �nd valid pointsfor matching.Multiple representations are used in shape estimation since at present there is no singlerepresentation that would be the most appropriate in all situations. In particular, NURBS,Delaunay triangulation and triangular spline representations are employed. The surface�tting results are re�ned by the algorithms to meet a user-de�ned tolerance value.The shape representations employed here are closely related to modeling primitivescommonly used in design and are included in a standard product data exchange format.This conformity allows the model to be imported into CAD systems and shared in aconcurrent engineering environment.The obtained shape model is often insu�cient for a good design. The design intention,the functionality of the part and the demands related to manufacturability, assembly, andmaintenance among others may be di�cult to determine from sensory measurements. Inorder to manufacture a functional part, such knowledge has to be incorporated into theproduct model and may cause changes in the shape model as well.11

Some problems that play an important role in geometric modeling from sensor dataremain to be solved. In particular, intelligent strategies for positioning the sensor withrespect the object are required. Furthermore, the segmentation of the data is an ill-posedproblem since the visible parts of an object do not necessary have a direct correspon-dence to the actual functional parts. Therefore, human intervention may be required insubdividing the data into parts.Acknowledgements. This work was partly done on a research visit to GRASP Laboratory, Uni-versity of Pennsylvania. The authors want to thank Prof. Bajcsy for allowing us to work in the GRASPLaboratory. Funding for V. Koivunen was provided by The Academy of Finland, the Foundation SuomenKulttuurirahasto, and the University of Oulu. The facilities were partly provided by Navy Grant N00014-92-J-1647, AFOSR Grant 88-0296; Army/DAAL 03-89-C-0031PRI; NSF Grants CISE/CDA 88-22719,IRI 89-06770, and ASC 91 0813; and Du Pont Corporation.References[1] N. Abdelmalek, \Algebraic error analysis for surface curvatures and segmentation of3-D range images", Pattern Recognition, Vol. 23, No. 8, (1990) 807-817.[2] \Alpha 1 User's Manual", University of Utah, 1992.[3] P. Besl, N. McKay, \A Method for Registration of 3-D Shapes", IEEE Transactionson Pattern Analysis and Machine Intelligence, Vol. 14, No. 2, (1992) 239-256.[4] L. Brown, \A Survey on Image Registration Techniques", ACM Computing Surveys,Vol. 24, No. 4., (1992) 325-376.[5] E. Cohen, T. Lyche, R. Riesenfeld, \Discrete B-Splines and Subdivision Techniquesin Computer Aided Geometric Design and Computer Graphics", Computer GraphicsAnd Image Processing, 14, (1980) 87-111.[6] L. De Floriani, E. Puppo, \An On-Line Algorithm for Constrained Delaunay Trian-gulation", CVGIP: Graphical Models and Image Processing, Vol. 54, No. 3, (1992)290-300.[7] T. DeRose, S. Mann, \An approximately G1 Cubic Surface Interpolant", in Math-ematical Methods in Computer Aided Geometric Design II, eds. T. Lyche, L. Schu-maker, Academic Press, 1992, pp. 185-196.[8] R. Deriche, \Using Canny's Criteria to Derive a Recursively Implemented OptimalEdge Detector", Int. J. of Computer Vision, (1987) 167-187.[9] G. Farin, \A Construction for Visual C1 Continuity of Polynomial Surface Patches"Computer Graphics and Image Processing, 20, (1982) 272-282.[10] G. Farin, \A Modi�ed Clough-Tocher Interpolant", Computer Aided Geometric De-sign, Vol. 2, No. 1-3, (1985), pp. 19-27.[11] G. Farin, \Curves and Surfaces for Computer Aided Geometric Design", AcademicPress, 1990.[12] O. Faugeras, M. Hebert, \The Representation, Recognition, and Locating of 3-DObjects", International Journal of Robotics Research, Vol. 5, No. 3, (1986) 27-52.[13] D. Field, \A Generic Delaunay Triangulation Algorithm for Finite Element Meshes",Adv. Eng. Software, 1991, Vol. 13, No. 5/6 combined, (1991) 263-272.[14] Initial Graphics Exchange Standard, Version 5.1., IGES/PDES Organization, 1991.[15] V. Koivunen, \A Robust Nonlinear Filter for Image Restoration", IEEE Transactionson Image Processing, (1995) in press.[16] V. Koivunen, \A robust approach to enhancement of multivariate images", 1st IEEEInternational Conference on Image Processing (ICIP), Austin, TX, USA, (1994) 462-466. 12

[17] V. Koivunen, M. Pietik�ainen, \Evaluating Quality of Surface Description UsingRobust Methods", 11th International Conference on Pattern Recognition, Vol. III,(1992) 214-218.[18] V. Koivunen, J-M. Vezien, \Multiple representation approach to geometric modelconstruction from range data", IEEE Cad-Based Vision Workshop '94, (1994) 132-139.[19] V. Koivunen, R. Bajcsy, \Spline Representations in 3-D Vision", In 1994 NSF, ARPAWorkshop on 3-D Object Representations, New York, to appear in Springer-VerlagLecture Notes on Computer Science (LNCS) Series, 1995.[20] M. Lipschutz, \Di�erential Geometry", McGraw-Hill, 1969.[21] M. Lounsbery, S. Mann, T. DeRose, \Parametric Surface Interpolation", IEEE Com-puter Graphics and Applications, Vol. 12, No. 5, (1992) 45-52.[22] N. Patrikalakis, \Surface-to-surface intersections", IEEE Computer Graphics andApplications, Vol. 13, No. 1, (1993) 89-95.[23] L. Piegl, \Modifying the shape of rational B-splines. Part 2: Surfaces", ComputerAided Design, Vol. 21, No. 9, (1989) 538-546.[24] L. Piegl, \On NURBS: a survey", IEEE Computer Graphics and Applications, Vol.11, No. 1, (1991) 55-71.[25] V. Pratt, \Direct Least-Squares Fitting of Algebraic Surfaces", Computer Graphics21, No. 4, (1987) 145-152.[26] M. Rioux, L. Cornoyer, \The NRCC Three Dimensional Image Data Files". CNRC29077, National Research Council of Canada, 1988.[27] D. Rogers, N. Fog, \Constrained B-spline curve and surface �tting", Computer AidedDesign, Vol. 21, No. 10, (1989) 641-648.[28] L. Schumaker, \Triangulations in CAGD", IEEE Computer Graphics and Applica-tions, Vol. 13, No. 1, (1993) 47-52.

13