ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27

Contents lists available at SciVerse ScienceDirect

ISPRS Journal of Photogrammetry and Remote Sensing

journal homepage: www.elsevier .com/ locate/ isprs jprs

Automatic roof model reconstruction from ALS data and 2D ground plans basedon side projection and the TMR algorithm

Jiann-Yeou Rau ⇑, Bo-Cheng LinDepartment of Geomatics, National Cheng Kung University, No. 1, University Road, Tainan City 70101, Taiwan, ROC

a r t i c l e i n f o

Article history:Available online 4 October 2011

Keywords:Roof model reconstructionTopology reconstructionAirborne laser scanningTIN-MergingReshaping

0924-2716/$ - see front matter � 2011 Internationaldoi:10.1016/j.isprsjprs.2011.09.001

⇑ Corresponding author. Tel.: +886 62757575x6383E-mail address: jyrau@maill.ncku.edu.tw (J.-Y. Rau

a b s t r a c t

This paper presents an automatic roof model reconstruction method based on the side projection of air-borne laser scanning (ALS) data. The proposed approach first detects the building’s primary orientationand decomposes multi-layer roofs into a single one. Then, 3D structural lines are detected and restoredfrom the projected point clouds. Finally, a line-based roof model reconstruction algorithm, namely TIN-Merging and Reshaping (TMR), is proposed. The originality for 3D roof modeling is to perform geometricanalysis and topology reconstruction from two 2D projections and then reshapes the roof using elevationinformation from the 3D structural lines or ALS data. Experimental results indicate a nearly 100% successrate for topology reconstruction can be achieved provided that the 3D structural lines can be enclosed aspolygons. However, the success rate of the Reshaping stage is dependent on the complexity of the rooftopstructure. With the exception of domed and multiple orientations roofs, which are not considered in thedeveloped method, we achieve success rates around 92–95%. As for absolute accuracy, less that 50 cm ofroot-mean-square error is observed in all X–Y–Z directions. The results demonstrate that the proposedscheme is robust and accurate even when a group of connected buildings with multiple layers and mixedroof types is processed.� 2011 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS) Published by Elsevier

B.V. All rights reserved.

1. Introduction eling the topology reconstruction among 3D structural lines is per-

Three-dimensional building model is one of the major compo-nents of a cyber-city and is vital for the realization of 3D GIS appli-cations. The building model is essential for true-orthophotogeneration (Habib and Kim, 2006; Rau et al., 2002), map revision,change detection, energy and property management, micro-climate and air pollution simulations, and many location-basedservices. In a photo-realistic city model, geometric building modelsare also required for the generation of façade and rooftop texture.Such models can be applied in virtual city tourism, urban planning,real-estate markets, smart cities, and among others. The generationof reliable and accurate 3D building models is crucial to accom-plish the above mentioned goals.

This study proposes an automatic roof model reconstructionmethod using ALS and 2D ground plans. The proposed roof model-ing method is designed to perform geometric analysis and topol-ogy reconstruction on two 2D projections. Then, the roof isreshaped using the elevation information obtained from the 3Dstructural lines or ALS data. The building’s primary orientationand roof structural lines are detected by projecting the ALS dataonto a pseudo vertical plane. On the other hand, during roof mod-

Society for Photogrammetry and R

9; fax: +886 62375764.).

formed on the 2D horizontal plane. The proposed scheme thusreduces the complexity of 3D roof modeling and makes the model-ing process easier. Literature related to our approach is briefly de-scribed in the following sections.

Most studies related to 3D building model reconstruction are ma-jorly discussed in the fields of computer vision, computer graphics,remote sensing and photogrammetry. A comprehensive literaturereview and comparisons can be found in Brenner (2005). They canbe categorized according to the types of user interactions (manual,semi-automatic, or fully automatic approaches), the data sourcesused (satellite, terrestrial or aerial photos, terrestrial or airborne la-ser scanning data, ground plans), the strategies adopted (data-driven, model-driven or hybrid), the extracted features (corners,ridges, eaves or regions), the generated roof types (flat, gable,hipped, complex, and so on), and the created building models (poly-hedral, B-Rep or parametric). In this study, the adopted data includeboth the ALS data and 2D ground plans. A fully automatic approach isdeveloped using a data-driven strategy for the detection of 3D struc-tural lines. The target is a group of connected buildings with mixedroof types which could be in the form of multiple or single layers. Theproduced roof models are in 3D B-Rep format.

Generally, the procedure for geometrical building modeling orcity modeling encompasses three main steps, namely (1) recogni-tion, (2) feature extraction, and (3) topology reconstruction with

emote Sensing, Inc. (ISPRS) Published by Elsevier B.V. All rights reserved.

S14 J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27

geometric modeling. Rather than automatic recognition, the mostreliable and accurate results can normally be achieved by a buildingreconstruction system that integrates human-assisted visual inter-pretation capability (Gruen and Wang, 1998). For a fully autono-mous system, the integration of multiple data sources, such asmultiple aerial images, ALS data, and 2D ground plans, might in-crease the reliability and degree of automation, but some constraintsor limitations in certain aspects are unavoidable. Examples includethe capability of handling a high density of built-up areas, occlusionsfrom trees or neighboring buildings, bad image quality due to shad-ows, weather conditions or digitized imagery, insufficient imageresolution or point clouds density, miscellaneous objects on therooftop, etc. (Baillard and Zisserman, 2000; Brenner, 2000; OudeElberink and Vosselman, 2009; Gruen et al., 2002; Haala andBrenner, 1998, 1999; Suveg and Vosselman, 2004).

The purpose of feature extraction is to retrieve 3D primitives ofbuilding structure from images or laser scanning data, includingcorners, ridges, eaves, faces, and so on. In the case of images, furtherimage matching is required to perform space intersection frommore than two images (Baillard and Zisserman, 2000). Canny(1986) Förstner (Förstner and Gülch, 1987) operators are the twomost commonly used methods in computer vision and digital pho-togrammetry for the purpose of extracting point- or line-based fea-tures (Suveg and Vosselman, 2004), while the Hough Transform(Hough, 1962) is often used for straight line detection after featurepoint detection.

In the bottom-up strategy (or data-driven method) (Brenner,2000; Rottensteiner et al., 2005; Vosselman and Dijkman, 2001),the lower level features extracted from image or ALS data are uti-lized to derive higher level models (e.g., buildings or roads). Theoperator does not required prior knowledge of the target.

In contrast, with the top-down strategy (or model-driven ap-proach) it is assumed that the operator already knows the struc-ture of the targets (Maas and Vosselman, 1999; Tseng and Wang,2003; Hammoudi and Dornaika, 2010). Therefore, a pre-defineddatabase storing several parametric building models is adoptedduring the modeling process. The operator is in charge of buildingrecognition task by selecting a suitable building model primitivefrom the database, such as gable roofs, flat roofs, hipped roofs,among others, and assigning initial values. The computation is con-ducted by a computer, meaning that some feature extraction andstereo matching techniques for finding the precise location, orien-tation and scale of the building model are performed automati-cally. Nevertheless, the model-driven strategy is constrained bythe limitations of the database. In order to create a complex build-ing model a series of building primitives need to be aggregated to-gether using a Constructive Solid Geometry (CSG)-tree.

The hybrid approach combines data-driven and model-driven ap-proaches. For example, Suveg and Vosselman (2004) utilized 2Dbuilding ground plans as well as aerial images as input data. The2D ground plans are used to constrain possible solutions by assign-ing approximate building heights and searching for conjugate fea-tures along the epipolar line between stereo images. During thebuilding reconstruction stage, some 3D volumetric primitives arecreated based on 3D corners that have been generated from featureextraction and stereo matching. Again, a complex building has tobe decomposed into several simple building model primitives thatare then aggregated together.

In automatic feature extraction using the data-driven strategy,image quality problems sometimes causes the extracted featuresdo not completely or correctly describe the roof structure. Forexample, the extracted ridges and eaves could be broken, the de-tected region might not cover the whole rooftop, there might begaps between two continuous faces, corners might not touch(shortening), or one corner might cross over its neighboring wall(Elaksher, 2002). Therefore, a strategy to rebuild this type of rela-

tionship to enclose a reasonable rooftop is essential for geometricbuilding modeling.

The purpose of topology reconstruction is to link or group thoseunrelated features to create 3D models. Most studies assume a roofmodel to be a composite of single or several planar faces. In thisstudy a TIN-Merging and Reshaping (TMR) algorithm is proposedfor the reconstruction of roof models from 3D structural lines. Asimilar approach can be found in Rau and Chen (2003) who devel-op a Split-Merge-Shape (SMS) algorithm for this purpose. In con-trast, the CyberCity Modeler (Gruen and Wang, 1998; Gruen andWang, 2001) utilizes ‘‘weakly structured point clouds’’ as input3D primitives. Plane–plane intersection for the detection of ridgeor concave structural lines from ALS data is another well-knowntechnology (Maas and Vosselman, 1999; Vosselman, 1999; Rotten-steiner et al., 2005; Chen et al., 2008). In which, Vosselman (1999)utilized roof outlines and ‘‘main building orientation’’ as con-straints to accomplish this goal. Brenner (2000) detected the planarregions from regularized DSM using the RANSAC (Random SampleConsensus) algorithm (Fischler and Bolles, 1981) and the 2Dground plan to assist in the filtering of non-reasonable faces. Onthe other hand, in the model-driven approach, since the topologyhas already been built within the primitives and the CSG-tree rep-resentation of building models, the geometric constraints areimplicitly maintained, which can reduce the complexity and ambi-guity during topology reconstruction. Therefore, the CSG represen-tation has been widely used in several research studies andcommercialized systems (Brenner, 2004; Gülch et al., 1999).

The rest of the paper is structured as follows. In the second sec-tion, the materials for the experiments are introduced. In Section 3,the method for automatic roof model reconstruction from ALS datawith 2D ground plans will be described, including a detailed expla-nation of the proposed TIN-Merging and Reshaping algorithm. InSection 4, several performance analyses are carried out using twotest datasets. Finally, some concluding remarks will be provided.

2. Materials

Two test datasets are utilized in the experiments to evaluate theperformance of the proposed methods. The first one consists of 3Dstructural lines manually measured from stereo-images while thesecond one involves the integration of 2D ground plans with theALS data.

2.1. Dataset I: manual stereo measured 3D lines

The 3D structural lines were manually measured from a pair ofaerial photographs with 60% overlap using a digital photogram-metric workstation. The photographs have a scale of 1:5000 andwere digitized with a scanner with a resolution of 25 lm which re-sults in a nominal ground sampling distance (GSD) of 12.5 cm. Thetest site is located around the Fu Jen Catholic University, Taiwan.The area is about 70 hectares in size and contains more than1000 buildings. The content of the test site can be roughly catego-rized into two parts. Part (I) is comprised of the university campusincluding mostly large and separated buildings with complexboundaries but simpler roof structures. Part (II) includes high-den-sity buildings with groups of complex roofs where industrial facto-ries, residential houses and apartments are located. Fig. 1 depictsthe measured 3D roof structure lines on the ground projection withthe university located in the upper-right region. For research pur-poses, most of the measured 3D structural lines are not complete.This means that due to the occlusion effect only the visible partsare measured and there can be large gaps between the roof eavesand neighboring buildings. This procedure is different from con-ventional stereo-mapping where the occluded eaves or corners

Fig. 1. Test dataset I of 3D structure lines on 2D horizontal projection.

J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27 S15

are inferred from one of the utilized stereo-images to minimize thegap between them. There are some examples of this situation dis-cussed in this paper.

2.2. Dataset II: ALS data and 2D ground plans

The test site is located in a rural area with industrial and resi-dential buildings. The area is about 30 hectares in size and contains115 buildings. The 2D ground plans were created in April 2002 forthe production of 1:1000 digital topographic maps. The buildingboundary was created by manual stereo-measurement from a ste-reo-pair acquired in March 2002. Major building boundaries aredescribed without detailed roof structure but include the roofoverhang and are recorded in polygon format. The planimetricaccuracy of the ground plans is around 30 cm but the height infor-mation was discarded during map compilation due to softwarelimitations. Instead, the number of floors and building type are de-noted within each building polygon, but they are not utilized inthis research.

The ALS data were acquired by Optech ALTM in April 2002 witha point density of 1.7 points/m2 and 15 cm of estimated verticalaccuracy. Fig. 2(a) illustrates the 2D ground plans and Fig. 2(b)the ALS data with false coloring distinguishing the elevation.

2.3. Reference dataset: measured from aerial stereo-images

For performance evaluation purposes, a stereo-pair of aerialimages acquired in October 2010 by an RMK-DX digital camerawith accurate aerial triangulation is used. The camera’s focal length

is 92 mm and its CCD cell size is 7.2 lm. Since, the images were ac-quired at an elevation of about 1200 m, the image scale is 1:13,000and the nominal ground sampling distance is 9.36 cm. After aerialtriangulation the root-mean-square error (RMSE) of the checkpoints is around 9 cm, thus is accurate enough for independentchecking of the generated 3D building models by using the 3Dcoordinates derived from space intersection.

3. Automatic roof model reconstruction using ALS data and 2Dground plans

There are several studies in the literature where ALS and 2DGIS/Map vector data are integrated for automatic building modelreconstruction, in which gable, pent, hipped (Haala et al., 1998;Haala and Brenner, 1999; Vosselman and Dijkman, 2001), domedand cylindrical roofs (Teo, 2008) have been treated successfully.However, the use of such methods for a group of connected build-ings with a combination of different roof types and multiple layershas not yet been discussed. The prerequisite for planar fitting usingALS data by the robust-least-squares adjustment technique is tosegment the approximate locations of each roof plane; otherwiseit is impossible to define two planes using one single plane func-tion. Thus, the goal of this research is to focus on (1) the recogni-tion and separation of each roof among a group of consecutiveroofs with/without multiple layers, and (2) the automatic recon-struction of each roof as a 3D polygon.

Fig. 3 denotes the work flow for the proposed method using ALSdata and 2D ground plans for automatic roof model reconstruction.The proposed scheme basically simulates the process that an

Fig. 2. Test dataset II: (a) 2D ground plans; and (b) corresponding ALS data indicated in false color.

S16 J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27

operator inspects the vertical profile of ALS data for ground objectrecognition and classification. The ALS data is projected onto apseudo vertical plane at a specific orientation (Schwalbe, 2004;Schwalbe et al., 2005) then extended for 3D structural line detec-tion and roof model reconstruction. There are three major stepsnamely (i) determination of the building’s primary orientation;(ii) structural line detection; and (iii) roof model reconstruction.In the beginning, the ALS data is extracted by building polygonsfrom the 2D ground plans to minimize the effect from non-buildingpoint clouds. Then, during primary orientation determination, allorientations are verified by calculating their Weighting Averageof Point Cloud Density (WD). The one which has the maximumWD is selected as the primary orientation. Furthermore, the detec-tion of multi-layer roofs is performed to separate the ALS data foroverlapped roofs into single layer roofs. Thus, the delimitation ofdifferent roof patches and roof types can be realized by means ofthe piece-wise slope variation method. Finally, the detected 2Dfeature points were restored into 3D lines and used in an innova-tive line-based 3D roof model reconstruction algorithm, i.e., TIN-Merging and Reshaping. In which, the Line-constrained DelaunayTriangulation is adopted to generate the TINs from all 3D lineson the 2D horizontal plane. The topology between these 3D linesis rebuilt by iteratively merging two TINs whose shared edge doesnot have any corresponding 3D line. The final roof type is deter-mined by a reshaping process using the ALS data with robust-least-squares adjustment.

In this study, the major target to be modeled is a group of singleorientation buildings that are continuous with the same height orconsecutive with different heights. The roof type could be a combi-nation of gable, cylindrical or flat roofs, but not including domedand hipped roofs with multiple orientations, which have been dis-cussed by other authors (Haala et al., 1998; Haala and Brenner,1999; Vosselman and Dijkman, 2001; Teo, 2008). Small roof struc-tures like dormers and chimneys are not considered as the mainbuilding structures and could be filtered during feature detectionand plane fitting.

3.1. Detection of primary building orientation

Initially, the point clouds within the polygon are collected. Thebuilding’s primary orientation is defined at a wall’s direction thatconstructs a vertical plane and the ALS data can be projected onto

it where the ALS data are clustered together with the highest pointdensity. In case of a group of buildings with different orientations,each polygon can be treated independently to reduce the complex-ity. The conception behind ALS data side projection transformationis illustrated in Fig. 4. One wall from the treated polygon(s) is cho-sen as the reference wall and is defined as the new X0-axis. The newY0-axis is defined as orthogonal in direction to the X0-axis on thehorizontal plane. This means that the Y0-axis is along the projectiondirection. The new Z0-axis remains the same as the original Z-axis.Along the X0-axis we construct a vertical X0–Z0 plane with all ALSdata projected onto it as 2D points. Along the X0-axis of the X0–Z0

plane, the Weighting Averaged of Point Cloud Density (WD) is cal-culated following the steps described below. Among all candidatewalls the one which has the maximum WD is defined as the build-ing’s primary orientation.

� Along the X0-axis, the point clouds are separated by one meterwide intervals. The number of points within the interval is Ai,where i is 1 �No. of intervals. The length of an interval is depen-dent on the width of a building, in order to distinguish the pri-mary one from the others. Generally, if the total of intervals foreach wall has more than five, it is enough for this purpose.� Centered at each ALS point, calculate the total number of points

within a search radius of 0.3 m to get NPk, where k = 1 � Ai. Alarge search radius will include too many irrelevant points forthe estimation of point density. Meanwhile, the density of thepoint clouds needs to be sufficient for the determination ofthe primary orientation, so a smaller search radius is suggested.� The range of all point clouds at the Y0-axis is estimated and

denoted as Ri.� TPi ¼

PAik¼1NPk: total number of points in each interval.

� Dj ¼ TPiAi

: point density for each interval.

� DNj ¼Dj

Rj: point density per one meter depth range (along the Y0-

axis).

� WD ¼P

DNi�DiPDi

: weighting average of point cloud density.

3.2. Detection of 3D structural line

In a congested urban environment roof structure and heightvariation is high. Some roofs may be higher than the others afterside projection transformation. On the other hand, in industrial

Fig. 3. Work flow for automatic roof model reconstruction using ALS data and 2D ground plans.

J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27 S17

areas, most of the buildings are larger and the roof structure is sim-pler. Therefore the roof structure, which appears as clusteredstraight lines after side projection transformation, is first detectedby a Global Hough Transform (Hough, 1962) using all of the pro-jected 2D ALS points. As shown in Fig. 3, the work flow in the struc-tural line detection stage is separated into two parts. One is formulti-layer roofs where some roof lines overlap others. The de-tected feature lines can be projected along the X0-axis to check ifany overlap exists. The second part is for single layer roofs whereno overlapped roofs exist. A semicircle (cylindrical roof) maybe de-tected and separated into several small lines in the current stage.

3.2.1. Robust-least-squares adjustmentAfter side projection transformation, the 2D ALS points are used

for the regression of a line equation by the robust-least-squaresadjustment (Werner, 1984) to minimize the deficiency arisingfrom these irrelevant points. For ordinary least-squares adjust-

ment, we need to minimize the weighted sum of residual squares,as shown in Eq. (1).

Xn

i¼1

piv2i ¼Min ð1Þ

In which, n is the total number of points; pi and vi are the weightand the residual for point i, respectively. Normally, the weight issetup as 1 for all points. However, for robust-least-squares adjust-ment the adopted weight is defined as a function of each point’sresidual, i.e., p(vi) which is modified according to an exponentialfunction as shown in Eq. (2). The constants used are decided byempirical examinations. The weighted sum of residual squaresnow becomes Eq. (3). The conception of robust estimation is thusto reduce the weight for those irrelevant points that have departedfrom the target geometric model, i.e., line, plane, circle, etc. Thepoint with larger residual will contribute lesser during the iterative

Fig. 4. Illustration of side projection transformation for ALS data.

S18 J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27

least-squares adjustment procedure, thus the obtained geometricmodel will be more accurate.

pðvÞ ¼1; if cvv > 300e�cvv ; if cvv � 300

�ð2Þ

where cvv ¼ 0:05�v4:4; if No: of iteration6 30:05�v3:0; if No: of iteration> 3

(and c¼ 0:05

Xn

i¼1pðv iÞv2

i ¼Min: ð3Þ

3.2.2. Single layer roofsIn this process we try to reconstruct a group of gable or cylindri-

cal roofs with irregular building boundaries. First of all, we need toseparate the ALS data into different categories that belong to oneroof patch. A slope-based feature detection method is thus devel-

Fig. 5. Slope variations for a g

oped. Two examples are illustrated in Figs. 5 and 6 for differentroof types. The upper portion shows the ALS points after side pro-jection transformation, while the lower one indicates the corre-sponding slope variation. The 2D ALS points are divided into twometer intervals along the X0-axis and move forward one meterstep-by-step which results in a one meter overlap between twoconsecutive steps. The size of the interval is dependent on thesmallest width of all target roofs and the original point cloud den-sity. A smaller value is suggested as long as high density pointclouds are available. Within each interval, the ALS points are fittedinto a single line by the robust-least-squares adjustment methodand the slope is also calculated. The resultant slope variation isused to differentiate between inclined, flat or cylindrical roofs. Asshown in Figs. 5 and 6, an inclined roof has a constant slope. Sodoes a flat roof, but with a slope close to zero, whereas a cylindricalroof has a constant change of slope. An abrupt change of slope indi-cates the change of roof patch and is used to locate rough breakpoints for dividing the ALS data with more accurate divisions thatcontain at most one roof patch only. The break points are denotedas red dashed lines in Figs. 5 and 6. Later, a corresponding functionis fitted according to its roof type. For example, the line equation isused for gable and flat roofs, while the equation for circles isadopted for cylindrical roofs and further divides it symmetricallyinto several small line segments. As a result, a set of lines are ex-tracted representing the roof structure on the projected plane.

3.2.3. Multi-layer roofsThis algorithm is designed to reconstruct a group of gable and

cylindrical roofs projected with multiple layers having rectangularbuilding boundaries. Since the slope-based feature detection meth-od described in the previous section is not suitable for the separa-tion of multi-layer roofs, a piece-wised Local Hough Transform issuggested. The ALS data are again divided into two meter intervals,overlapped by one meter for applying the Local Hough Transformto detect small lines embedded in the projected ALS data. Mean-while, the robust-least-squares adjustment is applied again to re-duce the deficiency arising from irrelevant points. These detectedfeature lines are aggregated and clustered into longer and morecomplete lines using the collinearity and overlap analysis. It meansthat if two feature lines are collinear and have overlap, they will beclassified as the same feature line. Furthermore, these clusteredfeature lines and corresponding ALS data are processed by the sin-gle layer roofs algorithm as described in the previous section. Thismeans that any consecutive gable or cylindrical roofs will be

roup of cylindrical roofs.

Fig. 6. Slope variations for a group of gable roofs.

Fig. 7. Detection of feature lines from multi-layer roofs.

J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27 S19

recognized by the slope-based feature detection technique andseparated into individual roof patches.

Fig. 7 shows an example demonstrating the performance of theproposed method. Fig. 7(a) shows the projected points. Fig. 7(b)shows the feature lines detected by the Local Hough Transform.Several small feature lines are detected, each one denoted by dif-ferent colors. However, after applying clustering they are catego-rized as the same group of feature line, as shown in Fig. 7(c).Each group is denoted by different colors.

One flat feature line is pointed out in Fig. 7(c) to emphasize theadvantage of the side projection method that a small roof withfewer point clouds and bears the occlusion effect can still be recog-nized. The reason is that the ALS points accumulate on a 2D planewhich can increase the number of observations for 2D feature linedetection and introduce higher capacity, reliability and accuracy.

3.2.4. Determination of feature pointsThe detected feature lines in the projected plane are planar roof

patches in the real world. The intersection of two neighboring in-clined feature lines (or a line-pair) at a point on the projected planecan be used to construct a 3D ridge or valley line. Meanwhile, thetwo endpoints of one feature line also represent structural lines inthe real world. These have to be considered, especially for those

structural lines that are higher and longer than their neighbors.This is to avoid duplicated lines between two roofs on the 2D hor-izontal plane to prevent the introduction of small polygons afterTIN-Merging. The results of feature point determination are shownin Fig. 8, where the blue points are the intersection points (ridgeand valley lines) and the red points are the line terminals (eaves).Please note that the two line terminals with heights lower thantheir neighbors are not considered due to the fact that the occlu-sion effect may introduce incorrect locations.

3.3. Roof model reconstruction

3.3.1. Determination of 3D structural linesPreviously detected feature points on the projected plane are

now treated by the reverse transform into 3D object space to getan infinite line. The range of this line in the object space has tobe determined in advance for further 3D roof modeling. For casesof single layer roofs, the range of structural lines can be obtainedfrom 2D ground plans. However, for multi-layer roofs, the roofmay not extend from the beginning to the end of the ground plans.The ALS data that belong to the detected 2D feature line are used toestimate its range at the Y0-axis. Two additional structural lines,which are parallel to the X0-axis, are added to confine their

Fig. 8. Results of feature point determination.

S20 J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27

boundary. They are located at the beginning and end of the ALSdata, except for those parallel and close to the ground plans onthe 2D projection and lower or shorter than its neighboring roof.On the other hand, one may detect the distribution of ALS dataand regularize the boundary as suggested by Dorninger and Pfeifer(2008).

One example of the extracted 3D structural lines with TINs gen-erated from ALS data is indicated in Fig. 9. The blue boundariesdelineate the 2D ground plans that are elevated by the averageheight of ALS data within the building. The extracted 3D structurallines are denoted as red lines in the figure. To assist with visual rec-ognition, two different 3D views are provided. One may notice thatthe additional lines parallel to the X0-axis are correctly localizedand some unnecessary structural lines are eliminated.

3.3.2. TIN-Merging and ReshapingIn this study, we assume that the roof patches can be described

by several planar regions and enclosed by roof structural lines ex-tracted from ALS data and 2D ground plans or digitized by manualstereo-measurement. The reason we propose the use of groundplans is because in this way we can specifically define the bound-ary of the major portion of a building and assist in the determina-tion of the building’s primary orientation. This method iscomparably more precise and reliable than boundaries directly de-rived from ALS data (Haala and Brenner, 1999; Vosselman andDijkman, 2001). Moreover, the complexity and diversity of the roofstructure makes a robust and reliable topology reconstructionalgorithm indispensible. Thus, we propose an algorithm based onthe derived structural lines for rebuilding roof polygons.

The TIN-Merging and Reshaping (TMR) algorithm is comprised offour main steps. The first one is a pre-processing step to repair anymeasurement errors in input structural lines or imperfect results

Fig. 9. Extracted 3D structural lines (red lines) with TINs from ALS data. (Forinterpretation of the references to color in this figure legend, the reader is referredto the web version of this article.)

from feature extraction. This can include performing right anglerectification, line collinearity adjustment, snapping of dangles fromshortening, and removal of overhanging dangles. The second step isto construct a Triangulated Irregular Network (TIN) using Con-strained Delaunay Triangulation (CDT) (Chew, 1987; Kallmannet al., 2003), where the vertices of structural lines are adopted aspoints and the structural lines themselves are used for constrainingthe generated TINs. Two neighboring TINs are iteratively merged byremoving the shared edges that have no corresponding structurallines. The resultant roof topology is reconstructed in a 2D projec-tion. Finally, we reshape the roof structure based on the rectifiedstructural lines that contain the third dimensional information(Z) to infer 3D roof models.

3.3.2.1. Definition of a line and related operations. Since the TMRalgorithm mentioned above is a line-based 3D roof modelingmethod. The definition of a line and its usage are described here.First of all, a line is determined from two end points that havethree-dimensional coordinates. However, for the purposes oftopology reconstruction on a 2D horizontal plane, the 2D normalform of the line equation is used in the TIN-Merging step, as shownin Eq. (4).

q ¼ xcoshþ ysinh ð4Þ

where h is the inclination angle of the normal to the x-axis and q isthe length of the normal. This formula is used before the adjustmentof two collinear-like lines by comparing the similarity of these twoparameters (h, q) between them. For the purpose of point-on-lineevaluation during the snapping and removal of dangles, the para-metric form of the line equation is adopted, as shown in Eq. (5),to calculate the ‘‘t’’ value. If ‘‘t’’ is within 0–1, it means that the ver-tex (x, y) from one end point of a line is between two end points (h,k) and (p, q) of the line.

X ¼ ðp� hÞt þ h; y ¼ ðq� kÞt þ k ð5Þ

For the purpose of judging whether two lines are overlapped or not,the collinearity and point-on-line conditions have to both besatisfied.

3.3.2.2. Pre-processing. Since the imperfect generation of structurallines is unavoidable during the feature extraction stage or duringmanual stereo-measurement, it is necessary to correct them beforethe construction of TINs; otherwise some illegal TINs will be gen-erated. Fig. 10(a) depicts several examples of such deficiencies, inwhich the red lines are the measured structural lines and the blue

Fig. 10. (a) Original measured (red) lines with dangles (blue); and (b) the generatedpolygons (each in a different color).

J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27 S21

dots denote the detected dangles. For example, a rectangular build-ing might be skewed, structural lines might pierce a wall or be dis-connected from a neighboring wall, two collinear-like lines mightbe distorted, multiple convergent lines might be detached, and soon. Meanwhile, when dangles exist, some illegal triangles will begenerated and the topology reconstruction will fail. Thus, thepre-processing of detected lines has to be robust and rigorous inorder to successfully reconstruct their topology. Fig. 10(b) showsthe pre-processing results, i.e., rectified structural lines, wherethe dangles were removed successfully. This procedure can be per-formed fully automatically in the developed system after rationaldetermination of several adopted parameters, such as h and q usedin Eq. (4) for collinearity verification, the maximum dangle length,and the maximum rotation angle for right-angle rectification.

3.3.2.3. Constrained Delaunay Triangulation. Delaunay Triangulation(Delaunay, 1934) is a well-known technique for constructing trian-gles from sparsely distributed points where there is no fourth pointinside its circumcircle to avoid spear-like triangles. Using this tech-nique unrelated points can be organized in such a way that neigh-borhoods are connected with topology. Delaunay Triangulation isthus useful for topology reconstruction of unrelated points. In thisstudy, the primitive for roof model reconstruction is derived fromstructural lines which have to be enclosed to define a polygon. Thegenerated TINs cannot intersect or cross over the structural lines.The endpoints of the structural lines act as points for constructingTINs on the 2D horizontal plane but are constrained by the struc-tural lines themselves using Constrained Delaunay Triangulation(Chew, 1987; Kallmann et al., 2003) to avoid triangles crossingthe structural lines. Fig. 11 illustrates the effect with and withoutapplying the line constraint in the generation of TINs. Fig. 11(a)shows the input lines and Fig. 11(b) shows the generated TINswithout the line constraint. One may notice that some created tri-angles have crossed over the original lines. Fig. 11(c) illustrates the

Fig. 11. Delaunay Triangulation wit

results after applying the line constraint. It is obvious that the useof the line constraint for TINs generation can achieve reasonableand correct topology.

3.3.2.4. TIN-Merging. After the generation of TINs, the relationshipamong the structural lines is created. The TINs are described inconvex hull. Some of them appear around concave building bound-aries do not exist in the real world and should be removed in ad-vance. Meanwhile, some shared edges between two TINs that donot exist should be eliminated as well. This can also reduce the vol-ume of data storage and present rational roof models. The clue forthe detection of existing edges is these rectified structural lines.We can merge two neighboring TINs by erasing the shared edgethat has no corresponding structural line. The TIN-Merging proce-dure is thus an iterative loop used to check for shared edges be-tween two TINs (or polygons) to verify whether there is anyoverlap or collinearity between the shared edges and the rectifiedstructural lines. If there is no corresponding line, the shared edgeswill be removed and those two TINs (or polygons) are merged asone polygon.

Fig. 12 shows an example of TIN-Merging. The CDT results areshown in Fig. 12(a). Fig. 12(b) is the results after applying TIN-Merging and Fig. 12(c) shows the results after removing the outerTINs that do not exist in the real world. It can be seen inFig. 12(c) that there is a small roof surrounded by another, thustwo additional pseudo edges (same location but different direc-tions) are added to connect each other. The edge sequence num-bers are denoted. Line numbers 8 and 13 are two pseudo edgesthat have no corresponding structural lines but are kept to describethis donut-type polygon. Since two polygons should not overlapafter topology reconstruction, the inner polygon has to be encircledby the outer one. It means that the outer polygon has to be cut bythe inner one resulting in a donut-type polygon. One may comparethe 3D view in Fig. 12(d) for clarification.

3.3.2.5. Reshaping. Before reshaping, recalling that all the aboveprocedures are processed in two-dimensional space. This reshapingprocedure utilizes the third dimensional information (Z) from theendpoints of rectified structural line or ALS data to infer the finalshape of the roof structure.

Reshaping from 3D lines: The basic idea behind constructing aroof shape (whether flat or inclined) from 3D lines is that two con-nected lines will have the same 3D coordinates at their joint andform a triangle. A triangle always located on a plane. The parame-ters of a planar function can thus be calculated by the vertices ofthe triangle.

In the beginning, all edges are classified as independent, shared orpseudo edges. The pseudo edges are created during the construction of

h and without line constraints.

Fig. 12. Example of the TIN-Merging process.

Fig. 13. Example of independent (yellow) and shared (white) edges.

S22 J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27

Delaunay Triangulation as described in the previous section. Anindependent edge means that no neighbor polygon is connected,but a shared edge has. For example, edge numbers 9–12 inFig. 12(c) are shared edges while the other edges (with the exceptionof edge numbers 8 and 13) are independent edges. The height valuefor an independent edge can be initialized and fixed by the Z-valueof its corresponding 3D line terminals. On the other hand, the heightvalue for the shared edges and pseudo edges can only be initialized,but is not yet fixed.

At the second stage, a coplanar verification process is appliedfor all edges within a polygon. This step is particularly essentialfor a roof that is taller than its surrounding roofs. In the previousstage they are assigned as shared edges, such as the inner roof de-picted in Figs. 12(c) and 12(d) highlighted by the red numbers 0 to3. If all the edges of this polygon are located on a plane, they will beclassified as independent edges with fixed heights. It is worth not-ing that currently edge numbers 9–12 in Fig. 12(c) are still re-mained as shared edges. On the other hand, for a gable roof, theridge lines are first considered to be shared edges. They will be con-sidered as independent edges by applying this coplanar verificationprocess. Fig. 13 illustrates a gable roof (the rightmost one) and aflat roof (the leftmost one) whose shape is determined at this pro-cedure. In the figure, the independent edges are depicted in yellowand the shared edges in white. The shape of the gable roof in themiddle and the small flat roof cannot be determined in the currentstage, because the initial heights of the shared edges are assignedby its neighborhood that is taller and will cause non-planarsituation.

The third step of reshaping is to search for the existence of inde-pendent edges within a polygon. Once more than two independentedges are found, the least-squares adjustment can be applied tocalculate the plane equation for this polygon and to determinethe heights of the other shared edges. These two independentedges can be connected or parallel to each other to form a triangleor a rectangle as long as they fall on a plane. The height value of theother shared edges will be adjusted and their attribute will be reas-signed as independent edges. The smaller flat roof shown in Fig. 13is an example of this case. Since three of the edges are indepen-dent, the height of the remaining shared edge can be decidedand fixed directly by least-squares adjustment.

The developed algorithm is designed for 3D structural lines de-rived from any source, such as manual stereo plotting (Rau andChen, 2003), stereo line matching (Schmid and Zisserman, 1997;Park and Zimmermann, 2000), ALS data (Chen et al., 2008), andso on. With the above inference procedures, in most cases, the roofmodels can be reconstructed successfully. However, if it is humanerror that causes structural lines to be missed, i.e., the roof bound-ary is not closed, the inference process may fail. One example willbe demonstrated in the case study with two exceptions are foundas stated below.

(i) The first is for a donut-type building, where the height of theinner roof is either lower than the outer roof or is empty.Occlusion problems make it difficult to recognize and todelineate its boundaries, in which case they will be keptand decided by the operator. One example will be discussedin the performance analysis.

(ii) The second is for a polygon with only one independent edgeremaining after the whole process. In this case, an exhaus-tive test will be performed to examine any two lines withinthe polygon, by checking their co-planarity. If any two ofthem fall on a plane, all of the other edges will be adjustedusing this plane equation and their attributes will be reas-signed as independent edges. The middle gable roof shownin Fig. 13 illustrates this situation. Although its ridge line isa shared edge, its initial height is assigned by its correspond-ing 3D line and is coplanar to the independent edge in 3Dspace. If the above conditions are not all satisfied, this roofwill normally be assigned as a flat roof using the only inde-pendent edge. However, it is suggested that the resultshould be checked again by visual inspection or utilize theALS data for reshaping, as will be discussed in the followingsection, in order to complete the roof modeling procedure.

Reshaping from ALS data by planar fitting: It is assume that whenall roofs have been separated into individual planar patches, planarfitting using ALS data can be performed accurately. However, somenon-roof points belonging to trees, fences, or small roof structuressuch as chimneys, dormers, and so on, will affect the final roofshape. Thus, assuming the ratio of irrelevant points is low, it is sug-gested to use the robust-least-squares adjustment for planar fitting(Werner, 1984) in order to minimize the deficiency arising fromthese irrelevant points and to increase the geometric accuracy ofthe roof’s structure. The robust-least-squares adjustment methodhas been explained in Section 3.2.1; however the planar equationis adopted here.

Following the example shown in Fig. 9, the TIN-Merging algo-rithm is further used to reconstruct the topology on the 2D hori-zontal plane. However, the final 3D roof shape cannot be decideddirectly from the extracted 3D structural lines and ground plansdue to the lack of accurate 3D information on the ground plans.Recalling the reshaping procedure in the TMR algorithm, the outerstructure lines are treated as independent edges, meaning that theirheight values are correct and reliable. However, currently the

Fig. 14. Reconstructed 3D roof models with two 3D viewing directions provided.

J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27 S23

height of ground plans is estimated from the average of the ALSdata within the whole group of buildings. Thus, reshaping fromALS data using the robust-least-squares adjustment is necessary.The result is shown in Fig. 14, where the small roof denoted inFig. 7(c) has also pointed out for comparison. This example alsodemonstrates that the proposed scheme can reliably and automat-ically perform 3D roof modeling from a group of connected build-ings with multiple layers and with a mixture of flat, gable andcylindrical roofs.

4. Experimental results and discussion

Before performance analysis of the proposed scheme using ALSdata and ground plans, the performance of the TMR algorithm willbe evaluated in advance by dataset I that is measured manuallyfrom the stereo-image. This ensures its suitability and stabilitywhen applying it to use 3D lines derived from the ALS data.

Fig. 16. (a) Original 3D lines; (b) after dangle removal.

Fig. 17. (a) Generated Delaunay Triangulation; (b) reconstructed polygons.

4.1. Performance analysis of the TMR algorithm using 3D lines

The reconstructed 3D building models for dataset I are illus-trated in Fig. 15. The number of dangles before pre-processingwas 1542, but after dangle removal at the pre-processing stage thisnumber was reduced to zero. This indicates that the developedpre-processing algorithm is robust and effective. The success ratefor TIN-Merging topology reconstruction is close to 100%, exceptfor the area where building boundary delineation has not beenmeasured to enclose the roof. To sum up, the total number of 3Droof models (polygons) generated is 1573. Nevertheless, the suc-cess rate for reshaping is dependent on the complexity of the roofstructure. For buildings in the university area (Part I), the reshapingsuccess rate can reach up to 98%, while for other areas (Part II, out-side the campus) it is only 90%. The average reshaping success ratefor the whole dataset is 92%. Most of the failures occurred with in-clined roofs surrounded by other buildings with varied heights androof types.

Fig. 15. Reconstructed 3D building models for dataset I.

Due to the occlusion effect, the structural lines of the roofs can-not be completely measured from the stereo-image. Their endpoints are not inferred from the other visible image or edited man-ually. Fig. 16(a) shows an example of the measured 3D lines pro-jected on the horizontal plane. The detected dangles are denotedas blue dots. The dangling effects are solved by applying the

Fig. 18. Reconstructed 3D roof models.

Fig. 19. Reshaping ambiguity due to measurement mistakes or complex roof structures.

Fig. 20. Generated 3D building models for test dataset II.

Fig. 21. Accuracy evaluation for corners and step edges.

Fig. 22. Accuracy evaluation for planar fitting.

S24 J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27

pre-processing steps, as shown in Fig. 16(b). The incomplete lineshave been extended to reach a wall or a vertex. However, some-times the extension from the dangle itself is not enough. Pleasenote that the dangles depicted in the blue dashed circles closestto the wall, which has already closed with other wall, also needto be extended in order to enclose the roof boundary.

All vertices are used as 2D points and the structural lines act asconstraints for the generation of Delaunay Triangulation. The re-sults after Constrained Delaunay Triangulation are illustrated inFig. 17(a). Several shared edges that do not exist in the real worldare removed during the TIN-Merging step. As a result, the topologyamong these structural lines can be rebuilt and the roof polygonsas well, as depicted in Fig. 17(b) with different colors. The resultsof reshaping the roof-top structure are shown in Fig. 18. ComparingFigs. 17(b) and 18, one may observe that the height of roof poly-gons A and B is the same as their neighboring polygon C. Actually,polygon C is an outdoor hallway and polygons A and B do not exist,but unfortunately the current algorithm cannot determinewhether they exist or not, or whether they are lower than theirneighbors, unless further auxiliary information is given, for exam-ple the ALS data.

The roof structures in the previous case are mostly flat. Thereshaping step is much easier when no measurement errors exist.However, human mistakes may occur. For example, in Fig. 19 thereare four gable roofs, one of which contains a measurement mis-take; more specifically is missing an eave. Thus, the inferred roofstructure could be wrong. Fig. 19 shows three possible roofs basedon the currently measured 3D lines, i.e., the red lines. This wouldbe better corrected before 3D roof modeling.

Table 1Summary of accuracy analysis I.

Number of measurements

Step edges 17Corners 14

Plane fitting Gable roof 20Cylindrical roof 24

4.2. Performance analysis of roof model reconstruction

Fig. 20 illustrates the generated 3D building models for datasetII. In total, 115 buildings with 350 roof polygons are created. Mostof the buildings are successfully generated by the proposedalgorithm, including 6 multi-layer roofs (the most complicated

RMSE (cm) Max./Min. (cm)

22.0 79.5/7.221.6 93.7/11.9

Average of Std. Dev. (cm) Max./Min. of Std. Dev. (cm)

14.9 64.3/2.15.6 38.5/1.2

Table 2Summary of accuracy analysis II.

Type of check point No. of check point (Unit: cm) X Y Z

Corners of eave 24 Mean �41.51 20.80 �34.42RMSE 37.68 27.19 41.05Max. 19.17 77.93 35.17Min. �141.88 �28.53 �127.00

Terminals of ridge line 20 Mean �41.98 16.23 �37.84RMSE 45.04 30.87 49.23Max. 32.71 79.33 29.70Min. �141.89 �59.75 �149.57

All 44 Mean �41.42 18.48 �38.56RMSE 40.69 28.67 44.45Max. 32.71 79.33 35.17Min. �147.89 �59.75 �149.57

Fig. 23. Error vectors on the horizontal plane.

J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27 S25

example is shown in Figs. 7, 9 and 14) and 15 single layer roofs(two examples are depicted in Figs. 5 and 6). However, due tothe limitations of the algorithm, reconstruction failed for 6 build-ings with multiple orientations, including one pyramidal roof.The overall success rate is 95% for the current test dataset. Twotypes of evaluation are carried out for accuracy analysis of the gen-erated 3D roof models.

4.2.1. Accuracy analysis ITwo buildings with interior roofs are purposefully removed

from the ground plans and reconstructed by the proposed scheme.The original interior roofs are used as ground truth. The accuracyevaluation results are summarized in Table 1. The positions ofthe check points for planimetric accuracy evaluation are illustratedin Fig. 21, in which ‘‘x’’ denotes the step edges and ‘‘o’’ denotes thecorners. The RMSE of the distance depart from the ground truth isabout 22 cm for both step edges and corners, but the maximum er-ror could up to 94 cm.The standard deviation after planar fitting isadopted for the evaluation of height accuracy. The averages ofstandard deviations from 20 gable roofs and 24 cylindrical roofsare 14.9 cm and 5.6 cm, respectively. Two examples are illustratedin Fig. 22, where the original point clouds and the generated 3Droof models are rendered together for comparison. The resultsdemonstrate that the proposed algorithm is effective and accurate.

4.2.2. Accuracy analysis IIIn the second type of absolute accuracy analysis, the generated

roof models are compared with the ground truth from space inter-section of manual measured conjugate points from the referencedataset. Forty-four 3D points are compared, in which 24 pointsare located at the roof eave corners and the other 20 points atthe terminal of gable roof ridge line. The statistics for accuracyanalysis are summarized in Table 2. Figs. 23 and 24 illustrate thedistribution and error vectors of those 44 points on the horizontalplane and in the vertical direction, respectively. One may observethat the overall means of the error in the X–Y directions are�41.42 cm and 18.48 cm, respectively. Meanwhile, the error vec-tors plot show a trend toward the N-W direction, meaning thatthere is bias between the generated roof model and its true posi-tion. This may be introduced due to the systematic error during da-tum transformation, because two datasets were createdindependently and separated for 8 years.

On the other hand, the RMSE is slightly greater for the ridge lineterminals than the corner of the eaves. This is acceptable andshows high accuracy of the inferred ridge line using the ALS data.In the meantime, from the overall RMSE we realize that the accu-racies of the generated roof models for each direction are all within50 cm and the absolute maximum discrepancy is less than 150 cm.This demonstrates that the proposed side projection method for

roof model reconstruction has high accuracy and good potentialin real applications.

5. Conclusions

This paper presents an automatic building modeling techniquefor certain types of buildings based on ALS data and 2D groundplans. The geometric analysis and topology reconstruction are per-formed on two different 2D projections in order to reduce the com-plexity of the problem. An innovative line-based 3D roof modelingalgorithm called TIN-Merging and Reshaping is the core to rebuildthe topology between roof structure lines and reshaping the rooftype.

The performance evaluation shows that an almost 100% successrate can be achieved with the proposed TIN-Merging step for 2Dtopology reconstruction, utilizing a high degree of incompletemanually measured 3D structural lines, provided that the roofboundary are well enclosed. However, the success rate for reshap-ing depends on the complexity of the roof structure. A 98% reshap-ing success rate is observed for buildings located within the

Fig. 24. Error vectors in Z direction.

S26 J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27

university, where the buildings are large and their roof structuressimpler. However, the success rate is only 90% for an area withmixed residential and industrial buildings where the roofs aremostly small and surrounded by others with great variation inheight and roof type.

The side projection method is originally from previous research(Schwalbe, 2004; Schwalbe et al., 2005) and is extended for singleor multi-layer roofs in this research. This method is proposed forroof structure line detection using automatic building modelingfrom ALS data and 2D ground plans. The advantage of side projec-tion is that the ALS point clouds will be clustered on the 2D planewhich can increase the number of observations and the capabilityfor feature line detection making more reliable and accurateperformance.

The 2D ground plan is used to select the ALS data within abuilding and to assist in the determination of the building’s pri-mary orientation. The Weighting Averaged of Point Cloud Densityis proposed for the determination of the building’s primary orien-tation. During the feature line detection stage, a slope-based fea-ture extraction technique is used for single layer roofs, while thepiece-wise Local Hough Transform is suggested. However, theyare treated as single layer roofs when the feature lines are detectedand clustered. In the end, the previously developed TIN-Mergingtechnique is adopted for topology reconstruction, whereas the roofshape is determined from the ALS data using the robust-least-squares plane fitting method.

The experimental results demonstrate that the side projectionmethod is effective for feature detection of a group of single orien-tation buildings with/without multiple layers but with mixed rooftypes. It is particularly advantageous for small roof patches withless point clouds and bear with the occlusion effect. Based on theresults of accuracy analysis I, the average accuracy of the standarddeviation after ALS planar fitting is better than 15 cm. This indi-rectly proves that the accuracy of the roof structure lines detectionand planar fitting is high. In accuracy analysis II, the ground truth ismeasured manually from a high-resolution and high-accuracy

stereo-image, to obtain an overall absolute accuracy of less than50 cm after roof model reconstruction.

In summary, the determination of the building’s primary orien-tation, the detection of the feature lines of roof patches and thereconstruction of the topology can be performed successfully.The proposed scheme is designed for single orientation buildingsand a 95% success rate is achieved for roof model reconstructionof dataset II.

In future research, the side projection method may be extendedfor buildings with multiple orientations especially with irregularbuilding boundaries. If the point cloud density is sufficient andthe required accuracy is lower, the neglect of 2D ground planscould be considered. The developed TIN-Merging and Reshapingalgorithm is useful for 3D roof modeling and flexible enough to de-rive structural lines from different data sources. As long as the 3Dstructural lines are enclosed to define a roof boundary withoutmisinterpretation of its structure, the reliability and success ratewill be high. This algorithm can also be integrated on any digitalphotogrammetric workstation (DPW) for digital topographicalmapping, particularly for polyhedral 3D object modeling, such asbuildings, lakes, land-use parcels, and so on.

Acknowledgments

This research was financially supported by the National ScienceCouncil, Taiwan. The authors are grateful to Prof. L.C. Chen of theCenter for Space and Remote Sensing Research, NCU, Taiwan, forproviding valuable advice and test datasets. Thanks also to Mr.Hsu of Asia GIS & GPS Co., Ltd., Taiwan, for providing the stereo-images in performance analysis II. Special thank to the editor andanonymous reviewers for their constructive criticisms and valu-able suggestions.

References

Baillard, C., Zisserman, A., 2000. A plane-sweep strategy for the 3D reconstruction ofbuildings from multiple images. International Archives of Photogrammetry andRemote Sensing 33 (PartB2), 56–62.

Brenner, C., 2000. Towards fully automatic generation of city models. InternationalArchives of Photogrammetry and Remote Sensing 33 (PartB3), 85–92.

Brenner, C., 2004. Modelling 3D objects using weak CSG primitives. InternationalArchives of Photogrammetry, Remote Sensing and Spatial Information Sciences35 (PartB3), 1085–1090.

Brenner, C., 2005. Building reconstruction from images and laser scanning.International Journal of Applied Earth Observation and Geoinformation 6 (3-4), 187–198.

Canny, J., 1986. A computational approach to edge detection. IEEE Transactions onPattern Analysis and Machine Intelligence 8 (6), 679–698.

Chen, L.C., Teo, T.A., Kuo, C., Rau, J.Y., 2008. Shaping polyhedral buildings by thefusion of vector maps and Lidar point clouds. Photogrammetric Engineering andRemote Sensing 74 (9), 1147–1157.

Chew, L.P., 1987. Constrained Delaunay Triangulations. In: Proceedings of the ThirdAnnual Symposium on Computational Geometry. Waterloo, Ontario, Canada,pp. 215–222.

Delaunay, B., 1934. Sur la sphère vide, Izvestia Akademii Nauk SSSRSur la sphèrevide, Izvestia Akademii Nauk SSSR. Otdelenie Matematicheskikh iEstestvennykh Nauk 7, 793–800.

Dorninger, P., Pfeifer, N., 2008. A comprehensive automated 3D approach forbuilding extraction, reconstruction and regularization from airborne laserscanning point clouds. Sensors 8 (11), 7323–7343.

Elaksher, A., 2002. Building Extraction from Multiple Images. ETD Collection forPurdue University.

Fischler, M.A., Bolles, R.C., 1981. Random sample consensus: a paradigm for modelfitting with applications to image analysis and automated cartography.Communications of the ACM 24 (6), 381–395.

Förstner, W., Gülch, E., 1987. A fast operator for detection and precise location ofdistinct points, corners and centres of circular features. In: Proceedings of ISPRSConference on Fast Processing of Photogrammetric Data. Interlaken,Switzerland, pp. 281–305.

Gruen, A., Wang, X., 1998. CC-Modeler: a topology generator for 3-D city models 1.ISPRS Journal of Photogrammetry and Remote Sensing 53 (5), 286–295.

Gruen, A., Wang, X., 2001. News from CyberCity-Modeler. In: Baltsavias, E.P., Grüen,A., Van Gool, L. (Eds.), Automatic Extraction of Man-Made Object from Aerialand Space Images (III). Birkhäuser Verlag, pp. 93–102.

J.-Y. Rau, B.-C. Lin / ISPRS Journal of Photogrammetry and Remote Sensing 66 (2011) S13–S27 S27

Gruen, A., Steidler, F., Wang, X., 2002. Generation and visualization of 3D-city andfacility models using CyberCity Modeler. In: Proceedings of MapAsia 2002.Bangkok, p. 12 (15 August, on CD-ROM).

Gülch, E., Müller, H., Läbe, T., 1999. Integration of automatic processes into semi-automatic building extraction. International Archives of Photogrammetry andRemote Sensing 32 (Part 3-2W5), 177–186.

Haala, N., Brenner, C., 1998. Interpretation of urban surface models using 2Dbuilding information. Computer Vision and Image Understanding 72 (2), 204–214.

Haala, N., Brenner, C., 1999. Virtual city models from laser altimeter and 2D mapdata. Photogrammetric Engineering and Remote Sensing 65 (7), 787–795.

Haala, N., Brenner, C., Anders, K., 1998. 3D urban GIS from laser altimeter and 2Dmap data. International Archives of Photogrammetry and Remote Sensing 32(Part 3), 339–346.

Habib, A., Kim, C., 2006. LIDAR-aided true orthophoto and DBM generation system.Innovations in 3D Geo-Information Systems, Lecture Notes in Geoinformationand Cartography. pp. 47–65.

Hammoudi, K., Dornaika, F., 2010. A featureless appraoch to 3D polyhedral buildingmodeling from aerial images. Sensors 11 (1), 228–259.

Hough, P., 1962. Methods and Means for Recognizing Complex Patterns US Patent3,069,654.

Kallmann, M., Bieri, H., Thalmann, D., 2003. Fully dynamic constrained Delaunaytriangulations. In: Brunnett, G., Hamann, B., Mueller, H. (Eds.), GeometricModelling for Scientific Visualization. Springer-Verlag, pp. 241–258.

Maas, H., Vosselman, G., 1999. Two algorithms for extracting building models fromraw laser altimetry data. ISPRS Journal of Photogrammetry and Remote Sensing54 (2-3), 153–163.

Oude Elberink, S., Vosselman, G., 2009. Building reconstruction by target basedgraph matching on incomplete laser data: analysis and limitations. Sensors 9(8), 6101–6118.

Park, H.J., Zimmermann, P., 2000. Colour image matching for DTM generation andhouse extraction. International Archives of Photogrammetry and RemoteSensing 33 (Part B3), 16–23.

Rau, J.Y., Chen, L.C., 2003. Robust reconstruction of building models from three-dimensional line segments. Photogrammetric Engineering and Remote Sensing69 (2), 181–188.

Rau, J.Y., Chen, N.Y., Chen, L.C., 2002. True orthophoto generation of built-up areasusing multi-view images. Photogrammetric Engineering and Remote Sensing 68(6), 581–588.

Rottensteiner, F., Trinder, J., Clode, S., Kubik, K., 2005. Automated delineation of roofplanes from lidar data. International Archives of Photogrammetry, RemoteSensing and Spatial Information Sciences 36 (Part 3/W19), 221–226.

Schmid, C., Zisserman, A., 1997. Automatic line matching across views. In:Proceedings of IEEE Computer Society Conference on Computer Vision andPattern Recognition. San Juan, Puerto Rico, 17–19 June, pp. 666–671.

Schwalbe, E., 2004. 3D building model generation from airborne laserscanner databy straight line detection in specific orthogonal projections. InternationalArchives of Photogrammetry, Remote Sensing and Spatial Information Sciences35 (Part B3), 249–254.

Schwalbe, E., Maas, H., Seidel, F., 2005. 3D building model generation from airbornelaser scanner data using 2D GIS data and orthogonal point cloud projections.International Archives of Photogrammetry, Remote Sensing and SpatialInformation Sciences 36 (Part 3/W19), 209–214.

Suveg, I., Vosselman, G., 2004. Reconstruction of 3D building models from aerialimages and maps. ISPRS Journal of Photogrammetry and Remote Sensing 58 (3-4), 202–224.

Teo, T.A., 2008. Parametric reconstruction for complex building from LiDAR andvector maps using a divide-and-conquer strategy. International Archives of thePhotogrammetry, Remote Sensing and Spatial Information Sciences 37 (PartB3a), 133–138.

Tseng, Y.H., Wang, S., 2003. Semiautomated building extraction based on CSGmodel-image fitting. Photogrammetric Engineering and Remote Sensing 69 (2),171–180.

Vosselman, G., 1999. Building reconstruction using planar faces in very high densityheight data. International Archives of Photogrammetry and Remote Sensing 32(Part 3/2W5), 87–92.

Vosselman, G., Dijkman, S., 2001. 3D building model reconstruction from pointclouds and ground plans. International Archives of Photogrammetry, RemoteSensing and Spatial Information Sciences 34 (Part 3/W4), 37–44.

Werner, H., 1984. Automatic gross error detection by robust estimators.International Archives of Photogrammetry and Remote Sensing 25 (Part A3),1101–1108.