Efficient registration of multitemporal and multisensor aerial images based on alignment of...

AansfidtetttdssetcatazaI

1Irhecmra

P42

1

Journal of Electronic Imaging 19(1), 013002 (Jan–Mar 2010)

J

Downloaded Fro

Efficient registration of multitemporal and multisensoraerial images based on alignment of nonparametric

edge features

Sokratis MakrogiannisGE Global Research

Visualization and Computer Vision LabOne Research Circle

Niskayuna, New York [email protected]

Nikolaos G. BourbakisWright State University

College of Engineering and Computer ScienceAssistive Technology Research Center

3640 Colonel Glenn HighwayDayton, Ohio 45435-0001

bstract. The topic of aerial image registration attracts consider-ble interest within the imaging research community due to its sig-ificance for several applications, including change detection, sen-or fusion, and topographic mapping. Our interest is focused onnding the optimal transformation between two aerial images thatepict the same visual scene in the presence of pronounced spatial,emporal, and sensor variations. We first introduce a stochasticdge estimation process suitable for geometric shape-based regis-ration, which we also compare to intensity-based registration. Fur-hermore, we propose an objective function that weights the L2 dis-ances of the edge estimates by the feature points’ energy, which weenote by sum of normalized squared differences and compare totandard objective functions, such as mutual information and theum of absolute centered differences. In the optimization stage, wemploy a genetic algorithm scheme in a multiscale image represen-

ation scheme to enhance the registration accuracy and reduce theomputational load. Our experimental tests, measuring registrationccuracy, rate of convergence, and statistical properties of registra-

ion errors, suggest that the proposed edge-based representationnd objective function in conjunction with genetic algorithm optimi-ation are capable of addressing several forms of imaging variationsnd producing encouraging registration results. © 2010 SPIE and

S&T. �DOI: 10.1117/1.3293436�

Introductionmage registration is the process of determining spatial cor-espondences between two images of the same scene thatave been acquired under varying conditions. It is usuallymployed as an intermediate stage in image analysis oromputer vision pipelines with applications in several do-ains, such as biomedical imaging,1–3 remote sensing,4,5

obot vision, motion detection and guidance systems,6–8

nd stereo vision.9 In the remote sensing field, it is fre-

aper 08160RR received Oct. 16, 2008; revised manuscript received Nov., 2009; accepted for publication Dec. 2, 2009; published online Feb. 17,010.

017-9909/2010/19�1�/013002/15/$25.00 © 2010 SPIE and IS&T.

ournal of Electronic Imaging 013002-

m: http://spiedigitallibrary.org/ on 03/07/2014 Terms of Use: http://spiedl.org

quently being used for change detection,10 topographicmapping, sensor fusion, and urban planning among others.More specifically, in change-detection applications, the ob-jective is to find the differences between two images of thesame scene that have been taken from variable viewpoints,at different times, using different sensors.11,12

The selection of the most suitable registration methodrelies on the nature of the problem and the imposedrequirements.13 The main categories of registration prob-lems have traditionally been divided into the following: �i�Template registration, which takes place when a specificpattern has to be matched with an image,14 �ii� viewpointregistration, which has to compensate for variations that arecaused by image acquisition from different viewpoints,8,15

�iii� multisensor registration, which is the registration ofimages acquired by different sensors,2,16–18 and �iv� multi-temporal registration, where the matching process is ap-plied to images taken from the same scene but at differenttimes.19,20

In general, the image-registration problem may be de-fined as follows:

Definition 1: Let SR be a set of points in the referenceimage R and SI the corresponding set in the misalignedinput image I so that SR,SI�S, where S is the domain ofthe image spatial coordinates S= ��x ,y� ,x ,y�N2� �in thenonrestrictive 2-D case�. The registration task is to deter-mine the optimal spatial transformation �ST� in the param-eter space P that minimizes the dissimilarity measureDST�R,I� between the two images that is defined in the setof real numbers R.

Existing registration approaches may be typically classi-fied into correlation-based, spatial frequency domain, andnumerical optimization-based approaches. The correlation-based methods, which appeared first in the literature, weremostly employed for template matching.14,21,22 According

Jan–Mar 2010/Vol. 19(1)1

/terms

tbbsdicTrHid

otfttdvvblctmatsdcn

otaemviafabn

1Aifbmsc

1Taiott

Makrogiannis and Bourbakis: Efficient registration of multitemporal and multisensor aerial images…

J

Downloaded Fro

o these methods, a statistical similarity measure is appliedetween the two test images. The objective is to find theest match of the template and the tested image usually byhifting and rotating the template in the reference spatialomain. Common template-matching metrics of the pastnclude the cross-correlation, cross-covariance, correlationoefficient, and the sum of absolute or squared differences.hese methods are more suitable for small translations and

igid transformations, even in the presence of white noise.owever, correlation-based measures are not sufficient for

mages corrupted by other types of noise or illuminationifferences.

Under these circumstances, it is more suitable to carryut the registration in the Fourier domain.23–26 According tohese approaches, specific properties of the Fourier trans-orm related to the shift theorem are utilized to account forranslation, rotation, and scale differences. In addition tohat, they perform well in the occurrence of frequency-ependent noise and handle the illumination changes andariations due to different sensors efficiently. Another ad-antage of these methods is that their execution times maye optimized using FFT. On the other hand, a usual prob-em of Fourier-based techniques is the aliasing effect thatan be resolved using windowing operations. Furthermore,hese methods are limited to specific well-defined transfor-

ations, mainly translation and rotation. When local mis-lignments are present �as, for example, in temporal regis-ration� or when the images contain different parts of theame scene, these methods present shortcomings. Theseifficulties can be resolved by proper utilization of numeri-al optimization-based registration methods as describedext.

The optimization-based registration methods make usef a numerical optimizer and an image similarity functiono determine the spatial correspondence between two im-ges. Previous reports have shown that methods of this cat-gory may efficiently address complicated forms ofisalignment.8,9,22,27,28 These approaches can be subdi-

ided into the intensity- and feature-based groups. Accord-ng to the first group, the original intensity or color-pixelttributes are readily employed, followed by a spatial trans-ormation and estimation of dissimilarity between theligned and reference images. On the contrary, feature-ased methods rely on a feature extraction step as describedext.

.1 Extraction of Salient Image Featurest this stage, specific features are extracted from the two

mages to facilitate the matching process. Usually theseeatures are related to intensity, edge, texture, or region-ased estimates.3–7,14,17,27,29–32 The goal of this step is toeasure features that can distinguish the points of interest,

uch as object corners, lines, or shapes, and can be effi-iently matched in the following steps.

.2 Spatial Transformationhe spatial transform is defined as the function that oper-tes on the domain of spatial coordinates, STP:SI→SR us-ng the parameter set P. The type of transformation dependsn the requirements set by the application of image regis-ration. This transform may also be defined as a functionhat maps the spatial coordinates to the n-dimensional real



domain Rn, where n is the dimensionality of the imagespace, followed by a resampling operation that producesthe transformed image values. Some common models arethe conformal, affine, projective, and polynomial trans-forms. Nevertheless, these global models cannot addressthe registration of nonrigid objects, such as human or ani-mal organs and body structures, with sufficient accuracy.These problems require the use of locally adaptive transfor-mation models,13 which are better known as deformationfields, and represent a popular research area in the medical-imaging and computer-vision community with typical ap-plications including computational anatomy3,31 and shape-based analysis.33

1.3 Estimation of the Dissimilarity MeasureThis function serves as an objective measure of the correct-ness and accuracy of a spatial transformation that maps theinput image to the reference spatial domain. In this context,our goal is to find the set of spatial transformation param-eters Pmin that minimizes the selected dissimilarity measureD

Pmin = arg minPi

�D�R��,I�STPi�� , �1�

where i is the index that spans the spatial transform param-eter space, Pi is the transformation parameter set at i,STPi�� is the transformed set of spatial coordinates �using Pi, and Pmin denotes the optimal set of transformationparameters.

Another significant component of every registrationscheme is the search strategy, which seeks the global opti-mum of the dissimilarity measure in the parameter spacedetermined by the transformation model. Previous optimi-zation strategies include the exhaustive search, gradient-based approaches, the Simplex algorithm, or more ad-vanced approaches, such as simulated annealing,Levenberg-Marquardt,34 TRUST,35 and geneticalgorithms.36,37

Further categorization of registration methodologiesoriginates from implementation considerations and user-defined requirements. For example, a potential solution fortime-sensitive applications that require optimization of alinear registration model would be the utilization of theRadon transform.23,38 This is the transform defined by sum-ming up the pixel values along multiple directions. In mostcases, the Radon transform is estimated over a number ofarbitrary spatial orientations and the optimization problemis projected from the original image space to a number of1-D feature spaces.

This work deals with registration of aerial images thatinclude viewpoint, temporal, and sensor variations. Be-cause of the different physical characteristics of varioussensors, the relationship between the intensities of match-ing pixels is often complicated and not known a priori.Features present in one image might appear only partiallyin the other image or not appear at all. Contrast reversalmay occur in some image regions while not in others, andmultiple intensity values in one image may map to a singleintensity value in the other. Furthermore, imaging sensorsmay produce considerably dissimilar images of the samescene when configured with different imaging parameters.The misaligned images might also include variable terrains

Jan–Mar 2010/Vol. 19(1)2

/terms

daItutststsetstrtfa

Setepctoia


J

Downloaded Fro

ue to temporal variability, and in addition to that, thelignment process is frequently required to be automated.n order to overcome difficulties related to intensity varia-ions, we first introduce a stochastic edge representationsing kernels for nonparametric probability density estima-ion. We then propose a registration cost function that ispecific to this geometric representation and compare it tohe mutual information �MI� and an L2 distance-based mea-ure for validation. An overview of the registration testbedhat we use in our comparisons is outlined in Fig. 1, wheretarting from the coarsest scale the spatial transform param-ters are estimated using genetic-algorithm-based optimiza-ion and the approximate solution serves as the center of theearch space in a multiscale optimization scheme. The con-ributions of this work are the multivalued edge-based rep-esentation based on nonparametric probability density es-imation and its associated dissimilarity function that aimor the alignment of image pairs characterized by consider-ble temporal, sensor, and viewpoint variations.

The structure of this paper is organized as follows: Inec. 2, we describe the proposed registration scheme withmphasis on the considerations of image representation andhe associated similarity measures. Section 3 presents ourxperimental methodologies for measuring different as-ects of the proposed scheme, such as the registration ac-uracy, convergence rate to the final solution, and compu-ational complexity. A discussion of the applicability andriginality of the previously presented ideas is also reportedn the same section. Finally, some conclusions are madend future goals are presented in Sec. 4.

Fig. 1 Outline of the employed registration metdifferent image representations and registration



2 Proposed Registration ApproachIn this section we introduce the main components of ourimage registration scheme �i.e., the data representation,similarity measure, search optimization algorithm, and mul-tiscale processing�. Our interest is concentrated on theoriginal parts of this work �i.e., the edge-based shape rep-resentation and the feature-weighting similarity function�.

2.1 Image RepresentationThe image data are regularly represented by the originalpixel values, wavelet parameters, or content-based descrip-tors such as edge or region maps and graphs. The originalpixel representation is sufficient for simpler registrationproblems; however, when multitemporal or multimodalvariations occur, other representations may be used to fa-cilitate the optimization process. In this work, we propose astatistical edge estimation approach for producing a shape-based representation of the image content that is detailednext.

2.1.1 Minimal density edge map estimatorThe fundamental problem of edge detection in a multivari-ate space is to find the boundary between two homoge-neous regions, where homogeneity may be defined andmeasured using intensity or color information. In contrastto conventional edge detection methods, which try to local-ize intensity or color gradients, we introduce a method thatproduces a probabilistic edge map using nonparametricprobability modeling and estimation. Assuming that the im-

gy that serves as a test bed for comparisons ofnctions.

hodolocost fu

Jan–Mar 2010/Vol. 19(1)3

/terms

agsaob

mucCasspitaleopeppmlippspMpwddsk

Fmf


J

Downloaded Fro

ge field can be locally approximated by homogeneous re-ions or edges, we proceed to the observation that thehape of the probability distribution of the image intensityt the edge areas is approximately bimodal, whereas areasf uniform intensity are characterized by a unimodal distri-ution as explained in Ref. 39.

Hence, homogeneous regions tend to form clusters in theultivariate space characterized by a high concentration of

niformly distributed intensity values. A basic feature indi-ating the degree of homogeneity is the cluster density.lusters of neighboring regions occupy different locations,nd the in-between space is identified by the absence ofamples and, thus, by low-probability density in the featurepace, as also indicated in Ref. 39, where an analogousrobability density function �pdf� model was employed formage denoising filters. The boundary edge line shouldherefore be positioned at these low-density areas and ide-lly located at minimum density. According to this formu-ation, instead of trying to locate intensity gradients, thedge detection problem is interpreted as seeking intensityr color points of discontinuity identified by their low-robability density values. Usually pixels are located onither of the two homogeneous levels—that correspond todf modes—and in general none of the existing sampleoints takes on low pdf values. In order to locate this mini-um, a resampling of the density function at an appropriate

ocation is required. In this work, the pdf of the imagentensity samples, which belong to a local area of the imagelane, is estimated on the sample mean, and this estimaterovides an implicit measure of the bimodality of theamples distribution, which consequently indicates theresence of an edge according to our previous assumption.ore specifically, in the case of a bimodal distribution, the

robability density of the mean will be relatively low,hereas it is expected to be significantly higher when theistribution is approximately unimodal. A case of bimodalistribution is depicted in Fig. 2, where the probability den-ity is approximated at the samples’ mean using Parzenernels as described next.

ig. 2 One-dimensional example of kernel �dotted lines� nonpara-etric density estimation �solid line� of five intensity sample points

orming an edge.



Edge detection filters are commonly implemented usinga k�k window W that slides over the image plane. If W ispositioned over a homogeneous area of the image, then thehistogram of the samples within W is expected to be uni-modal, whereas if W lies on the border between two uni-form areas, the histogram should have bimodal shape be-cause it will include intensity samples from two differentregions. In the unimodal case, a high-density value is ex-pected everywhere within the sample domain defined by W.In the second case, although the pdf might take on rela-tively high values at the sample points, it is expected tohave low values in the valley between the two peaks �asseen in Fig. 2�. Now, regarding the estimation of probabil-ity density of the intensity samples within W, there is alarge number of methods40 to choose from. Although thehistogram calculation might suffice for several applications,here we employ a nonparametric kernel method because ofthe small sample size and the presence of noise in our data.Nonparametric techniques have the advantage that they donot make any explicit assumptions about the underlyingdistribution, but the data sample itself is used as a model. Anonparametric method widely used by the statistical com-munity is the Parzen window or multivariate kernel.40 Theunknown density p�X� is estimated by a sum of radial ker-nel functions centered at the sample points, and all contri-butions are averaged to produce a smoothed version of thetrue pdf. A probability density estimate is thus available forevery point in the sample space.

Let Xi, i=1:N, be intensity samples that are locatedwithin the spatial window W in the image’s spatial domain,and correspond to independent and identically distributed

observations. The density value pN�X� produced by the N

sample vectors Xi in position X may be estimated by

pN�X� =1

Nhd�i=1

N

K� X − Xi

h , �2�

where K is the employed kernel function, h is the standarddeviation of the kernel, and d is the dimensionality of the

employed feature space. In our application, d=1 and X is

determined by the samples’ mean estimate �i.e., X=1 /N�i=1

N Xi�. A common choice for the kernel function Kis the second-order multivariate Gaussian, which may alsobe regarded as a spherically symmetric pdf,

K�X� = �2��−d/2� exp�−X2

2 . �3�

The performance of the kernel density estimator definedin Eq. �2� is measured using the distance between the trueand estimated densities. The consistency of this operatordepends on the L1 or L2 error criterion, and it was ob-served that, in the L2 approach, the tail behavior of a den-sity function becomes less important. It also turns out thatthe choice of h is much more significant for the quality ofthe estimate than the choice of K. The h parameter deter-mines the kernel’s decrease rate with distance, in otherwords, the smoothness of the kernel function. In the presentapplication, the image can be considered as the superposi-tion of a deterministic signal plus noise; therefore, the ker-

Jan–Mar 2010/Vol. 19(1)4

/terms

numm

bstwtptH

pdttwa

2Atcififwbbo�m

2Ai


J

Downloaded Fro

el width must be smaller than the sharpest detail in thenknown target density. If h has a big value, then the esti-ate will be too smooth, causing undesirable blurring andight not reveal structural features, such as an existing

imodality. If it is too small, then the estimate pN�X� willuffer from statistical instability, not being able to filter outhe random variability of the sample �due to noise�. In thisork, a global value for h is used. Its value is mainly de-

ermined by the finest detail in the image that needs to bereserved �signal characteristics� and is increased thereaf-er, when noise is present, according to noise variance.ere, we typically use h=20 for kernel width k=5.

Each density estimate pN�X� is attributed to the centerixel of window W, which slides over the complete imageomain to generate the final edge map. This edge estima-ion algorithm produces a geometric image representationhat will be used in conjunction with a shape feature–eighting cost function to address multisensor variability

s described next.

.2 Similarity Measuresccording to the previous section, a fitness function needs

o be optimized in order to establish the optimal spatialorrespondence. Because our goal is to achieve efficientmage registration, an effective cost function has to be de-ned in order to express the �dis�similarity of the trans-ormed image in the reference image spatial domain. Heree introduce a cost function that estimates the similarityetween two edge maps by means of L2 distances denotedy sum of normalized squared differences �SNSD�, and weutline the MI and sum of absolute centered differencesSACD� functions, two commonly used image similarityeasures that will be used for comparisons in Sec. 3.

.2.1 SNSD�dis�similarity measure for the edge-based representation

s introduced here. It originates from the idea that the con-

Fig. 3 Graphs of �a� the normalized squaredfunctions, applied to the edge- and intensity-bnoting that NSD introduces feature weighting in



tribution of each point to the objective function should beweighted by the significance of the corresponding feature.Therefore, points with higher edge probability or strengthshould be attributed bigger weights in the calculation of thetotal image similarity cost.

Assuming that � is a point in the reference coordinatespace � and STp� . �, the spatial transformation model thatmaps �� to STp��, the local dissimilarity estimateat each point � is defined as the L2 distance between theedge estimates of the spatially transformed input imageIE�STp� . �� where IE� . � denotes the input image� and thereference image RE� . �, divided by the sum of their energy.The summation of the local �i.e., pixelwise� dissimilarityestimates over � defines the overall cost function SNSD

SNSDSTp�RE,IE� = �

��

�IE�STp�� − RE��2

IE�STp��2 + RE��2 . �4�

This objective function may be considered as a hybridfeature-and image-based criterion. This is because the sig-nificance of each pair of points is weighted by an energyterm that is explicitly derived from their probability of be-ing feature points, while all image pixels are still used forcalculating the overall degree of similarity. Therefore, anadvantage of this approach is that it does not require afeature selection process, which is often complicated andsensitive to noise and outliers. It is also worth noting thatSNSD is expected to be more efficient when applied tomultivalued edge representations. In this application, wegenerate the edge maps from the reciprocal of Eq. �2� suchthat larger values correspond to higher edge likelihood. Theeffect of the energy weighting term renders SNSD suitablefor registration of intermodal imaging data because it willbe minimized when the edge estimates are aligned. Thegraph of the pixelwise normalized squared differencesfunction, which is utilized in Eq. �4�, is depicted in Fig.3�a�.

ces �NSD� and �b� absolute differences �AD�mage representations respectively. It is worthsimilarity estimation process.

differenased ito the

Jan–Mar 2010/Vol. 19(1)5

/terms

2Tdbitrsa

M

ssaeiv

2T

S

w

attpE

2

AtfottmSctcadcvtfistmif


J

Downloaded Fro

.2.2 MIhis is a statistical similarity measure that estimates theegree of dependence of A and B by measuring the distanceetween the joint distribution PA,B�a ,b� and the statisticallyndependent case of PA�a� and PB�b�.2 In the image regis-ration field, A and B denote the variables of intensity of theeference R and input I image. The best matching corre-ponds to the maximum value of this quantity. The MI isccordingly estimated by the following expression:

ISTp�R,I� MIA,BSTp = �

a�

b

pAB�a,b�log� pA,B�a,b�pA�a� · pB�b� .

�5�

This estimation of MI may produce several artifacts,uch as multiple local optima, that complicate the task ofearch optimization.19 More robust pdf estimation may beccomplished by interpolation, or nonparametric densitystimation. MI is considered to be a reliable image similar-ty measure for radiometric, multisensor, and multitemporalariations.2,19

.2.3 SACDhis measure is expressed by

ACDSTp�R,I� = �

��

�I�STp�� − ISTp− R�� + R� , �6�

here R and I are the reference and input images, while R

nd ISTpare the respective expected values calculated over

he overlapping area of the image pair. The reciprocal ofhis function operates as a fitness measure. The graph of theixelwise absolute differences function that is calculated inq. �6� is shown in Fig. 3�b�.

.3 Genetic Algorithm (GA)-Based OptimizationApproach

significant stage of the registration process is related tohe search for the optimal parameter set of the spatial trans-orm. This is frequently complicated by the occurrence ofutliers, digitization effects, noise, and illumination varia-ions that affect the imaging process. In order to overcomehe above difficulties, a numerical optimization scheme

ay be employed to define the optimal transformation.ome early approaches to this problem, such as relaxation,lustering, and hierarchical search algorithms, were limitedo simple variations or were characterized by impracticalomputational complexity. Although several optimizationnd feature extraction methods have been proposed to ad-ress these limitations,13,35,36 the optimization task becomeshallenging when a significant amount of uncorrectedariations occurs; therefore, it remains an open researchopic. In our scheme, the genetic algorithm41 is used fornding the optimal registration parameters in the globalearch space. The employed search space corresponds tohe class of rigid spatial transformations combined with iso-

etric scaling, which is sufficient for a number of aerialmage registration problems. Moreover, the complicatedeature selection process is not necessary in this approach;



the overlapping areas of the examined images are comparedinstead. A more detailed explanation of this methodologyfollows.

2.3.1 GA principlesThe Genetic Algorithm approach is a stochastic method,very well suited for optimization problems in severalfields.41,42 The initial search space is reduced using prin-ciples of the evolution theory in order to optimize a user-defined fitness function as defined next.

Definition 2: Let f be the function that we seek optimi-zation for, G the domain of genes or chromosomes, and X agene that belongs to G, X�G. The function f maps valuesfrom the parameter space G to the domain R of real values,f :G→R. The goal of optimization is to find the gene Xminthat minimizes f , Xmin=arg min�f�Xi��, where i spans thesearch space.

It is obvious that the optimization problem follows thesame formulation as that of image registration �see Defini-tion 1�. Previous theoretical studies show that genetic tech-niques provide a highly efficient heuristic for informationgathering in noncontinuous search spaces.41 It has also beenindicated that GAs produce reliable optimization results inpractice. The classic gradient search is more efficient forproblems that satisfy tighter constraints �i.e., continuity,low dimensionality, unimodality�. Nevertheless, the GAsoutperform both gradient search and other forms of sto-chastic search, e.g. simulated annealing, on more compli-cated problems, close to real conditions, such as discontinu-ous, noisy, high-dimensional and multimodal objectivefunctions.35,36,41 In an interesting connotation, the imageregistration field includes discontinuous problems due topartial area overlapping, difficulties caused by the noiseand illumination variations that take place during imageacquisition, multimodal complexities that are introduced bysensor variations, and multidimensional difficulties due tothe use of several image cues in the registration process.Next, we explain the details of GA application to this opti-mization problem.

2.3.2 Parameter settings and encodingThe input variables of the genetic algorithm known as chro-mosomes or genes are bit strings that contain the param-eters of the spatial transformation. In order to constructthese bit strings, we need first to define the parametermodel of the search space, which in our application is theconformal mapping model, a member of the family of lin-ear transformations. The parameters may now be encodedfrom the real value format to the bit-string representationby means of

Bj =�2Lj−1 − 1� · �Cj − Cj

min�Cj

max − Cjmin , �7�

where j is the index in the parameter space, Bj is the deci-mal number that is subsequently represented by binary for-mat, Lj is the granularity of input parameter Cj that takes onreal values in the employed search space, and �Cj

min,Cjmax�

is the range of values of Cj. Figure 4 displays an exampleof the affine parameter representation and coding for onegeneration of k genes. After the range and granularity of the

Jan–Mar 2010/Vol. 19(1)6

/terms

icst

2TtcbeTaApgvitrlgobrttvvhefacctnt

2Itreefmst

Frp


J

Downloaded Fro

nitial search space has been determined, the population ofhromosomes is selected randomly in the initial searchpace using a uniform probability distribution. A fraction ofhe good solution is selected while the rest is eliminated.

.3.3 GA operationshe chromosomes that remain are combined according to

he three basic operations of reproduction, mutation, androssover. More specifically, reproduction is the operationy which the strings that produce high fitness values �orquivalently, lower costs� remain in the next generation.his is carried out by ranking the gene performance andssigning higher selection probabilities to the best genes.fter the reproduction has been completed, a mutationrobability is assigned to each chromosome of the currenteneration. When a mutation operation occurs, a randomalue is chosen for the corresponding bit string. Crossovers the random combination of the best strings coming fromhe previous generation that is accomplished by exchangingandom parts between successive genes in the current popu-ation. Crossover is executed for the first N genes of theeneration, where N is proportional to probability of cross-ver. Two points are selected, and the parts of the bit stringsetween these points are exchanged. We note here that theeproduction and crossover operations are responsible forhe searching capability of genetic algorithms. The muta-ion operation prevents the genetic algorithms from con-erging to a local solution by randomly changing the binaryalue at a location in the bit string. After each generationas been created, the fitness values of the current genes arestimated. In the image registration context, the fitnessunction corresponds to an image similarity measure. Thebove optimization process is iterated until the algorithmonverges to the final solution according to the terminationriterion, which is typically related to convergence proper-ies, for example, when the same solution is repeated for aumber of consecutive generations that was specified byhe user.

.4 Transformation Modeln most of the literature, the registration problem is limitedo addressing relatively small variations in translation andotation. In this work, an effort is carried out to solve morextended variations in scale, rotation, and translation. Themployed search space is therefore defined by a rigid trans-orm plus isometric scaling that is also known as conformalapping. This transform originates from a simplified ver-

ion of the affine model without the skew and shear opera-ions, and has proven to be sufficient for the image mis-

ig. 4 The spatial transformation parameters are encoded and rep-esented by binary strings to produce the gene populations that arerocessed by the GA algorithm.



alignments of our application. Both these transformationmodels are outlined next for the sake of completeness.

2.4.1 Affine transformA point of the input image spatial domain SI—representedin homogeneous coordinates �i.e., �x ,y ,1��—is mappedonto the registered image domain SR by means of spatialtransform STp, such that SR=STp�SI�. This operation mayalso be expressed by the matrix product

SR = �Tr� · �Sc� · �Rt� · �Sk� · SI, �8�

where Tr, Sc, Rt, and Sk represent the matrices of transla-tion, scaling, rotation, and skew, respectively.6 The param-eter space AP= �tx , ty ,sx ,sy ,� , sk� is the domain of affineparameters that cover several forms of misalignments �i.e.,translation, scale, rotation, shear, and skew. It should benoted that affine transforms are linear because they pre-serve straight lines. When the shear and skew operationsare omitted, the corresponding transform is called confor-mal mapping and its parameter set is represented by CMP= �tx , ty ,s ,��. A synthetic example of this transformationmodel is illustrated in Fig. 5.

Another consideration when applying a spatial transformis the implementation of the resampling process. An inter-polation scheme is normally required for several arbitraryvalues of the transform coefficients. In this work, the resa-mpling is carried out using B-spline interpolation.34 It isalso worth noting that the presented registration scheme canbe readily extended to other linear transformation models.The selection of a transformation model is contingent onthe nature and requirements of each application.

2.5 Hierarchical Multiscale StructureAs mentioned in the previous sections, the GA-based opti-mization scheme is considered suitable for ill-posed regis-tration problems that include extensive scaling and rotationdifferences. Even so, the results of this process still need tobe refined to achieve sub-pixel registration, that is essentialfor processes that typically follow the image registrationstage, such as change detection.12,33 This level of accuracymay be accomplished by increasing the number of genera-tions in the GA optimization; nevertheless, this policywould introduce additional computational cost. In thiswork, we employ a multiscale representation scheme usinga hierarchical Gaussian pyramid. In previous works, it wasconcluded that the generation of multiple scales is advan-tageous for various image analysis processes, includingregistration.43 Some standard hierarchical multi-resolutionapproaches of the literature are Wavelet schemes and

Fig. 5 A synthetic example of two aerial images that are character-ized by scale, rotation, and translation misalignments.

Jan–Mar 2010/Vol. 19(1)7

/terms

GstGst

s

iaccociotTfdtsrpbpwtt

3

Ttttntpgwltaabtn


J

Downloaded Fro

aussian or Laplacian pyramids.34 Multiscale processingchemes have been used in biomedical imaging applica-ions with outstanding results as well.3 Here we employ aaussian pyramid that has two goals: �i� find the coarse

olution �affine parameter set� in reduced time and �ii� op-imize the registration results as the scales become finer.

The overall process may be divided into the followingteps:

1. A Gaussian image pyramid is constructed.2. Start from the coarsest toward the finest scale.3. The GA registration algorithm is applied.4. The Affine Parameters are stored.5. The search space range is reduced by a factor of 2

using as reference point �center value� the detectedregistration solution.

6. The overall process is completed after the finest scaleis processed, and the input image is resampled basedon the final parameters.

One of the main advantages of this hierarchical schemes the computational efficiency because it is a top-downpproach �i.e., an approximate optimum is found in theoarsest scale, while enhanced registration accuracy is ac-omplished in finer scales�. At the intermediate scale�s�, inrder to further localize the optimization process in the vi-inity of the previously found solution, a portion of thenitial gene population is randomly generated around theptimized parameters. These genes are generated by addi-ion of Gaussian noise to the previous optimization result.he standard deviation of this noise was determined to be a

raction of the search space range �typically, 1 /8�. We pro-uce 30%of the initial population using this strategy, whilehe remaining genes are generated in the previously definedearch space. At the finest scale �i.e., the original image�, aeduced number of iterations may be used because an ap-roximate solution of the optimization function has alreadyeen found. The multiscale processing often leads to sub-ixel capability—even for extensive misalignments—asill be shown in Sec. 3. A disadvantage of this strategy is

hat the initial solution is critical for the final result; thushe final accuracy relies on the initial estimation.

Experiments and Discussion

he proposed scheme was tested on several aerial imageshat include viewpoint, temporal, and multisensor varia-ions. In addition to that, the test images include differenterrain parts and, in several cases, the image quality is sig-ificantly different between the reference and input images;hus, the registration task is further complicated. Here weresent results on three sets of test images that belong toroups denoted by A–E, which include images acquiredith different sensors �visual and thermal� of variable reso-

utions and intrinsic parameters. Table 1 displays the sensorypes and years of acquisition for a subset of our test im-ges that are used here for evaluating the aforementionedpproaches. This evaluation was carried out qualitativelyy visual assessment of alignment correctness and quanti-atively by a measure of registration accuracy as describedext.



3.1 Registration Consistency MeasureThe assessment of registration accuracy is relatively simplewhen ground-truth data are available �i.e., the affine param-eters are previously known�. Conversely, in the absence ofground truth, alternative strategies have to be followed. Anindicative measure of registration accuracy in this case isthe registration consistency1 �RC�. This measure requiresthe estimation of forward and inverse affine registrationparameters that is followed by a composition operator. Theinput image is subsequently resampled using the resultingparameters, and a spatial distance measure is averaged tocalculate the final difference on the overlapping area. Theestimation of RC between the reference R and the input Iimage is carried out by

RC�R,I� = �1/NR� · ��

�� − STR,I � STI,R�� , �9�

where STR,I and STI,R are the forward and inverse spatialtransformations between the input and reference images, ois the composition operator, � is a point in the referencecoordinate space �, and NR is the area of overlapping pix-els in the reference image space.

Our experimental comparisons were carried out on thebasis of data representation, similarity measures, and theuse of single or multiple scale information. Considering the

Table 1 Image-acquisition information that indicates the type of tem-poral and sensor variations in the utilized data set.

Image Year Mode

A1 1974 Visual

A2 1974 -��-

A3 1978 -��-

A4 1974 -��-

T_A1 1974 Thermal

B1 1978 Visual

B2 1978 -��-

B3 1985 -��-

B4 1978 -��-

C1 1975 -��-

C2 1974 -��-

D2 1974 -��-

D4 1975 -��-

D5 1978 -��-

T_D1 1985 Thermal

E1 1974 Visual

E2 1974 -��-

Jan–Mar 2010/Vol. 19(1)8

/terms

dntAStomrmscssdea


J

Downloaded Fro

ata representation, we tested the algorithm using the origi-al intensity values and geometrical features derived fromhe edge estimation process that was previously introduced.part from that, we compared the similarity measures ofACD, MI, and SNSD. The final comparison element was

he potential use of a pyramid hierarchical scheme to carryut registration in multiple scales. Following this experi-ent methodology, Figs. 6–9 depict qualitative registration

esults, while Figs. 10–14 show graphs of our quantitativeeasures. More specifically, in Figs. 6–9 are displayed

ome representative test images from our data sets, theirolor-coded, edge-based representations, and the corre-ponding registration results. It can be seen that althougheveral variations are present in the original images due toifferent sensors, times of acquisition, and viewpoints, thedge-based representation considerably reduces the sensornd temporal variations in order to assist the registration

Fig. 6 �a� First set of images denoted by A1, Aedge map representations. �Color online only.�

Fig. 7 �a� Representative registration results us�left to right, reference image A3 and aligned Ausing multiple scales �left to right, reference im



process. Besides that, Figs. 10–14 provide an objectiveoverview of registration capabilities and facilitate compari-sons described next.

3.2 Similarity Measure ComparisonsFigures 10 and 11 display the graphs of objective functionsversus registration parameters in the search space. For theintensity representation, the MI and SACD were compared�Figs. 10�a� and 11�a��. We conclude from these graphs thatthe MI measure produces a smoother curve with fewer lo-cal minima than SACD. In addition, the SNSD functionwas tested on the edge-based image representation andcompared to the previous two with respect to registrationaccuracy. SNSD seems to be more amenable for optimiza-tion using this representation �Figs. 10�b� and 11�b��. Thisobservation becomes more obvious for the image pair A1,

4, T_A1 �left to right� and �b� their color-coded

and original pixel values using multiple scales4� and �b� SNSD on edge map representationand aligned A2 and T_A1�.

2, A3, A

ing MI1 and Aage A1

Jan–Mar 2010/Vol. 19(1)9

/terms

Aapa

Raigapvn


J

Downloaded Fro

3 in Fig. 11, which corresponds to multiple types of vari-bility �i.e., scale, rotation, translation, and temporal� com-ared to A1, A2 �Fig. 10�, which includes translational vari-bility only.

In another series of experiments, we used the quantity ofC to evaluate the accuracy of our registration algorithmpplied to pairings �Nimagepairs=15� coming from the 17 testmages in Table 1. The image pairs were determined afterrouping the original data with respect to the image scenesnd the types of variations �i.e., temporal, sensor, and view-oint�. The images in Table 1 are grouped according to theisual scene; for example, images with identifiers begin-ing with the letter A correspond to the same scene. A

Fig. 8 �a� Second set of images denoted bycolor-coded edge map representations. �Color o

Fig. 9 �a� Representative registration results us�left to right, reference image B1 and aligned Busing multiple scales �left to right, reference im

ournal of Electronic Imaging 013002-1


statistical evaluation of these results is displayed in the boxplots of Fig. 12. The boxes denote the range of RC mea-surements with respect to the utilized cost function. Thecentral line of each box plot denotes the median estimate,and the box boundaries correspond to the median absolutedifferences. The statistical outliers are symbolized by plussigns. The outlier detection criterion calculates the distanceof each measurement from the median divided by the stan-dard deviation of the samples. Figure 12�a� corresponds tothe original intensity-based image representation, whereasFig. 12�b� shows results from the shape-based approach. Itis worth noting that SACD was not included in the com-

, B3, and B4 �from left to right� and �b� theirnly.�

and original pixel values using multiple scales3� and �b� SNSD on edge map representationand aligned B2 and B4�.

B1, B2nline o

ing MI2 and Bage B1

Jan–Mar 2010/Vol. 19(1)0

/terms

patiae


J

Downloaded Fro

arison of edge-based registration �Fig. 12�b��, because it issimilarity measure developed for original image registra-

ion only and SNSD was not included in Fig. 12�a� becauset is an edge-based cost function. From this experiment, welso concluded that the MI and SNSD are more efficient,specially when using the multiscale scheme; nevertheless,

Fig. 10 Comparison of the objective function vation of images A1 and A2 on the �a� original andspace. The MI-based registration scheme in thishas a smoother curve and fewer local minima—

Fig. 11 Comparison of the objective function vation of images A1 and A3 on the �a� original andspace. SNSD shows favorable optimization procomplexity.



MI is computationally demanding and more sensitive tonoise. We have also carried out experiments to compare thecomputational loads of SNSD and MI and the average com-putational times from repeated experiments on our datasets, where Ntest=Nimagepairs�Ntrials=15�10=150, are

resented as normalized costs� for the registra-e-based representation in the employed searchple is better suited for optimization—because itACD and, to a smaller extent, SNSD.

resented as normalized costs� for the registra-e-based representation in the employed searchcompared to MI in this example of increased

lues �p�b� edg

examthan S

lues �p�b� edgperties

Jan–Mar 2010/Vol. 19(1)1

/terms

TwsTiwotsbwt

csrttsio


J

Downloaded Fro

SNSD=67�16 s and TMI=217�41 s on a PC platformith Intel Xeon processor at 2.4 GHz and 3 GB of RAM

uggesting 224% improvement in computational speed.he reason for this substantial improvement is that SNSD

nvolves computationally efficient arithmetic operations,hereas MI requires the calculation of intensity histogramsf the two compared images. The total execution time ofhis system may be significantly reduced by replacing theophisticated GA optimization algorithm with a simpler oneased on gradient descent or Marquardt-Levenberg, whichould nevertheless compromise the convergence resilience

o noise and discontinuities in the search space.The experimental results depicted in Figs. 10–12 indi-

ate that the combined use of the edge-based shape repre-entation and SNSD is capable of producing as efficientegistration results as MI at a considerably lower computa-ional cost. An additional advantage of this edge represen-ation is that it captures the basic geometry of the imagecene, alleviating the need for radiometric calibration ofmages coming from different sensors. Another interestingutcome is that the global parametrization of the transfor-

Fig. 13 GA convergence estimated by the Biasusing different settings of image representationedge-based representation leads to faster cintensity-based.

Fig. 12 Box plots of the registration consistencrepresentation for the dataset of Table 1. It becacterized by better registration accuracy. In adshape-based representation and the multiscalevalues and reduced variability.



mation in conjunction with the edge representation leads toefficient registration of temporal discrepancies that includeterrain, shadow, and viewpoint variations. Nevertheless, itshould be noted that the edge detection parameters mightneed to be calibrated according to the application domain,because the accuracy of edge estimation is crucial for thefinal registration results especially in the coarser scales.Overall, these experimental comparisons suggest thatSNSD represents a reliable registration cost function withfeature-based properties that can be readily employed in ahierarchical scheme.

3.3 Search AlgorithmAs stated previously, when no ground-truth data are avail-able, the registration process is further complicated. In thiswork, to facilitate the search optimization, an approximateestimation of the ground truth was carried out by means ofground control point registration �i.e., manual selection ofpoint correspondences on the reference and input data�. Thesearch space was defined by a percentage of variations

re for the image pairs �a� A1, A2 and �b� B1, B2age similarity measures. We observe that the

ence of the optimization algorithm than the

ure using �a� original pixel values and �b� edgeobvious that the multiscale schemes are char-the multiscale SNSD approach applied to theoriginal pixel intensities produce lower median

measuand im

onverg

y measomesdition,MI on

Jan–Mar 2010/Vol. 19(1)2

/terms

a3zpttitTtevptpcmave0dmtoSoct

3Arcvctsds2dtm


J

Downloaded Fro

round the approximate ground truth and was initially set to0% of the overall search space in order to test the optimi-ation capabilities. Considering the GA implementation, theopulation consists of 120 chromosomes. Efficient registra-ion results were also produced when the range was raisedo 50% by increasing the generation size. If each parameters encoded using 9 bits—as was typically the case—thenhe length of the total bit string would be 4�9=36 bits.he reproduction and crossover operations are applied with

he corresponding probabilities Pc and Pm, iteratively. Afterxtensive experimentation, it was concluded that faster con-ergence can be reached for Pc=0.9 and Pm=0.05. Theroposed method regularly converged within 5000 trials tohe final solution using Elite search, which preserves andasses the best matching gene to the next generation. Theonvergence rate of the GAs may be evaluated by the Biaseasure, which indicates the average percentage—or prob-

bility of each position to have the most prominent binaryalues. For example, a Bias of 0.8 means that on averageach position takes the value “0” or “1” with probability.8. In Fig. 13 are displayed the graphs of bias for twoifferent image pairs, using the SNSD, MI, and SACDeasures combined with original or edge map representa-

ion. One should note that because GAs seek maximizationf the fitness function we use the reciprocals of SACD andNSD, whereas MI does not need to be reciprocated in theptimization step. From Fig. 13, it is interestingly con-luded that the edge representation is more efficient inerms of convergence rate than the original intensity values.

.4 Single versus Multiple Scale Processings expected, the use of visual information from multiple

esults enhances the registration accuracy and reduces theomputational load. Table 2 displays the final optimizationalues as a function of the number of scales in the hierar-hical scheme and total number of trials in GA optimiza-ion. From Table 2 is concluded that as the number ofcales grows, more accurate registration results are pro-uced. It is interesting to note that even for a relativelymall number of trials—around 2000, which corresponds to8 generations—efficient registration results may be pro-uced by the multiscale methodology. Moreover, in Fig. 14,he convergence of GAs is displayed for single-scale and

ultiscale registration. A number of 20,000 trials is used

Fig. 14 Optimization values �SACD function� fo�from scales� structure for the same number oscale strategy shows improved optimization cap



for the single-scale version, while the multiscale employsfour scales with 5000 trials each to carry out a fair com-parison. It is obvious that the multiscale version yields bet-ter optimization results using the SACD function and simi-lar results were also observed for SNSD and MI.Considering the execution time, the single-scale versiontakes approximately 273�52 s, while the multiscalescheme 67�16 s, leading to significant improvement inthis regard, too. Similar conclusions are drawn from theother images of the employed data set. On the other hand,this pyramidal scheme is dependent on the initial estimationof affine parameters from the coarsest scale, which may notinclude some meaningful visual detail. As a result, thenumber of generated scales should be determined from thespatial resolution and level of details in the acquired im-ages. It is worth noting that the presented scheme showssubpixel capabilities in many cases, which is quite encour-aging considering the type and extent of variations that arepresent in the employed test sets. The edge-based shaperepresentation is more effective for addressing sensor andtemporal variability and the registration accuracy is furtherenhanced by the introduction of multiscale processing.

4 Conclusions and Future Research

A linear registration scheme is proposed in this work andtested on aerial imaging data. The registration problem inthis case comprises of combined viewpoint, multitemporal,and multisensor registration that include local variations.We propose to address this problem by the use of a multi-valued edge representation using nonparametric probabilitydensity estimation coupled with an adaptive feature weight-ing objective function denoted by SNSD. To the best of ourknowledge, this pairing of a multivalued stochastic edgeestimator with a feature weighting function has not beenreported previously for image registration applications. Dif-ferent data representations �intensity, edge-based� and simi-larity measures �MI, SACD� were implemented and com-pared in a registration test bed that uses GA optimization ina hierarchical fashion. From these experiments, it was con-cluded that the proposed elements of this work producedencouraging results at a moderate computational cost. It isalso worth noting that the proposed methodology has beensuccessfully tested for single- and multimodal rigid regis-

pair A1, A2 using �a� single- and �b� multiscaletrials �registration cost evaluations�. The multi-

r imagef totalability.

Jan–Mar 2010/Vol. 19(1)3

/terms

tr

tards

R

1


J

Downloaded Fro

ration of biomedical brain images by further delimiting theange of the search space in the optimizer input.10,32

Our future goals include the use and comparison of spa-ial graph structures and statistical tests to estimate the im-ge similarity, the consideration of different multiscale rep-esentation schemes �scale-space, feature domain�, and theevelopment of a change detection approach based on thischeme.

eferences1. M. Holden, D. L. G. Hill, E. R. E.Denton,J. M. Jarosz, T. C. S. Cox,

T. Rohlfing, J. Goodey, and D. J. Hawkes, “Voxel similarity measuresfor 3-D serial MR brain image registration,” IEEE Trans. Med. Im-aging 19, 94–102 �2000�.

2. F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens,“Multimodality image registration by maximization of mutual infor-mation,” IEEE Trans. Med. Imaging 16, 187–198 �1997�.

3. D. Shen and C. Davatzikos, “HAMMER: hierarchical attributematching mechanism for elastic registration,” IEEE Trans. Med. Im-aging 21�11�, 1421–1439 �2002�.

4. Y. Bentoutou, N. Taleb, K. Kpalma, and J. Ronsin, “An AutomaticImage Registration for Applications in Remote Sensing,” IEEETrans. Geosci. Remote Sens. 43�9�, 2127–2137 �2005�.

5. D. Liu, P. Gong, M. Kelly, and Q. Guo, “Automatic registration ofairborne images with complex local distortion,” Photogramm. Eng.Remote Sens. 72�9�, 1049–1060 �2006�.

6. L. Shapiro and G. C. Stockman, Computer Vision, Chapter 11, Pren-tice Hall, 2001.

7. L.-K. Shark, A. A. Kurekin, and B. J. Matuszewski, “Developmentand evaluation of fast branch-and-bound algorithm for feature match-ing based on line segments,” Pattern Recogn. 40�5�, 1432–1450�2007�.

8. S.-D. Wei and S.-H. Lai, “Robust and efficient image alignmentbased on relative gradient matching,” IEEE Trans. Image Process.15�10�, 2936–2943 �2006�.

9. B. D. Lucas and T. Kanade, “An iterative image registration tech-nique with an application to stereo vision,” in Proc. of 7th Int. JointConf. on Artificial Intelligence, Vancouver, pp. 674–679 �1981�.

0. X. Dai and S. Khorram, “The effects of image misregistration on the

Table 2 Optimization results �using SACD� witstructure and the number of trials in GA for ima

A1,A2

Number of trials

6000 96

4000 96

2000 95

B1,B2

Number of trials

6000 97

4000 97

2000 97

E1,E2

Number of trials

6000 95

4000 95

2000 89



accuracy of remotely sensed change detection,” IEEE Trans. Geosci.Remote Sens. 36�5�, 1566–1577 �1998�.

11. N. Rowe and L. Grewe, “Change detection for linear features in aerialphotographs using edge-finding,” IEEE Trans. Geosci. Remote Sens.39�7�, 1608–1612 �2001�.

12. J. R. G. Townshend, C. O. Justice, C. Gurtney, and J. McManus,“The impact of misregistration on change detection,” IEEE Trans.Geosci. Remote Sens. 30�5�, 1054–1060 �1992�.

13. L. G. Brown, “A survey of image registration techniques,” ACMComput. Surv. 24�4�, 325–376 �1992�.

14. T. Kim and Y.-J. Im, “Automatic satellite image registration by com-bination of matching and random sample consensus,” IEEE Trans.Geosci. Remote Sens. 41�5�, 1111–1117 �2003�.

15. Q. Zheng and R. Chellappa, “A computational vision approach toimage registration,” IEEE Trans. Image Process. 2�3�, 311–326�1993�.

16. G. Hermosillo, C. Chefd’Hotel, and O. Faugeras, “Variational meth-ods for multimodal image matching,” Int. J. Comput. Vis. 50�3�, 429–343 �2002�.

17. M. Irani and P. Anandan, “Robust multi-sensor image alignment,” inProc. of Int. Conf. on Computer Vision, pp. 959–966 �1998�.

18. Y. Keller and A. Averbuch, “Multisensor image registration via im-plicit similarity,” IEEE Trans. Pattern Anal. Mach. Intell. 28�5�,794–801 �2006�.

19. H.-M. Chen, P. K. Varshney, and M. K. Arora, “Performance of mu-tual information similarity measure for registration of multitemporalremote sensing images,” IEEE Trans. Geosci. Remote Sens. 41�11�,2445–2454 �2003�.

20. V. Murino, U. Castellani, A. Etrari, and A. Fusiello, “Registration ofvery time-distant aerial images,” in Proc. of Int. Conf. on ImageProcessing (ICIP 2002), Vol. 3, pp. 989–992 �2002�.

21. R. Brooks, T. Arbel, and D. Precup, “Anytime similarity measures forfaster alignment,” Comput. Vis. Image Underst. 110�3�, 378–389�2008�.

22. B. K. P. Horn, Robot Vision, MIT Press, Cambridge, MA �1986�.23. C. Cain, M. M. Hayat, and E. E. Armstrong, “Projection-based image

registration in the presence of fixed-pattern noise,” IEEE Trans. Im-age Process. 10�12�, 1860–1872 �2001�.

24. C. D. Kuglin and D. C. Hines, “The phase correlation image align-ment method,” in Proc. of IEEE Conf. on Cybernetics and Society,pp. 163–165 �1975�.

25. B. S. Reddy and B. N. Chatterji, “An FFT-based technique for trans-lation, rotation and scale-invariant image registration,” IEEE Trans.

ect to the number of scales in the hierarchicalrs A1-A2, B1-B2, and E1-E2.

Number of scales

3 2 1

959.2 954.1 943.5

957.8 953.4 938.9

952.0 944.5 928.9

Number of scales

3 2 1

970.5 970.5 967.8

970.3 969.6 967.5

970.5 965.6 964.1

Number of scales

3 2 1

950.5 934.4 912.9

951.1 952.0 910.6

944.4 943.4 906.6

h respge pai

4

1.0

0.8

9.2

4

0.6

0.7

0.6

4

1.7

2.4

9.9

Jan–Mar 2010/Vol. 19(1)4

/terms

http://dx.doi.org/10.1109/42.836369

http://dx.doi.org/10.1109/42.836369

http://dx.doi.org/10.1109/42.563664

http://dx.doi.org/10.1109/TMI.2002.803111

http://dx.doi.org/10.1109/TMI.2002.803111

http://dx.doi.org/10.1109/TGRS.2005.853187


http://dx.doi.org/10.1016/j.patcog.2006.10.022

http://dx.doi.org/10.1109/TIP.2006.877500

http://dx.doi.org/10.1109/36.718860

http://dx.doi.org/10.1109/36.718860

http://dx.doi.org/10.1109/36.934092

http://dx.doi.org/10.1109/36.175340

http://dx.doi.org/10.1109/36.175340

http://dx.doi.org/10.1145/146370.146374

http://dx.doi.org/10.1145/146370.146374



http://dx.doi.org/10.1109/83.236535

http://dx.doi.org/10.1023/A:1020830525823

http://dx.doi.org/10.1109/TPAMI.2006.100


http://dx.doi.org/10.1016/j.cviu.2007.09.011

http://dx.doi.org/10.1109/83.974571

http://dx.doi.org/10.1109/83.974571

http://dx.doi.org/10.1109/83.506761

2

2

2

2

3

3

3

3

3

3

3

3

3

3


J

Downloaded Fro

Image Process. 5�8�, 1266–1271 �1996�.6. H. Stone, M. Orchard, E.-C. Chang, and S. Martucci, “A fast direct

Fourier-based algorithm for subpixel registration of images,” IEEETrans. Geosci. Remote Sens. 39�10�, 2235–2243 �2001�.

7. J. W. Hsieh, H. Y. M. Liao, K. C. Fan, M. T. Ko, and Y. P. Hung,“Image registration using a new edge-based approach,” Comput. Vis.Image Underst. 67�2�, 112–130 �1997�.

8. M. D. Pritt, “Automated subpixel image registration of remotelysensed imagery,” IBM J. Res. Dev. 38�2�, 157–166 �1994�.

9. B. Ma, A. Hero, J. Gorman, and O. Michel, “Image registration withminimum spanning tree,” in Proc. of IEEE Int. Conf. on Image Pro-cessing (ICIP 2000), Vol. 1, pp. 481–484 �2000�.

0. M. R. Sabuncu and P. Ramadge, “Using Spanning Graphs for Effi-cient Image Registration,” IEEE Trans. Image Process. 17�5�, 788–797 �2008�.

1. M. I. Miller, “Computational anatomy: shape, growth, and atrophycomparisons via diffeomorphisms,” Neuroimage 23�1�, S19–S33�2004�.

2. J. Shi and C. Tomasi, “Good features to track,” in Proc. of IEEEConf. on Computer Vision and Pattern Recognition (CVPR 1994), pp.593–600 �1994�.

3. G. E. Christensen, R. D. Rabbitt, and M. I. Miller, “Deformabletemplates using large deformation kinematics,” IEEE Trans. ImageProcess. 5�10�, 1435–1447 �1996�.

4. P. Thevenaz, U. E. Ruttimann, and M. Unser, “A pyramid approach tosub-pixel registration based on intensity,” IEEE Trans. Image Pro-cess. 7�1�, 27–41 �1998�.

5. Y. Chen, R. R. Brooks, S. S. Iyengar, N. S. V. Rao, and J. Barhen,“Efficient global optimization for image registration,” J. Plant Nutri-tion 14�1�, 79–92 �2002�.

6. P. Chalermwat and T. El-Ghazawi, “Multi-resolution image registra-tion using genetics,” Proc. of IEEE Int. Conf. on Image Processing(ICIP 1999), Vol. 2, Oct. 24–28, Kobe, Japan, pp. 452–456 �1999�.

7. S. Makrogiannis, N. G. Bourbakis, and S. Borek, “A stochastic opti-mization scheme for automatic registration of aerial images,” in Proc.of IEEE Int. Conf. on Tools with Artificial Intelligence (ICTAI 2004),pp. 328–336 �2004�.

8. D. Robinson and P. Milanfar, “Fast local and global projection-basedmethods for affine motion estimation,” J. Math. Imaging Vision 18,35–54 �2003�.

9. S. Fotopoulos, A. Fotinos, and S. Makrogiannis, “Fuzzy rule-basedcolor filtering using statistical indices,” in Fuzzy Filters for Image



Processing, pp. 72–97, Springer-Verlag, Heidelberg �2003�.40. E. Parzen, “On estimation of a probability density function and

mode,” Ann. Math. Stat. 33, 1065–1076 �1962�.41. D. Goldberg, Genetic Algorithms in Search, Optimization and Ma-

chine Learning, Addison-Wesley, New York �1989�.42. S. Makrogiannis, G. Economou, and S. Fotopoulos, “A fuzzy dissimi-

larity function for region based segmentation of color images,” Int. J.Pattern Recognit. Artif. Intell. 15�2�, 255–267 �2001�.

43. J. L. Barron and M. Khurana, “Determining optical flow for largemotions using parametric models in a hierarchical framework,” inProc. of Vision Interface (V197), May, Kelowna, BC, pp. 47–56�1997�.

Sokratis Makrogiannis received his PhDin 2002 from the University of Patras,Greece. His research interests are mainlyrelated to image analysis and computer vi-sion. He has authored several journal andconference papers in these areas. Dr. Mak-rogiannis is an image processingdeveloper-consultant at the Visualizationand Computer Vision Laboratory in Gen-eral Electric’s Global Research Center inNiskayuna, New York.

Nikolaos G. Bourbakis received his PhDfrom the University of Patras, Greece, in1983. His research interests are in assis-tive technology, information security, intelli-gent systems and robotics, computer vi-sion, and wearable systems. He has beenpublished in international refereed journalsand conference proceedings. Dr. Bour-bakis currently is an OBR DistinguishedProfessor of IT and the director of the As-sistive Technology Research Center

�ATRC� at the College of Engineering and Computer Science atWright State University, in Ohio.

Jan–Mar 2010/Vol. 19(1)5

/terms

http://dx.doi.org/10.1109/83.506761

http://dx.doi.org/10.1109/36.957286

http://dx.doi.org/10.1109/36.957286

http://dx.doi.org/10.1006/cviu.1996.0517

http://dx.doi.org/10.1006/cviu.1996.0517

http://dx.doi.org/10.1109/TIP.2008.918951

http://dx.doi.org/10.1016/j.neuroimage.2004.07.021

http://dx.doi.org/10.1109/83.536892

http://dx.doi.org/10.1109/83.536892

http://dx.doi.org/10.1109/83.650848

http://dx.doi.org/10.1109/83.650848

http://dx.doi.org/10.1023/A:1021841127282

http://dx.doi.org/10.1214/aoms/1177704472

http://dx.doi.org/10.1142/S0218001401000861

http://dx.doi.org/10.1142/S0218001401000861

Efficient registration of multitemporal and multisensor aerial images based on alignment of...

Documents

Transcript of Efficient registration of multitemporal and multisensor aerial images based on alignment of...