Fringe-adjusted joint transform correlation

Fringe-adjusted joint transform correlation

Mohammad S. Alam and Mohammad A. Karim

Improved correlation discrimination is achieved by using a fringe-adjusted joint transform correlator(JTC). This technique is found to yield significantly better correlation output than the classical andbinary JTC's for input scenes involving single as well as multiple objects. It also avoids thecomputation-intensive Fourier-plane joint power spectrum binarization processing of a binary JTC.Two optical implementations for the proposed technique are also suggested.

Introduction

Recent techniques used for optical pattern recogni-tion may be broadly classified into the VanderLugt-type filter-based correlation' and the joint transformcorrelation.2 A VanderLugt-type correlator requiresa priori fabrication of the filter used in the correlationprocess, thereby prohibiting real-time operation.In addition, the filter must be accurately alignedalong the optical axis in the Fourier plane andrequires close positioning between the filter and theFourier transform of the input. On the other hand,a joint transform correlator (JTC) can be operated atvideo frame rates3 and does not require the referenceimage to be known substantially in advance of per-forming the correlation process.

One of the main problems associated with theclassical JTC is the presence of a strong zero-orderpeak in the output plane that corresponds to the sumof the autocorrelations of the reference and the inputsignals and almost overshadows the desired correla-tion peaks.4 5 For a single noise-free target, forexample, the zero-order peak is at least four timesstronger than the cross correlation peaks. This situ-ation becomes more bizarre in the presence of noise inthe input scene. In a real implementation, such azero-order peak may oversaturate the output detectorand cause strong spurious reflections. In specificsituations where the zero-order term is confined to anarrow region, however, an optical stop can be effec-tively used at the center of the output plane toovercome this problem.

Recently Javidi and Kuo6 proposed a binary JTC

M. S. Alam is with the Department of Engineering, PurdueUniversity, Fort Wayne, Indiana 46805-1499. M. A. Karim iswith the Department of Electrical Engineering and the Center forElectro-Optics, University of Dayton, Dayton, Ohio 45469-0227.

Received 16 June 1992.0003-6935/93/234344-07$06.00/0.t© 1993 Optical Society of America.

where the joint power spectrum (JPS) is binarized,based on a hard-clipping nonlinearity in the Fourierplane, to only two values (+ 1 and -1) before applyingthe inverse Fourier-transform operation. When com-pared with the classical JTC, a binary JTC is found toyield superior correlation peak intensity, correlationwidth, and discrimination sensitivity. The mainproblem with a binary JTC is the computation timerequired for the determination of the threshold valueused for binarizing the JPS, which acts as a con-straint on the system-processing speed.4 Also thebinarization process introduces harmonic correlationpeaks, and a portion of the correlation plane energy isdistributed among these higher-order harmonic terms.In addition, the higher-order harmonic terms mayyield false alarms or may result in misses,7' 8 therebycomplicating the target-detection process.

Most recently a JTC based on an amplitude-modulated filter9 (AMF) was reported. In this tech-nique, the JPS is multiplied by the AMF before theinverse Fourier transform is applied to yield thecorrelation output. For single noise-free targets, theAMF-based JTC is found to yield better correlationperformance than the classical and binary JTC's.However, the proposed AMF may produce high opti-cal gain for smaller values of the reference signalpower spectrum, which may actually degrade thenoise performance of the JTC. Johnson et al., 5

reported a JTC that uses Fourier-plane JPS apodiza-tion. In this technique, one uses an expensive phase-only spatial light modulator (SLM) at the Fourierplane, and the analysis/simulation results are applica-ble to only those reference image JPS's that do notcontain any zeros. To alleviate the problems and atthe same time to increase the autocorrelation peakintensity, accordingly, we propose in this paper afringe-adjusted JTC in which a real-valued filtercalled a fringe-adjusted filter (FAF) is used. Theperformance of the fringe-adjusted JTC is investi-gated with computer simulation that uses both noise-

4344 APPLIED OPTICS / Vol. 32, No. 23 / 10 August 1993

free and noisy input images. The proposed schemehas been found to yield better results than the classi-cal and binary JTC's while avoiding the computation-intensive Fourier-plane processing of a binary JTC.

Analysis

Joint Transform Correlation

The proposed real-time JTC is shown in Fig. 1, wherethe reference and the input scene are introduced inthe input plane by use of an SLM such as a liquidcrystal television. Assume that r(x, y + y') repre-sents the reference image and that t(x, y - y') repre-sents the input scene in the input plane separated bya distance 2y' along they axis. The input joint imagef(x, y) can be expressed as

f(x, y) = r(x, y + y') + t(x, y - y'). (1)

Lens LI performs the Fourier transform of f(x, y),which is given by

F(u, v) = IR(u, v) exp[kr(u, v)]exp(jvy')

+ I T(u, v) lexp[,t(u, v)]exp(-jvy'), (2)

where IR(u, v) I and I T(u, v) I are the amplitudes andkr(U, v) and t(u, v) are the phases of the Fouriertransforms of r(x, y) and t(x, y), respectively, u and vare mutually independent frequency domain vari-ables scaled by a factor 2,rr/Xf, is the wavelength ofthe collimating light, and f is the focal length of theFourier-transforming lenses LI and L2. The inten-

sity of the complex light distribution produced in theback focal plane of Li, called the JPS, is then detectedby a square-law detector like a CCD array or a liquidcrystal light valve (LCLV), given by

IF(u, ) 12 = IR(u, v) 12 + I T(u, v) 12

+ 2 R(u, v) I I T(u, v) I

x cos[-~(u, v) - 4(u, v) + 2vy']. (3)

In a classical JTC, the JPS is Fourier transformed bylens L2 to yield the correlation output. However, ina binary JTC the JPS is first binarized according to athreshold value before taking the inverse Fouriertransform of the JPS:

IF(u, v)12 = { 1 if IF(u,v)12 > Tf

otherwise(4)

where Tf is the JPS binarization threshold, defined by

Tf = median[ IF(u, v) 12]. (5)

In general, one selects the threshold value by makingthe histogram of the pixel values of the JPS and thenpicking the median.6

Recently, a JTC based on an amplitude-modulatedfilter9 (AMF) has been proposed in which the AMF isdefined as

1H~f(u, v) = jR~u v) (6)

The JPS is multiplied by Haf(u, v) before the inverseFourier transform operation is applied to produce thecorrelation output. This scheme is found to yieldbetter correlation results than a binary JTC. Howev-er, the fact that IR(u, v) -2 may be associated withone or more poles may contribute to other problems.The presence of poles may force the gain of the AMFto approach infinity, thereby imposing a seriousrestriction on the realization of this technique.Also, smaller values of the AMF may actually accentu-ate the noise, thereby degrading the noise perfor-mance of the system.

Fringe-Adjusted Joint Transform Correlation

To overcome the aforementioned problems of AMF-based JTC, we propose a fringe-adjusted JTC forwhich the fringe-adjusted filter (FAF) is defined as

Hfaf(u, v) = B(u, v)H~~u, ) -A(u, v) + I R(u, v) 12

Fig.1. Fringe-adjusted JTC implementation-1. BS, beam split-ter.

(7)

where A(u, v) and B(u, v) are either constants orfunctions. When B(u, v) is properly selected, one canavoid having an optical gain greater than unity.With a very small value of A(u, v), the pole problem isovercome, while at the same time it is possible toachieve a very high autocorrelation peak. The func-tion A(u, v) may be used to suppress noise or bandlimit the signal or both. For example, if the noisepower spectrum is known, the A(u, v) factor may be

10 August 1993 / Vol. 32, No. 23 / APPLIED OPTICS 4345

chosen to suppress the noise spectrum at the Fourierplane. Therefore, in a fringe-adjusted JTC, the am-plitude matching is used more effectively to producesharper and larger correlation peak intensity.

Notice that the filters used in VanderLugt-typecorrelators involve both magnitude and phase, thuscomplicating the filter fabrication process. On theother hand, the FAF is a real-valued function becauseit involves only the intensity (i.e., the JPS) and has nophase terms. Therefore a FAF is more suitable foroptical implementation. Also, the computations in-volving the FAF may be completed long before theinput scene is introduced in the input plane of theJTC. Thus the inclusion of the filter does not haveany significant detrimental effect on the processingspeed of the system. However, an additional spatiallight modulator is necessary to display the FAFfunction, as shown in Fig. 1.

The fringe-adjusted JPS is obtained by multiplyingthe filter function with the JPS. This multiplicationis achieved by displaying the JPS and the FAF in twoseparate SLM's placed side by side and then illuminat-ing the SLM's with the same laser, using a beamsplitter and mirror combination, as shown in Fig. 1.Thus the fringe-adjusted JPS may be expressed as

G(u, v) = Hff(u, V) IF(u, v) 12

= [ B~u, v) lIR(u, v)I21 + I T(u, v) 12[A u, v) + I R(u, V) 12_

+ 2 IR(u, v) I I T(u, v) I

x cos[4,(u, v) - lt(u, v) + 2vy']}.

and the corresponding JPS is given by

n

IF(u, v) 12 = IR(u, v) 12 + I T,(u, v) 12

n

+ 2 IR(u, v) ITi(u, v) Ii=1

x cos[kr(u, v) - 4td(u, v) + v(y' + Ai)]

n n

+ Ti(u, v) I I Tk(U, V) Ii= k=-

X COSH4)tA(U V) - (|)k(U, V) - V(Yi k)];

i • k, (12)

where I Ti(u, v) I is the amplitude and ti(u, v) is thephase of the Fourier transform of ti(x, y). The corre-sponding correlation output will contain the follow-ing terms: the autocorrelations of the referenceimage and the input scene objects, the cross correla-tions between the reference image and the inputscene objects; and the cross correlations between thedifferent input scene objects. The autocorrelationterms are diffracted on the optical axis at the correla-tion plane. Using Eq. (8) and relation (9), one mayexpress the fringe-adjusted JPS as

Gn(U, V) = [Ru I2 1I R(u, v)12I n~, ) 1

(8)

When B(u, v) = 1 and IR(u, v) 12 >> A(u, v), the FAFapproaches a perfect real-valued inverse filter, andEq. (7) may be written as

IHfaf(u, v) = ~

IR(u, V) 12(9)

+ E I Ti(u, v) I2

i=1

n+ 2 1 IR(u, v)IITi(u, v)I

i=l

x cos[tk(u, v) - ckti(u, V) + v(y' + Y)]

n n

+ i I T(u, ) I I Tk(u, v) Ii=1 k=1

X Cos[0t4(u, V) - 4)tk(U, V) - V(Yi -k)];

If the reference is the same as the target, i.e.,r(x, y + y') = t(x, y - y'), the fringe-adjusted JPS forthe autocorrelation output, when using Eq. (8), isgiven by

G(u, v) 21 + cos[r(u, v) - t(u, v) + 2vy']}. (10)

Lens L2 performs a Fourier transform of the fringe-adjusted JPS, and from relation (10) it is evident thatthe autocorrelation output will consist of two deltafunctions located at ±2y' and a zero-order term.Notice that the zero-order term is also a delta-func-tionlike output. Therefore it is simpler to block thisterm by using an optical stop in the correlation plane.

Multiobject Fringe-Adjusted Joint Transform Correlator

If the input scene contains n objects t(x, y - yl),t2(x, y - Y2), * * tn(X, Y - Yn), the input joint imagemay be expressed as

(X, y) = (X, y y') I t, Y i),i=1

(11)

i k. (13)

Assuming the input scene object t(x, y - yl) is thesame as the reference r(x, y + y'), Eq. (13) becomes

G,(u, v) = 2 + 2 cos[v(y' + yi)] + [IR(i U) 121

n n

2 IT(u,V)12 + 2 E IR(u,v)IlTi(u,v)Ii=2 i=2

X coS[r(U, V) - (kA, V) + V(y' + Yi)]

n n

+ ITA, (u, v)I Tk(U, V) Ii=1 k=1

x COS[ki~(U, V) - 4tk(U, V) - V(Yi - Yk)]1

i # k. (14)

An inverse Fourier transform of Eq. (14) yields the


correlation output. In Eq. (14) the first and thirdterms correspond to the zero-order terms diffractedon the optical axis at the correlation plane, the secondterm corresponds to the autocorrelation between thereference and the input scene object t(x, y - yi),while the last two terms represent the cross correla-tion between the reference and nontarget objects andbetween the input scene objects themselves. In gen-eral, the autocorrelation peak intensity decreaseswith the increase in the number of objects in theinput scene.'0

An all-optical implementation with one SLM andone LCLV may also be used to implement the pro-posed fringe-adjusted JTC, as shown in Fig. 2. TheLCLV is used to record the JPS. A portion of theinput light is passed through an SLM containing theFAF filter. The light beam originating from theSLM is then used as the readout light beam for theLCLV to yield the fringe-adjusted JPS, as shown inFig. 2. A subsequent Fourier transform by lens L2produces the correlation output.

Simulation Results

To investigate the performance of the proposed fringe-adjusted JTC, we consider (a) an input scene involv-ing a single object and (b) another input scene involv-ing multiple objects. The simulation tests wereperformed with a two-dimensional fast Fourier trans-form routine, and the results were plotted with athree-dimensional plotting routine. The correlationexperiments were performed next for the classical,the binary, and the proposed fringe-adjusted JTC's.In all the cases, the correlation peak height is normal-ized with respect to the total energy of the outputplane" over a scale of 0-255 so that it can be easilyrepresented by 8-bit quantizer levels. For the fringe-adjusted JTC, both A(u, v) and B(u, v) are taken to beunity to ensure that the gain of the FAF is less thanor equal to unity. Notice that smaller values ofA(u, v) and I R(u, v) 12 may accentuate the noise compo-

(a)

(b)

Fig. 3. (a) reference image; (b) input scene containing single object.

nent of the input signal power spectrum, therebydegrading the correlation performance.

Input Scene With Single Object

For the single-object input scene, a 28 x 28 pixelnoise-free image of a tank was used as the referenceimage, as shown in Fig. 3(a), and a 28 x 28 noisy

Fringe-adjusted JTC implementation-2. BS, beam split- qao

Fig. 4. Correlation output corresponding to the classical JTC.


Fig.2.ter.

image of the same tank was used as the target image,as shown in Fig. 3(b). These images were combinedand zero padded to form a 512 x 512 pixel input jointimage so that there was no circular convolutionartifact in the correlation output.

The correlation output for the classical JTC isshown in Fig. 4, which shows that it is difficult todetect the target because the autocorrelation outputsare almost buried in the noise floor. Because theclassical JTC yields poor correlation results for noisyimages, we limit the comparison of the fringe-adjusted JTC with the binary JTC. In a binary JTC,generally the hard-clipping nonlinearity is used forbinarizing the JPS. Accordingly, in the simulationfor binary JTC, the JPS median is used as thethreshold for binarization. The binarized JPS isshown in Fig. 5(a), and the correlation outputs corre-sponding to the first-order harmonic of the binarizedJPS are shown in Fig. 5(b). When binary JTC iscompared with the classical JTC, we observe from

(a)

3 ,0

Fig. 5(b) that the autocorrelation peak intensity hasan increased manifold and that the target can bedetected without ambiguity. However, binary JTCrequires computation-intensive Fourier-plane process-ing for the determination of the threshold value andsubsequent JPS binarization, which may act as aconstraint on system-processing speed.

The FAF used in the fringe-adjusted JTC is shownin Fig. 6(a). The fringe-adjusted JPS, obtained bymultiplying the JPS by the FAF, is then Fouriertransformed to produce the correlation output, asshown in Fig. 6(b). Comparing Fig. 5(b) with Fig.6(b), we observe that the autocorrelation peak inten-sity produced in the fringe-adjusted JTC is approxi-mately 27% higher than that in the binary JTC. Theautocorrelation peak-to-sidelobe ratio in the fringe-adjusted JTC is found to be approximately six timeshigher than that of a binary JTC.

In general, the peak-to-noise ratio is defined as theratio of the correlation peak intensity to the average

(a)

0.0'

(b)

Fig. 5. (a) binarized joint power spectrum corresponding to thebinary JTC; (b) correlation output corresponding to Fig. 3(b) as theinput scene for the binary JTC.

(b)

Fig. 6. (a) fringe-adjusted filter; (b) correlation output correspond-ing to Fig. 3(b) as the input scene for the fringe-adjusted JTC.


q ."

(a)

i

(b)

(c)Fig. 7. (a) input joint image containing single reference andmultiple objects; (b) correlation output corresponding to Fig. 7(a)for the binary JTC; (c) correlation output corresponding to Fig. 7(a)for the fringe-adjusted JTC.

value of the noise intensity. 2 ,13 For the given inputjoint image, the fringe-adjusted JTC is found to yielda 55% higher peak-to-noise ratio than that of thecorresponding binary JTC.

Input Scene With Multiple ObjectsNext we investigated the performances of the fringe-adjusted and binary JTC's when the input scenecontained multiple objects. The 512 x 512 pixelinput joint image, shown in Fig. 7(a), consists of anumber of aircraft, with the reference image sepa-rated from the object scene, which contains a group ofseveral different aircraft. Note that one of the input

Table 1. JTC Results for Single Object Case

Type of JTC Input Scene APIa PSRb PNRc FWHM

Binary Single object 2.46 20.5 166.76 1 x 1Fringe-adjusted Single object 3.12 120 258.43 1 x 1

aAutocorrelation peak intensity.bAutocorrelation peak-to-sidelobe ratio.cAutocorrelation peak-to-noise ratio.

Table 2. JTC Results for Multiple Object Case

Type of JTC Input Scene APIa PSRb ACIc FWHM

Binary Multiple objects 2.56 123.67 73.14 1 x 1Fringe-adjusted Multiple objects 3.45 3000 113.45 1 x 1

aAutocorrelation peak intensity.bAutocorrelation peak-to-sidelobe ratio.cAutocorrelation peak intensity to cross correlation peak inten-

sity ratio.

scene aircraft is identical with the reference aircraft.The symbols in this image were chosen because theyhave complex shapes, assuming that a more complexshape would cross correlate less with dissimilar shapesand correlate more with a similar shape. The corre-lation outputs corresponding to Fig. 7(a) for thebinary and fringe-adjusted JTC's are shown respec-tively in Figs. 7(b) and 7(c). Comparing Fig. 7(b)with Fig. 7(c), we observe that the fringe-adjustedJTC yields 35% higher correlation peak intensity andan autocorrelation peak-to-sidelobe ratio 24 timeshigher than that of the binary JTC. Also, the ratioof the autocorrelation peak intensity to the highestcross-correlation peak intensity is 85% higher in thefringe-adjusted JTC.

Finally the full width of the autocorrelation peakintensity at half of its maximum is found to be 1 x 1for both binary and fringe-adjusted JTCs for inputscenes involving single and multiple objects. Thesimulation results for these input scenes are summa-rized in Tables 1 and 2, respectively. The numericalresults obtained in this research are obviously uniqueto the images shown in Fig. 3 and Fig. 7(a), respec-tively. However, our efforts with many other imageshave yielded similar numerical results.

Conclusion

In this paper we have presented a fringe-adjustedfilter based JTC for target detection. This techniqueis found to yield substantially better correlationoutput than the classical and binary JTC's for inputscenes involving single as well as multiple objects.The FAF is designed such that it avoids the problemsassociated with an inverse filter, while producing ahigh autocorrelation peak intensity. It may also beused to attenuate the noise that is present in theinput scene provided that the factor A(u, v) is selectedproperly. Computer simulation results show thatfor noisy input images, the fringe-adjusted JTC yieldsbetter correlation peak intensity, peak-to-sideloberatio, and peak-to-noise ratio than the classical andbinary JTCs. For input scenes involving multipleobjects where two or more objects are identical, falseautocorrelation peaks may be produced at the outputplane. This problem may be alleviated by subtract-ing the input-only JPS from the JPS at the expense ofan additional processing step. The input-only JPSmay be obtained by displaying only the input scene inthe input plane SLM in the absence of referenceimage and then recording the JPS. By using theproposed technique, we are able to avoid the computa-


tion intensive JPS binarization process associatedwith binary JTC and the false alarms and misses7"14

often associated with the multiobject binary JTCbecause of JPS binarization. The fringe-adjustedJTC, however, requires an extra SLM to display theFAF filter. Inexpensive SLM's such as liquid crystaltelevisions may be used for this purpose. Note thatthe computation associated with the FAF can becompleted long before the input scene is actuallyintroduced into the input plane. Therefore the useof this additional filter may not have any detrimentaleffect on the system-processing speed.

References1. A. B. VanderLugt, "Signal detection by complex spatial filter-

ing," IEEE Trans. Inf. Theory IT-10, 139-145 (1964).2. C. S. Weaver and J. W. Goodman, "Technique for optically

convolving two functions," Appl. Opt. 5, 1248 (1966).3. J. C. Kirsch and D. A. Gregory, "Video rate optical correlation

using a magneto-optic spatial light modulator," Opt. Eng. 29,1122-1128(1990).

4. M. S. Alam, A. A. S. Awwal, and M. A. Karim, "Improved

correlation discrimination using joint Fourier transform opti-cal correlator," Micro. Opt. Tech. Lett. 4, 103-106 (1991).

5. F. T. J. Johnson, T. H. Barnes, T. Eiju, T. G. Haskell, and K.

Matsuda, "Analysis of a joint transform correlator using a

phase-only spatial light modulator," Opt. Eng. 30, 1947-1957(1991).

6. B. Javidi and C. Kuo, "Joint transform image correlation usinga binary spatial light modulator at the Fourier plane," Appl.Opt. 27, 663-665 (1988).

7. F. T. S. Yu, F. Cheng, T. Nagata, and D. A. Gregory, "Effect of

fringe binarization of multi-object joint transform correla-tion," Appl. Opt. 28, 2988-2990 (1989).

8. J. A. Davis, E. A. Merrill, D. M. Cotrell, and R. M. Bunch,"Effects of sampling and binarization in the output of the jointFourier transform correlation," Opt. Eng. 29, 1094-1100(1990).

9. D. Feng, H. Zhao, and S. Xia, "Amplitude-modulated JTC for

improving correlation discrimination," Opt. Commun. 86,260-264(1991).

10. M. S. Alam and M. A. Karim, "Improved correlation discrimi-nation in a multiobject bipolar joint Fourier transform correla-tor," Opt. Laser Tech. 24, 45-50 (1992).

11. A. A. S. Awwal, M. A. Karim, and S. R. Jahan, "Improvedcorrelation discrimination using an amplitude modulatedphase-only filter," Appl. Opt. 29, 233-236 (1990).

12. J. L. Horner, "Metrics for assessing pattern recognitionperformance," Appl. Opt. 31, 165-166 (1992).

13. B. Javidi, F. Parchekani, and Q. Tang, "Gray-scale nonlinearjoint transform correlator," Opt. Eng. 31, 888-895 (1992).

14. W. B. Hahn and D. L. Flannery, "Design elements of a binaryjoint transform correlator and selected optimization tech-niques," Opt. Eng. 31, 896-905 (1992).


Fringe-adjusted joint transform correlation

Documents

Transcript of Fringe-adjusted joint transform correlation