A new block-based motion estimation algorithm

Signal Processing: Image Communication 4 (1992) 507 517 507 Elsevier

A new block-based motion estimation algorithm

Kan Xie, Luc Van Eycken and Andr60oster l inck University of Leuven, ESA T-MI2, Kardinaal Mercierlaan 94, B-3001 Heverlee, Belgium

Received 19 June 1991 Revised 2 December 1991

Abstract. The conventional motion estimation algorithms used in digital television coding can roughly be classified into two categories, namely the block-matching method and the recursive method. Each of them has its own strong points. In this paper, a new type of block-based motion estimation algorithm is presented, which is based on the block-recursive (gradient) method and makes use of some of the merits of the block-matching method. For a moderate translational motion, motion estimation with a subpel precision can conveniently be obtained with only a couple of recursive searches, and for a violent or complicated motion which cannot be estimated by any block-based algorithm, the local minimum of prediction errors can always be found. Our experiments show that the proposed algorithm is efficient and reliable, and obviously superior to the conventional block-recursive algorithms and the fast block-matching algorithms. The performance of the proposed algorithm tends almost to the optimum of the full search algorithm with the same estimation precision, but the computational effort is much less than that of the full search algorithm.

Keywords. Motion estimation; motion-compensated prediction.

1. Introduction

In digital television coding, one attempts to mini- mize the temporal redundancy in a sequence of images so as to reduce the transmission data rate. Motion-compensated (MC) interframe prediction is a most efficient approach to realize this goal. In motion-compensated interframe prediction, the frame to frame displacement of a moving object in a television scene is estimated, and the prediction is formed by displacing the previous frame elements.

It is well known that the success of motion-compensated prediction depends on the ability to estimate the displacement of moving objects. Several mathematical models have been proposed to estimate the motion vector fields in a sequence of television pictures. They can be classified basically into two groups : the block-matching motion estimation algorithms [4, 5, 7] and the recursive (gradient) motion estimation algorithms [ 1, 2, 9, 15].

The fundamental idea behind the block-matching algorithms is that the motion estimation can be

obtained by searching the position of the best match in the previous frame. The brute force search procedure (full search algorithm) is straight- forward to find the optimal match in a search region, but an enormous amount of computation is required. Therefore, most of the efforts in the block-matching method are to seek an efficient search procedure to reduce the computational complexity but without an obvious effect on the estimation accuracy, such as some proposals like the 2-D logarithmic algorithm, the 3-step algorithm, the conjugate direction algorithm, etc. Most of these fast algorithms are based on the assumption that the cross correlation function in the search region is a single-peaked and quadrant- monotonic model. However, the assumption is not always true for actual TV signals. It has been noted that sometimes a fast search procedure slides to a wrong direction or is trapped by a false correlation peak, and as a result we get a non-optimum match or an erroneous estimate which could affect the visual perception and the coding efficiency. In

0923-5965/92/$05.00 © 1992 Elsevier Science Publishers B.V. All rights reserved

508

addition, it is known that motion estimation with a subpel precision is effective to improve the estimation performance and to reduce the prediction errors, but the computational complexity will increase tremendously even more. For example, suppose that the possible maximum displacement in the horizontal and the vertical directions is Omax = 7 pel, the full search procedure with a subpel

1 precision P = z pel requires to evaluate a cross- correlation function (or a mismatch function) at (2Dmax/P+l)2=3249 shifts and to calculate additionally the spatial interpolation at [ (2Dmax /P+ 1) 2 - (2Dmax+ 1) 2] =3024 points of subpel. Such a tremendous amount of computation is a big difficulty for the real time applications.

The recursive algorithms are based on the gradient (differential) method: suppose that the luminance intensity changes in successive frames are due to the translation of an object, the motion estimation will iterate in the luminance gradient direction so that the consecutive approximations converge towards the actual displacement. The recursive motion estimation algorithms can be executed either block by block or pel by pel, and denoted as the block-recursive algorithms and the pel-recursive algorithms, respectively. For a slow motion, the performance of the recursive algorithms is quite good and the convergence speed is high. Moreover, it is very convenient for the recursive algorithms to estimate a motion vector field with a subpel precision. However, experiments show that the recursive algorithms are not very stable and reliable especially when there are some violent and complicated motions in television scenes.

Starting from the investigation of the different motion estimation algorithms published, a new type of block-based motion estimation algorithm is developed. The algorithm is a hybrid version of the block-recursive method and the block- matching method.

2. The proposed block-based motion estimation algorithm

The proposed algorithm is based on the block- recursive method. Let I(z, k) denote the luminance Signal Processing: Image Communication

K. Xie et al. / A new motion estimation algorithm

intensity of a pel at spatial location z "r = (x, y) in frame k, where superscript T denotes transpose. If lighting is uniform and luminance intensity changes are due to the translation of an object, then

I(z,k)=I(z- D,k-1),

where D is the motion vector of a moving object during the time interval between two successive frames. If/~i is the/th estimate of the actual motion vector, the displaced frame difference DFD can be defined as

DFD(x, b i) =I(z, k ) - I ( z - D i, k - 1). (1)

Suppose that the displacement of a moving object is constant within a block. The motion estimation of a block can be approximated by the following recursive equation:

[-~DFD(z,/~i 1) sign(Vlx(z_/~t-,,k_ 1)).]

b,=b,_l_| ~,lVl~(z-b'-l,k-1)l +o I L Z l V l y ( z - D ' - I , k - 1 ) l + ~ r

(2)

where VIx, VIy are the vector components of the luminance gradient VI with respect to z - /~ i -1 in frame k - 1, and

0, if x = 0 ,

= 1, if x sign(x) > 0,

-1 , if x < 0 ,

Equation (2) is similar to one of the block-recursive algorithms given by Netravali and Robbins [9]. A small positive constant ~ is included to account for the case of the denominator becoming zero (here ~ = 10 is used for the block size 8 x 8). Since (2) has a comparable performance with other block- recursive algorithms but with a much simpler hard- ware implementation, it is adopted in this study.

The performance of a gradient algorithm is good as long as the displacement D is small. Lamb and Murphy reported [6] that the gradient algorithm (without iteration) performs accurately for a moving object up to a speed of 2.5 pel per frame. How- ever, as motion D increases, the performance of the

K. Xie et al. , /A new motion estimation algorithm

gradient algorithm degenerates and errors increase because the quality of approximation becomes poor. Although the estimation errors can be allevi- ated to a certain extent by introducing the recursive method, the block-recursive algorithms are still unstable and unreliable for a large displacement. In addition, the effective range of estimation and the stability of a block-recursive algorithm are related to the size and the speed of the estimated moving object. It has been observed that sometimes an estimated vector is far away from the actual displacement, because the mathematical model of the gradient method fails to describe the real motion. For example, sometimes there is no reasonable luminance gradient due to fast motion of a small object, or sometimes the luminance gradient is very steep and the gradient region is very narrow, which often occurs in the neighborhood of edges and contours of moving objects, so that recursive estimates are not able to follow the correct gradient direction and converge to the real motion. As a result, an erroneous estimation or so- called 'oscillation' may occur, and the variance of prediction errors may be very large. In the proposed algorithm, an alternative approach similar to the block-matching method will be employed to the regions where the recursive method is invalid.

For a recursive algorithm, a basic assumption is that an initial displacement estimate/)~-~ is used to produce a new improved estimate b i according to

b '=b ' - ' +u,, (3)

where ui is the updated term of the ith iteration. If the algorithm is convergent, then for a large enough number of iterations u; will tend towards zero. In actual computations, if ui is below a pre- determined value, then the motion estimation is terminated and the vec to r / ) ; is considered as the estimated motion. However, it should be kept in mind that (ui-- 0) is only a necessary condition but not a sufficient condition for a correct convergence. It is not difficult to understand from (2) that a small u; does not guarantee a small variance value of prediction errors. In addition, it has been noted

509

that sometimes the prediction errors in the last estimate is slightly larger than that in the previous estimates, even though the recursive algorithm is convergent. Therefore, an additional test for evalu- ating MC prediction errors is measured at every iteration point in the proposed algorithm,

1 D , k - 1)l, (4) MAD(b;)= B Y I I ( z , k ) - I ( z - ^i

xE~¢

where ~ is the subset of the integer pels in a block, and B ( = M x N) is the total number of the pels in the block. The MAD is identical to the mean of the absolute displaced frame differences, a matching criterion in the block-matching algorithms. The purpose of measuring MAD is to locate the small- est mean of prediction errors over all operation points, especially when the block-recursive algorithm fails to estimate the real motion. That is, motion estimation is obtained by searching the optimal match along the prevalent luminance gradient direction rather than along a pre-determined search route.

It is known that the convergent behavior of a recursive estimation algorithm depends strongly on the motion natures and the local image details. If the recursive method is not suitable to describe the real motion, oscillation may occur and the prediction errors may be very large. In the proposed algorithm, if the performance of the bloek,recursive algorithm is not good enough, a so-called 'force search procedure' which is similar to the block-matching method is applied to search the best match in the whole search region. Then, the block-recursive algorithm performs again from the position of the best match to get a higher estimation precision. This approach is efficient to remedy some erroneous estimated vectors. In addition, sometimes it is impossible to obtain a correct motion vector by any of block-based algorithms for the actual signals (for example, motion estima, tion will not make any sense if a block contains multiple motions or deformed objects), then the force search procedure can at least help to mini- mize the prediction errors. The reported block- matching algorithms like the full search algorithm

Vol. 4, No. 6, November 1992

510 K. X ie et aL / A new motion estimation algorithm

can be used as the force search procedure. Addi- tional computation does not increase too much because the force search procedure is only used for a small number of blocks, while a considerable improvement can be obtained.

It is known that a reasonable initial estimate is beneficial to improving the performance of a motion estimation algorithm. Since the motion vector field contains a considerable correlation in both the spatial and the temporal dimensions, one can expect that a motion estimation algorithm con- verges faster if a correlative motion vector from the previous nearby blocks is taken as the initial motion estimate of the present block. Our experiments have shown that the initial estimate is favor- able to improving not only the convergent speed but also the estimation accuracy of the motion estimation algorithm.

In summary, the proposed block-based motion estimation algorithm is implemented in the following steps:

1. Init&l estimation. The initial estimate is obtained by selecting a correlative motion vector from the previous nearby blocks. The motion vectors of the block above and the block to the left of the present estimated one, and of the block at the same position in the previous frame, if they are available and good-estimated, are taken as the candidates of the initial estimation. A good-estimated vector means the one which is within the maximum possible displacement region and the corresponding mean of absolute displace frame differences MAD(D) ~< T2 (a pre-determined threshold). These candidates are used to predict the present block, and the one corresponding to MADmin, the small- est value of MAD, is chosen as the initial motion estimate /~0, where the superscript 0 denotes the initial state. But if MAD(0)-%< MADmi°,/~0 is reset to zero. If MADmin is below a small pre-determined threshold T~, the corresponding vector is considered as the estimated motion vector of the present block, and no further computation is necessary. Otherwise, go to the next step.

2. Iteration. Starting from the initial motion estimate /~0, the block-recursive motion estimation algorithm (2) is executed iteratively until MAD(/ ) i) ~< T1 or the updated term ui is below a pre-determined value or the maximum iteration limit is reached. If the estimated vector exceeds the maximum limited displacement Dmax or if the minimal mean of prediction errors MADmin > T,, (a pre-determined threshold) after the block-recursive estimation algorithm, then go to Step 3. Otherwise, the optimal estimated vector Dop which corre- sponds to the minimum MADmin instead of the last estimate/~i is taken as the estimated motion vector of the current block. 3. Force search procedure. If an estimated vector exceeds Dmax or if MADmin is higher than Tin, the force search procedure is applied to find the best match by searching all integer pels in the searching region of the previous frame. If the minimal mismatch value MAD is smaller than the MADmin in Step 2, then go back to Step 2 and the recursive algorithm (2) is executed once again from the best matching point to get a subpel estimation precision. Otherwise, the computation is terminated and the optimal vector Dop estimated in Step 2 is taken as the output motion vector.

3. The performance evaluation

The performance of the proposed algorithm is evaluated and compared with the conventional block-based motion estimation algorithms. Since the actual motion vector field of television signals is unknown, the accuracy of motion estimation obtained by any algorithm could not be determined directly. Several indirect measures which are usu- ally used to evaluate the estimation performance are adopted in this investigation:

1. The peak-to-peak signal-to-noise ratio (SNR) of prediction image,

SNR = 10 log10

x (peak to peak value of signal) 2 (dB),

var[I(z, k) - I(z, k)] Signal Processing: Image Communication

K. Xie et al. / A new motion estimation algorithm 511

where I(z, k) is the actual luminance intensity and I(z, k) the predicted value.

2. The entropy of displaced frame differences DFDs (i.e., the MC prediction errors),

sp H(b) = - Y. P(b) log2 P(b) (bits/pel),

b----gp

where Sp is the possible maximum value of DFDs, and the absolute values of DFDs below the threshold 7"1 are lumped into zeros.

3. The number of blocks with the mean of absolute prediction errors MAD > Tp (a pre-determined threshold.

The luminance components of two television sequences, 'Miss America' and 'Claire', which are typical teleconference scenes, are used in the com- puter simulation. The luminance signals are quan- tized uniformly using 8 bit/pel, and each frame has 288 × 352 samples. The frame frequency of 'Miss America' is 30 frame/s, and that of 'Claire' is 10 frame/s.

The following block-based motion estimation algorithms have been investigated, where the block-matching algorithms are enhanced with a subpel precision for the purpose of comparison: Algorithm 1: A block-recursive algorithm (eq. (2)). Algorithm 2: A modified 3-step algorithm with a subpel precision (i.e., the 3-step procedure is used to estimate motion vector with 1-pel precision, and then the subpel precision is obtained by full-searching all subpel positions in the 2 x 2 region centered at the last integral estimate). Algorithm 3: The full search block-matching algorithm with a subpel precision. Algorithm 4: The proposed block-based motion estimation algorithm.

Some implementation details in the simulation are described as follows: 1. The performance of the block-based motion

estimation relates strongly to the block size and the motion estimation precision. In this investigation, the block size is 8 x 8 (64 pels) and the

1 motion estimation precision is P = z pel.

.

.

4.

5.

The maximum displacement Dmax in the horizontal or the vertical direction is set to 7 pel per frame. The maximum number of iteration limitation for the block-recursive algorithms is 10. The pre-determined thresholds TI = 2, T2 = 15, Tp = 8 and Tm = 5 (out of 255). The calculation of the horizontal and the vertical gradient components and the spatial interpolation are illustrated in Fig. 1. Let I(z, k - 1) = Ic be the luminance intensity of the present pel at position z in the previous frame k - 1 . Suppose that I ( z - [ D i - l ] , k - 1 ) = I e , where [D] denotes the integral approximation of a motion vector D by rounding both the components of the vector, then the spatial gradients of the displaced pel are given by

V/x(Z-- D i-1, k - 1) = Ih-- Ib, 2

h-/d V/Az-b'-l, ~ - 1)- 2

The corresponding luminance intensity of the displaced pel in frame k - 1 is obtained by a two-dimensional interpolation as

- D , k - l ) I ( Z ~ i - I

=(1 - dy)[(1 -dx)Ie+dxlh]

+ dy[(1 -dx)Id+d~Ig],

- -8 8 8 I ( z - D i - l , k - 1)

g 6 Fig. 1. Position of the picture elements in frame k - 1 used for

the computat ion of interpolation and gradient values.


512 K. XW et aL / A new motion estimation algorithm

where dx, dy are the fractional parts of the motion vector in the horizontal and vertical directions, respectively, and

dx = Dx - [Dx], dy = By - - [By].

The following figures show in detail the performance of the proposed algorithm and the conventional motion estimation algorithms for the sequences of 'Miss America' and 'Claire'. Figure 2

42

~. 40

.... 38

36

34

32

.............. algo.l

. . . . algo.2 , , , , algo.3

• ~ 1 " - - algo.4

, / V i~ , . ,: i ! i ! ! :'V,~ ./

,'~ il i/i: : i iL ,

V i i v : :

10 20 30 40 50

(a) No, of frame

illustrates the signal-to-noise ratios of prediction images of the different algorithms for 'Claire' and 'Miss America'. Figure 3 shows the entropies of the motion-compensated prediction errors and the numbers of blocks with MAD>Tp for the sequence 'Claire'. It can be observed that the performance of the conventional block-recursive algo-

t l algo.l ..... algo.2

. algo.3 1.4 i :\/ ~ a]go.4

t .3 : i

i i2L~ , , , , ~ ::

: itI[i'~ ' " ' ~ :ff~I!:l/~. , q~:t¢, / i ~ \ : i/ i : . . . . ' I t ?'i,",X/I, , ];"v: ,,'/, l *~ I ili/A\ v ~/i :",.i ",! Fii:ll VV~il

0.7 I0 20 30 40 50

(a) No. of f r a m e

42 [ ............. algo. 1

41.5 1 ..... algo.2 J lt algo.3 _ _ algo.4

4z[-

Z 40 /"

39 / ... "'-./ ......... \/ ......

38.5 i .

38 I 0 20 30 40 50

(b) No, of frame

Fig. 2. The S N R of predict ion images o f the different algo- r i thms (a) for the sequence 'Claire ' an d (b) for the sequence

'Miss Amer ica ' .

Signal Processing: Image Communication

80 ............. algo.l

,5 ..... algo.2 : { , , t , algo.3 : _ _ algo.4

j '. J !

70

6O A

50 = :

i i / i

i : A( ~ : ,' i t "

of" 10 20 30 4.0 50

(b) No. of frame

Fig. 3. (a) The entropies o f M C predict ion errors and (b) the n u m b e r o f unpredic table blocks o f the different a lgor i thms for

the sequence 'Claire ' .

K. Xie et al. / A new motion estimation algorithm

rithm (Algorithm 1) is obviously not stable and varies with motions in the image scenes. For the scenes containing slow motions, the performance of Algorithm 1 is passable. However, if there is a relatively fast or complicated motion in image scenes, the estimation performance of Algorithm 1 will degenerate seriously. By contrast, the stability of the 3-step block-matching algorithm (Algorithm 2) is much better, but the prediction errors sometimes are still very large because the fast search is strongly related to the image features. The graphs show that the proposed algorithm has a pretty good performance which almost tends to the optimum (by the standard of prediction errors) of the full search algorithm with the same estimation precision. However, the amount of computation of the proposed algorithm is much lower than that of the full search algorithm (the average number of iterations per block of Algorithm 4 is only 2-5). It is also noted from Fig. 2(b) that the SNR of Algo- rithm 4 is a little bit higher than that of algorithm 3. The reason is that there are some small differences in the procedure of initial estimation in this investigation. For Algorithm 4, besides D = 0, some previous motion vectors are used in the initial test, and if MmDmin ~< T~ (it is obvious that MADmin~<MAD(0)), motion estimation is fin- ished. For Algorithm 3 only MAD(0) is tested, and if MAD(0)~< T1 the block is considered stationary and no further estimation is executed.

It should be noted that the above indirect measures are not sufficient and are sometimes ambigu- ous to give an objective evaluation of the accuracy of motion estimation algorithms. A subjective evaluation Of course is more desirable than the evaluation of these quantitative measures. Figures 4-7 show the prediction images of the different algorithms for frame 20 of 'Claire', which are the enlarged pictures on the face, in order to give a subjective impression of the performance of the different algorithms. It is obvious that there are some annoying distortions caused by erroneous motion estimation in the prediction images of Algorithms 1 and 2 (Figs. 4-5). However, there is no visible distortion for Algorithms 3 and 4 for this picture.

513

Fig. 4. The prediction image of Algorithm 1.


The simulation is also performed on other television sequences which contain more complicated figures and motions. 'Baltimore' is a sequence of high detailed pictures with moderate motion (including camera zooming), and 'Teeny' is a


514 K. X i e et al. / A new motion estimation algorithm

6 ,

5.5

5

4.5

4

3.5 ¢zl

3

2.5

2 10

(a)

f , ~ , .......... algo.l + . ~ + a l g o . 3

_ _ a l g o , 4 ( D m = 7 )

i , . . . . . . . algo.4(Dm=14) f'i

i i i

20 30 40 50

No, of f r a m e

Fig. 6. The predic t ion image of A lgo r i t hm 3.

J~

3.4

3.3

3.2

3.1

3

2.9

2.8

2.7

2.6

2.5

2.4

.... algo.l ....... algo.2

~ algo.3

- - algo.4

r i i i i

2 4 6 8 10 12

( b ) N o . o f f r a m e

Fig. 8. The entropies of M C predic t ion errors of the different a lgor i thms for the sequences (a) 'Teeny ' and (b) 'Bal t imore ' .


sequence of pictures of a close-up view of the head of a young girl with some violent and complicated motions and brightness changes. The motion estimation is performed on the central part of 288 × 352 samples in the frames for the purpose Signal Processing: Image Communication

of comparison and saving computation. Figure 8 shows the entropies of DFD for the sequences 'Teeny' and 'Baltimore'. It can be observed that for the sequence 'Baltimore', the performance of the block recursive algorithm is pretty good. How- ever, for the scenes containing some violent and complicated motions like those in the sequence 'Teeny', it becomes very difficult and sometimes


impossible to estimate the real motion. It can be seen that the entropy of the proposed algorithm tends to the optimum of the full-search algorithm (the performance of Algorithm 2 is approximate to that of Algorithms 3 and 4 and is not included in Fig. 8(a) to simplify the graph). Since there are some very violent motions in the sequence 'Teeny', extending the search region, i.e., increasing D . . . .

can effectively reduce the prediction errors of an estimation algorithm in some scenes. The behavior of the proposed algorithm with Dmax = 14 pel is also shown in Fig. 8(a). By the way, it should be pointed out that in the prediction coding applications it is not worthwhile to seek so sedulously for a correct estimate in the violent motion areas, since the image of violent moving objects is blurred by the integration effect of a TV camera and the correlation between neighboring pels becomes larger. On the contrary, an intraframe coder like intraframe prediction may have a better behavior in these areas. Therefore, an adaptive MC inter/intra scheme will be more efficient to cope with a large class of television signals.

the proposed block-based motion estimation algorithm is capable of working efficiently and stably with different block sizes. Figure 9 illustrates the SNR of prediction images of Algorithm 4 with different block sizes for 'Claire' and 'Miss America'. It can be observed that SNR is remark- ably improved as the block size reduces, because a

z

(a )

4 4

4 2

4 0

38

34 i r

32

30

' 6 * , .,' V \ ,;'. 4 x 4 ,', f / ~ 8 x 8

1 6 x 1 6

, 3 2 x 3 2

i,' I

' I'J

i i J i

10 20 3 0 4 0

N o . o f f r a m e

50

4. E x p e r i m e n t a l resu l t s for d i f ferent b l o c k s i z e and

di f ferent e s t i m a t i o n prec i s ion

It is obvious that the performance of block- based motion estimation algorithms is firmly related to the block size used to measure the motion vector. A large block size could result in an increase in the variances of prediction errors because the assumption of constant displacement in a block may no longer hold, whereas a small block size could lead to an unstable estimate or an erroneous match. It has been reported that, for the block-recursive (gradient) algorithm, the estimation of the motion components in the direction orthogonal to the prevalent gradient can be carried out only when the block size is 8 x 8 or larger [2, 3]. Experiments also show that the performance of the conventional block-recursive algorithms becomes very poor as the block size decreases to smaller than 8 x 8. However, it is of interest to note that

41

~" 40

Z;

39

38

37

36

' ',, ! i .... '~ A~ / l ', ' ,' ' " '~,' ~,- ,,. / ,,,' ,/.~ i \ vi',..' . .

',!~' i / 'J

4 x 4

8 x 8

1 6 x 1 6

3 2 x 3 2

i i i i

10 2 0 3 0 4 0 5 0

( b ) N o . o f f r a m e

F i g . 9. T h e S N R o f p r e d i c t i o n i m a g e s o f t h e p r o p o s e d a l g o -

r i t h m w i t h d i f f e r e n t b l o c k s izes ( a ) f o r ' C l a i r e ' a n d (b ) f o r ' M i s s

A m e r i c a ' .


516 K. Xie et aL ,/A new motion estimation algorithm

smaller block size gives a better approximation for complicated motion like rotation or deformation. However, it should be kept in mind that in the forward motion-compensated prediction, a smaller block size also means requiring more overhead data for the motion information.

As mentioned above, increasing the estimation precision can improve the estimation performance and reduce the prediction errors. Our experiments show that the performance of Algorithm 4 with different precisions approximates the optimum of the full search algorithm with the same estimation precisions. Figure 10 indicates the SNR of prediction images of Algorithm 4 with different estimation precisions for 'Claire' and 'Miss America'. The SNR of the full search algorithm with pel- integer estimation precision is also presented in Fig. 10 as a comparison. It can be seen that there is a significant improvement as the estimation pre-

1 1 cision increasing from 1-pel to ~ and z pel, but little further improvement is obtained as the estimation precision increases to higher than ~ pel. In addition, there is a trade-off between the motion estimation precision and the overhead data if the motion vector field must be transmitted. A more precise motion estimation results in a reduction of the prediction errors which leads to a reduction of transmission bit-rate, but it also results in an increase of the motion information overhead. Therefore, the estimation precision chosen in this study is ¼ pel.

44

43

42

41

40

39

38

37

A

i

,i

/i / .... i V ~ . / Li 'v t

/

i J

5. Conclusion

1/8 p e l

f 1/4 p e l

1/2 pel_

~ / ~ algo3( lp~ " ! pel _

/

i 3O

3~ i i i

l0 20 40 50

(a)

40.5

~ 40

Z 39.5

m 39

38.5

38

N o . o f f r a m e

42

41"5 t

41 I- ~ / . . ] ~ , ~, 1/8 pel

/ / 'V /X /X~ .- , ! " \ \ . ,~.- . ,~ t / 4 pe~

. . . . . . .

' t,]

37.5

37 i i i i

10 20 30 40 50

(b) No. o f f r a m e

Based on the investigation of the conventional motion estimation algorithms, a new block-based motion estimation algorithm is developed. It is a combination of the block-recursive (gradient) method and the block-matching method. For a small or moderate motion, a block-recursive algorithm is used to estimate the motion vector with a subpel precision. For a fast or complicated motion for which the gradient method is not valid, the block-matching method is employed to search the Signal Processing: Image Communication

Fig. 10. The SNR of prediction images of the proposed algorithm with different estimation preeisions (a) for 'Claire' and

(b) for 'Miss America'.

best match in the search region. Our experiments show that the proposed algorithm is superior to the conventional block-recursive algorithms and the fast block-matching algorithms. The performance of the proposed algorithm approximates the optimum of the full search algorithm with the same estimation precision, but the computational effort is much less than that of the full search algorithm.


References [1] M. Bierling, "A differential displacement estimation algo-

rithm with improved stability", SPIE Vol. 594 Image coding (1985), 1985, pp. 170-174.

[2] C. Cafforio and F. Rocca, "The differential method for image motion estimation," in: T.S. Huang, ed., Image Sequence Processing and Dynamic Scene Analysis, NATO ASI Series, Vol. 12, Springer, New York, 1983, pp. 104-124.

[3] C. Cafforio, F. Rocca and S. Tubaro, "Motion compensated image interpolation", IEEE Trans. Comm., Vol. COM-38, No. 2, February 1990, pp. 215-222.

[4] J.R. Jain and A.K. Jain, "Displacement measurement and its application in interframe image coding", 1EEE Trans. Comm., Vol. COM-29, No. 12, December 1981, pp. 1799- 1808.

[5] T. Koga et al., "Motion-compensated interframe coding for video conferencing", NTC 81, Proc., New Orleans, LA, December 1981, pp. G5.3.1-G5.3.5.

[6] J.O. Limb and J.A. Murthy, "Estimating the velocity of moving images in television signals", Comput. Graph. Image Process., 1975, pp. 311-327.

[7] H.G. Musmann, P. Pirsch and H.J. Grallert, "Advances in picture coding", Proc. 1EEE, Vol. 73, No. 4, April 1985, pp. 523 548.

[8] A.N. Netravali and B.G. Haskell, Digital Pictures, Plenum, New York, 1988.

[9] A.N. Netravali and J.D. Robbins, "Motion compensated television coding- Part I", Bell Syst. Tech. J., Vol. 58, 1979, pp. 631-670.

[10] S. Sabri, "Movement compensated interframe prediction for NTSC color TV signals", IEEE Trans. Comm., Vol. COM-32, No. 8, August 1984, pp. 954-968.

[11] K. Xie, L. Van Eycken and A.J. Oosterlinck, "Motion- compensated interframe prediction", Proc. SPIE Vol. 1349, Applications of Digital Image Processing )(111 (1990), San Diego, USA.

[12] K. Xie, L. Van Eycken and A.J. Oosterlinck, "Some new techniques in displacement estimation algorithms", Proc. SPIE 1Iol. 1199 Visual Communication and Image Process- ing I V (1989), 1989, pp. 18-29.

[13] K. Xie, L. Van Eycken and A.J. Oosterlinck, "Motion- compensated adaptive inter/intra frame prediction", Proc. SPIE Vol. 1360 Visual Communication and Image Process- ing I V (1990), 1990, pp. 1798 1809.

[14] H. Yamaguchi, "Iterative method of movement estimation for television signals", IEEE Trans. Comm., Vol. 37, No. 12, December 1989, pp. 1350 1358.

[15] H. Yamaguchi, T. Sugi and K. Kinuhata, "Movement- compensated frame-frequency conversion of television signals", IEEE Trans. Comm., Vol. COM-35, No. 10, October 1987, pp. 1069-1082.


A new block-based motion estimation algorithm

Documents

Transcript of A new block-based motion estimation algorithm