Image Based Visual Servoing: A New Method for the Estimation of the Image Jacobian in Dynamic...

12
A. Campilho and M. Kamel (Eds.): ICIAR 2006, LNCS 4141, pp. 850 861, 2006. © Springer-Verlag Berlin Heidelberg 2006 Image Based Visual Servoing: A New Method for the Estimation of the Image Jacobian in Dynamic Environments L. Pari 1 , J.M. Sebastián 1 , C. González 1 , and L. Angel 2 1 Departamento de Automática, Ingeniería Electrónica e Informática Industrial (DISAM) Escuela Técnica Superior de Ingenieros Industriales, Universidad Politécnica de Madrid C/ José Gutiérrez Abascal, 2, 28006 Madrid, Spain 2 Facultad de Ingeniería Electrónica, Universidad Pontificia Bolivariana Km. 7 Via de Piedecuesta, Bucaramanga, Colombia {jsebas, lpari, cgpascual, langel}@etsii.upm.es Abstract. This paper describes a innovative algorithm for the on-line estimation of the image jacobian based on previous movements. It has been utilized for the tracking of an uncalibrated 3 DOF joint system, using two weakly calibrated fixed cameras. The algorithm incorporates the epipolar restriction of the two cameras conveyed in the fundamental matrix, and a reliability factor has been defined for the previous movements. The tests carried out show that the pro- posed algorithm is more robust in presence of noise than those obtained from already existing algorithms in specialized literature. Keywords: Visual servoing, Jacobian estimation, Fundamental matrix. 1 Introduction Visual servoing consists in the use of visual information given by visual sensors (i.e. cameras) to control a robotic system. This kind of control turns out to be very useful in dynamic environments, as it allows us to know which objects are present in the scene with high accuracy, as well as their position, orientation and velocity. It makes possible to use robots in new domains where the workspace is not known a priori. However it also proposes to solve new problems (in terms of accuracy and robust- ness) currently unresolved [6]. The propose algorithm in this paper increases the accu- racy and robustness without requiring a thorough knowledge of the involved systems: calibration of the joint system and kinematic calibration of the vision system, both common sources of accumulative errors. Visual servoing has been widely estudied in the lastest years, there are in the litera- ture interesting surveys [5] and [7]. Among the existing clasiffications of visual ser- voing, one of the most known is the way visual information is used to define the sig- nal error to control the system: position based visual servoing (PBVS or 3D) and the image based visual servoing (IBVS or 2D). In position based visual servoing, features are extracted from the image and used to reconstruct the 3D position of the target, whereas in image based visual servoing the task is defined in the image plane directly

Transcript of Image Based Visual Servoing: A New Method for the Estimation of the Image Jacobian in Dynamic...

A. Campilho and M. Kamel (Eds.): ICIAR 2006, LNCS 4141, pp. 850 – 861, 2006. © Springer-Verlag Berlin Heidelberg 2006

Image Based Visual Servoing: A New Method for the Estimation of the Image Jacobian in Dynamic

Environments

L. Pari1, J.M. Sebastián1, C. González1, and L. Angel2

1 Departamento de Automática, Ingeniería Electrónica e Informática Industrial (DISAM) Escuela Técnica Superior de Ingenieros Industriales, Universidad Politécnica de Madrid

C/ José Gutiérrez Abascal, 2, 28006 Madrid, Spain 2 Facultad de Ingeniería Electrónica, Universidad Pontificia Bolivariana

Km. 7 Via de Piedecuesta, Bucaramanga, Colombia {jsebas, lpari, cgpascual, langel}@etsii.upm.es

Abstract. This paper describes a innovative algorithm for the on-line estimation of the image jacobian based on previous movements. It has been utilized for the tracking of an uncalibrated 3 DOF joint system, using two weakly calibrated fixed cameras. The algorithm incorporates the epipolar restriction of the two cameras conveyed in the fundamental matrix, and a reliability factor has been defined for the previous movements. The tests carried out show that the pro-posed algorithm is more robust in presence of noise than those obtained from already existing algorithms in specialized literature.

Keywords: Visual servoing, Jacobian estimation, Fundamental matrix.

1 Introduction

Visual servoing consists in the use of visual information given by visual sensors (i.e. cameras) to control a robotic system. This kind of control turns out to be very useful in dynamic environments, as it allows us to know which objects are present in the scene with high accuracy, as well as their position, orientation and velocity. It makes possible to use robots in new domains where the workspace is not known a priori. However it also proposes to solve new problems (in terms of accuracy and robust-ness) currently unresolved [6]. The propose algorithm in this paper increases the accu-racy and robustness without requiring a thorough knowledge of the involved systems: calibration of the joint system and kinematic calibration of the vision system, both common sources of accumulative errors.

Visual servoing has been widely estudied in the lastest years, there are in the litera-ture interesting surveys [5] and [7]. Among the existing clasiffications of visual ser-voing, one of the most known is the way visual information is used to define the sig-nal error to control the system: position based visual servoing (PBVS or 3D) and the image based visual servoing (IBVS or 2D). In position based visual servoing, features are extracted from the image and used to reconstruct the 3D position of the target, whereas in image based visual servoing the task is defined in the image plane directly

Image Based Visual Servoing 851

through image features. In the latter a matrix is defined called the Image Jacobian, which linearly relates changes in image features and changes in Cartesian coordinates or changes in joints (in this case, image jacobian is called visual-motor jacobian).

This work, belongs to IBVS and contributes a new method based on the use of the epipolar restriction for the estimation of the Image Jacobian: this estimation can be carried out without the need for an accurate calibration, in these conditions the ana-lytical calculation of the Jacobian is impossible. An uncalibrated 3 DOF joint system with two fixed cameras has been used. The tests carried out consisted of tracking curve trajectories in the 3D space and comparig them with other existing algorithms. In previous works, [11], tests of static positioning have been described with very posi-tive results. In both cases a increase of robustness in presence of noise is achieved.

This paper is organized as follows: after the present introduction, section two de-tails the terminology and theoretical concepts used in the paper. Section three puts forward the innovative algorithm proposed, whilst section four describes the applied workspace and the results, and finally section five reflects our conclusions.

2 Image Jacobian

Assume that a robot or positioning system is observed from one or various fixed

views. Let [ ]Tp21 rrr=r be the p-dimensional vector that represents the posi-

tion of the end effector in a Cartesian coordinate system. Let [ ]Tn21 qqq=q

be the n-dimensional vector that represents the joint position of the robot. Let

[ ]Tm21 sss=s be the m-dimensional vector that represents the image features

(for example the coordinates of a point in one or both images).

The relation between joint velocity of the robot [ ]Tnqqq 21=q and its cor-

responding velocity in task space, [ ]Tp21 rrr=r , is captured in terms of the

robot Jacobian, rqJ , as qJr rq= . The relation between feature velocities

[ ]Tm21 sss=s and task space velocities, is given by rJs sr= .

The velocity of the image features can be directly related to joint velocities in terms of a composite Jacobian, also known as the full visual-motor Jacobian [2], [12]

JJJJqqJs ==

⎥⎥⎥⎥⎥

⎢⎢⎢⎢⎢

∂∂

∂∂

∂∂

∂∂

== rqsrsqmm

n

sq

qn

s

q

s

q

s

q

s

where;

1

1

1

1

(1)

Determining this matrix analytically is not simple. It is necessary to remark that in its calculation there must be considered: the intrinsic parameters of the camera cali-bration (focal distance, image center coordinates), the 3D reconstruction of the point or an approximation ( Z coordinate), the kinematic calibration of the camera (relation between camera coordinates and joint space origin), and the kinematic calibration of

852 L. Pari et al.

the robot. Most of the previous works on visual servoing assume that the system structure and the system parameters are known, or the parameters can be identified in an off-line process. A control scheme with off-line parameter identification is not ro-bust for disturbance, change of parameters, and unknown environments. One ap-proach to image-based visual servoing without calibration is to dynamically estimate the full visual-motor Jacobian during motion.

2.1 Estimation of the Jacobian

Specialized literature ([2], [12])gathers four methods for estimating the jacobian described in equation (1). In all cases an initial jacobian is obtained by making n linearly independent small movements.

Estimation Based on the Last Moves. If the change in image features and the change in joint position at moment k are represented respectively by 1−−=Δ kkk sss

and by 1kkk −−=Δ qqq , and the image jacobian is assumed to be constant for n

movements, it can then be estimated as the matrix that simultaneously satisfies n or more movements:

T Tk n 1 k n 1

Tk

T Tk k

− + − +⎡ ⎤ ⎡ ⎤Δ Δ⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥Δ Δ⎣ ⎦ ⎣ ⎦

s q

J

s q

(2)

where kJ is the image jacobian at moment k.

This method may not work well when several subsequent joint motions are gener-ated in the same direction, a not unusual situation. In these cases a badly-behaved ma-trix is obtained, and the consequent problems arise. Sutanto [12] solves the problem by detecting when this situation will occur and adding small exploratory motions.

Broyden Method. In this method the jacobian is estimated recursively, combining the information supplied by the last movement with the previous jacobian. Regarding the former method, it has the advantage of gathering information from all the movements. The equation for the Broyden method [2], [9] is:

kTk

Tkkk

kk

kk

tt

qq

qqJe

eJJ

ΔΔ

Δ⎟⎠⎞

⎜⎝⎛ Δ−Δ

∂∂+Δ−

+=−

1

1 (3)

being kkk sse −= * image features error, and *ks the desired features

Recursive Least Squares (RLS) Method. In this method the jacobian is estimated recursively by a least squares algorithm [9], [1], the equations of which, for a positioning task, are:

kkTk

kTkkk

kk

kk

tt

qPq

PqqJe

eJJ

ΔΔ+

Δ⎟⎠⎞

⎜⎝⎛ Δ−Δ

∂∂+Δ−

+=−

−−

−1

11

1 λ (4)

Image Based Visual Servoing 853

where

Tk 1 k k k 1

k k 1 Tk k 1 k

P q q P1P P

q P q− −

−−

⎛ ⎞Δ Δ= −⎜ ⎟λ λ + Δ Δ⎝ ⎠

(5)

The behaviour of this method depends on parameter λ , which varies in a range from 0 to 1, and ponders previous movements. λ settles a compromise between the information provided by old data from previous movements and new data, possibly corrupted by noise. In the presence of moderate noise, values of λ close to 0.9 are of-ten used.

Kalman Method. Another way to estimate the jacobian recursively is by using the Kalman filter [10]: the system is modelled through its state variables as follows:

υ⎧⎨⎩

x x

y C xk+1 k k

k k k k

= +

= +

η (6)

where ,υk k

η are the state noise and observer noise respectively, x is an ( )mn 1×

vector formed by concatenations of the row elements of the jacobian:

T

1 2 ms s s

q q q

⎡ ⎤∂ ∂ ∂= ⎢ ⎥∂ ∂ ∂⎣ ⎦

x … (7)

where is

q

∂∂

is the i-th row of the jacobian kJ . The measurement vector is

k1k1k ssy −= ++ (8)

and ( )m mn× matrix kC is the measured joint increase matrix:

Tk

kTk

q 0

0 q

⎡ ⎤Δ⎢ ⎥= ⎢ ⎥⎢ ⎥Δ⎣ ⎦

C (9)

The Kalman equations are:

[ ][ ]

k 1 k

1T Tk 1 k 1 k k k 1 k

k 1 k 1 k k 1

k 1 k k 1 k 1 k k

P P R

K P C C P C R

P I K C P

ˆ ˆ ˆx x K y C x

η

υ

−+

−− −+ + +

−+ + +

+ + +

= +

⎡ ⎤= +⎣ ⎦= −

= + −

(10)

where kP is the error covariance matrix at moment k, k 1P−+ is the prediction of kP ,

R ,Rη υ are the state and output covariance matrices respectively and k 1K + is the

Kalman gain.

854 L. Pari et al.

2.2 Multiple-View Jacobian

When several views are used, the full visual-motor Jacobian can be defined as the concatenation of the partial Jacobians of each view (more detailed in [8], [1]). All the Jacobians share the same joint increments, although visual features are managed in-dependently. Here lies main contribution of this paper: if the features are points, the epipolar constraint is incorporated in the estimation of the Jacobian. The notation we used is detailed in section 3.2.

In previous works [11], we carried out experiments comparing the results given us-ing one of the cameras and those given using both cameras: our results showed that using two cameras instead of one improved the behaviour of all the methods we tested. In many applications, improvement in the performance more than justifies the possible disadvantages: increased equipment cost or calculation time. Therefore, the experiments described in this work have been performed using both cameras.

2.3 Control Law

A control law have been utilized for tracking a twisted quadratic trajectory. It adds to the proportional control law [9] a predictive term from the last two references accord-ing to the equation:

( )*1

**1 −

++ −+−+= kkkkkk ssssJqq (11)

being ( ) T1T JJJJ−+ = the pseudoinverse for the image jacobian.

3 Proposed Algorithms

The research line we have followed is the estimation of the jacobian based on already performed movements, not necessarily the n last ones. We use the visual information supplied by two cameras. We contribute two innovations: on one hand, each move-ment has been endowed with a certain reliability, so that the most adequate or reliable movements can be used. On the other hand, the epipolar constraint has been taken into account in the calculation of the jacobian, which significantly increases the ro-bustness of the method, as will be seen in section four. It must be remarked that al-though we propose the joint application of both innovations, using them independ-ently of each other also improves the results.

The visual features we use are centroids of points. The proposed task consists in making these visual features follow a trajectory set in advance.

3.1 Reliability Estimation

The jacobian matrix, equation (1), strongly depends on the position of the point in the image (u,v), so the assumption that it is constant will only be valid in the surroundings of the point in the image. For this reason, a first possibility would be to promote the consideration of those already performed movements with a short path in image fea-tures. However, these movements are too sensitive to noise, so an agreement must be reached between both effects. Movements performed in the joint surroundings of the

Image Based Visual Servoing 855

desired movement also seem more adequate. The proposed algorithms rank the al-ready performed movements according to a reliability which depends on two factors that assemble these concepts:

kiiki FactorFactoryreliabilit 2/1 = (12)

Where subindex k represents the last movement performed and subindex i will vary amongst those already performed movements which have been stored.

Factor1i promotes movements in which features vary within a range: they are not too large so that the jacobian can be considered a constant, nor too small so they will not be too sensitive to noise. The formula we used is:

( )R

F

Fabs

11Factor

ii

+−Δ

=s

(13)

where R represents the estimated error in feature calculation (for example, for 5 points detected with subpixel precision, R would have a value close to 1.0 pixels), and F represents the most reliable feature increment (for example, 20 pixels). We must re-mark that Factor1i does not depend on the last performed movement, so it will be cal-culated and stored together with its corresponding movement.

Factor2ki promotes those already stored movements produced in the joint sur-roundings of the desired movement, comparing for this purpose the current joint posi-tion with the mid point of already reached joint positions. Its expression is:

⎟⎠⎞

⎜⎝⎛ +

−= −

22Factor 1ii

kkiqq

q (14)

Greater reliability will be given to those performed movements that have greater value for the expression (12).

3.2 Adding the Epipolar Constraint

A point in space [ ]1T

X Y Z=S , where represents homogeneous coordinates,

is projected in the first image in point [ ]' ' ' 1 'T

x y= =s P S , where 'P represents

the (3x4) projective matrix of the first camera. This point will likewise be projected in

the second image in point [ ]'' '' '' 1 ''T

x y= =s P S where ''P represents the (3x4)

projective matrix of the second camera. Points are expressed in homogeneous coordi-nates and an image acquisition model without distortion is assumed. The Euclidean calibration of both cameras implies the knowledge of matrices ', ''P P .

The projection of the same point on both images must satisfy the equation:

'' ' 0T =s F s (15)

where F is a (3x3) matrix known as the fundamental matrix or epipolar constraint. Its knowledge is known as weak or projective calibration, and its detection is much more

856 L. Pari et al.

robust than Euclidean calibration. A more detailed description can be found in [3] and [4].

Our method considers the epipolar constraint, equations (15) in the calculation of the image jacobian, equation (1). If the considered visual features are centroids of points, and if we note a point in the first camera by ‘, and by “ in the second camera, we have the following model:

⎥⎦

⎤⎢⎣

⎡′′′

=⎥⎦

⎤⎢⎣

⎡′′′

=⎥⎦

⎤⎢⎣

⎡′′′

=−

−− J

JJ

s

ss

s

ss ;;

1

11

k

kk

k

kk (16)

kkkkkk qJssqJss Δ′′+′′=′′Δ′+′=′ −− 11 ; (17)

[ ] 01

1 kTk =⎥

⎤⎢⎣

⎡ ′′′

sFs (18)

[ ] 01

1 1kT1k =⎥

⎤⎢⎣

⎡ ′′′ −−

sFs (19)

Substituting in (18) the values obtained in (17) and considering that equation (19) must be satisfied, we have the following non-linear equation for ( )JJ ′′′ , :

[ ] 011 1

1 =Δ′′′+⎥⎦

⎤⎢⎣

⎡ ′′′Δ+Δ′′′Δ −

−k

Tk

kTTkk

TTk qJFs

sFJqqJFJq (20)

The linear equations in (2) and the non-linear equations in (20) have been jointly solved applying the Levenberg-Marquadt method.

3.3 Glossary of Tested Algorithms

In the implementation described in the present article, the following algorithms have been tested:

• 3LAST: Last three movements performed. Since our joint system has three degrees of freedom, equation (2) can be solved using the information on joint and feature velocities provided by three movements.

• 3+REL: Three most reliable movements amongst the last ten performed. • 10LASTW: Last ten movements performed, weighted by their reliability. Weight-

ing is introduced multiplying each row in equation (2) by its corresponding reli-ability.

• 10+REL: Ten most reliable movements performed. • FUNDMAT: Ten most reliable movements performed, adding the epipolar con-

straint. • BROYDEN: Broyden method. • RLS: Recursive least squares method. • KALMAN: Kalman filter estimation.

Image Based Visual Servoing 857

Algorithms 3+REL, 10LASTW, 10+REL and FUNDMAT are original, innovative methods founded on the estimation of the jacobian based on the last moves.

4 Experiments

In this section we describe our experimental equipment and results.

4.1 Experimental Setup

The system used in the experiments consists of:

• A joint system composed of a high precision positioning device and its controller, model Newport MM3000 (see Fig. 1). The system has 3 DOF with a prismatic and two revolute joints, and its theoretical precision is of a thousandth of a millimeter and a thousandth of a degree. The visual control object, made out of five black dots on a white background, the projection of which on the image will be the control features, has been attached to the last link of the joint system.

Fig. 1. Experimental setup

• An image acquisition and processing system composed by two CV-M50 analogic cameras and a Matrox Meteor II-MC image acquisition board, which allows simul-taneous acquisition from both cameras. The cameras, fixed in the working envi-ronment, are separated by about 300 millimeters, both their axes converge towards the joint system, and they are separated from it by about 700 millimeters. Visual features are detected with subpixel precision, and given the simplicity of the image, the error is estimated to be of less than 0.2 pixels. Communication with the joint system controller is established through a serial RS-232C cable.

We must remark that the joint system is a closed system: once a position has been selected, control will not be handed over until it is reached. This course of action is unac-ceptable for a visual control system that must use the information supplied by the cam-eras as soon as possible. In order to simulate a more adequate behaviour, we have chosen to limit joint motion according to the estimated time it would take the vision system to acquire and process images. On the other hand, the high accuracy of the employed sys-tem allows us to perform a thorough comparative analysis of the proposed algorithms.

858 L. Pari et al.

4.2 Control Objective

The task entrusted to the system is to achieve the tracking of a trajectory set in ad-vance (see Fig. 2) by using the information obtained on detection of certain image fea-tures (centroids of projected dots). We intend to contrast the performance of the pro-posed methods of estimation of the image jacobian using the control law (section 3.2.)

Visual features must be reachable and the visual object must be visible from both points of view. To ensure coherence, we decided to obtain the desired visual features previously by acquiring images in reference joint positions, chosen randomly within the workspace (teach-by-showing). Due to the dependency of the jacobian on visual features and on the point in joint space, we have chosen to link a high number of tra-jectories (50) in order to obtain more representative results for the comparative study of the proposed algorithm.

4.3 Evaluation Indices

To evaluate the effectiveness of each method, we consider three indices, defined as follows:

• Index 0: Sum of Euclidean distances between desired and current visual features. Weighted by number of points, number of cameras and number of trajectories.

• Index 1: Sum of Euclidean distances in joint space for all of the performed move-ments, divided by one thousand. Weighted by number of trajectories.

• Index 2: Sum of Euclidean distances between desired and current joint positions. Weighted by number of points.

Table 1. Values for three indices, with and without added noise. Predictive law.

WITHOUT NOISE WITH NOISE

ALGO-RITHM 2

POINTS 3

POINTS 4

POINTS 5

POINTS 2

POINTS 3

POINTS 4

POINTS 5

POINTS 3LAST 16.6 17.9 18.6 18.5 20.4 21.8 21.5 22.1 3+REL 16.8 18.2 18.0 17.8 17.2 21.0 19.6 22.0

10LASTW 16.5 17.5 17.9 17.7 18.4 17.8 20.7 19.9 10+REL 16.5 17.7 17.9 17.7 18.5 20.9 19.4 21.4

FUNDMAT 16.5 17.5 17.7 17.6 16.6 17.7 17.9 17.7 BROYDEN 16.4 18.2 18.7 17.8 16.9 19.6 22.8 19.0

RLS 16.5 17.7 18.1 17.7 16.7 18.8 18.4 18.4

IND

EX

0

KALMAN 16.0 17.3 17.7 17.6 17.4 18.0 18.7 18.5 3LAST 3.43 3.48 3.68 3.63 2.22 2.55 2.29 2.39 3+REL 3.68 3.47 3.44 3.42 3.60 2.45 3.66 2.40

10LASTW 3.40 3.37 3.37 3.37 3.39 3.57 2.92 3.40 10+REL 3.37 3.32 3.34 3.35 3.18 2.42 3.44 2.49

FUNDMAT 3.35 3.33 3.34 3.35 3.36 3.34 3.31 3.34 BROYDEN 3.37 3.43 3.66 3.38 3.34 2.86 3.84 3.32

RLS 3.34 3.31 3.38 3.34 3.44 3.59 3.49 3.44

IND

EX

1

KALMAN 3.37 3.41 3.48 3.42 3.65 3.53 3.39 3.46

3LAST 0.77 0.98 1.84 0.99 7.08 5.70 5.34 5.55 3+REL 0.94 0.67 0.58 0.41 1.88 5.66 1.97 5.36

10LASTW 0.59 0.33 0.39 0.34 4.11 0.78 4.99 3.65 10+REL 0.46 0.69 0.53 0.37 4.14 6.01 2.97 5.14

FUNDMAT 0.42 0.50 0.38 0.29 0.45 0.60 0.45 0.29 BROYDEN 0.38 0.80 1.60 0.45 1.42 3.64 6.19 2.38

RLS 0.50 0.43 0.58 0.34 1.17 2.81 1.06 1.17

IND

EX

2

KALMAN 0.21 0.42 0.62 0.56 3.45 1.63 1.73 1.58

Image Based Visual Servoing 859

4.4 Results

A comparative study was conducted on the proposed algorithms. The comparison covers visual features calculated with an estimated error of 0.2 pixels, therefore con-sidered without noise, as well as visual features with artificially added Gaussian noise with a standard deviation of 0.5 pixels. Also, the effect of increasing the number of points considered as visual features from 2 to 5 is analyzed.

Table 1 displays the results obtained with the control predictive law, with or with-out added noise for all algorithms analyzed according to the three defined indexes and varying the number of points taken into account. The methods which give better re-sults are KALMAN, RLS, FUNDMAT and 10LASTW. With added noise the most robust method is FUNDMAT, especially on index 2, although RLS and KALMAN methods also show a good behavior. Increasing the number of points has two opposite effects: we have more information to control the system, but also stricter requirements to fulfill, which causes variations in the indices.

Fig. 2 ( FUNDMAT), Fig. 3 (RLS) and Fig. 4 (KALMAN) represent the evolution in joint space for each method of estimation of the jacobian. The red points represent points to be reached in the reference trajectory, and the blue line shows the evolution of the joint system. For the two laws considered, with and without added noise, the method that performs the best tracking is the one that considers the epipolar restric-tion (FUNDMAT).

Fig. 2. Evolution for five points, FUNMAT algorithm without and with noise

Fig. 3. Evolution in joint space for five points, RLS algorithm without and with noise

860 L. Pari et al.

Fig. 4. Evolution in joint space for five points, KALMAN algorithm without and with noise

5 Conclusions

The on-line estimation of the image jacobian is a flexible and versatile method for the visual control of a joint structure, since it isolates the obtained results from errors in the calibration of the camera and the joint system. This paper contributes the defini-tion of a reliability to calculate the jacobian, and the inclusion of the epipolar con-straint or fundamental matrix in its calculation. This aspect is not considered in spe-cialized literature, and it improves the results significantly when the noise level in feature detection increases. The knowledge of the fundamental matrix is no objection, as its calculation has been proven to be much more simple, robust and reliable than that of the complete calibration of the cameras and joint system.

Some aspects not dealt with in the present paper which are being currently studied are the analysis of the system stability with a control law generated from the jacobian estimation, and the analytic calculation of the image jacobian. Regarding the first as-pect, we must remark that algorithm FUNDMAT has never made the system unstable throughout the many tests performed.

This work was supported by the Comisión Interministerial de Ciencia y Tecnología of the Spanish Government under the Project DPI2004-07433-C02-02.

References

[1] M. Asada, T. Tanaka, K. Hosoda, Adaptive Binocular Visual Servoing for Independently Moving Target Tracking, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’00), (2000) 2076-2081.

[2] Z. Deng, M. Jägersand, Evaluation of Model Independent Image-Based Visual Servoing, Canadian Conference on Computer and Robot Vision, (2004), 138-144

[3] O. Faugeras, Q.T. Luong, The Geometry of Multiple Images, The Massachusetts Institute of Technology Press, 2001

[4] R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge University Press, 2000

[5] S.A. Hutchinson, G.D. Hager, P.I. Corke, A tutorial on visual servo control, IEEE Trans. Robotics and Automation, 12 (1996) 651-670.

Image Based Visual Servoing 861

[6] D. Kragic, H.I. Christensen, Advances in robot vision, Robotics and Autonomous Sys-tems. 52 (1) (2005) 1-3

[7] D. Kragic, H.I. Christensen, Survey on visual servoing for manipulation, Technical Re-port ISRN KTH/NA/P-02/01-Sen, CVAP259, 2002

[8] B. Lamiroy, B. Espiau, N. Andreff and R. Horaud, Controllling Robots With Two Cam-eras: How to Do it Properly, Proceedings of the International Conference on Robotics and Automation, ICRA’00, (2000), 2100-2105

[9] J.A. Piepmeier, G.V. McMurray, H. Lipkin, Uncalibrated Dynamic Visual Servoing, IEEE Transactions on Robotics and Automation, 20 (1) (2004) 143-147.

[10] J. Qian, J. Su, Online estimation of Image Jacobian Matrix by Kalman-Bucy filter for un-calibrated Stereo Vision Feedback, International Conference on Robotics & Automation (ICRA’02), (2002), 562-567

[11] J.M. Sebastián, L. Pari, C. González, L. Ángel, A New Method for the Estimation of the Image Jacobian for the Control of an Uncalibrated Joint System, Lecture Notes in Com-puter Science, Vol. 3522, 2005, pp. 631-638.

[12] H. Sutanto, R. Sharma, V. Varma, The role of exploratory movement in visual servoing without calibration, Robotics and Autonomous Systems 23 (1998) 153-169.