Humanoid Oculomotor Control Based on Concepts

of Computational Neuroscience

Tomohiro Shibata∗1, Sethu Vijayakumar∗2, Jorg Conradt∗3, and Stefan Schaal∗2

*1 Kawato Dynamic Brain Project, ERATO, JST

*2 Computer Science & Neuroscience, University of Southern California

*3 Institute of Neuroinformatics, University/ETH Zurich

Keywords: oculomotor control, computational neuroscience, learning control, VOR-OKR, visual attention

Abstract

Oculomotor control in a humanoid robot faces sim-ilar problems as biological oculomotor systems, i.e.,the stabilization of gaze in face of unknown pertur-bations of the body, selective attention, the complex-ity of stereo vision and dealing with large informationprocessing delays. In this paper, we suggest controlcircuits to realize three of the most basic oculomotorbehaviors - the vestibulo-ocular and optokinetic reflex(VOR-OKR) for gaze stabilization, smooth pursuit fortracking moving objects, and saccades for overt visualattention. Each of these behaviors was derived frominspirations from computational neuroscience, whichproves to be a viable strategy to explore novel controlmechanisms for humanoid robotics. Our implementa-tions on a humanoid robot demonstrate good perfor-mance of the oculomotor behaviors that appears natu-ral and human-like.

1 Introduction

1.1 Research Objectives

The goal of our research is to understand the prin-ciples of information processing of the human brain,with a focus on basic sensorimotor control and thehope to expand this scope increasingly more towardsmore cognitive topics. As a research strategy, we chosean approach that emphasizes the interplay betweencomputational neuroscience and humanoid robotics.In this approach, research topics are initially inves-tigated from the present stage of knowledge of neu-robiology and, subsequently, abstract computationalmodels are created that can be implemented on ahumanoid robot to accomplish interesting behavioralgoals. Control theory and learning theory are em-

ployed to examine the validity of the models and thesuccess of the models in actual behavior is investi-gated. Theoretical and experimental insights are thenused to re-evaluate biological data and the presentstage of modeling, which usually leads to suggestionsfor improvement in both neuroscientific research andcomputational modeling. For the purpose of this re-search strategy, we developed a humanoid robot sys-tem with 30 degree-of-freedoms (DOFs), each of whichis hydraulically operated and mimics the complianceof humans by means of impedance control in eachjoint. Kinematics and dynamics of the robot are asclose as possible to the human counterpart.In this paper, we present results of our research

in the field of oculomotor control, i.e., learning thevestibulo-ocular and optokinetic reflex (VOR-OKR),smooth pursuit, and saccades for overt visual atten-tion. Oculomotor control is one of the best inves-tigated areas in computational neuroscience due tothe relative simplicity of the primate oculomotor sys-tems and rich set of oculomotor behaviors, includingreflexes and adaptation. For example, the monkey’soculomotor system can be approximated by a secondorder linear system. It has only 3 DOFs per eye, andusually only 1 DOF is used in neurobiological exper-iments. Furthermore, in the case of reflex behaviors,experimenters can eliminate the effects of attention,and easily elicit the reflex behavior by simple stimulisuch as rotation of the head, spot lights, random dotdisplays, etc. Such simple experimental paradigms areideally suited to draw comparisons between biologicalknowledge, computational models, and empirical eval-uations in robotic experiments.

1.2 Robotic Head Setup

In the next sections, we will present computationalmodels for the three oculomotor behaviors we exam-

Figure 1: Humanoid vision head

ined and the corresponding experimental results. Inall experiments, the same platform - the vision head(see Fig. 1) of our Humanoid Robot[1] - was used.

The robot head has 7 DOFs in total, a neck with3 DOFs and two camera eyes, each equipped with 2independent DOFs, arranged as pan and tilt. In orderto provide high-resolution vision simultaneously withlarge-field peripheral vision, the robot employs twocameras per eye, a foveal camera (24 deg view-anglehorizontally) and a wide-angle camera(100 deg view-angle horizontally). This setup mimics the foveatedretinal structure of primates, and it is also essentialfor an artificial vision system in order to obtain highresolution vision of objects of interest while still beingable to perceive events in the peripheral environment.

The learning controller is implemented with thereal-time operating system VxWorks using severalparallel Motorola PowerPC processors in a VME rack.Visual processing is performed out of specialized hard-ware, a Fujitsu tracking vision board and QuickMag(Japan) color vision tracking system. The Fujitsutracking vision board calculates retinal error (positionerror) and retinal slip (velocity error) information ofeach eye at 30Hz. The QuickMag system returns thecentroid of blobs of pixels of pre-specified colors in theenvironment. Up to six different colors can be trackedsimultaneously at 60Hz sampling rate.

The oculomotor control loop runs at 420 Hz, whilethe vision control loop runs at 60 Hz due to the Quick-Mag video processing rate. The oculomotor controlloop implements a strong spring and damping termsuch that the nonlinearities of the oculomotor systemdue to hydraulics and attached cables become negligi-ble.

2 Vestibulo-Ocular-Reflex (VOR)

This section outlines the computational model ofVOR-OKR we developed [1], shown schematically inFig. 2. The VOR reflex serves to keep the eyes fixedon a target in case that there is a mechanical pertur-bation of the head, e.g., as caused by locomotion. TheOKR has a similar functionality, just that it is trig-gered by a movement of the entire visual field, whichit tries to compensate for - a typical movement thatwould be elicited in a movie theater when the entirescene on the screen moves. The inputs to the VOR-OKR system are i) the visual target in camera coor-dinates and ii) an angular velocity signal generatedfrom a gyroscopic sensor due to perturbations of therobot’s body; since the sensor is attached to the head,the signal is referred to as “head angular velocity”.From the target position and eye position, retinal er-ror and retinal slip can be computed. In the simplestcase, the ideal compensatory desired movement of theeyes would be the negative of the retinal slip, but, ingeneral, a nonlinear transformation from retinal sliperror velocity to eye movement is needed due to off-axis effect, i.e. the fact that head axis and eye axis arenot collinear. The retinal error signals are also used asinput to a PD controller in the bottom part of Fig. 2.The gains of this PD controller have to be kept rathersmall due to the delays incurred in visual informationprocessing. The output of the PD controller servesprimarily as a teaching signal to the feedback errorlearning system. However, it is also needed to stabilizethe crude feedforward controller in the shaded blockof Figure 2. Without the feedback input, the feedfor-ward controller would only be marginally stable due tothe floating integrator. As described in a later section,eligibility traces, realized by a second order filter, arealso used as inputs to the learning system.The entire control systems is quite similar to what

has been discovered in the primate oculomotor sys-tem. The a priori existing feedforward controller pro-vides some crude functionality of the VOR while theexisting feedback controller provides acceptable OKRperformance for slowly changing visual targets andacts as a compensatory negative feedback for the VORmodule. These systems form what is called the “di-rect pathway” of oculomotor control in biology. Byadding a learning controller trained with the feedback-error-learning [2] strategy in the indirect pathway (seeFig. 2), excellent VOR performance can be accom-plished even if the feedback pathway has large delays(see Section 2.1). Furthermore, the OKR performanceis improved to some extent. This learning network,known to be located in the primate cerebellum, ac-

��������������������

�

�

� ���������� ����

��

�������������������

�

�����������������

�

�

�

�

������������� ��������������

�������

��������

��������������

�����

���������������

�������������

Figure 2: Our VOR-OKR model

quires an inverse dynamics model of the oculomotorplant during the course of learning that compensatesfor the missing performance of the crude feedforwardcontroller in the shaded box (c.f. Fig. 2). The coor-dination of a direct and indirect pathway is analogousto how the cerebellar pathway acts in parallel to thebrainstem pathways [3]. As discussed in [1], this con-trol system is equally suitable for both biological androbotic oculomotor control. However, for the roboticcontrol, we augment the biologically plausible modelwith a fast nonlinear learning scheme based on non-parametric regression networks [4].

2.1 Learning with the delayed-error sig-nal

For successful feedback-error-learning, the time-alignment between input signals and the feedback-error signal is theoretically crucial, and, thus, addi-tional techniques are required in the case of delayedsensory feedback. For instance, if a perturbation of thehead or body has frequency components that are muchfaster than the delay in the feedback pathway duringVOR learning, the phase delay in the feedback path-way gets large such that learning speed decreases, orlearning can even become unstable in the worst case.To solve this “temporal credit assignment prob-

lem”, the concept of eligibility traces has been sug-gested in both biological modeling and machine learn-ing [5]. For neurons in the brain, it is assumed that asecond messenger would tag a synapse as eligible formodification. This “tag” would decay with an appro-priate time constant, thus forming a temporal eligibil-ity window. Schweighofer et al. proposed a biologi-cally plausible learning model for saccadic eye move-ment, and modeled the second messenger as a secondorder linear filter of the input signals to the learning

0 5 10 15 20 25 300.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

Time [sec]

Rec

tifie

d m

ean

retin

al s

lip [r

ad]

Figure 3: Time course of the rectified mean retinalerror: While the dashed line represents data obtainedwithout eligibility trace, the solid line shows data ac-quired with eligibility trace.

system[6]. For this purpose, note that second orderfilter is better than first order filter. The impulse re-sponse of second order filter has a unimodal peak at adelay time determined by the time constant of the fil-ter. For successful learning, the delay time only has toroughly coincide with the actual delay of the sensoryfeedback. In related work, Fagg proposed a cerebel-lar learning model where eligibility traces modeled bysuch a second order filter are employed[7]. Applyingthis technique to feedback-error learning, we completeour final learning control system (see Fig. 2) where theimpulse response of a second order linear filter is putjust before the “learning” box.

2.2 Experimental result

In our system, the visual processing introduces adelay of more than 60 ms in the retinal signals. Inthis experiment, we test how the eligibility traces canimprove the efficiency of VOR learning. For this pur-pose, head movement was generated by three super-imposed sinusoidal signals with frequencies of 0.6, 2.0,and 3.0 Hz and amplitude of 0.1 rad, respectively. Afrequency of 3.0 Hz is high enough to obtain sustainedblurred visual images. Fig. 3 shows the time courseof the rectified retinal error during learning obtainedfrom a moving average using a one second time win-dow. While the dashed line represents data obtainedfrom learning without eligibility traces, the solid lineshows data acquired with eligibility traces. This figureshows that eligibility traces are necessary for success-

ful learning as the the retinal error does not decreasewithout using the traces. It should also be noted thatlearning proceeds quite rapidly such that in less than ahalf a minute, the initial errors are reduced by a factorof two. Longer learning results in a further reductionof the error [1].

3 Smooth Pursuit

Smooth pursuit refers to the oculomotor behaviorof tracking a moving target on the fovea - a task whichrequires high accuracy; for instance such behaviors areneeded to visually inspect moving object. Strong evi-dences (e.g.,[8]) support the idea that smooth pursuitemploys some predictive control mechanism. From thecontrol theoretical view, this observation is not sur-prising since smooth pursuit cannot be achieved by asimple visual negative feedback controller due to thelong delays inherent in visual information processing(e.g. around 70 ms in our humanoid vision system,and around 100 ms in the human brain).In the field of robot vision, many projects inves-

tigated visual servoing, but, to our knowledge, with-out ever examining a smooth pursuit controller thathas similar features and performance as that in pri-mates. One of the most related pieces of research is inBradshaw et al. [9], which employed a Kalman filterfor prediction. However, these authors assumed priorknowledge of the target dynamics and, thus, avoidedto address how unknown target motion can be trackedaccurately. In contrast, in this paper, we present abiologically motivated smooth pursuit controller thatlearns to predict the visual target velocity in head co-ordinates based on fast on-line statistical learning ofthe target dynamics. In the following sections, we willfirst explain the setup of our smooth pursuit model,explain the learning component, and, in Section 3.2,describe how to accelerate learning speed. The sug-gested control system is evaluated in simulations andexperiments with our humanoid robot.

3.1 Cascade of predictor and controller

Fig. 4 presents our smooth pursuit model. It con-sists of two subsystems: one is a target velocity pre-dictor, and the other is an inverse model controller ofthe oculomotor system. In this control diagram, s and1/s are Laplace transform operators denoting differ-entiation and integration, respectively. ∆ stands fora constant delay element. e, e, E, and E are the reti-nal error and the retinal slip, the eye angular position,and the eye angular velocity, respectively.

The predictor outputs an estimate of the currenttarget velocity ˆT(t) out of a history of past estimatedtarget angular positions T(t−∆) and velocities T(t−∆). In linear systems, the state predictor of a n-thorder linear system can be defined as:

xt+1 = Axt (1)

where x is the n × 1 state vector and A is the n × nstate transition matrix. As we are only interested invelocity prediction in this paper, we reduce equation 1to focus only on the the states that are identified withtarget velocities, not positions:

ˆTt+1 = A2xt (2)

where A2 is the the appropriate submatrix of A cor-responding to the target velocity component.Since, similar to the primates’ eye plant, many ar-

tificial oculomotor systems can be approximated bya second order linear dynamical system, a linear in-verse dynamics model suffices for feedforward controlof the eye system. It receives as input the desiredeye angular position, velocity, and acceleration. Thefeedforward controller is stabilized by a PD feedbackpathway whose gains need to be rather low due to itsoperating out of strongly delayed signals.

�

�

�

Figure 4: Our smooth pursuit model

3.2 Learning the discrete predictor fromthe delayed signals

The learning scheme in Fig. 4 may appear difficultto implement as it has to learn the target dynamicsout of the history of past estimated target states andthe delayed retinal error. As will become apparent inthe next paragraphs, however, a straightforward de-velopment allows us to solve this learning problem.At time t, the predictor can only see the delayed

estimated target state xt−∆. The corresponding dis-crete target velocity prediction is represented as

ˆTt = f(xt−∆,wt) (3)

where w is a parameter vector. Let ξ, the velocityprediction error, equal T− ˆT, and let the loss functionJ be the simple squared error:

J =12ξ2t (4)

Thus, a gradient descent learning rule for w can bewritten as:

(dwi

dt)t = −ε( ∂J

∂wi)t = ε(

∂f

∂wi)t−∆ξt (5)

with ε denoting the learning rate. If we can make theassumption that the predicted target velocity ˆT willbe tracked accurately by the robot without delay, wecan regard the retinal slip as the prediction error givenby ξ = T− ˆT � T− E = e. The learning rule can thusbe rewritten as:

(dwi

dt)t = ε(

∂f

∂wi)t−∆et (6)

Note that the time alignment of the predictor output fand the error ξ (� e) needs to be correct for successfulminimization of the loss function J . Since the predic-tor has no access to et at time t, a modified learningrule is required by introducing a delayed form of Eq. 6:

(dwi

dt)t = ε(

∂f

∂wi)t−2∆et−∆ (7)

Thus, the predictor is required to keep the information∂f/∂wi in memory for the duration of ∆.In summary, it is important to use the most recent

information for prediction, but to use one delayed by∆ for learning in order to achieve successful learningand control.

3.3 Experimental result

We implemented online learning of the smooth pur-suit behaviour on our humanoid robot using the cir-cuit decribed above. Our model does not rely on anyspecific or particular learning method. For trackingan target having a simple linear dynamics such as aswinging pendulum, we employed an adapted versionof recursive least squares (RLS), and obtained excel-lent results. For more complex target motion withnonlinear dynamics, we replaced RLS in our smoothpursuit controller with nonparametric regression net-works we developed. This system was used to learna periodic motion generated by van der Pol equationsimplemented on a separate industrial robot in our lab-oratory. Fig. 5 shows the excellent learning results ofthis experiment.

0 5 10 15 20 25 30 35 40−0.5

0

0.5

posi

tion

[rad

]

0 5 10 15 20 25 30 350

0.1

0.2

0.3

0.4

rect

ified

ret

inal

err

or [r

ad]

time [s]

Figure 5: Smooth pursuit of a target following a tra-jectory that is generated by Van der Pole equation.The upper figure shows the time course of the angu-lar position of the visual target (dotted) and the eye(solid). The lower figure presents the time course ofthe rectified mean retinal error (smoothed with mov-ing average of time window 1 s)

4 Saccade for Overt Visual Attention

Visual attention involves directing a “spotlight” ofattention [10] to interesting areas, extracted from amultitude of sensory inputs. Most commonly, atten-tion will require to move the body, head, eyes, or acombination of these in order to acquire the targetof interest with high-resolution foveal vision, referredto as ‘overt’ attention, as opposed to covert attentionwhich does not involve movement.There has been extensive work in modeling atten-

tion and understanding the neurobiological mecha-nisms of generating the visual “spotlight” of atten-tion [11], both from a top-down[12] and a bottom-upperspective [13, 14] - albeit mainly for static images.From the perspective of overt shift of foci, there hasbeen some work on saccadic eye motion generationusing spatial filters [15], saccadic motor planning byintegrating visual information [16], social robotics [17],and humanoid robotics [18]. In contrast to this previ-ous work, our research focus lies on creating a biologi-cally inspired approach to visual attention and oculo-motor control by employing theoretically sound com-putational elements that were derived from models ofcortical neural networks, and that can serve for com-parisons with biological behavior. We also emphasizereal-time performance and the integration of the at-tention system on a full-body humanoid robot that is

not stationary in world coordinates. As will be shownbelow, these features require additional computationalconsideration such as the remapping of a saliencymap for attention after body movement. In the fol-lowing sections, we will first give an overview of theattentional system’s modules, then explain the com-putational principles of each module, before we pro-vide some experimental evaluations on our humanoidrobot.

�������������� �����

�� ��������������

�����������������

��������

��������

���������������

������� �����

������������ �����

���� ������������ �� �����

��������������

!��� �" ������������

����� �������

�����������������

���� ���������������#�� $����������%

������������

��$�������&$������%

Figure 6: A schematic block diagram of the variousmodules involved in the system for implementing overtvisual attention

4.1 Sensor pre-processing and integration

The key element of our Sensory Pre-Processingblock (Fig. 6) is a competitive dynamical neural net-work, derived in Amari and Arbib’s [19] neural fieldsapproach for modeling cortical information process-ing. The goal of this network is to take as input spa-tially localized stimuli, have them compete to becomethe next saccade target, and finally output the win-ning target. For this purpose, the sensory input pre-processing stage takes the raw visual flow VF (x, t) asinputs to the stimulus dynamics, a first order dynam-ical system. Using x to denote the position of a stim-ulus in camera coordinates, the stimulus dynamics is:

S(x) = −αS(x) + V isInp(x, t) (8)

where

V isInp(x, t) =∫

R

G(x, t) ∗ exp(−x2/2σ2)dx (9)

G(x, t) = VF (x, t) + γ ∗ �VF (x, t)�+(10)

0 5 10 15 20 25

05

1015

200

1

2

3

0 5 10 15 20 25

05

1015

20−1

−0.5

0

0.5

STIMULUS DYNAMICS

ACTIVATION DYNAMICS

Population Code Computationof Saccade Target

(a)

0 5 10 15 20 25

05

1015

200

1

2

3

0 5 10 15 20 25

05

1015

20−1

−0.5

0

0.5

STIMULUS DYNAMICS

ACTIVATION DYNAMICS

Inhibition of Return

(b)

Figure 7: A snap shot of the stimulus and activationdynamics just (a) before and (b) after the saccade

Eq.(10) enhances the raw visual flow vector when it isincreasing to emphasize new stimuli in the scene, whileEq.(9) implements a Gaussian spatial smoother of thestimuli to reduce the effects of noise. The variable αwas set to a value of 100 in our experiments. Thetop of Fig. 7a shows an example of a typical stimuluspattern in the two dimensional neural network due toa moving object at the top-left of the camera image. Ingeneral, we could have multimodal sensory inputs, e.g.from color detectors, edge detectors, audio input, etc.,feeding into Eq.(10) as a sensory signal. As suggestedby Niebur, Itti and Koch [13, 14], it would be useful toweight these inputs according to their importance inthe scene, usually based on some top-down feedbackor task-specific biasing (e.g., if we know that color ismore important than motion).This stimulus dynamics feeds into a saliency map

[10], essentially a winner-take-all (WTA) networkwhich decides on a winning stimulus from many si-multaneous stimuli in the camera field. The winningstimulus will become the next saccade target or fo-cus of overt attention. The WTA network is realizedbased on the theory of neural fields, a spatial neuralnetwork inspired by the dynamics of short range ex-citatory and long range inhibitory interactions in the

neo-cortex [19, 20]. The activation dynamics u(x, t)of the saliency map is expressed as:

τu(x) = −u(x) + S(x) + h

+∑x′

w(x,x′)σ(u(x′)) (11)

Here, h is the base line activation level within the field,S(x, t) is the external stimulus input (Eq.8), w(x,x′)describes the coupling strength between all the unitsof the network, and σ(u) controls the local thresholdof activation. Depending on the choice of parameterh and the form of σ and w, the activation dynamicsof Eq.(11) can have various stable equilibrium points[19]. We are interested in a solution which has uniformactivation at base line level in the absence of externalstimuli, and which forms a unimodal activation pat-tern at the most significant stimulus in the presenceof stimuli that are possibly dispersed throughout thespatial network. This is achieved by choosing a trans-fer function:

σ(u) = 1/(e(−cu) + 1) (12)

with constant c >> 1 and an interaction kernel withshort range excitation and long-range inhibition termH0:

w(x,x′) = ke−(x−x′)2/σ2w − H0 (13)

The constants were fixed at τ = 0.01, h = −0.5,H0 = 0.75, k = 4, σ2

w = 1.4, and c = 5000, thevalues of which were decided based the magnitude ofthe stimulus dynamics S(x, t), as outlined in [19].In addition to the stimulus driven dynamics, we

also suppress the activation of the most recently at-tended location by adding a large negative activationin Eq.(10) at the location of the last saccade target.This strategy implements an inhibition of return [14]and ensures that the robot does not keep attendingto the same location in the continuous presence ofan interesting stimuli. The plots at the bottom ofFig. 7(a)(b) illustrate the behavior of the activationdynamics just before and after an attention shift, in-cluding the effect of the negative activation after thesaccade.

4.2 Planning and generation of motorcommands

Given a new saccade target, extracted from thesaliency map, the direction of gaze needs to be shiftedto the center of this target. Since fifth order splines are

a good approximation of biological movement trajec-tories [21, 22], we use this model to compute a desiredtrajectory from the current position x0 to the targetxf , all expressed in camera coordinates.The camera-space trajectory is converted to joint

space by inverse kinematics computations based onResolved Motion Rate Control (RMRC) [23]. We as-sume that only head and eye motion is needed to shiftthe gaze to the visual target, an assumption that isjustified given that the target was already visible inthe peripheral field of view. For the time being, theinverse kinematics computation is performed for theright eye only, while the left eye performs exactly thesame motion as the right eye. Thus, we need to mapfrom a 2D camera space of the right eye to a 5D jointspace, comprised of pan and tilt of the camera, and 3DOFs of the robot’s neck. To obtain a unique inverse,we employ Liegeois [23] pseudo-inverse with optimiza-tion:

θ = J#x+ (I − JJ#)knull (14)where J# = JT(JJT)−1

knull is the gradient of an optimization criterion w.r.t.the joint angles θ. The second term of the Eq.(14) isthe part that controls the movement in the null spaceof the head-eye system. Any contribution to θ fromthis term will not change the direction of gaze but willonly change how much we use the head or eye DOFs torealize that gaze. As optimization criterion we chose:

L =12

∑i

wi(θi − θdef,i)2 (15)

resulting in

knull,i =∂L

∂θi= wi(θi − θdef,i) (16)

(17)

This criterion keeps the redundant DOFs as close aspossible to a default posture θdef . Adding the weightswi allows giving more or less importance to enforcingthe optimization criterion for certain DOF–this fea-ture is useful to create natural looking head-eye coor-dination.Once the desired trajectory is converted to joint

space, it is tracked by an inverse dynamics controllerusing a learned inverse dynamics model [24].

4.3 Experimental result

We implemented the visual attention system on ourhumanoid robot. The stimulus dynamics and saliencymap had 44x44 nodes, i.e., twice the length and width

Figure 8: Snap shots of the robot’s peripheral viewbefore, during, and after an attentional head-eye sac-cade, taken at 30 Hz sampling rate. Superimposed onthe images is the visual flow field.

of the 22x22 nodes of the visual flow grid of the pe-ripheral vision. This extended size assured that after asaccade, the remapping of the saliency map and stim-ulus dynamics could maintain stimuli outside of theperipheral vision for some time. The Jacobian neededfor the inverse kinematics computation was estimatedwith linear regression from data collected from mov-ing the head-eye system on randomized periodic tra-jectories for a few minutes. Due to the limited rangeof motion of the eye and head DOFs, the Jacobiancould be assumed to be constant throughout the en-tire range of motion of head-eye system, which wasconfirmed by the excellent coefficient of determinationof the regression of the Jacobian. The saliency mapwas able to determine winning targets at about 10Hz,which is comparable to the capabilities of the humanattentional system.

An illustration of the working of the attentional sys-tem is provided in Fig. 8. The top image shows therobot’s right eye peripheral view of the lab, focusingon the person in the middle of the image. At the bot-tom left part of the image, another person was wavinga racket to attract the robot’s attention. This mo-tion elicited a saccade, recognizable from the middleimage of Fig. 8 which shows the visual blur that therobot experienced during the movement. The bottomimage of Fig. 8 demonstrates that after the saccade,the robot was correctly focusing on the new target.Note that the three images were sampled at 30Hz,indicating that the robot performed a very fast head-eye saccade of about 60ms duration, which is againcomparable to human performance. Our future workwill augment the attentional system with more sen-sory modalities, including the learning of weighting ofsensory modalities for different tasks.

5 Conclusion

In this paper, we presented our research on hu-manoid oculomotor control, focusing on models ofthe VOR-OKR reflex system, smooth pursuit, andsaccades based on concepts of computational neuro-science. We demonstrated the performance of theseoculomotor behaviors on the vision head that is anintegral part of our humanoid robot - a specializedrobotic platform developed with a strong emphasison brain science research. In all given examples, therobot control mechanisms were derived from principlesof computational neuroscience that was proved to beable to generate viable solutions for robotic controlwith good performance. Our efforts not only presentnovel control design of the humanoid robot, but alsoaim to contribute to brain science by proposing newbiological control models and circuits and posing in-teresting problems towards exploratory neuroscience.Currently, our work is dedicated to the integration

of the VOR-OKR, smooth pursuit and saccadic sys-tems to give our humanoid the basis of the visual per-ception. In preliminary studies, we have been success-ful in implementing the VOR-OKR and smooth pur-suit behaviour by accesing only the retinal velocityinformation and integrating it to compute the reti-nal positions. This is more robust than trying tocompute with retinal error information and any po-sitional errors that acculmulate are periodically cor-rected through the saccade behavior. We will work onthese issue more elaborately in general cases, and hopeto move on to more cognitive topics in the future.

References

[1] T. Shibata and S. Schaal. Biomimetic gaze stabi-lization based on feedback-error-learning with non-parametric regression networks. Neural Networks,14(2):201–216, 2001.

[2] M. Kawato. Feedback-error-learning neural networkfor supervised motor learning. In R. Eckmiller, editor,Adv Neural Comput, pages 365–372. Elsevier, North-Holland, 1990.

[3] H. Gomi and M. Kawato. Adaptive feedback controlmodels of the vestibulo-cerebellum and spinocerebel-lum. Biological Cybernetics, 68:105–114, 1992.

[4] S. Vijayakumar and S. Schaal. Fast and efficient incre-mental learning for high dimensional movement sys-tems. In IEEE Int Conf Robot and Automat, pages1894–1899, 2000.

[5] A.G. Barto, R.S. Sutton, and C.W. Anderson. Neu-ronlike adaptive elements that can solve difficultlearning control problems. IEEE Trans Systems,Man, and Cybern, SMC-13:834–846, 1983.

[6] N. Schweighofer, M.A. Arbib, and P.F. Dominey. Amodel of the cerebellum in adaptive control of sac-cadic gain. Biol Cybern, 75:19–28, 1996.

[7] A.H. Fagg, N. Sitkoff, A.G. Barto, and J.C. Houk.Cerebellar learning for control of a two-link arm inmuscle space. In Proc IEEE Conf Robot and Automat,pages 2638–2644, 1997.

[8] S.G. Whittaker and G. Eaholtz. Learning patterns ofeye motion for foveal pursuit. Assoc Res Vis Ophthal,23(3):393–397, 1982.

[9] K.J. Bradshaw, I.D. Reid, and D.W. Murray. TheActive Recovery of 3D Motion Trajectories and TheirUse in Prediction. IEEE Trans on Pattern Analysisand Machine Intel., 19(3), 1997.

[10] C. Koch and S. Ullman. Selecting one among themany: A simple network implementing shifts in selec-tive visual attention. A.I. Memo 770, MassachusettsInstitute of Technology, January 1984.

[11] E. Neibur and C. Koch. Computational architecturesfor attention. In R. Parasuraman, editor, The At-tentive Brain, pages 163–186. MIT Press, Cambridge,MA, 1998.

[12] R. Parasuraman. The Attentive Brain. MIT Press,Cambridge, MA, 1998.

[13] L. Itti and C. Koch. A comparison of feature com-bination strategies for saliency-based visual attentionsystems. In Proc. SPIE Human Vision and ElectronicImaging IV (HVEI’99), San Jose, CA, volume 3644,pages 473–82, January 1999.

[14] L. Itti and C. Koch. A saliency-based search mech-anism for overt and covert shifts of visual attention.Vision Research, 40(10-12):1489–1506, 2000.

[15] R. Rao and D. Ballard. Learning saccadic eye move-ments using multiscale spatial filters. In G. Tesauro,D. S. Touretzky, and T. K. Leen, editors, Advances inNeural Information Processing Systems 7 (NIPS 94),pages 893–900. MIT Press, 1995.

[16] K. Kopecz and G. Schoner. Saccadic motor planningby integrating visual information and pre-informationon neural dynamic fields. Biological Cybernetics,73:49–60, 1995.

[17] C. Breazeal and B. Scassellati. A context dependentattention system for a humanoid robot. In Proc ofIJCAI-99, pages 1146–1151, 1999.

[18] J.A. Driscoll, R.A. Peters II, and K.R. Cave. A visualattention network for a humanoid robot. In Proc IntConf Intell Robot Syst (IROS-98), pages 1968–1974,1998.

[19] S. Amari. Dynamics of pattern formation in lateral-inhibition type neural fields. Biological Cybernetics,27:77–87, 1977.

[20] S. Amari and M.A. Arbib. Competition and coop-eration in neural nets. In Metzler, editor, Systemsneuroscience, pages 119 – 165. Academic Press, NewYork, 1977.

[21] M. Kawato. Internal models for motor control andtrajectory planning. Current Opinion in Neurobiol-ogy, 9:718–727, 1999.

[22] G.R. Barnes. Visual-vestibular interaction in the con-trol of head and eye movement: The role of visualfeedback and predictive mechanisms. Prrogress inNeurobiology, 41:435–472, 1993.

[23] A. Liegeois. Automatic supervisory control of theconfiguration and behavior of multibody mechanisms.IEEE Transactionsn on Systems, Man, and Cybernet-ics, 7:868–871, 1977.

[24] S. Vijayakumar and S. Schaal. LWPR : An O(n) al-gorithm for incremental real time learning in high di-mensional space. In Proc. International Conferenceon Machine Learning (ICML 2000), pages 1079–1086,2000.