Combining Dynamical Systems Control and Programmingby Demonstration for Teaching Discrete Bimanual

Coordination Tasks to a Humanoid Robot

Elena Gribovskaya and Aude BillardLASA Laboratory – Ecole Polytechnique Federal

de Lausanne(EPFL)http://lasa.epfl.ch

CH-1015 Lausanne, Switzerlandelena.gribovskaya@epfl.ch,aude.billard@epfl.ch

ABSTRACTWe present a generic framework that combines DynamicalSystems movement control with Programming by Demon-stration (PbD) to teach a robot bimanual coordination task.The model consists of two systems: a learning system thatprocesses data collected during the demonstration of thetask to extract coordination constraints and a motor systemthat reproduces the movements dynamically, while satisfy-ing the coordination constraints learned by the first system.We validate the model through a series of experiments inwhich a robot is taught bimanual manipulatory tasks withthe help of a human.

Categories and Subject DescriptorsI.2.6 [Artificial Intelligence]: Learning; I.2.9 [ArtificialIntelligence]: Robotics.

General TermsAlgorithms, Performance.

KeywordsProgramming by Demonstration(PbD), Learning by Imita-tion, Human-Robot Interaction(HRI), Dynamical Systems,Bimanual Coordination, Humanoid Robot

1. INTRODUCTIONRecent advances in Robot Programming by Demonstra-

tion (PbD) endow robots with abilities to learn differenttasks from human demonstrators in a natural and effectiveway. PbD operates at different levels of the task represen-tation: from copying low-level features of the motion [3, 19]to inferring the user’s intention and using a symbolic repre-sentation [5]. We follow a low-level approach to PbD, which

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.HRI’08, March 12–15, 2008, Amsterdam, The Netherlands.Copyright 2008 ACM 978-1-60558-017-3/08/03 ...$5.00.

encapsulates the task constraints at the trajectory-level andapply it to the study of learning of bimanual coordinationskills in humanoid robots.

Discrete bimanual coordination in Robot Programming byDemonstration (PbD) has received little attention so far. Bi-manual coordination is not treated as a constraints per se,but emerges from requiring that the robot reproduces ex-actly the demonstrated joint trajectories [21], or some givenobject/hand relation [4]. However, none of these works ad-dress the problem of inter-limbs coordination explicitly byextracting high-level features of coordination.

From the viewpoint of the conventional robot trajectoryplanning, two-arms manipulation is a relatively researchedtopic, yet under certain limited assumptions. E.g., in [6,20]the authors propose the approach to planning of both kine-matic and dynamic aspects of linear movements; however, ithas some restrictions: only horizontal movements are con-sidered, and the model itself doesn’t reflect coupling betweenthe two arms. The latter drawback excludes the control oversynchronization and mutual adaptation between the arms.The method is not meant to deal with external perturba-tions and learning. Some researchers adopt behavior-basedapproach to manipulation [7], it is advantageous in the sensethat a robot has several hard-coded models of behavior, be-tween which it can switch depending on a situation, pro-viding a versatile feedback on changing conditions. Never-theless, this approach needs algorithms implementing eachparticular behavior, i.e., grasping, coordination, etc.

In our work we try to overcome the limitations of theexisting engineering and RbD approaches to bimanual ma-nipulation, specifically, to find constraints responsible forcoordination and the way to learn them; we are also lookingfor the algorithm for bimanual task reproduction allowingto adhere to a learned model of a task and generate naturaland robust trajectories. We were inspired by the intensiveresearch in bimanual coordination in human movement sci-ence. The past few years, theories of coordinative structureand coordination dynamics of human motion [13] have beengaining ground. These theories successfully explain and pre-dict the emergence of coordination patterns in rhythmicalmovements, see e.g. Haken-Kelso-Bunz model [9]. In [18]the nonlinear dynamical system framework of rhythmicalpattern formation was extended to explain a tendency to-wards synchronization in discrete bimanual movements withdifferent amplitudes to account for experimental data [13].

In this paper, we follow such a dynamical system approach

to encoding coordinated motion patterns. Specifically, weconsider that a discrete coordinated movement can be de-scribed as a set of coordination patterns that change eachother dynamically. We extend this approach to allow thesystem to be trained through a PbD paradigm.

According to dynamical system theory the movement pat-terns are described by collective variables, which contain in-formation about the coordination patterns – spatio-temporalconstraints typical of certain movements and governing co-operative behavior of the two arms. One part of the problemof explaining pattern formation in a coordinated task is tofind an appropriate set of those variables. E.g., in [9] therelative phase between the two limbs is a suitable candidatefor explaining the collective variable in rhythmical move-ments. Here, we hypothesize that, for discrete goal-directedmovements, by analogy to the rhythmic case, the relative po-sition between the two arms is an appropriate candidate forthe collective variable. Stable positions (attractors) in thisvariable’s state space represent stable coordinated postures(coordination patterns), that must be reached in sequenceto perform a task, see fig. 1 for an illustration.

Figure 1: Top: Example of a 3-step sequence of co-ordinated postures through which the robot transitwhen performing a Tea task. The three posturesconsist in: initial position, putting a piece of sugarin a cup, and back to the initial position. Bottom:Relative trajectory of the two hands along the x co-ordinate. The arrows superimposed to the trajec-tories show the location of the attractors (specifiedby ellipses), that correspond to the stable posturesvisible in the top pictures.

2. SYSTEM OVERVIEWOur model is composed of two systems (see fig.2): 1)

a learning system, responsible for extracting the task con-straints and, thus, for skill acquisition; 2) a motor system,that dynamically generates the movements while satisfyingthe coordination constraints.

To generate unambiguous values for the two arms’ trajec-tories during the reproduction we have to specify in additionto the relative position between the two arms, the positionof one of the arms. In bimanual tasks, one arm is usuallymore limited in its motion than the other. Therefore, weextract the key postures by analyzing the trajectories of themost constrained arm and of the relative position betweenthe two hands, see fig. 4. We then infer the key postures ofthe less constrained arm based on this data.

In our experiment training of the robot was done throughkinesthetic teaching, i.e. by a human demonstrator guidingpassively the robot’s arms through the motions required toperform the task. Teaching the robot this way is advanta-geous in that it solves the correspondence problem. Kines-thetic coaching can be especially relevant for transferringcoordination skills. Studying a coordinated movement in-volves the adoption of an optimal movement pattern. How-ever, the optimality of such patterns is still a matter of con-tention in human movement science [11]; due to the fact thatoptimality to a great extent is different for each person, sim-ply mimicking the postures will not necessarily be optimal.This problem is even more difficult in robotics because ofthe robot’s limited workspace. Robots cannot move theirlimbs into all the configuration available to humans. Whenthe demonstrator moves the robot’s arms, she can feel therobot’s limitations, and properly adapt her motions to suitthe robot’s particular abilities.

2.1 The Learning ProcessDuring the learning process a robot builds a model of a

skill by observing several demonstrations.

2.1.1 Data acquisitionWe use a Fujitsu HOAP-3 humanoid robot with 28 de-

grees of freedom (DOFs), 4 DOFs per arm. As we wereinterested solely in the arms’ movements (8 DOFs total), allother DOFs were set on constant positions. In the experi-ments reported here we produced 5 demonstration of eachtask. The recorded set of data for each task consists of Nobservations of joint angles values: Θ = {θR

t , θLt ; t = 1..N},

where θRt , θL

t ∈ R4 (here and further upper indexes ”R” and”L” refer to either the right or the left arm).

2.1.2 Data preprocessingData collected during kinesthetic teaching must be pro-

cessed in order to harmonize a dimensionality between differ-ent trials, reduce the noise and the temporal dissimilarities.Thus the joint angles data Θ recorded from the demonstra-tions are smoothed with 1D Gaussian filter of size 5; resam-pled to a unified number of observations M ; and temporallyaligned by Dynamic Time Warping (DTW) [17].

During the experiments it was observed, that the tempo-ral dissimilarities between the different trials have a nega-tive impact on reliability of temporal constraints extractedby the learning system. A human demonstrator cannot pre-serve the same velocity along the analogous parts of com-plex movements, therefore misalignment between the differ-ent demonstrations are frequent. As a result, it becomesdifficult at the subsequent stages of learning to match thepatterns and get relevant time properties. Hidden MarkovModels (HMMs) are known for their ability to deal withtemporally heterogeneous data. However in our case, as wewant not only to learn the correct sequence of postures, butalso to learn their duration, we have to apply DTW priorHMMs to harmonize the signals in the time domain.

After pre-processing the recorded set of joint trajectoriesΘ, the trajectories of the end-effectors {xR

t , xLt ; t = 1..M},

xRt , xL

t ∈ R3 and the relative position dt = xRt − xL

t acrossthe two arms are calculated by vectorial projection.

Figure 2: System overview: the arrows show the input-output flow across the system. The original signalΘ is preprocessed to reduce the dimensionality and align temporally the signal. The resulting signal X isthen segmented into a set Π of stable postures, which are further encoded in a HMM, to be reduced to ageneralized set of spatio-temporal constraints P . The robot’s motion between each generalized posture aregenerated by a dynamical system, taking into account the robot’s geometry given by J the Jacobian. Controlapplies to both the position of arms xt and to the joint angle configuration θt at each time step.

2.1.3 Key postures extractionIn our work we consider a coordinated bimanual move-

ment as a dynamical transition between stable coordinatedpostures. To automatically extract these postures from thedata set X = {xR

t , dt; t = 1..M} we apply Mean Square Ve-locity analysis in Cartesian space to the trajectory of therelative position d.

In [14], a method was proposed to automatically segmenta trajectory of a complex movement into episodes. We mod-ified this approach to suit our task: each coordination pat-tern is described by the set of stable states of the relativeposition variable d. Stability is used here in a loose senseand corresponds to period of time during which d stay ap-proximately constant (see fig.1) and the velocity decreases.In the ideal case it falls to zero. However, in practice, we canonly observe it decreasing below a certain threshold. Thuswe look in the Mean Square Velocity function {Vt; t = 1..M}specified by (1) for points tb

i (an upper index ”b” indicatesthat a variable refers to the beginning of a segment) at whichVtb

i< V ∗, where V ∗ is specified by (2).

Vt = (dxt )2 + (dy

t )2 + (dzt )

2, t = 1..M ; (1)

where d is the rate of change of the relative position d;

V ∗ =< V > −0.5σV ; (2)

where < V > and σV are accordingly the mean value andthe standard deviation of Mean Square Velocity.

We then consider tbi as the beginning of the corresponding

ith segment, and start to look for a point tei (an upper index

”e” indicates that a variable refers to the end of a segment)at which Vte

i> V ∗. If we find such point it means that the

values of the relative position d on a interval [tbi , t

ei ] describes

the key posture di.To specify the key postures of the right arm xR

i , we set upa correspondence between them and the key postures of therelative position di. As a result the key postures of the rightarm take the following values: xR,b

i = xRtbi, xR,e

i = xRtei, i =

1..Π. The data set used for stochastic posture encoding hasthe following structure:Π = {di, x

R,bi , xR,e

i , tbi , ∆ti; i = 1..NΠ}.

2.1.4 Stochastic postures encodingWe have several motivations to encode the postures stochas-

tically with HMMs before their reproduction. These are: 1)to get rid of the spurious postures (the postures that occa-sionally appear as a result of an involuntary deceleration ofa demonstrator’s motion, they are not relevant for the repro-

duction)(see fig.4 for illustration)1; 2) to join the posturesclose to one another (in the sense of time and distance); 3)to extract typical spatial and temporal characteristics of thepostures.

We represent the key postures for the relative position dby a single continuous HMMs that encodes the set of thekey postures in terms of both the time of their appearanceand their duration {di, t

bi , ∆ti; i = 1..NΠ}.

The most general approach is to use a fully-connectedmodel. Training this model needs a large set of trainingdata, however the PbD paradigm assumes that a humandemonstrator should not be asked to repeat a task an un-bearable number of times. Hence, we incorporate prior knowl-edge about the data structure into the model through biasedtransitional probabilities. The states in our model repre-sent certain events in the trajectory. Thus to guaranteethe smoothness of the trajectories at the reproduction stageand to avoid knots, we constrain the possible transitions, bypreventing backward transitions. We use a left-right model,with the particularity that self-transitions are not-allowed.Each emission probability is approximated with a single mul-tivariate Gaussian distribution, and represents a key posturewith its time properties.

The initialization of the parameters in HMMs plays a cru-cial role in the convergence of training: here we apply theBaum-Welsh algorithm [16], that locally maximizes the like-lihood of data to obtain the estimation of HMMs parame-ters. Therefore if we initialize them randomly, the algo-rithm in most cases will converge to a wrong solution. Thewidely used method for initialization of HMMs is the K-means method [15]. However, it assumes that the numberof clusters (corresponding to hidden states) is known, andconsequently, before applying this method to initialize theHMMs, we first have to decide how many hidden states NP

our model should have.The problem one encounters after segmentation is a dif-

ferent number of segments extracted for each trial. For K-means different cluster validation methods were proposed[12] in order to estimate whether a found partition properlyexplains the data and how reliable it is. We adopted the

1Given the accidental nature of such postures, the probabil-ity to transit into such a state is comparatively small, andwe can thus eliminate them. Here, we fixed a threshold forthe transitional probabilities of 0.2, under which transitionswill not be allowed.

Dunn index according to speculations in [1] and the struc-ture of our data, to validate the number of clusters our datacontains. The final criterion reads as follows:

Γ(U) = max1≤i≤m

{ max1≤j≤m

δ(Mi, Mj)

m max1≤l≤m ∆(Ml)}; (3)

where U is a current partition, consisting of a set of clusters{Mi, i = 1..m}, δ(S, G) = maxs∈S,g∈G(‖s−g‖) is a between-cluster distance, ∆(S) = maxs,p∈S(‖s− p‖) - within-clusterdistance. The maximum value of the criterion (3) points outto the optimal partition.

The set of spatio-temporal constraints P has the follow-ing structure: P = {di, x

R,bi , xR,e

i ,tbi , ∆ti; i = 1..NP }, where

di, xR,bi , xR,e

i are, accordingly, updated values of the relativeposition and positions of the right arm in the beginning andin the end of a ith stable posture; tb

i , ∆ti are the time ofemergence and the duration of a ith posture.

2.2 Reproduction processNext, we explain how the generalized set of postures learned

by the HMM is used to control the reproduction of the taskby the robot (see fig.2).

2.2.1 Hybrid controllerIn previous work of ours [10], we proposed a hybrid con-

troller for reaching movements in humanoid robots. Thissystem is based on Grossberg’s model of human reachingmovement – Vector Integration to Endpoint (VITE) [2], andfollows a current trend in human movement control: move-ments are not planned in a single frame of reference; rather,several frames of reference are involved in planning. Thisconcept brings a certain degree of redundancy to the sys-tem, but we showed how such redundancy can be efficientlyexploited to resolve the joint limit avoidance problem. Inthe work we report here, we took this controller as the basisfor our motor system, see fig. 3.

The hybrid controller operates as follows. In the beginningof each part of a movement, the target Cartesian position ofthe left arm is reconstructed from the target position of theright arm and the stable posture that should be reached:xL,b

i = xR,bi − di; x

L,ei = xR,e

i − di, the target joint angles

θR, θL are computed by inverse kinematics from the targetCartesian positions.

Then, the desired manipulator configurations θR,d(t+1), θ

L,d(t+1)

and xR,d(t+1), x

L,d(t+1) are obtained at each time step by the

VITE controllers. This is done through the equations (4)-(6) (we present here only the equations for the right arm, asthe equations for the left arm are identical):

θR,d(t+1) = θR

t + αRθ (−θR

t + βRθ (θR

i − θRt )); (4)

xR,d(t+1) = xR

t + αRx (−xR

t + βRx (xR

i − xRt )); (5)

θR,d(t+1) = θR

t + θR,d(t+1); x

R,d(t+1) = xR

t + xR,d(t+1); (6)

where αRx , αR

θ , βRx , βR

θ are empirically derived constants.Generally, the desired arm configuration θd

t+1 is incompat-ible with the desired end-effector position xd

t+1, i.e. xdt+1 6=

K(θdt+1). However, consistent positions exist in the certain

neighborhood of the desirable position. To find such posi-tions we minimize a function H(θR

t , θLt , xR

t , xLt ) (7) imposing

on it coherent constraints (8), that are mathematical expres-

sions of the robot’s body constraints.

H(θRt , θL

t , xRt , xL

t ) = (θRt − θR,d

t )′W Rθ (θR

t − θR,dt )+

(xRt − xR,d

t )′W Rx (xR

t − xR,dt ) + (θL

t − θL,dt )′W L

θ (θLt − θL,d

t )+

(xLt − xL,d

t )′W Lx (xL

t − xL,dt ); (7)

where W Rθ , W L

θ , W Rx , W L

x are positive diagonal matrices, thatcontrol the influence of each controller 2.

xRt = K(θR

t ); xLt = K(θL

t ); (8)

where K(.) is the kinematic function of the robot arm.Solving (7) and(8) using the Lagrange multipliers, we get

coherent values for the robot’s joint angles that can then beused to control the robot:

θR(t+1) = θR

t + (W Rθ + J ′Rt W R

x JRt )−1(J ′Rt W R

x (xR,d(t+1) − xR

t )

+ W Rθ (θR,d

(t+1) − θRt )); (9)

where JR, JL are the Jacobians of the right and the left arm:xR = JRθR, xL = JLθL

To avoid the joint limits, we can gradually move froma cartesian controller to a joint angle controller when ap-proaching the workspace boundaries, using time-dependentmatrices W R

θ , W Lθ , W R

x , W Lx , see [10]:

wx,t

wθi,t= 0.5γ(1− cos(2π

θi,t − θi,min

θi,max − θi,min)); (10)

where wx,t is the cartesian weight (i.e., a diagonal elementof Wx), wθi,t is the weight of angle i at a given time t (i.e.,a diagonal element of Wθ), θi,min and θi,max are the cor-responding joint angle boundaries, θi,t is the correspondingangular position at time t, and γ is a constant setting themaximum value for wx/wθi .

2.2.2 Spatial constraintsTo preserve the coordination pattern learned from the

demonstrations at given stages of the movement, we addto the above system a spatial constraint, according to:

xRt − xL

t = 0 (11)

Solving jointly the constraint optimization problem (7), (8)and (11) we get:

θR(t+1) = θR

t + (M1)−1M2; (12)

θL(t+1) = θL

t + [(JLt )−1JR

t ]θR(t+1); (13)

M1 = W Rx JR

t + (J ′Rt )−1W Rθ + W L

x JRt

+ (J ′Lt )−1W Lθ (JL

t )−1JRt ; (14)

M2 = W Rx (xR,d

(t+1) − xRt ) + (J ′Rt )−1W R

θ (θR,dt+1 − θR

t )

+ W Lx (θL,d

(t+1) − θLt ) + (J ′Lt )−1W L

θ (θL,d(t+1) − θL

t ) (15)

2.2.3 Time constraintsFurther, we add a time constraint to guarantee the syn-

chronization of the arms (adaptation of one arm’s velocityto another one) and the timing of the whole movement.

In general a discrete reaching movement is defined by itsstarting and ending positions. It can also be characterizedby the mean movement time, as well as its precision [8].Varying the parameters of the VITE model (4)-(6), while

2”′” refers to the transpose operator

Figure 3: Overview of the robot’s Motor System

the initial and target position are fixed, one can either in-crease or decrease the velocity or the movement duration.Here, we want on the contrary to find such values of thesystem’s parameters that will guarantee that we have thedesired time properties. We thus compute α and β to makeboth arms reach the target positions simultaneously, evenif their amplitudes are different (in agreement with experi-mental data on human motion, see [13]). We consider thecontinuous form of the VITE model and apply the followingboundary conditions:

x(0) = x0; v(0) = v0; x(∆t) = x1; v(∆t) = v1; (16)

where ∆t is the duration of the movement (extracted by thelearning system), x0, v0, x1, v1 – respectively the Cartesiantrajectories and velocities of the arm at the onset and end ofthe movement. The boundary conditions have to satisfy thefollowing requirement: ||(x1 − x)/(x0 − x)|| ¿ 1, where x –the target position. In addition, for each stable posture tosatisfy the formal definition of an attractor of a dynamicalsystem the velocity at the end of the movement must satisfy:||v1|| ¿ 1.

The VITE model in the form we consider is a linear dy-namical system. Therefore it has one attractor that canbe either stable or unstable. We choose the eigenvaluesλ1 and λ2 for our system that are complex conjugate withnegative real parts: λ1 = −m + ni, λ2 = −m − ni, wherem, n ∈ R, m > 0 – are constants that must be specified, i– imaginary unity. Given the above, the attractor of theVITE model is asymptotically stable and the parameters αand β take the following values:

α = 2m; β = (4n2 + α2)/(4α). (17)

Taking into account the above mentioned boundary condi-tions, we get for m, n:

n = π/∆t; m = log((x0 − x)/(x1 − x))/∆t. (18)

If during the motion the robot encounters an external per-turbation that affects the target position, the motor systemhas to react on it by adapting the velocity of both armsappropriately. In our system this goal is achieved by recom-puting the parameters α and β.

Table 1: A number of the postures found automati-cally by the Learning system

Name of task Maximumnumberof seg-ments

Numberof pos-tures

Tea task 6 4Cube task 5 3Tray task 7 4

3. EXPERIMENTAL RESULTSWe conducted three experiments to test our model for

teaching the humanoid robot discrete coordinated move-ments. This procedure aimed to demonstrate how the robot’smotor system could adapt to fulfill the coordination con-straints of the tasks.

After learning, the robot has the implicit information aboutan object involved in a task, as its size and the initial po-sition occur encoded in the collective variable and positionof the leading arm. During the reproduction of a task thepositions of the objects on the scene are tracked with therobot’s stereovision system and partly with external stereo-vision system to increase the angle of view. The robot alsohas information concerning the time that should be spent foreach part of the task, and about the fact that during certainparts of the movement, it should preserve the relative posi-tion between the two arms. All trajectories of the robot’sarms are computed with respect to the frame of referencelocated at the center of the robot’s waist.

Fig. 4 shows an example of posture encoding for the Teatask. The learning system determined four stable posturesalong the relative position d of the two arms. Only threeof these postures (number 1,3,4) are relevant. Posture 2 isspurious, as it does not correspond to any specific move-ment pattern and appears only in a small number of trials.We can then exclude it from the reproduction because theprobability to transit through it is lower than the specifiedthreshold.

Table 1 collects information about the automatic selectionof a number of key postures candidates in the three tasks. Ingeneral the segmentation procedure tends to extract moresegments than there exist in reality. Statistical encodingwith HMMs allows to optimize their number.

Figure 4: Encoding of the Tea task. On the pictures are plotted: the demonstrated trajectories of the relativeposition d with beginnings of the extracted postures 1-4 (they are represented with the corresponding varianceinformation – bold ellipses); the demonstrated trajectories of the right arm. For the right arm, we extractboth the beginnings and the ends of the postures.

Figure 5: Set-ups of the 3 experiments. Tea task :put the piece of sugar into the cup. Cube task : graspthe cube, lift it, and put it on top of the pedestal.Tray task : grasp the tray with two arms, lift it, andmove it forward.

Fig. 6 - 8 share the same legend: each figure contains thepicture with a projection of reproduced trajectories into anaxial plane, an accessible workspace of the robot in a currenttask and photos of an experiment with the superimposedtrajectories. On the robot’s workspace light areas point outto reachable positions.

Fig.6 shows the results of the Cube Task. To test themotor system adaptability to external perturbations we firstchanged the position of the cube while the robot was tryingto grasp it, and then we changed the position of the pedestalonto which the robot had to put the cube. During the secondperturbation the robot was carrying the cube and thus hadto preserve the relative position between its arms so as notto lose the cube. All positions of the objects on the scenewere chosen so that they were reachable and thus within therobot’s workspace.

Fig. 7 illustrates the results of reproducing the Tea task.In this experiment we also applied a perturbation twice:both times we simulated the situation where the right armof the robot was suddenly pushed. This was achieved bysending the perturbed command to the arm’s joints. Thefirst time the jerk was initiated while the robot was bring-ing the two arms together, the second time, it was appliedwhen the robot was putting the piece of sugar into the cup.The sudden changes were detected by measuring a discrep-ancy between the planned position of the robot’s arm jointsand the feedback from the motors. In each case, the robotreadapts the position of both arms to make sure that thepiece if sugar will not fall outside the cup.

Fig. 8 illustrates the results of reproducing the Tray task.We changed the position of the tray, while the robot wastrying to grasp it.

Fig. 9 illustrates the preservation of the synchronizationfeature of the movement of the two arms of the robot dur-ing the Cube task. At time t1 while the robot was movingthe arms towards the cube, we changed the position of thecube. Both arms adapted their trajectory simultaneouslyto handle this perturbation and reached the target simul-taneously. In the same figure we show the velocity profilesof both arms in both Cartesian and joint-angle spaces. Wesee that the motor system produce smooth and bell-shapedvelocity profiles similar to that of humans.

4. DISCUSSIONThis work attempted to shed some light on what are the

main aspects of bimanual coordination to guarantee satis-factory robot’s performance in simple manipulatory tasks.We investigated two types of constraints: spatial constraints(e.g. the two arms must adopt a specific spatial relation toone another) and temporal constraints (the two arms mustsynchronize and should reach a target at the same time).Satisfactory performance was deemed achieved when therobot managed to go through the set of required postures,specifying bimanual constraints. The robot was, however,free to depart significantly from the arm trajectories shownduring the demonstration in between each of these postures.

During the reproduction we applied several external per-turbations to test the robustness of the system against those.While we could show that in some restricted parts of theworkspace, the robot did handle well the perturbations, wemust also point out two factors that may deteriorate therobot’s performance: 1) the bimanual spatial constraintslimit the robot’s movements and, thus, reduce importantlythe part of its workspace; 2) the magnitude of a pertur-bation should be sufficiently small and fast that the robotcan still reach the target during the required duration ofthe movement; failing this, the system can potentially reactwith jerky movements. The above considerations determinethe directions of our future work, namely: 1) to investigatebimanual coordination tasks with more pronounced asym-metry between the two arms; 2) to develop a mechanismin the motor system to automatically modify the time con-straints, when those may become harmful for the robot.

Figure 6: Cube task. Top-left: the robot tries tograsp the cube while its position is changed fromA to B and it simultaneously adapts both armsto grasp the cube at position B. Top-right: whilethe robot is carrying the cube, the position of thepedestal is changed from C to D, the robot putsthe cube on the new location preserving the relativeposition between its arms.

5. CONCLUSIONSWe presented a novel approach to learn discrete biman-

ual coordination skills in a humanoid robot. Its main as-pects are: 1) a system that can automatically extract spatio-temporal coordination constraints across the two end-effectors;and 2) a motor system consisting of several coupled dynam-ical systems that is able to generate coordinated movementson-line, handle perturbations, while satisfying the learnedcoordination constraints.

The system was validated in three experiments where ahumanoid robot was taught simple discrete bimanual coor-dinated tasks. We showed that the system could successfullyreproduce the tasks under various external perturbations.Moreover, we identified a number of basic features of discretebimanual coordination essential for effective manipulation.

6. ACKNOWLEDGMENTSThis work is supported by the European Commission through

the EU Projects Robot@CWE and Feelix-Growing.

Figure 7: Tea task Top-left: the robot is bringingthe two arms together, while the sudden change inposition of the right arm is simulated, the robot si-multaneously adapts both arms to put a sugar intothe cup from the other position. Top-right: whilethe robot’s arms are in the stable posture (the rel-ative position between the arms is preserved) theright arm changes its position abruptly, however theMotor system preserves the posture and adapts theleft arm.

7. REFERENCES[1] J. Bezdek and N. Pal. Some new indexes of cluster validity.

IEEE Transaction on System, Man, and Cybernetics,28:301–315, 1998.

[2] D. Bullock and S. Grossberg. Neural dynamics of plannedarm movements: Emergent invariants and speed-accuracyproperties during trajectory formation. PsychologicalReview, 95(1):49–90, 1988.

[3] S. Calinon and A. Billard. Stochastic gesture productionand recognition model for a humanoid robot. InProceedings of IROS, pages 2769–2774, September 2004.

[4] S. Calinon and A. Billard. Incremental learning of gesturesby imitation in a humanoid robot. In Proceedings of theInternational Conference on HRI, pages 255–262, March2007.

[5] Y. Demiris and B.Khadhouri. Hierarchical attentivemultiple models for execution and recognition of actions.Robotics and Autonomous Systems, 54:361–369, 2006.

[6] J. Desai, M. Zefran, and V.Kumar. Two-arm manipulationtasks with friction assisted grasping. Proceedings of theIROS, 1:189–195, 1997.

[7] A. Edsinger and C. Kemp. Manipulation in humanenvironments. Proceedings of the 6th IEEE-RASInternational Conference on Humanoid Robots, pages102–109, 2006.

[8] P. M. Fitts. The information capacity of the human motorsystem in controlling the amplitude of movement. Journalof Experimental Psychology, 47:381–391, 1954.

Figure 8: Tray task : Tray task: while the robotwas trying to grasp the tray, it was moved asideto the boundaries of the workspace, even though,the perturbation applied was of big magnitude, therobot adapted successfully.

[9] H. Haken, J. Kelso, and H. Bunz. A theoretical model ofphase transitions in human hand movements. BiologicalCybernetics, 51:347–356, 1985.

[10] M. Hersch and A. Billard. A biologically-inspired model ofreaching movements. In Proceedings of the InternationalConference on Biomedical Robotics and Biomechatronics,pages 1067–1072, 2006.

[11] N. Hodges and I. Franks. Modelling coaching practice: therole of instruction and demonstration. Journal of SportSciences, 20:793–811, 2002.

[12] L. Hubert and P. Arabie. Comparing partitions. Journal ofClassification, 2:193–218, 1985.

[13] J. Kelso, D. Southard, and D. Goodman. On thecoordination of two-handed movements. Journal ofExperimental Psychological Human Perception,5(2):229–238, 1979.

[14] J. Lieberman and C. Breazeal. Improvements on actionparsing and action interpolation for learning throughdemonstration. In Proceedings of International Conferenceon Humanoid Robots, 1:342– 365, 2004.

[15] J. B. MacQueen. Some methods for classification andanalysis of multivariate observations. In Proceedings of 5-thBerkeley Symposium on Mathematical Statistics andProbability, 1967.

[16] L. Rabiner. A tutorial on hidden markov models andselected applications in speech recognition. Proceedings ofIEEE, 77:257–286, 1989.

[17] H. Sakoe and S. Chiba. Dynamic programming algorithmoptimization for spoken word recognition. IEEETransactions on Acoustic, Speech, and Signal Processing,pages 43–49, 1978.

[18] G. Schoner. A dynamic theory of coordination of discretemovement. Biological Cybernetics, 63:257–270, 1990.

[19] A. Ude, C. Atkeson, and M.Riley. Programming full-bodymovements for humanoid robots by observation. Roboticsand Autonomous Systems, 47:93–108, 2004.

[20] M. Zefran, V. Kumar, E. Henis, and J. Desai. Two-armmanipulation: What can we learn by studying humans? InProceedings of the IROS, 1995.

[21] R. Zollner, T. Afour, and R. Dillman. Programming bydemonstration: Dual-arm manipulation tasks for humanoidrobots. In Proceedings of the IROS, 2004.

Figure 9: Synchronization in the Cube task : att1 = 150 the position of the cube was moved. Si-multaneously, the robot’s Motor system adapted thetrajectories of the two arms to reach the cube. Thevelocity profiles are smooth and bell-shaped. Thetime instances t2..t6 refer to the boundaries of stablepostures